S

Wednesday, February 9th, 2022 10:03 AM

Denodo Collibra Spring Boot Integration Build error

Hello
I am trying to use the Denodo Collibra Spring boot Integration
When i try to build the POM, it shows multiple errors on various methods like below :-
“Method getCollibraCommunityName() is undefined for type Applicationconfig”
Here are the steps i have followed so far -

  1. Installed Java 1.8
  2. Installed Maven
  3. Installed Collibra_Integration_Library jar in maven repository via command line
  4. Imported the package as Maven project
    When i try to build the above errors are showing up.
    Any idea on this

1.2K Messages

3 years ago

Which command did you use to import the integration library into Maven?

28 Messages

Used the command as specified in the snipped you shared from document.
Updated the version of this jar file also in pom.xml
Updated project, restarted eclipse . still same issue

14 Messages

3 years ago

Hi all!
Any suggestions here? I’m facing exactly the same issue and even Collibra support pointed me to this forum…
Thanks in advance,
Koen

1.2K Messages

3 years ago

Have you correctly defined the properties in the application.properties file?
Specifically this bit here:

# Names of the 'community' and 'domain' in the collibra instance where the metadata will be imported.
collibra.domain=
collibra.community=

If those properties are left empty, that might explain the error. They should be filled like:

# Names of the 'community' and 'domain' in the collibra instance where the metadata will be imported.
collibra.domain=Spring boot
collibra.community=Schemas

There are many more properties you need to define.

28 Messages

Hello,

Thanks a lot for your response.
We have not done that , We were of the opinion that we should be able to build the jar file and then we can update the properties file. But as per your suggestion we will try to add these properties and see if it resolves our issue.

Regards
Simar

1.2K Messages

3 years ago

Yes, I agree with you. I’m quite puzzled with the integrations as they are today, and I’m very much waiting for the webinar on feb 24th to clarify the integration roadmap.

I saw some very good examples of custom integration frameworks, such as the Sinequa search engine.
There is a default template to follow:

  • xml to define the form in the main application to fill in the configuration
  • C# code to script the business logic of the integration

The result is a seamless integration into the application. Example here: plugins/SlackConnector at master · sinequa/plugins (github.com)

368 Messages

3 years ago

Hello everyone,

Thanks for trying the Denodo to Collibra integration.

Regarding the missing getter methods when using Eclipse IDE, it could be due to Lombok which is not provided as part of the Eclipse IDE. (More information regarding Lombok can be found here.)

Therefore, can you please try downloading the latest version from here and then follow the steps explained in this page?

Thank you

28 Messages

Hello Team,

Thanks a lot . We are able to successfully compile and build jar file with Lombok. But when we try to run this jar it throws below error “this integration requires spring boot integration library from collibra”
Now we have installed this library while building the jar file
What could be the issue here ?

368 Messages

3 years ago

Hello Simar,

Thanks for the update.

Regarding the error being encountered, can you please provide more details / the full error message or stack trace?

Additionally, can you please confirm that the following Spring Boot Integration Library was installed using the specified Maven command where the integration is being deployed/run?

28 Messages

After reinstalling the spring boot library and then triggering the application we encountered below error -


Attached file with full error stackcollibra_denodo_jar error stack.txt (30.5 KB)


368 Messages

3 years ago

Hello Simar,

Regarding the following error encountered, can you please confirm that the collibra.url does not contain an extra equals sign?

The string \"=https\" for the URI component SCHEME contains an invalid character, '=', at index 0.

It should have the following format:

collibra.url=https://<instance>.collibra.com

where:

  • <instance> should be replaced with your subdomain.

28 Messages

We rectified the URL but seems like we have again errors.
collibra.url=https://.collibra.com/
Do we need to remove the backslash at the end of URL as well ?
attaching the error stack.collibra_denodo_jar error stack.txt (30.1 KB)

368 Messages

3 years ago

Hello Simar,

Can you please check whether you Collibra instance has build number 20220302161146?

If it is the case, it is due to the REST API login request not returning the CSRF token (more details here).
To address this, a new version of the Spring Boot Integration Library should be released this week. Thank you

28 Messages

Yes it is the same build version
Please update us once the new integration library is published

368 Messages

3 years ago

Hello Simar,

Just to let you know that the updated version of the Spring Boot Integration Library, having version 1.1.4, has been released and is available on Collibra’s Marketplace.

28 Messages

Reinstalled new integration library , Rebuilt the jar using
mvn clean package
Inititated this jar file but found new error logs. Attached full error stack.
FYI Collibra authentication shows successful now
collibra_denodo_jar error stack.txt (20.3 KB)

368 Messages

3 years ago

Hello Simar,

Regarding this error, can you please confirm that the following properties were set (i.e. not empty) in the application.properties file? Thank you

  • trigger.api.username
  • trigger.api.password

28 Messages

These are empty. Can you share some information what is expected ? Which user name and passwords as we are trying to trigger this by executing jar

368 Messages

3 years ago

Hello Simar,

Sure, as specified in the Denodo to Collibra documentation, these properties are used to define the basic authentication credentials that is enabled on the HTTP listening endpoint used to trigger the integration.

Therefore, these properties should be set to a custom username and a strong password. Additionally, for security purposes, these can be encrypted as specified in section “Encrypt Configuration Properties” of the Denodo documentation.

28 Messages

3 years ago

We disabled SSL
We were able to successfully deploy the integration.
Here are the logs working_logs_deployment.txt (11.5 KB)

Now we are trying to use this integration as Linux service by creating .service file.
But we are getting the below logs and no metadata is getting ingested.error_logs_description.txt (46.0 KB)
Can you please help us ?

1.2K Messages

That looks like a permission issue. Have you given rwx access to the application user on the current folder (and subfolders)?
The application is telling you it cannot create the file logs/denodo-to-collibra-integration.log

28 Messages

We are already working on permission issue for log file creation. But if we scroll down we can see that the integration has been successfully running. But we do not see any metadata ingested.
Does that mean that first we need to resolve error on logging or this should work fine except creating log files ?

1.2K Messages

3 years ago

What makes you think anything has completed successfully? I cannot find any such message in your logs.

28 Messages

Please find attached snippet of last lines of log file after logging error was encountered. I see that attached log file in previous comment was not showing full lines of code.

These are logs when i run systemctl status command

28 Messages

@arthur.burkhardt
We changed the permission of user to create log files. Now there is no error while starting this servica via linux.Also please find attached logs denodo-to-collibra-integration.txt (8.7 KB)

But now we are not sure about next steps. We try to trigger this by calling end point HTTPS POST: https://:<server.port>/sync (Host and port where integration is running)
But it is not a valid url

28 Messages

3 years ago

Update ------
When we trigger the integration now and run the sync command , it logged below error -


2022-03-23 15:58:34,198 [http-nio-8442-exec-1] ERROR com.collibra.marketplace.denodo.exception.GlobalExceptionHandler - Internal error during execution.
org.springframework.web.HttpRequestMethodNotSupportedException: Request method ‘GET’ not supported
at org.springframework.web.servlet.mvc.method.RequestMappingInfoHandlerMapping.handleNoMatch(RequestMappingInfoHandlerMapping.java:253) ~[spring-webmvc-5.3.9.jar!/:5.3.9]
at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.lookupHandlerMethod(AbstractHandlerMethodMapping.java:438) ~[spring-webmvc-5.3.9.jar!/:5.3.9]
at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.getHandlerInternal(AbstractHandlerMethodMapping.java:379) ~[spring-webmvc-5.3.9.jar!/:5.3.9]
at org.springframework.web.servlet.mvc.method.RequestMappingInfoHandlerMapping.getHandlerInternal(RequestMappingInfoHandlerMapping.java:125) ~[spring-webmvc-5.3.9.jar!/:5.3.9]
at org.springframework.web.servlet.mvc.method.RequestMappingInfoHandlerMapping.getHandlerInternal(RequestMappingInfoHandlerMapping.java:67) ~[spring-webmvc-5.3.9.jar!/:5.3.9]
at org.springframework.web.servlet.handler.AbstractHandlerMapping.getHandler(AbstractHandlerMapping.java:498) ~[spring-webmvc-5.3.9.jar!/:5.3.9]
at org.springframework.web.servlet.DispatcherServlet.getHandler(DispatcherServlet.java:1258) [spring-webmvc-5.3.9.jar!/:5.3.9]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) [spring-webmvc-5.3.9.jar!/:5.3.9]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963) [spring-webmvc-5.3.9.jar!/:5.3.9]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) [spring-webmvc-5.3.9.jar!/:5.3.9]
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) [spring-webmvc-5.3.9.jar!/:5.3.9]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:655) [tomcat-embed-core-9.0.50.jar!/:?]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) [spring-webmvc-5.3.9.jar!/:5.3.9]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:764) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:228) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) [tomcat-embed-websocket-9.0.50.jar!/:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163) [tomcat-embed-core-9.0.50.jar!/:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:121) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:178) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.3.9.jar!/:5.3.9]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.3.9.jar!/:5.3.9]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.3.9.jar!/:5.3.9]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183) [spring-security-web-5.5.1.jar!/:5.5.1]
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) [spring-web-5.3.9.jar!/:5.3.9]
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) [spring-web-5.3.9.jar!/:5.3.9]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163) [tomcat-embed-core-9.0.50.jar!/:?]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) [spring-web-5.3.9.jar!/:5.3.9]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.3.9.jar!/:5.3.9]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163) [tomcat-embed-core-9.0.50.jar!/:?]
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) [spring-web-5.3.9.jar!/:5.3.9]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.3.9.jar!/:5.3.9]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163) [tomcat-embed-core-9.0.50.jar!/:?]
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) [spring-boot-actuator-2.5.3.jar!/:2.5.3]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.3.9.jar!/:5.3.9]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163) [tomcat-embed-core-9.0.50.jar!/:?]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) [spring-web-5.3.9.jar!/:5.3.9]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.3.9.jar!/:5.3.9]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:382) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1723) [tomcat-embed-core-9.0.50.jar!/:?]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.50.jar!/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_312]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_312]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.50.jar!/:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]

368 Messages

3 years ago

Hello Simar,

Regarding the following error, the HTTP listening endpoint of the Denodo to Collibra integration listens for request having type POST instead of GET.

Amongst others, this could be achieved through Postman (https://www.postman.com/) or using the following cURL request (X-Use-Mock-Data -> when set to true, the mock data would be used):

curl --location --request POST 'https://<host>:<port>/sync?denodoDatabase=test' \
--header 'X-Use-Mock-Data: true' \
--header 'Authorization: Basic <encoded-credentials>'
Loading...