9 Messages
DBT to Collibra Integration(v1.2.2) Error writing to server
Hello @spring-team.collibra.com ,
We’re getting a Error writing to server error.
We installed and tested the dbt to Collibra Integration (v1.2.2) in January; It ran and created assets in Collibra.
We then waited to run it again until we had installed Edge. (They are on separate machines)
Now when we try to run the dbt to Collibra Integration it crashes on an exception. It fails when it tries to create assets.
We cleaned up our environment by removing all previously created assets before rerunning it.
2023-03-15T09:54:16.379Z INFO 3392 --- [ main] com.collibra.dbt.Application : Started Application in 39.15 seconds (process running for 47.239)
2023-03-15T09:54:56.376Z INFO 3392 --- [0.0-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2023-03-15T09:54:56.381Z INFO 3392 --- [0.0-8080-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2023-03-15T09:54:56.383Z INFO 3392 --- [0.0-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 2 ms
2023-03-15T09:54:57.224Z INFO 3392 --- [0.0-8080-exec-1] c.c.dbt.controller.EntryPointController : Extract SQL for all accounts
2023-03-15T09:55:00.405Z INFO 3392 --- [0.0-8080-exec-1] c.c.dbt.component.DBTLineageExtraction : Processing Account id:18748 Project id:43985
2023-03-15T09:55:02.053Z INFO 3392 --- [0.0-8080-exec-1] c.c.dbt.component.DBTLineageExtraction : Job is running! JobId:195960
2023-03-15T10:11:31.461Z INFO 3392 --- [0.0-8080-exec-1] c.c.dbt.component.DBTLineageExtraction : Job run has been completed
2023-03-15T10:11:31.493Z INFO 3392 --- [0.0-8080-exec-1] c.c.dbt.component.DBTLineageExtraction : Snapshots extraction performing
2023-03-15T10:11:32.340Z INFO 3392 --- [0.0-8080-exec-1] c.c.dbt.component.DBTLineageExtraction : Extracted Compiled Sql files are writing to the local lineage directory
2023-03-15T10:11:32.342Z INFO 3392 --- [0.0-8080-exec-1] c.c.dbt.component.DBTLineageExtraction : ETL operations successfully has been completed
2023-03-15T10:11:32.343Z INFO 3392 --- [0.0-8080-exec-1] c.c.dbt.component.DBTLineageExtraction : Models extraction performing
2023-03-15T10:11:38.239Z INFO 3392 --- [0.0-8080-exec-1] c.c.dbt.component.DBTLineageExtraction : Extracted Compiled Sql files are writing to the local lineage directory
2023-03-15T10:11:46.689Z INFO 3392 --- [0.0-8080-exec-1] c.c.dbt.component.DBTLineageTransformer : Transforms DBT database entities are performing
2023-03-15T10:13:21.032Z INFO 3392 --- [0.0-8080-exec-1] c.c.dbt.component.DBTLineageTransformer : Database assets transformer operation has been completed
2023-03-15T10:13:24.149Z ERROR 3392 --- [0.0-8080-exec-1] c.c.dbt.component.CollibraOperations : Import has been failed
com.collibra.marketplace.library.integration.exception.CollibraIntegrationLibraryException: {"message":"An unexpected error occurred during execution.","details":"java.io.IOException: Error writing to server","cause":"Error writing to server"}
at com.collibra.marketplace.library.integration.CollibraImportApiHelper.internalImportAssetsInBatches(CollibraImportApiHelper.java:663) ~[collibra-integration-library-1.2.1.jar!/:1.2.1]
at com.collibra.marketplace.library.integration.CollibraImportApiHelper.importAssets(CollibraImportApiHelper.java:129) ~[collibra-integration-library-1.2.1.jar!/:1.2.1]
at com.collibra.dbt.component.CollibraOperations.createAssets(CollibraOperations.java:68) ~[classes!/:1.2.2]
at com.collibra.dbt.component.DBTLineageTransformer.createDatabaseAssets(DBTLineageTransformer.java:144) ~[classes!/:1.2.2]
at com.collibra.dbt.component.DBTLineageExtraction.extractSql(DBTLineageExtraction.java:207) ~[classes!/:1.2.2]
at com.collibra.dbt.component.DBTLineageExtraction.extractSqlByJob(DBTLineageExtraction.java:149) ~[classes!/:1.2.2]
at com.collibra.dbt.component.DBTLineageExtraction.lambda$processProjectsInAccounts$13(DBTLineageExtraction.java:306) ~[classes!/:1.2.2]
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) ~[na:na]
at com.collibra.dbt.component.DBTLineageExtraction.lambda$processProjectsInAccounts$14(DBTLineageExtraction.java:280) ~[classes!/:1.2.2]
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) ~[na:na]
at com.collibra.dbt.component.DBTLineageExtraction.processProjectsInAccounts(DBTLineageExtraction.java:280) ~[classes!/:1.2.2]
at com.collibra.dbt.component.DBTLineageExtraction.findAllProjectsInAccount(DBTLineageExtraction.java:241) ~[classes!/:1.2.2]
at com.collibra.dbt.component.DBTLineageExtraction.requestAllAccounts(DBTLineageExtraction.java:125) ~[classes!/:1.2.2]
at com.collibra.dbt.controller.EntryPointController.requestAllAccounts(EntryPointController.java:38) ~[classes!/:1.2.2]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
patrichjalmarsson
9 Messages
2 years ago
We think we’ve solved it.
We changed from Collibra Integration Library v1.2.1 to Spring Boot Integration Library (Spring Boot 3.x) (installed it, updated the
pom.xml
and rebuild the integration).Our best guess is that some breaking changes must have been introduced somewhere in the back-end.
0
0
marcinnowicki
8 Messages
•
710 Points
1 year ago
@spring-team.collibra.com
I’ve encountered similar problem when using CollibraImportApiHelper.class to import assets on our server application.
library:
collibra-integration-library-sb3-1.0.1.jar
tag:
community-offering
Problem appears when import is called after some time of inactivity (few hours) - the only way to fix it is to reboot the application.
For short periods of inactivity (about 1 hour), I see that library is able to re-login in case of token expiration and everything works fine.
stacktrace logs below:
0
0