Skip to main content
S
scottnichols
Employee

12 Messages

Friday, October 28th, 2022 4:21 PM

Integration Library Import Errors

The latest Collibra Integration Library 1.1.9 doesn’t seem to work with the latest version of Collibra DIC. My guess there has been a model change with Collibra and the import. When using its import with the Open API Swagger .
To fix this my guess is the Collibra Integration Library should upgraded to the latest version of the Collibra Rest spec.
Marketplace asset DGC returns the following:

See DGC import error:

2022-10-28 01:57:48.415 [pool-20-thread-1] INFO c.c.c.d.i.s.DatabaseIngestionControllerService - The periodical lifecycle check is completed after PT0.000983S [session_hash=11b42c1875b4fbf96a87526f95732aa3, authenticated_id=00000000-0000-0000-0000-000000900001, trace_id=44fd687688d629e273b00fd40da96f61, trace_flags=01, span_id=64c7a6f6a49c4f60]
2022-10-28 01:57:51.352 [http-nio-0.0.0.0-4400-exec-13] INFO c.c.d.c.s.i.i.OutputViewServiceImpl - TableViewConfig: {

  • “TableViewConfig”: {*
  •    "displayLength": -1,*

  •    "displayStart": 0,*

  •    "Resources": {*

  •        "Term": {*

  •   		"Signifier": {*

  •   			"name": "assetName"*

  •   		},*

  •   		"Vocabulary": {*

  •   			"Name": {*

  •   				"name": "domainName"*

  •   			},*

  •   			"Community": {*

  •   				"Name": {*

  •   					"name": "communityName"*

  •   				}*

  •   			}*

  •   		},*

  •            "ConceptType": {*

  •                "Id": {*

  •                    "name": "assetTypeId"*

  •                },*

  •                "Signifier": {*

  •                    "name": "assetTypeName"*

  •                }*

  •            },*

  •            "Filter": {*

  •                "AND": [*

  •                    {*

  •                        "Field": {*

  •                            "name": "assetName",*

  •                            "operator": "IN",*

  •                            "values": ["Canary REST API","Canary REST API>GET/browseNodes","Canary REST API>GET/browseNodes>200","Canary REST API>POST/getTagData","Canary REST API>POST/getTagData>200","Canary REST API>GET/getUserToken","Canary REST API>GET/getUserToken>200","Canary REST API>GET/getAggregates","Canary REST API>GET/getAggregates>200","Canary REST API>POST/browseTags","Canary REST API>POST/browseTags>200"]*

  •                        }*

  •                    },*

  •                    {*

  •                        "Field": {*

  •                            "name": "domainName",*

  •                            "operator": "IN",*

  •                            "values": ["API Assets","Data Assets","Code Values"]*

  •                        }*

  •                    },*

  •                    {*

  •                        "Field": {*

  •                            "name": "communityName",*

  •                            "operator": "IN",*

  •                            "values": ["Open Api Spec"]*

  •                        }*

  •                    }*

  •                ]*

  •            }*

  •        }*

  •    },*

  •    "Columns": [*

  •        {*

  •            "Column": {*

  •                "fieldName": "assetName",*

  •                "index": 0*

  •            }*

  •        },*

  •        {*

  •            "Column": {*

  •                "fieldName": "assetTypeId",*

  •                "index": 1*

  •            }*

  •        },*

  •        {*

  •            "Column": {*

  •                "fieldName": "assetTypeName",*

  •                "index": 2*

  •            }*

  •        },*

  •        {*

  •            "Column": {*

  •                "fieldName": "domainName",*

  •                "index": 3*

  •            }*

  •        },*

  •        {*

  •            "Column": {*

  •                "fieldName": "communityName",*

  •                "index": 4*

  •            }*

  •        }*

  •    ]*
  • }*
    } [authenticated_id=bd1704ee-625f-4def-a9ae-c0400e22e0ef, session_hash=11d050c92a09cc0b68f05e6c1ec1eca6, ehcache-txid=10791456, trace_id=506b1503a7bffbfe7cee878c88676c45, trace_flags=01, span_id=a0341f8939cdb19a]
    2022-10-28 01:57:51.585 [jobExecutor-5] INFO c.c.d.c.s.job.impl.JobServiceImpl - Job ActiveJob{id=‘1e0f73fa-61c9-469f-b629-4019f931e367’, name=‘Export to “exportd206635faef8433f9c500c6c3b6fb208.csv”.’, user=‘bd1704ee-625f-4def-a9ae-c0400e22e0ef’, state=WAITING} started at 2022-10-28T01:57:51.585562Z [session_hash=ed4a004220b3685e16e1fed5a4517c3a, authenticated_id=bd1704ee-625f-4def-a9ae-c0400e22e0ef, trace_id=08b2e1fd778b1ea729ee8f7248a308a1, trace_flags=01, span_id=21128e975839bf6e]
    2022-10-28 01:57:51.591 [jobExecutor-5] INFO c.c.d.c.s.job.impl.JobServiceImpl - Preprocessing of job ActiveJob{id=‘1e0f73fa-61c9-469f-b629-4019f931e367’, name=‘Export to “exportd206635faef8433f9c500c6c3b6fb208.csv”.’, user=‘bd1704ee-625f-4def-a9ae-c0400e22e0ef’, state=RUNNING} done. [session_hash=ed4a004220b3685e16e1fed5a4517c3a, authenticated_id=bd1704ee-625f-4def-a9ae-c0400e22e0ef, trace_id=08b2e1fd778b1ea729ee8f7248a308a1, trace_flags=01, span_id=21128e975839bf6e]
    2022-10-28 01:57:51.592 [jobExecutor-5] INFO c.c.d.c.s.job.impl.JobServiceImpl - Transaction started for job ActiveJob{id=‘1e0f73fa-61c9-469f-b629-4019f931e367’, name=‘Export to “exportd206635faef8433f9c500c6c3b6fb208.csv”.’, user=‘bd1704ee-625f-4def-a9ae-c0400e22e0ef’, state=RUNNING}. [session_hash=ed4a004220b3685e16e1fed5a4517c3a, authenticated_id=bd1704ee-625f-4def-a9ae-c0400e22e0ef, ehcache-txid=10791462, trace_id=08b2e1fd778b1ea729ee8f7248a308a1, trace_flags=01, span_id=21128e975839bf6e]
    2022-10-28 01:57:51.592 [jobExecutor-5] INFO c.c.d.c.io.job.OutputViewJobRunner - Parsing configuration [session_hash=ed4a004220b3685e16e1fed5a4517c3a, authenticated_id=bd1704ee-625f-4def-a9ae-c0400e22e0ef, ehcache-txid=10791462, trace_id=08b2e1fd778b1ea729ee8f7248a308a1, trace_flags=01, span_id=21128e975839bf6e]
    2022-10-28 01:57:51.594 [jobExecutor-5] INFO c.c.d.c.io.job.OutputViewJobRunner - Parsing configuration took: 2ms [session_hash=ed4a004220b3685e16e1fed5a4517c3a, authenticated_id=bd1704ee-625f-4def-a9ae-c0400e22e0ef, ehcache-txid=10791462, trace_id=08b2e1fd778b1ea729ee8f7248a308a1, trace_flags=01, span_id=21128e975839bf6e]
    2022-10-28 01:57:51.595 [jobExecutor-5] INFO c.c.d.c.io.job.OutputViewJobRunner - Executing viewconfig and querying data [session_hash=ed4a004220b3685e16e1fed5a4517c3a, authenticated_id=bd1704ee-625f-4def-a9ae-c0400e22e0ef, ehcache-txid=10791462, trace_id=08b2e1fd778b1ea729ee8f7248a308a1, trace_flags=01, span_id=21128e975839bf6e]
    2022-10-28 01:57:51.605 [jobExecutor-5] INFO c.c.d.c.s.file.impl.FileServiceImpl - Calculated MD5 checksum d41d8cd98f00b204e9800998ecf8427e for the file exportd206635faef8433f9c500c6c3b6fb208.csv /collibra/collibra_data/dgc/temp-files/3136d3eb-f3d8-4d9b-8563-aa89670055dd [authenticated_id=00000000-0000-0000-0000-000000900001, session_hash=4b43e111f47986d8a4e8623892fd35cc, ehcache-txid=10791462, trace_id=08b2e1fd778b1ea729ee8f7248a308a1, trace_flags=01, span_id=21128e975839bf6e]
    2022-10-28 01:57:51.608 [jobExecutor-5] ERROR c.c.d.c.io.job.OutputViewJobRunner - An error occurred during the export [authenticated_id=bd1704ee-625f-4def-a9ae-c0400e22e0ef, session_hash=ed4a004220b3685e16e1fed5a4517c3a, ehcache-txid=10791462, trace_id=08b2e1fd778b1ea729ee8f7248a308a1, trace_flags=01, span_id=21128e975839bf6e]
    com.collibra.common.exception.CollibraIllegalArgumentException: fileResourceSizeTooSmall
  • Params: [exportd206635faef8433f9c500c6c3b6fb208.csv]*
  • at com.collibra.dgc.core.service.file.impl.FileServiceImpl.checkFileSize(FileServiceImpl.java:541)*
  • at com.collibra.dgc.core.service.file.impl.FileServiceImpl.addFile(FileServiceImpl.java:202)*
  • at com.collibra.dgc.core.service.file.impl.FileServiceImpl.addFile(FileServiceImpl.java:185)*
  • at jdk.internal.reflect.GeneratedMethodAccessor3992.invoke(Unknown Source)*
  • at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)*
  • at java.base/java.lang.reflect.Method.invoke(Unknown Source)*
  • at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)*
  • at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)*
  • at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)*
  • at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)*
  • at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)*
  • at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)*
  • at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)*
  • at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)*
  • at com.sun.proxy.$Proxy367.addFile(Unknown Source)*
  • at com.collibra.dgc.core.io.connector.output.impl.file.AbstractFileWrapper.lambda$addFileAndSkipChecksIfNeeded$1(AbstractFileWrapper.java:103)*
  • at org.springframework.security.concurrent.DelegatingSecurityContextRunnable.run(DelegatingSecurityContextRunnable.java:82)*
  • at com.collibra.dgc.core.application.security.SecurityContextHolderUtils.runAs(SecurityContextHolderUtils.java:55)*
  • at com.collibra.dgc.core.application.security.SecurityContextHolderUtils.runAsSystemUser(SecurityContextHolderUtils.java:67)*
  • at com.collibra.dgc.core.io.connector.output.impl.file.AbstractFileWrapper.addFileAndSkipChecksIfNeeded(AbstractFileWrapper.java:99)*
  • at com.collibra.dgc.core.io.connector.output.impl.file.chunked.CsvChunkedFileWrapper.closeFile(CsvChunkedFileWrapper.java:55)*
  • at com.collibra.dgc.core.io.connector.output.impl.CSVOutputConnector.finishExportToFile(CSVOutputConnector.java:103)*
  • at java.base/java.lang.Iterable.forEach(Unknown Source)*
  • at com.collibra.dgc.core.graph.query.engine.GraphQueryEngineToOutputConnectorAdapter.executeDataFetchingInChunks(GraphQueryEngineToOutputConnectorAdapter.java:250)*
  • at com.collibra.dgc.core.graph.query.engine.GraphQueryEngineToOutputConnectorAdapter.executeQueryAndPopulateOutputConnectors(GraphQueryEngineToOutputConnectorAdapter.java:128)*
  • at com.collibra.dgc.core.io.job.OutputViewJobRunner.executeDataFetching(OutputViewJobRunner.java:212)*
  • at com.collibra.dgc.core.io.job.OutputViewJobRunner.run(OutputViewJobRunner.java:177)*
  • at com.collibra.job.api.IJobRunner.execute(IJobRunner.java:50)*
  • at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob$1.doInTransactionWithoutResult(JobServiceImpl.java:1255)*
  • at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:36)*
  • at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)*
  • at com.collibra.dgc.core.exceptions.ExceptionResolvingWithCachingTransactionTemplate.execute(ExceptionResolvingWithCachingTransactionTemplate.java:28)*
  • at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob.runJobInTransaction(JobServiceImpl.java:1249)*
  • at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob.postInitRun(JobServiceImpl.java:1117)*
  • at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob.run(JobServiceImpl.java:1085)*
  • at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:89)*
  • at io.micrometer.core.instrument.Timer.lambda$wrap$0(Timer.java:152)*
  • at org.springframework.security.concurrent.DelegatingSecurityContextRunnable.run(DelegatingSecurityContextRunnable.java:82)*
  • at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)*
  • at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)*
  • at java.base/java.lang.Thread.run(Unknown Source)*
    2022-10-28 01:57:51.609 [jobExecutor-5] ERROR c.c.d.c.s.job.impl.JobServiceImpl - Job ActiveJob{id=‘1e0f73fa-61c9-469f-b629-4019f931e367’, name=‘Export to “exportd206635faef8433f9c500c6c3b6fb208.csv”.’, user=‘bd1704ee-625f-4def-a9ae-c0400e22e0ef’, state=ERROR} error: {} [authenticated_id=bd1704ee-625f-4def-a9ae-c0400e22e0ef, session_hash=ed4a004220b3685e16e1fed5a4517c3a, ehcache-txid=10791462, trace_id=08b2e1fd778b1ea729ee8f7248a308a1, trace_flags=01, span_id=21128e975839bf6e]
    com.collibra.common.exception.CollibraIllegalArgumentException: fileResourceSizeTooSmall
  • Params: [exportd206635faef8433f9c500c6c3b6fb208.csv]*
  • at com.collibra.dgc.core.service.file.impl.FileServiceImpl.checkFileSize(FileServiceImpl.java:541)*
  • at com.collibra.dgc.core.service.file.impl.FileServiceImpl.addFile(FileServiceImpl.java:202)*
  • at com.collibra.dgc.core.service.file.impl.FileServiceImpl.addFile(FileServiceImpl.java:185)*
  • at jdk.internal.reflect.GeneratedMethodAccessor3992.invoke(Unknown Source)*
  • at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)*
  • at java.base/java.lang.reflect.Method.invoke(Unknown Source)*
  • at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)*
  • at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)*
  • at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)*
  • at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)*
  • at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)*
  • at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)*
  • at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)*
  • at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)*
  • at com.sun.proxy.$Proxy367.addFile(Unknown Source)*
  • at com.collibra.dgc.core.io.connector.output.impl.file.AbstractFileWrapper.lambda$addFileAndSkipChecksIfNeeded$1(AbstractFileWrapper.java:103)*
  • at org.springframework.security.concurrent.DelegatingSecurityContextRunnable.run(DelegatingSecurityContextRunnable.java:82)*
  • at com.collibra.dgc.core.application.security.SecurityContextHolderUtils.runAs(SecurityContextHolderUtils.java:55)*
  • at com.collibra.dgc.core.application.security.SecurityContextHolderUtils.runAsSystemUser(SecurityContextHolderUtils.java:67)*
  • at com.collibra.dgc.core.io.connector.output.impl.file.AbstractFileWrapper.addFileAndSkipChecksIfNeeded(AbstractFileWrapper.java:99)*
  • at com.collibra.dgc.core.io.connector.output.impl.file.chunked.CsvChunkedFileWrapper.closeFile(CsvChunkedFileWrapper.java:55)*
  • at com.collibra.dgc.core.io.connector.output.impl.CSVOutputConnector.finishExportToFile(CSVOutputConnector.java:103)*
  • at java.base/java.lang.Iterable.forEach(Unknown Source)*
  • at com.collibra.dgc.core.graph.query.engine.GraphQueryEngineToOutputConnectorAdapter.executeDataFetchingInChunks(GraphQueryEngineToOutputConnectorAdapter.java:250)*
  • at com.collibra.dgc.core.graph.query.engine.GraphQueryEngineToOutputConnectorAdapter.executeQueryAndPopulateOutputConnectors(GraphQueryEngineToOutputConnectorAdapter.java:128)*
  • at com.collibra.dgc.core.io.job.OutputViewJobRunner.executeDataFetching(OutputViewJobRunner.java:212)*
  • at com.collibra.dgc.core.io.job.OutputViewJobRunner.run(OutputViewJobRunner.java:177)*
  • at com.collibra.job.api.IJobRunner.execute(IJobRunner.java:50)*
  • at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob$1.doInTransactionWithoutResult(JobServiceImpl.java:1255)*
  • at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:36)*
  • at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)*
  • at com.collibra.dgc.core.exceptions.ExceptionResolvingWithCachingTransactionTemplate.execute(ExceptionResolvingWithCachingTransactionTemplate.java:28)*
  • at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob.runJobInTransaction(JobServiceImpl.java:1249)*
  • at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob.postInitRun(JobServiceImpl.java:1117)*
  • at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob.run(JobServiceImpl.java:1085)*
  • at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:89)*
  • at io.micrometer.core.instrument.Timer.lambda$wrap$0(Timer.java:152)*
  • at org.springframework.security.concurrent.DelegatingSecurityContextRunnable.run(DelegatingSecurityContextRunnable.java:82)*
  • at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)*
  • at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)*
  • at java.base/java.lang.Thread.run(Unknown Source)*

Questions

 • 

Updated 

2 years ago

43

1

0

0

springboot_team

368 Messages

2 years ago

Hi @scott.nichols,

It seems that logs provided are of the Collibra Data Intelligence Cloud instance, where the export failed with an exception:

com.collibra.common.exception.CollibraIllegalArgumentException: fileResourceSizeTooSmall
Params: [exportd206635faef8433f9c500c6c3b6fb208.csv]*

The instance appears to throw an error when there is no data that is exported – i.e. the result is empty for the specified TableViewConfig.

However, the Collibra Integration Library should handle this and return an empty list. Just to note that is when the CollibraExportApiHelper class.

Therefore, can you please confirm what is the error with the Collibra Integration Library and maybe the stack trace? Thanks

0

0

Related Tags

openapi

community-offering

springboot

Related Conversations

Loading...
Loading...

Was this helpful?