Read from a file and import into Collibra

Question

Using the Collibra Integration Library set of Java classes to access to Collibra APIs, I’m trying to build custom API-based integrations around the Collibra platform.
I would need some support to import code sets and code values properly using this template.
Using the template available in the marketplace, I managed to import some code sets and assicoated code values but encoutered an issue in transformAssets function, as the asset type hierarchy described in the xsd are not taken into account, all asset types and assets are imported at the same level
Could you please help solve this issue ?

jeanlucgarnier · Answer

Hello, thanks for your feed back. It should be OK with your explanation. I will check this asap

springboot_team · Answer

Hello Jean-Luc,

Thanks for the confirming.

Yes, the respective communities and domains should be created prior to triggering the integration. Additionally, if the created community and domain names are different than the default values, then the following properties should be updated (from the application.properties file) prior to deploying the integration.

collibra.domain.csv
collibra.domain.xml
collibra.community

Regarding the ordering, there are two options:

Either when adding the assets to the list that should be imported, they are ordered;
Otherwise, use a list for each asset type, add the assets (based on their type) to the respective list, and then merge all list according to the required order.

Re. point 2, example:

Databases List: DB1, DB2
Schemas List: SC1, SC2
Tables List: TB1, TB2

Then, since the table assets depend on the schema assets, and the schema assets depend on the database assets, the import list should be merged as follows:

Import List:

Databases List
Schemas List
Tables List

jeanlucgarnier · Answer

Hello, thanks for your feed back
Yes this the [“Read from a file and import into Collibra”] Spring Boot integration template
Your reply did helped, if I understand well, the relative communities and domains shall be created as first step prior to or within the first execution or/and it is recommanded to ensure that tags beeing ordered in the XML Schema, I don’t know if an additionnal id attribute could help for that purpose

springboot_team · Answer

Hello Jean-Luc,

Thanks for trying the Collibra Integration Library.

Can you please confirm that the Spring Boot integration being referred to is the “Read from a file and import into Collibra” one?

Regarding the issue encountered and assuming the above integration, currently, the XML file parsing logic is structured in such a way that the root/parent tags are added to the list of assets that should be imported before the child tags.

As an example, if the XSD file has the following structure:

Root tag
- Tag 1
  - Tag 2
- Tag 3
  - Tag 4

Then, when preparing the import payload (i.e. the list of assets), it is important that the “Root tag” asset is the first one since all the other assets depend on it; “Tag 1”, in the list of assets, is after “Root tag” and before “Tag 2”; Similarly, “Tag 3”, in the list of assets, is after “Root tag” and before “Tag 4”.

Therefore, as a general rule, if an asset has a dependency (e.g. a relation) on another asset, the related asset should:

Either already exist on the Collibra instance from a previous import
Or be located before in the list of assets that should be imported

Hope it helps.

1	Grant Rollerson SUNCORP CORPORATE SERVICES PTY LTD
2	Tom Friesen Algonquin College
3	SB S Benson Plus3 IT
4	S stefan_busch
5	Brian Lloyd USA TODAY Co.
6	D daverichard