FU

Thursday, December 3rd, 2020 8:20 PM

DB2 z/os Catalog Ingestion

I would be interested if anyone has been successful ingesting using the DB2 for z/os into Collibra for just one database. We haven’t been able to figure out how to ingest by database. We’ve tried using specialRegisters, but haven’t yet been successful. The database property ($database) in the connection string points to the sub-system, not a particular database. Our databases are using the same schema name, so we can’t just sync by schema without getting metadata from multiple databases.

This is the connection string: jdbc:db2://$host:$port/$database

1.2K Messages

4 years ago

Can’t you use the “Tables excluded from registration” option?
https://productresources.collibra.com/docs/cloud-user/2020.11/#Catalog/RegisterDataSource/ta_register-data-source-own-driver.htm?Highlight=exclude

If you have 5000 tables in the same schema, you probably have some logic/prefix to identify tables per “database”.
Identify the minimum set of prefixes, and add them here, like so:
“TA*”, “TC*”, etc.

1.2K Messages

4 years ago

Unfortunately that wouldn’t help much, as each database/application uses the same naming convention. We have multiple databases/applications that use the same schema. First, we want one application per Collibra domain. Second, we wouldn’t be able to put together an exclusion list that would identify just one application.

1.2K Messages

Have you contacted your customer account manager to set up a meeting with the data catalog product team on the “staging” repository?
I know nothing about it beyond having heard the concept. That would allow to import all the metadata into a staging repository before importing into the data catalog. Maybe that would be a solution to your problem.

1.2K Messages

I’ve heard it is in the roadmap. Working with my rep to find out more.

36 Messages

4 years ago

Hi @cxlarsen.larsen,
did you find a solution for ingesting metadata by database in the meanwhile? We are facing exactly the same problem for z/OS in the moment.

36 Messages

We did a proof-of-concept using Catalog for our DB2-catalog, and found the same issue. (We ended with 125000 columns in one domain).
There isn’t an ‘out-of-the-box’ solution at the moment. We had a call with Peter Princen about this (and other issues), so they are aware of it

We decided a to build a custom solution, which isn’t so difficult (ODBC-connection, query on the DB2 catalog-tables, import in Collibra). You loose the profiling/sample data in that case, but that wasn’t an issue for us

Loading...