What does the Collibra DQ Scala API com.owl.core.Owl register(OwlOptions) function do?
A question was raised, What does the Collibra DQ Scala API com.owl.core.Owl register(OwlOptions) function do?
When we read this Scala code in the Collibra DQ Scala API:val cdq = OwlUtils.OwlContext(df, options)cdq.register(options)
There are a number of things happening here:
that cdq is an object of type com.owl.core.Owl
And that object has the register()function on it.
- It makes sure there is a
datasetname,runId,host, andportfor the DQ Metastore. If there is not, it throws an Error, and you can’t proceed. - If you do have all that, it creates two things:
A. It creates aDatasetScanDaoand an SQL INSERT into the DQ Metastore.
B. It creates aCatalogDaoand puts something in the DQ Catalog.
Don’t skip this step when using the Scala API.
BigDataBear
Posted 3 years ago · Edited 1 year ago
0 comments
