L

Friday, December 17th, 2021 7:23 PM

TO BE REVIEWED: How do I ensure my Collibra DQ BigQuery ("BQ") Connection will work?

Question: How do I ensure my Collibra DQ BigQuery (“BQ”) Connection will work?
Answer: There were three BQ connections, with three different Auth JSON files. I took Brian Mearns’ advice: “This is likely due to an updated handling of bigquery tables. The root cause (in my estimation) is your project/dataset in your query do not line up with the project dataset in your auth json file.”

I made sure:

  1. Each BQ connection URL had the correct JSON in the URL, and that the JSON existed in that location.:
    OAuthPvtKeyPath=/opt/ext/owl-gcp.json

  2. Each connection URL’s JSON was the JSON against which we ran the base64 command to generate the password. Then we ran a fresh base64 encryption against the JSON file at that location:
    base64 your_json.json -w 0

  3. We added viewsEnabled=true to the Driver Properties text field at the bottom of the BigQuery Connection form.

  4. Validated the http://<URL> in the BQ Connection JDBC URL was the correct server for this project.

  5. Double check the project ID in the JDBC URL was accurate for the Connection: ProjectId=<project>

Then we saved the Connection (which said “Success”), and ran a BQ DQ Check and it worked.

See also: https://dq-docs.collibra.com/v/2021.09/dq-job-examples/owlcheck-bigquery

1 Message

2 years ago

Just to add, we need to make sure the userid: projectid and the password is the base64 your_json.json -w 0

Loading...