TO BE REVIEWED: Can I set up Collibra DQ multi-tenancy to have each tenant running on a different Google Cloud Dataproc cluster?

Question: Dataproc architecture & sizing. We are considering multi-tenancy with Collibra DQ – having different teams in the tool running their own data profiling jobs. We’re working on our Prod sizing, so can we have different Dataproc clusters running, one for each tenant? If yes, how can we set that up in the tool?

Answer from Milind Pandit:
Yes you can run different compute for your different tenants. See the example below

1. Tenant A : Say you have data source A, and data source B.

2. Tenant B : data source C, and data Source D.

3. Deploy Agent 1 for Dataproc cluster 1 ( For Tenant A) : Map data source A and B to Agent 1 .

4. Deploy Agent 2 for Dataproc Cluster 2 ( For Tenant B) : Map data source C and D to Agent 2.

All Tenant A users will see Agent 1 option to run the compute for Dataproc cluster 1. ( Same will apply for Tenant 2 users).