L

Thursday, March 10th, 2022 4:14 PM

REVIEWED-PENDING DOCUMENTATION: What is the purpose of the Collibra DQ Parallel JDBC functionality?

Question: What is the purpose of the Collibra DQ Parallel JDBC functionality?
42%20AM
Answer: It is for speeding up the DQ initial “Scope SELECT” query against the datastore. Parallell JDBC splits the initial query to the source data into multiple parallel threads so the fetch to memory (Apache Spark DataFrame) becomes faster. It has nothing to do with any processing downstream after the data is fetched and in a Spark DataFrame.

No Responses!
Loading...