Connect Collibra to Dynamo DB

Hi folks,

Has anyone successfully connected to Dynamo DB through the Collibra provided JDBC jar? I’m hitting errors where I’m assuming I’m not specifying a property correctly and was curious to know if anyone was able to connect.

Context: We have our job server hosted in AWS (account owned by my team) and are trying to connect to Dynamo DB hosted in a separate AWS account. We are utilizing role based approach and are utilizing AuthScheme = AwsEC2Roles property along with only other property Tables. This ingests the table without any columns but gives us:
java.sql.SQLException: [com.amazon.coral.validate#ValidationException] The parameter ‘TableName’ is required but was not present in the request.

So we switched the Tables property with TableName and this helps get rid of the above error but returns no data. Also, not sure what the URL is supposed to be in the Other property as specified on the driver details page: https://productresources.collibra.com/docs/collibra/2021.11/Content/Catalog/RegisterDataSource/JDBC/ref_db-connection-details-collibra-provided-driver.htm?catalog-connector-details=amazon-dynamodb

Thanks,
Ritesh

Ritesh Mody

Posted 4 years ago · Edited 1 year ago·Last reply 3 years ago

6 comments

Laura Beale

·3 years ago · Edited

@riteshmody.northwesternmutual.com were you able to get the connection working and if so, do you have any advice you would be willing to share here? We’re also looking into setting up a connection with Dynamo, but on Edge rather than Jobserver.

Laura Beale

·3 years ago · Edited

Ritesh Mody

OP3 years ago · Edited

Hi @laura.beale.radancy.com: I was configuring my access using IAM Cross Account Authentication as our job server is installed in a separate AWS account as compared to where the DynamoDB is hosted. Collibra (CData provided) Dynamo JDBC doesn’t support pure IAM Cross Account Authentication and we were recommended to use MFA credentials to connect after assuming external IAM role. I’m still waiting for more insights from my platform team but I don’t think this would be a feasible route as MFA credentials expire every hour killing the purpose of continuous access via cross account access. If you have Edge and DynamoDB in the same account AuthScheme and Tables connection properties should do. But you can test around using other properties from: https://cdn.cdata.com/help/DDH/jdbc/Connection.htm

I was able to ingest DynamoDB Table from same AWS account using these two properties.

Ritesh Mody

OP4 years ago · Edited

@paulo.taylor @kristen.freer - Apologies for the delay! Yes, I have a ticket open with Collibra support on this. I’m working with them on this but wanted to check in parallel if anyone else had been able to use the driver to ingest data.

Alexandra Jorgenson

Admin4 years ago · Edited

@riteshmody.northwesternmutual.com, I agree with Paulo, this would be a good candidate for submission to our Support Portal. When submitting, I would reference the URL of this thread.

Paulo Taylor

·4 years ago · Edited

Hi @riteshmody.northwesternmutual.com ,
This driver is certified by Collibra and is therefore supported.
Have you thought about opening a support ticket?