I am trying to configure a JDBC connection to a blob in Azure Data Lake Gen2 using a jobserver. We are using the Service Principal Client OAuth. When trying to connect to a parquet file I get the error message Permission mismatch I have checked in Azure and the service principal we are using have read and execute permission on the file I am trying to read and on every folder from the file and to the root folder in the blob. The Storage account is using Access Control Lists and the service princi...

Permission Mismatch using ACL when reading from Azure Data Lake gen2 | Integrations

Permission Mismatch using ACL when reading from Azure Data Lake gen2

I am trying to configure a JDBC connection to a blob in Azure Data Lake Gen2 using a jobserver.

We are using the Service Principal Client OAuth. When trying to connect to a parquet file I get the error message Permission mismatch MismatchError

I have checked in Azure and the service principal we are using have read and execute permission on the file I am trying to read and on every folder from the file and to the root folder in the blob.

The Storage account is using Access Control Lists and the service principal has these permissions either on its own or through membership in a group with these permissions.

Any suggestions to what could be wrong?

knutge_hofseth

Posted 4 years ago · Edited 1 year ago·Last reply 4 years ago

2 comments

noor

·4 years ago · Edited

Hi Knut,

I am still trying to comprehend the benefit of documenting a file system like AWS S3/Azure Storage for the consumers of the catalog. Can you please share your thoughts on this as you seem to have already decided to ingest the metadata.

There could be several containers in Azure Storage & each container could have several folders & sub-folders that store files by date/month/year. And hence, thousands of objects to be brought into the catalog.

knutge_hofseth

OP4 years ago · Edited

Hello Noor,

this is the pilot stages of this Data Catalog Project. We are at the moment looking to ingest from a Data Lake to see if the output we get to the catalog ends up looking like it will be useful for our analysts. The final decision on ingesting those types of Storage awaits the analysis of the pilot.