S

Monday, December 20th, 2021 8:50 PM

Parquet JDBC directory level

Hello - Looking into using the Parquet JDBC connector and am curious to know if the connector can be pointed at a directory with multiple sub directories containing parquet files, or does the connector need to be specified to the lowest level of folder where the parquet files exist. For instance we have the following organization structure in our blob containers: system > snapshot > parquet > *folder per subject area …can I just point it to the parquet/ level or do I need a connection for each /parquet/subject area 1, /parquet/subject area 2, etc. Thanks!

4 Messages

 • 

50 Points

3 months ago

Hello - I do not know if you are still looking for this, but yes you can use the following variable to include sub directories.

IncludeSubdirectories Whether to read files from nested folders. In the case of a name collision, table names are prefixed by the underscore-separated folder names.
Loading...