Download a data snapshot from Collibra DGC environment

Question

Hi,
I am trying to download data snapshots from the Collibra DGC environment in order to create dashboards.
I follow the steps documented here:
https://productresources.collibra.com/docs/reporting/1.5/#Reporting/ta_create-reporting-data-layer-aws.htm

I face 2 challenges:

I use CURL statement to download snapshot.
But it is ending intermittently with message: " curl: (18) transfer closed with outstanding read data remaining".
How to fix this?
Above error happened after it downloaded 2.8GB of data.
How big will be the snapshot? Are there any ways to restrict to a particular community?
Our intention is to create a dashboard for the community we are responsible.
I am doubtful that the API call to download the snapshot is downloading the whole Collibra’s communities?
Thanks in advance.

Alexandra Jorgenson · Answer

@venkata.ramani , I notice Roche has also raised a support ticket for this so can you report back here anything helpful you get from support/engineering from that?

tomkuppens1 · Answer

Hi @venkata.ramani ,
we are also using the Reporting Data Layer for dashboarding and reporting.

have you tried to download it via a browser or a different client? How much time did it take to download the 2.8GB?
AFAIK, there is only one endpoint that allows you to download the whole snapshot with all the RDL objects. We are also particularly interested having more flexibility in the RDL snapshot. The attribute object is becoming increasingly large (~70% of total size) and difficult to manage because of its size.

Could be interesting to have read access to the S3 bucket (where I presume the snapshots are stored).

Best,
Tom

1	Grant Rollerson SUNCORP CORPORATE SERVICES PTY LTD
2	Tom Friesen Algonquin College
3	SB S Benson Plus3 IT
4	S stefan_busch
5	Brian Lloyd USA TODAY Co.
6	D daverichard