Last sync date on S3 file System

Hi,

Is there a possibility to update the attribute Last sync Date on an S3 file system ? If yes how can I do that ? Is there a possibility tu update it automaticaly once the crawler job is completed ?

In advance thanks for your help

Daphné OZBEK

Posted 3 years ago · Edited 1 year ago·Last reply 3 years ago

6 comments

Daphné OZBEK

OP3 years ago · Edited

Hi Kristen, thanks for your answer !

@martin.sorel a new idea for the ideation portal

martinsorel

·3 years ago · Edited

Hi, it seems both https://ideas.collibra.com/ideas/CAT-I-476 and https://productresources.collibra.com/ideation-platform/?id=CAT-I-159 are similar to what we have in mind.
There is an Out of the Box " Last Sync date" attribute, it feels strange that it can not be updated using built-in ingestion capabilities.

Daphné OZBEK

OP3 years ago · Edited

Hi Kristen, thanks for your answer !

@martin.sorel a new idea for the ideation portal

Ann Wuyts

·3 years ago · Edited

Hi @daphne.ozbek and @Martin - I am not sure if this is what you meant (you talk about updating, the idea about capturing the last sync date) but ‘ Capture “Last Sync Date” during catalog metadata ingestion and refresh’ already exists.

If this is identical or related, you can maybe add some info about your specific use case there? If this is specifically for Data Quality, please submit a new idea.

Ann Wuyts

·3 years ago · Edited

@Martin @daphne.ozbek
Thank you for that clarification. Based on your reply, I’ve done some housekeeping and merged CAT-I-159 into Automatically populate “Last Sync date” for ingested metadata (CAT-I-476). I realize 159 was marked as ‘planned’, where 476 is marked as ‘under review’. Under review (CAT-I-476) is most accurate, as this status was updated in July.

If you haven’t done so yet, please add your vote and any additional comments you would have to the parent idea.

Alexandra Jorgenson

Admin3 years ago · Edited

@daphne.ozbek I asked someone here in Product Management and this was their response: “I’m not 100% sure I understand the question — but in short — No, I do not believe this is something possible for DQ w/o a feature enhancement. They could use an external job control process to run a DQ check, but we don’t listen on S3 natively within DQ.”

If you haven’t already,this would be a good candidate for submission to our Ideation Portal. If/When submitting, I would reference the URL of this thread. @ann.wuyts FYI.