Profiling & Sampling stats disappeared suddenly

Hi all

Did you notice this in your PROD environments (2022.04) -

We did Snowflake schema refresh in the first week of May via Jobserver with sampling & profiling turned ON. The samples & profiling stats got saved in DGC. Last week, we did a simple schema refresh (i.e. without sampling & profiling turned ON). Now, we no longer see sampling & profiling stats for those schemas. I don’t know when exactly they disappeared but we found just yesterday. I see on 15th an infra update was scheduled. I am not sure if it caused the issue. I raised a support ticket, need to check what comes out.

noor

Posted 4 years ago · Edited 1 year ago·Last reply 4 years ago

5 comments

Ritesh Mody

·4 years ago · Edited

Hi Noor, I believe if you re-run a schema refresh without selecting the Store Sample/Profile Data checkboxes then all the profiling information/stats get removed with that run.

noor

OP4 years ago · Edited

Hi Ritesh
This was not the behavior till now. We run profiling & sampling once in a quarter & then resume simple schema refresh as usual. The stats never got deleted so far. Are you sure they got deleted in your environment?

Alexandra Jorgenson

Admin4 years ago · Edited

@noor.shaik I check internally and confirmed what @riteshmody.northwesternmutual.com experienced: “Yes, that is the expected behavior. Uncheck and run removed that sample and profiling data.”

noor

OP4 years ago · Edited

Oh :(

Then I would raise an idea immediately. Our schemas (each one) with sampling & profiling take 48 hrs, and at times, time out with 504 error. So, we cannot afford sampling & profiling all the time.

Alexandra Jorgenson

Admin4 years ago · Edited

Wow. Then definitely submit to the Ideation Portal. When submitting, I would reference the URL of this thread. @ann.wuyts FYI.