[This is for standalone installations, but can be modified for K8s] Customer noticed that disk space was filling up. The spark /tmp location can be redirected per this procedure:https://dq-docs.collibra.com/installation/standalone/full-install#redirecting-spark-scratch But that doesn’t solve the problem of these logs accumulating and taking up disk space. The default is that cleanup is not enabled and they are retained for 7 days. The number of seconds to retain application work directories on e...

How do I modify the settings for Spark to manage logs (and the disk space used) | Data Quality & Observability

How do I modify the settings for Spark to manage logs (and the disk space used) | Data Quality & Observability | Collibra Community