Restart Collibra Data Quality automatically on GCP

A startup script is a file that performs tasks during the startup process of a virtual machine (VM) instance. We are going to configure our DQ instance to be able to start Collibra DQ (Postgre, Spark, OwlAgent OwlWebapp) automatically when the instance is started. For more information about how this works, check this link.

A startup script is passed to a VM from a location that is specified by a metadata key. A metadata key specifies whether the startup script is stored locally, stored in Cloud Storage, or passed directly to the VM. The metadata key that you use might also depend on the size of the startup script.

In this case, we are going to create a script to start DQ and later we are going to configure the instance

1. We are going to create the dq_start.sh script. The dq_start.sh script looks like this:

   #!/bin/bash

   #Postgres

   export INSTALL_PATH=/home/dq_user/owl

   export PGPATH=/usr/pgsql-11/bin/

   export PGDBPATH=$INSTALL_PATH/postgres/data

   export LOG_PATH="$INSTALL_PATH/log"

   sudo -u postgres $PGPATH/pg_ctl -D $PGDBPATH start >$LOG_PATH/postgres-start.log 2>$LOG_PATH/postgres-start-error.log & echo $! >/dev/null

   #Spark

   $INSTALL_PATH/spark/sbin/start-master.sh

   $INSTALL_PATH/spark/sbin/start-slave.sh spark://$(hostname -f):7077

   #Owl

   $INSTALL_PATH/bin/owlmanage.sh start=owlweb

   $INSTALL_PATH/bin/owlmanage.sh start=owlagent

2. Now, we can configure the call of the script in the instance. We can do this during the creation of the instance or edit the instance.

3. During the creation, inside “Management”-> “Automation”, add the call to our script:

   #!/bin/bash

   /home/dq_user_/dq_start.sh

4. If you have the instances created, click on the instance and click on “Edit”. In “Custom Metadata” section add the pair key/value “startup-script” and

   #!/bin/bash

   /home/dq_user_/dq_start.sh

Note: you can redirect the call of the script into a log if you need more information during the process.

Hope this helps!