How to launch a Collibra DQ installation in GCP from an image

GCP allows us to create Images. These images contain the configuration for creating instances. You can launch multiple instances from a single image when you need multiple instances with the same configuration. This post explains how to configure Collibra DQ from a GCP image with DQ installed. Click here to have more information about GCP images.

Prerequisites: Have a GPC image created with Collibra DQ installed.

Follow these steps to configure an instance from a GCP image with DQ installed:

1. Within GCP, select the image and click on Actions, selecting “Create instance”.

2. Configure the image with the desired configuration. In this example, we will keep the configuration of the image.

Note: Related to the network configuration, we are keeping the configuration of the image and we are using the network (in our case “default” network). This network has ports 9000, 9001, and 8080 open. If you want to use another network configuration, these ports have to be opened.

3. When the configuration is finished click on “Create”.

4. Once the instance is running, connect through ssh.

ssh -i “pemfile” user@ipaddress

Note: Keeping the configuration of the image, the credentials of the original instance will be configured in this new instance too. Click on the instance name to change this configuration if needed.

5. Check that there isn’t any owl process working.

6. Start Postgres

   #Postgres

   export INSTALL_PATH=$(pwd)/owl

   export PGPATH=/usr/pgsql-11/bin/

   export PGDBPATH=$INSTALL_PATH/postgres/data

   export LOG_PATH="$INSTALL_PATH/log"

   sudo -u postgres $PGPATH/pg_ctl -D $PGDBPATH start >$LOG_PATH/postgres-start.log 2>$LOG_PATH/postgres-start-error.log & echo $! >/dev/null

Check Postgres is working:

7. Start Spark

   #Spark

   $INSTALL_PATH/spark/sbin/start-master.sh

   $INSTALL_PATH/spark/sbin/start-slave.sh spark://$(hostname -f):7077

Check Spark is up and running:

Also, connect using the browser: http://awsip:8080

8. Update the old Ip address with the new one. To do that, the following files have to be modified:

• $HOME/owl/config/owl-env.sh

Update export SPRING_DATASOURCE_URL= with the new private IP address.

• $HOME/owl/config/owl.properties

Update spring.datasource.url= and spring.agent.datasource.url= with the new private IP address.

9. Change the permission on the PID file.

sudo chmod 777 $HOME/owl/pids/owl-agent.pid

10. Start DQ Agent and Web app. The scripts are in $HOME/owl/bin

    ./owlmanage.sh start=owlagent

    ./owlmanage.sh start=owlweb

11. Check the agent and the web app are working properly.

Open http://ipaddress:9000:

12. Go to Admin Console and Remote Agent. Make sure the agent number 2 appears in green:

Note: The name of the agent can be updated by clicking on the chain.

13. Click on the pencil to update the settings. In this case, the Default Master has to be updated with the new IP address.

14. Once everything is changed, Collibra DQ is ready to be used.