How to Launch Collibra DQ from an AWS AMI

AWS allows us to create AMI (Amazon Machine Images). This AMI contains the configuration for creating an instance. You can launch multiple instances from a single AMI when you need multiple instances with the same configuration. This document explains how to configure Collibra DQ installed in an AMI(Amazon Machine Images). To create this AMI, you have to install DQ first and create the AMI in AWS.

Follow these steps to configure an instance from an AMI with DQ installed:

1. Within AWS, go to your image and click on “Launch”

2. Select the type of instance.

3. Configure the ports in your security group: Ports 9000, 9001 and 8080 have to be opened.

4. Once the instance is running, connect through ssh.

ssh -i "pemfile" user@ipaddress

5. Check that there isn’t any owl process working.

6. Start Postgres

   #Postgres
   export INSTALL_PATH=$(pwd)/owl
   export PGPATH=/usr/pgsql-11/bin/
   export PGDBPATH=$INSTALL_PATH/postgres/data
   export LOG_PATH="$INSTALL_PATH/log"
   sudo -u postgres $PGPATH/pg_ctl -D $PGDBPATH start >$LOG_PATH/postgres-start.log 2>$LOG_PATH/postgres-start-error.log & echo $! >/dev/null
   
   
   
   
   

Check Postgres is working:

7. Start Spark

   $INSTALL_PATH/spark/sbin/start-master.sh
   $INSTALL_PATH/spark/sbin/start-slave.sh spark://$(hostname -f):7077
   

Check Spark is up and running:

Also, connect using the browser: http://awsip:8080

8. Update the old Ip address with the new one. To do that, the following files have to be modified:

• $HOME/owl/config/owl-env.sh: Update export SPRING_DATASOURCE_URL= with the new IP address.

• $HOME/owl/config/owl.properties: Update spring.datasource.url= and spring.agent.datasource.url= with the new IP address.

9. Change the permission on the PID file.

sudo chmod 777 $HOME/owl/pids/owl-agent.pid

10. Start DQ Agent and Web app:

    ./owlmanage.sh start=owlagent

    ./owlmanage.sh start=owlweb

11. Go to Admin Console and Remote Agent. Make sure the agent number 2 appears in green:

Note: The name of the agent can be updated by clicking on the chain.

12. Click on the pencil to update the settings. In this case, the Default Master has to be updated with the new IP address.

13. Once everything is changed, Collibra DQ is ready to be used.

Hope this helps!