How to Launch Collibra DQ from an AWS AMI

AWS allows us to create AMI (Amazon Machine Images). This AMI contains the configuration for creating an instance. You can launch multiple instances from a single AMI when you need multiple instances with the same configuration. This document explains how to configure Collibra DQ installed in an AMI(Amazon Machine Images). To create this AMI, you have to install DQ first and create the AMI in AWS.

Follow these steps to configure an instance from an AMI with DQ installed:

Within AWS, go to your image and click on “Launch”

Select the type of instance.

Configure the ports in your security group: Ports 9000, 9001 and 8080 have to be opened.

Once the instance is running, connect through ssh.

ssh -i "pemfile" user@ipaddress

Check that there isn’t any owl process working.

Start Postgres

#Postgres
export INSTALL_PATH=$(pwd)/owl
export PGPATH=/usr/pgsql-11/bin/
export PGDBPATH=$INSTALL_PATH/postgres/data
export LOG_PATH="$INSTALL_PATH/log"
sudo -u postgres $PGPATH/pg_ctl -D $PGDBPATH start >$LOG_PATH/postgres-start.log 2>$LOG_PATH/postgres-start-error.log & echo $! >/dev/null

Check Postgres is working:

Start Spark

$INSTALL_PATH/spark/sbin/start-master.sh
$INSTALL_PATH/spark/sbin/start-slave.sh spark://$(hostname -f):7077

Check Spark is up and running:

Also, connect using the browser: http://awsip:8080

Update the old Ip address with the new one. To do that, the following files have to be modified:

$HOME/owl/config/owl-env.sh: Update export SPRING_DATASOURCE_URL= with the new IP address.
$HOME/owl/config/owl.properties: Update spring.datasource.url= and spring.agent.datasource.url= with the new IP address.

Change the permission on the PID file.

sudo chmod 777 $HOME/owl/pids/owl-agent.pid

Start DQ Agent and Web app:

./owlmanage.sh start=owlagent

./owlmanage.sh start=owlweb
Go to Admin Console and Remote Agent. Make sure the agent number 2 appears in green:

Note: The name of the agent can be updated by clicking on the chain.

Click on the pencil to update the settings. In this case, the Default Master has to be updated with the new IP address.

Once everything is changed, Collibra DQ is ready to be used.

Hope this helps!

anasalas

Posted 4 years ago · Edited 1 year ago·Last reply 4 years ago

4 comments

noor

·4 years ago · Edited

Good job Ana. These kind of How-Tos will be really helpful. We need to preserve them in a central place or have tags attached for retrieval (which I think you already did).

noor

·4 years ago · Edited

Good job Ana. These kind of How-Tos will be really helpful. We need to preserve them in a central place or have tags attached for retrieval (which I think you already did).

Alexandra Jorgenson

Admin4 years ago · Edited

We are trying to document more How-Tos here in Discussion forums, yes. Tagging is helpful too for sure but anyone should be able to search for keywords and suggested threads will come up.

arthurburkhardt

·4 years ago · Edited

I was desperately looking for the thumbs up button, but HEY, there’s a new VOTE button! That’s amazing!
Thanks for sharing, @ana.salas.collibra.com!