Troubleshooting the Collibra DQ AMI

Hi Community,

Sharing my experience below in trying to stand up the AMI instance in the Collibra AWS environment in case helpful for others.
https://owl-analytics.gitbook.io/user-guide/-La5sAxci8GhitOM0qCp/learning-the-ropes-of-owldq-ami

I was told that the AMI must be updated with a new SPRING_DATASOURCE_URL in owl-env.sh.

I used the value for the internal IP address of the instance (as below) and it did not appear to work. I also tried the external IP address.

export SPRING_DATASOURCE_URL=jdbc:postgresql://ip-172-31-13-190.ec2.internal:5432/postgres

Spark master appears to be running: http://3.93.36.144:8080/
DQ not responding: http://3.93.36.144:9000/

You can view the instance values in Collibra AWS > EC2 > case_cdq_test

Any help much appreciated.

#!/bin/bash
#export SPARK_CONF_DIR="/home/owldq/owl/cdh-spark-conf"
export ORIENTDB_HOST=“localhost”
export ORIENTDB_PORT=“2424”
export INSTALL_PATH="/home/owldq/owl"
export JAVA_HOME="/usr"
export LOG_PATH="/home/owldq/owl/log"
export BASE_PATH="/home/owldq"
export SPARK_MAJOR_VERSION=2
export OWL_LIBS="/home/owldq/owl/libs"
export USE_LIBS=0 #1 is on, 0 is off
export SPARKSUBMITEXE=“spark-submit”
export ext_pass_manage=0 #0 to disable 1 to enable
export ext_pass_script="/opt/owl/bin/getpassword.sh"
TIMEOUT=1200 #20 minutes in seconds
PORT=9000 #owl-web port NUMBER
SPRING_LOG_LEVEL=ERROR
PGDBPATH=/home/owldq/owl/postgres/data
export SPRING_DATASOURCE_PASSWORD=o0YqGpK+m6sDc2haKsHmuPUldmaEmru0uHvnzhN01tNRYBGBXPrzd5dFhXcgEyXt
export ORIENTDB_PASS=
export SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.postgresql.Driver
export SPRING_DATASOURCE_URL=jdbc:postgresql://ip-172-31-13-190.ec2.internal:5432/postgres
export SPRING_DATASOURCE_USERNAME=owldq
SPARKCONF=“spark.dynamicAllocation.enabled=false”
export OWL_CORE_PROPS=/home/owldq/owl/config/
export DATASETS_PER_ROW=8000
export RUNS_THRESHOLD=30
export ROW_COUNT_THRESHOLD=300000
export OWL_SPARKLOG_ENABLE=false
export MULTITENANTMODE=FALSE
export multiTenantSchemaHub=owlhub
#export SERVER_HTTP_ENABLED=true
#export HTTP_SECONDARY_PORT=9001
#export SERVER_PORT=9000
#export SERVER_HTTPS_ENABLED=true
#export SERVER_SSL_KEY_TYPE=PKCS12
#export SERVER_SSL_KEY_STORE=/home/owldq/owl/keystoredsktp.p12
#export SERVER_SSL_KEY_PASS=t2lMFWEHsQha3QaWnNaR8ALaFPH15Mh9
#export SERVER_SSL_KEY_ALIAS=owl

casefarley

Posted 5 years ago · Edited 1 year ago·Last reply 5 years ago

1 comment

casefarley

OP5 years ago · Edited

Spoke with Brian and Matt about this offline and they shared some config tips. This should get you closer if you are starting with the AMI.

https://owl-analytics.gitbook.io/user-guide/-La5sAxci8GhitOM0qCp/learning-the-ropes-of-owldq-ami

Start the Instance
Get access to DQ AMI. We need your AWS Account ID and then we can whitelist the image.

Spin it up from N. Virginia (us-east-1) region (only available there).
Instance size: t2.large or t2.xlarge or above

Open the following ports so that you can connect to the instance. Use whatever Source restrictions make sense for your environment.

9000 TCP
9091 TCP
8080 TCP
22 TCP

Configure the Instance

ssh to box to do the config and start the services.
Switch to root after you connect, or run sudo as part of your edit commands. Root is not password protected, so:
# sudo -i

You must update
/home/owldq/owl/config/owl-env.sh

with the correct SPRING_DATASOURCE_URL IP address so that it matches the internal IP address of your instance.

Update this line:
export SPRING_DATASOURCE_URL=jdbc:postgresql://ip-172-31-13-190.ec2.internal:5432/postgres

to:
export SPRING_DATASOURCE_URL=jdbc:postgresql://*your private IP address*.ec2.internal:5432/postgres

Change the permission on the PID file.
sudo chmod 777 /home/owldq/owl/pids/owl-agent.pid

Start the services from /home/owldq/owl/bin:
./owlmanage.sh start=owlweb
./owlmanage.sh start=owlagent

Connect to DQ
http://*public ip address*:9000/

admin admin123