Post method returning 401
Hello,
I am trying to create a script which can auto sync data sources through Edge. The steps I am doing are:
- refresh the schema list
- wait for the job to complete
- fetch the new schema names
- create domains for the new ones
- create schema configurations.
I am at step 4 and the post method call fails without giving any error message. The logs are not helpful either. It only says HTTP 401 unauthorized.
All the previous API calls on the script are running.
Appreciate any help.
Script below:
#from calendar import c
import requests
import json
#**************Configure Variables
collibra_session_rest_url = “https://teck-dev.collibra.com/rest/2.0/auth/sessions”
collibra_session_rest_content_type = ‘application/json’
collibra_report_rest_headers = {“accept”:"/"}
schemaRefresh_url = “https://teck-dev.collibra.com/rest/catalogDatabase/v1/schemaConnections/refresh?databaseConnectionId=7a0ab2d1-2856-4f7b-b257-db65b2436420”
schemaRefreshJob_url = “https://teck-dev.collibra.com/rest/2.0/jobs?offset=0&limit=0&countLimit=-1&name=Schemas%20list%20of%20database%20connection%20hive_metastore&states=WAITING&maxVisibility=2147483647&sortOrder=DESC&sortField=CREATED_ON”
refreshedSchemaList_url = “https://teck-dev.collibra.com/rest/catalogDatabase/v1/schemaConnections?databaseConnectionId=7a0ab2d1-2856-4f7b-b257-db65b2436420&offset=0&limit=0”
domainCreation_url = “https://teck-dev.collibra.com/rest/2.0/domains”
schemaMetadataConfiguration_url = “https://teck-dev.collibra.com/rest/catalogDatabase/v1/schemaMetadataConfigurations”
#******************retrieve secrets
collibra_username = “svc.8888888888”
collibra_password = “c888888888”
#****************log into Collibra
collibra_session_rest_headers = {“Content-Type”:collibra_session_rest_content_type}
collibra_session_rest_content = {“username”:collibra_username, “password”:collibra_password}
session = requests.post(collibra_session_rest_url,json=collibra_session_rest_content,headers=collibra_session_rest_headers)
response_cookies = session.cookies
#********************API Calls :
#Run the refresh schema list
schemaRefreshResponse = requests.get(schemaRefresh_url, stream=True, cookies=response_cookies, headers=collibra_report_rest_headers)
result = schemaRefreshResponse.content
#track to see the refresh job completes
schemaRefreshJob = requests.get(schemaRefreshJob_url,stream=True,cookies = response_cookies, headers=collibra_report_rest_headers).json()
print(‘refresh Job completion check:’)
print(schemaRefreshJob)
schemaRefreshJob_results = schemaRefreshJob[‘results’]
print(schemaRefreshJob_results)
schemaRefreshJob_total = schemaRefreshJob[‘total’]
print(schemaRefreshJob_total)
name = []
if (schemaRefreshJob_total==0) :
#get the list of new schemas
refreshedSchemaResponse = requests.get(refreshedSchemaList_url,stream=True,cookies = response_cookies, headers=collibra_report_rest_headers).json()
print(type(refreshedSchemaResponse))
for each in refreshedSchemaResponse['results']:
if 'schemaId' not in each:
print(each['name'])
datasource = each['name']
domainPayload = {
"name": datasource ,
"communityId": "a88e4cdb-2e64-46f9-a0da-aba7b235dfe2",
"typeId": "00000000-0000-0000-0000-000000030011"
}
print(domainPayload)
#domainCreationResponse = requests.post(domainCreation_url,headers=collibra_session_rest_content_type,data=domainPayload)
domainCreationResponse = requests.request("POST",domainCreation_url,json=collibra_session_rest_content_type,headers=collibra_report_rest_headers)
print(domainCreationResponse.status_code)
# headers = {
‘Authorization’: 'Basic ',
‘Content-Type’: ‘application/json’,
‘Cookie’: ‘AW6f2e521c’
#}
domainCreationResponse = requests.request(“POST”,domainCreation_url,headers=headers,data=json.dumps(domainPayload)).json()
#print(domainCreationResponse.text)
#print( + "of the domain" + domainCreationResponse[ ]['name'])
#id = 'id' in domainCreationResponse
#schemaMetadataConfigurationPayload = {
# "schemaConnectionId": "7a0ab2d1-2856-4f7b-b257-db65b2436420",
# "synchronizationRules": [
# {
# "include": "*",
# "exclude": "",
# "targetDomainId": id,
# "skipViews": "true"
# }
# ]
# }
#schemaMetadataConfigurationResponse = requests.post(schemaMetadataConfiguration_url,stream=True,cookies=response_cookies, header = collibra_report_rest_headers,json = schemaMetadataConfigurationPayload).json()
#print(schemaMetadataConfigurationResponse)
Mehjabeen Sultana
OP4 years ago · EditedHi Stefan, Thank you for sharing your thoughts on the error.
I tried to give the syntax of the call as: domainCreationResponse = requests.request(“POST”,domainCreation_url,json=collibra_session_rest_content_type,headers=collibra_report_rest_headers,cookies = response_cookies)
It returned me 400 without creating the domain.
I am keen to understand if the sequence of parameters in the API call affect the response?
Best regards,
Mehjabeen
stefan_busch
·4 years ago · EditedMy guess it that the first API calls get authenticated via the cookies you send with the call. Because you don’t send them with the domain creation call you do not get authenticated there.