I have loaded the azure-purview-to-collibra-integration code in the eclipse IDE and getting a pom.xml error
Description Resource Path Location Type
Non-resolvable parent POM for com.collibra.template:sbi-azure-purview:1.2.2: org.springframework.boot:spring-boot-starter-parent:pom:2.5.12 failed to transfer from https://repo.maven.apache.org/maven2 during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of central has elapsed or updates are forced. Original error: Could not transfer artifact org.springframework.boot:spring-boot-starter-parent:pom:2.5.12 from/to central (https://repo.maven.apache.org/maven2): transfer failed for https://repo.maven.apache.org/maven2/org/springframework/boot/spring-boot-starter-parent/2.5.12/spring-boot-starter-parent-2.5.12.pom pom.xml /sbi-azure-purview line 6 Language Servers
I am not sure what I am doing wrong here. This is the first integration code that I am trying from the marketplace.
Please advise.
abhishekdutta1
OP3 years ago · EditedHi @spring-team.collibra.com,
azure.datalake.gen2.inferred.directories what does this do… I am trying to understand this more. I thought default is true so did not add but now explicitly I made that true and is working. Can you give some examples and help here.
I faced another issue where I was getting error when inserting data, and after debugging I got that schema needs to be mentioned as well.
collibra.asset.file.schema=SchemaNow I am getting all the assets, fields, schema, etc…
Thank you for your support.
Regards,
Abhishek
springboot_team
·3 years ago · EditedHi @abdutta.geico.com,
The following is a description of the azure.datalake.gen2.inferred.directories property:
A flag to determine whether Gen2 Directories are constructed by the connector from file paths or explicitly retrieved via Purview API.
When true, the directories are constructed from file paths (which requires less API calls and processing, but details in Directory asset attributes will be left empty).
When false, the directories are explicitly retrieved via Purview API (which requires more API calls and processing, but details in Directory asset attributes will be included).
You can find the description of the rest of the properties in the documentation. However, please let us know if you have any questions on a specific property.
Thanks
abhishekdutta1
OP3 years ago · EditedHi @spring-team.collibra.com,
Any insights to move forward from here will be useful.
Regards,
Abhishek
springboot_team
·3 years ago · EditedHi Abhishek,
After investigating your logs, we notice that the NullPointerException is occurring because you have a property that is not set, which in your case is; azure.datalake.gen2.inferred.directories (a boolean).
Please reference the documentation for more information about each property.
Thanks
abhishekdutta1
OP3 years ago · EditedI changed the below variable and after that it worked and added some of the assets
purview.rest.api.base.path=/catalog/api/atlas/v2Added /catalog before /api
It added the storage account, service and container names for the adls gen2 and after that its failing with NullPointer Exception…
What else can cause this…
abhishekdutta1
OP3 years ago · EditedYeah thanks I got that resolved. Now I have everything setup as false and enabled only ADLS gen 2
Yet when I am running the application I am getting 404 Not Found
What else is missing
springboot_team
·3 years ago · EditedHi @abdutta.geico.com,
Thanks for the update.
Regarding the error, it seems that the issue might be that from Postman HTTPS is being used, while the Purview Spring Boot integration is configured to allow HTTP requests – e.g.
server.ssl.enabledis set to false.Therefore, can you please update the request protocol from Postman to use HTTP instead of HTTPS?
In case you would like to use HTTPS, you should:
server.ssl.enabledtotrueserver.ssl.*propertiesFor information on generating a self-signed certificate, you can refer to the documentation, section “Generate a Self-Signed Certificate using the Java Keytool”.
Thanks
abhishekdutta1
OP3 years ago · EditedHi @spring-team.collibra.com,
On a separate note I was able to resolve the maven dependencies and run, but when I am running this from Postman its failing
Below is the curl command
abhishekdutta1
OP3 years ago · EditedHi @spring-team.collibra.com,
Thanks. I tried that and got this error
I have set -
azure.uri.limit=60This is what I ran -
java -jar sbi-azure-purview-1.2.2.jar -Dspring.config.location=C:\Code\sbi-template-azure-purview-1.2.2\sbi-template-azure-purview-1.2.2\src\main\resources\application.propertiesspringboot_team
·3 years ago · EditedHi @abdutta.geico.com,
The issue might be occurring because your machine is being blocked from fetching the integration’s dependencies.
To test this, can you try accessing this link in a browser from the same machine you are running the integration on:
https://repo.maven.apache.org/maven2/org/springframework/boot/spring-boot-starter-parent/2.5.12/spring-boot-starter-parent-2.5.12.pom.
Kindly note that if you don’t want to perform any code modifications and you want to use the integration directly, you can try running this integration from the provided jar file by running the following command:
java -jar sbi-azure-purview-1.2.2.jar -Dspring.config.location=src/main/resources/application.properties
Thanks
springboot_team
·3 years ago · EditedHi @abdutta.geico.com,
From the error details provided it might be that Maven is not installed or configured correctly, therefore, it is not able to download the Maven dependencies required by this integration.
Can you please try following the steps in the Installation section of the Azure Purview integration documentation?
Azure Purview to Collibra Integration - Collibra Marketplace
A Spring Boot integration that invokes Azure Purview API, transforms it, and upserts it to the Collibra Platform
Since the Eclipse IDE is being used, you can also review the Development with the Eclipse IDE [Optional] section.
Thanks
abhishekdutta1
OP3 years ago · EditedHi @spring-team.collibra.com,
I tried in local maven and also via eclipse both are still giving the same error.
>mvn -v
Apache Maven 3.8.7 (b89d5959fcde851dcb1c8946a785a163f14e1e29)
Maven home: C:\Code\mvn\apache-maven-3.8.7
Java version: 1.8.0_352, vendor: Temurin, runtime: C:\Program Files\Eclipse Adoptium\jdk-8.0.352.8-hotspot\jre
Default locale: en_US, platform encoding: Cp1252
OS name: “windows 10”, version: “10.0”, arch: “amd64”, family: “windows”