Lineage Harvester Error for Power BI Source

Hi,

Did anyone receive below error using lineage harvester in generate technical lineage with power BI.
2023-03-21T00:14:10-04 INFO Main batch (techlin-aws-us.collibra.com/c3e037f5-d9ae-4c3c-b7c6-6707779c88f7) is failed (0 m 0.00 s)…
[INFO] [03/21/2023 00:14:13.181] [zio-default-async-10] [CoordinatedShutdown(akka://default)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason]
2023-03-21T00:14:13-04 ERROR ShowcaseClientError Showcase returning client error:
PROCESSING ERROR: “org.sqlite.SQLiteException: [SQLITE_CONSTRAINT_UNIQUE] A UNIQUE constraint failed (UNIQUE constraint failed: COLLIBRA_MAPPINGS.ASSET_ID, COLLIBRA_MAPPINGS.EXT_SYSTEM_ID)”

Full error message: Received ‘410 Gone’ response code to GET https://techlin-aws-us.collibra.com/api/batch/c3e037f5-d9ae-4c3c-b7c6-6707779c88f7/status/ request. Response body:
{“slug”: “c3e037f5-d9ae-4c3c-b7c6-6707779c88f7”, “status”: “failed”, “message”: “PROCESSING ERROR: “org.sqlite.SQLiteException: [SQLITE_CONSTRAINT_UNIQUE] A UNIQUE constraint failed (UNIQUE constraint failed: COLLIBRA_MAPPINGS.ASSET_ID, COLLIBRA_MAPPINGS.EXT_SYSTEM_ID)””}

I understand the reson for this error from below URL, but it does not provide solution or how to rectify this error.

https://support.collibra.com/hc/en-us/articles/8983085603607-Tech-Lin-Harvester-error-A-unique-constraint-failed-when-using-Power-BI-1-0?auth_token=eyJhbGciOiJIUzI1NiJ9.eyJhY2NvdW50X2lkIjoxNTc5NDgsInVzZXJfaWQiOjEyNTk5OTk0MDE0NjE1LCJ0aWNrZXRfaWQiOjExMTA5OCwiZGVmbGVjdGlvbl9pZCI6MTMyNTE1ODY5OTM1NTksImFydGljbGVzIjpbODk4MzA4NTYwMzYwNyw0NDEyMzYxMDA3NjM5LDExNTcwNTQ2MzExNzAzXSwidG9rZW4iOm51bGwsImV4cCI6MTY4MTk4Mzg1MH0.2mbvleBwNsqpOXeTS8_BWc8UEOejlKh6pYxJXSmIyCk

I’ve reset all my configurations but still this issue is coming up. Can someone help?

krishnachaitanya_p

Posted 3 years ago · Edited 1 year ago·Last reply 3 years ago

9 comments

krishnachaitanya_p

OP3 years ago · Edited

Hi @graham.pearman,

Yes it is fixed. Please follow below steps:

Download the latest harvester
Run the command ./lineage-harvester list-sources
If 2 or more same PowerBI sources are listed but under 2 different source IDs, this can cause the error you are seeing
The solution then is to ignore the problem sources by running ./lineage-harvester ignore-source [sourceID] .
After removing all unwanted sources, the sync should succeed

Regards,
Krishna Chaitanya P

Graham Pearman

·3 years ago · Edited

Hi Both - coincidentally I am now facing this issue with another client. Was this fixed/ patched in harvester version 2023.04?

krishnachaitanya_p

OP3 years ago · Edited

Thanks @anouk.gorris!

Support ticket #111098 is in place.

anoukgorris

·3 years ago · Edited

Hi @krishna.chaitanya.p,

This error means that two column names were found with the exact same name. We are working on a solution and expect a fix early next week. Sorry for the inconvenience.

For future errors, I recommend creating a support ticket.

Kind regards,

Anouk

krishnachaitanya_p

OP3 years ago · Edited

Hi @anouk.gorris - Can you please help with this issue?

krishnachaitanya_p

OP3 years ago · Edited

Hi Graham,

We are using 2022.08 version. We wanted to move to new redhat linux server from existing one (cent OS). So making sure all configurations run with existing version as expected before migrating to redhat.

Do you suggest that this issue will be resolved if we install latest versions?

We are running it with admin access.
We have technical lineage enabled in our DGC

Regards,
Krishna Chaitanya

Graham Pearman

·3 years ago · Edited

Is this a new installation or an error on a pre-existing version? - Which version of harvester are you running? Have you tried the latest versions (post 2022.09?)

the other things to ensure are :
you are running cmd as an Administrator?
Technical Lineage has been enabled for your client?

krishnachaitanya_p

OP3 years ago · Edited

Hi Graham,

Thank you for the response!

I’ve tried your suggestion. But the issue still persists.

Regards,
Krishna Chaitanya

Graham Pearman

·3 years ago · Edited

Hey Krishna - try running this code first before running the harvester.
set JAVA_OPTS="–add-opens=java.base/java.nio=ALL-UNNAMED"

if it works, suggest adding this to the first line of the lineage-harvester.bat file