Use pre-instanced Collibra variables locally in workflows

I am a workflow developer and want to improve my development and testing process.

My problem is that there are some pre-instanced variables in the Collibra runtime environment (e.g. “loggerApi”, “assetApi”, and so on) that are just magically available when the workflow runs in Collibra. I want to use these variables locally, so that I can test my code without the overhead of uploading it. How do I do this?

I already installed the Collibra library jar so that I can use code completion and type checking in my IDE. Unfortunately the mentioned variables are not part of the library.

I also noticed that the library only contains interfaces for some classes (e.g. “LoggerApi”, “AssetApi”) while the Collibra Runtime Environment contains the implementations (e.g. “LoggerApiImpl”, “AssetApiImpl”). It would already help much to have these implementations locally or to have mocks that implement easier versions of these implementations, but return usable data (not correct, just usable) so that the developed code can be executed and tested locally.

The current process of uploading and running the workflow in Collibra and getting the logs from the console takes about 5 to 10 minutes of manual work for each test iteration. If I would be able to run the code of script tasks locally this would reduce to a few seconds.

Thanks for any help!

marvinschulze

Posted 3 years ago · Edited 1 year ago·Last reply 3 years ago

1 comment

arthurburkhardt

·3 years ago · Edited

Ah… always happy to introduce a fellow collibra user to the wonderful world of autocomplete in eclipse! https://datacitizens.collibra.com/forum/t/the-best-autocomplete-trick-for-eclipse-get-hints-to-java-apis-directly/1485

In addition, I have automated the deployment with a bunch of extra features: https://datacitizens.collibra.com/forum/t/workflow-manager-for-eclipse/560/2?u=arthur.burkhardt

Still not possible to catch errors locally, I don’t think it will be possible…
I still have many errors that are actually thrown by the postgres database rather than by the code!!!

Anyways, the development feedback loop is now reduced from those 4-5 minutes down to seconds.
I’m both enthusiastic and afraid of the workflow designer: it should better integrate with the metamodel and deployment, but the IDE experience is still very poor (no autocomplete, error checking, etc.), so it might disrupt my semi-functional setup today.