Most effective way to bulk delete assets
Evening All
Can anyone please shed light on what is the most effective way to bulk delete Collibra assets via workflows written in Groovy. An example would be nice
My code thus far looks like this
loggerApi.info(“Script1 : CP 010 : Start : Remove Group Data Assets”)
def vFindAsset1 = assetApi.findAssets(FindAssetsRequest.builder()
.typeIds([string2Uuid("9517f636-c8b6-46bc-9d0b-ca36703694f7")])
.limit(1000)
.build() )
.getResults()
loggerApi.info("The number of Asset found is is ${vFindAsset1.size()}")
int vCounter=1
for (vAsset in vFindAsset1)
{
vCounter=vCounter+1
if (vCounter%10 == 0)
{
loggerApi.info("We have processed ${vCounter} Group Data Assets from a total of ${vFindAsset1.size()}")
}
assetApi.removeAsset(vAsset.getId())
}
loggerApi.info("Script1 : CP 020 : End : Group Data Assets")
robertinnes11
Posted 2 years ago · Edited 1 year ago·Last reply 2 years ago
4 comments
stefan_busch
·2 years ago · EditedHi,
there is a bulk delete function in the AssetApi removeAssets(List assetIds), though I do not know the real world performance difference between the bulk delete and iteratively deleting every asset. Maybe I will have the time to do some testing next week and would then give an update.
robertinnes11
OP2 years ago · EditedMany thanks for your reply, I saw that use of the restAPI. I am not sure looping thru the assets to get their UUID and then load this into List for subsequent removal would be any faster. I guess I will just have to try it and see
stefan_busch
·2 years ago · EditedHi,
there is a bulk delete function in the AssetApi removeAssets(List assetIds), though I do not know the real world performance difference between the bulk delete and iteratively deleting every asset. Maybe I will have the time to do some testing next week and would then give an update.
stefan_busch
·2 years ago · EditedHi,
so I did some tests to compare the performance. I measured the performance for the deletion of 100, 300, 500, 1000, 2000 and 3000 assets from one domain. The results are as follows:
For single deletion:
For bulk deletion:
As can be seen, bulk deletion is faster by a considerable margin.
For the sake of transparency I will append the code used.
Single deletion
Bulk deletion