70 Messages
Duplicate Attribute Value Gets Created If more than 1 user clicks on "Submit" button in workflow
Hi Data Citizens,
We have recently encountered a bug, where in a duplicate attribute is getting created when more than 1 user clicks on “Submit” button.
Context:
A workflow is created which loops in an attribute (integer) and gets the highest int and increment it by 1. The workflow works fine, if they are triggered at different times.
Requirement:
I used thread, timer to include in the workflow, however, the WF still populates duplicate integer values for the attribute. I need unique values if more than 1 user clicks on "Submit " button (I know this is rare, but this shouldn’t happen)
Has anyone faced such issue? How could we overcome it?
Providing script for reference. The variable is prefixNum
def dqRuleList = assetApi.findAssets(FindAssetsRequest.builder().domainId(domainId).typeIds([dqRuleUUID]).limit(Integer.MAX_VALUE).build()).getResults().collect{it.getId()};
ArrayList prefixes = [];
for(dq in dqRuleList) {
def ruleIdval = attributeApi.findAttributes(FindAttributesRequest.builder().assetId(dq).typeIds([ruleID]).build()).getResults();
if (ruleIdval.size() > 0){
ruleId = ruleIdval.get(0).getValueAsString()
prefixes.add(ruleId.toInteger());
}
}
def maxruleIdList = prefixes.max();
prefixNum=maxruleIdList+1;
String newName = prefixNum.toString().concat("_").concat(dqRuleName);
def newAsset = assetApi.addAsset(AddAssetRequest.builder()
.name(newName)
.displayName(newName)
.typeId(assetTypeId)
.domainId(domainId)
.build())
def newAssetUuid = newAsset.getId();
Regards,
Ravi
arthurburkhardt
1.2K Messages
2 years ago
Ah, the old incremental signifier.
Here’s how we solve it: We define a prefix, a scope of a unicity (a domain / community / asset type, etc.) and we parse the previous signifiers and increment by one.
Implement an ID generator? - Developers - the Data Citizens community (collibra.com)
We haven’t had any issues so far.
If you want to retain your technique, I would suggest a more computationally efficient query by using the output module:
you could query for all DQ rules that have that attribute, sort by descending and pick the max.
0
0
ravi
70 Messages
2 years ago
Hi Arthur,
Thank you for your response. I tried this way, but still unicity is not maintained.
Since, you have defined a prefix, we as such do not have it defined for DQ Rule asset type, we instead follow a naming convention where asset name = “01234_SomeName_”, at the same time, we are using the integer (01234) for this stored in a int attribute type.
Initially, I was getting all the asset id for the asset type, then getting this attribute value, fetching the highest value, and incrementing it by 1 . It all worked fine, until we found out that, when more than 1 user clicked on the “Submit” button of workflow, unicity is not maintained.
I tweaked the code, but it didn’t work.
def dQRuleList = 1 + (
assetApi.findAssets(FindAssetsRequest.builder()
.typeIds([DQRuleUUID])
.limit(Integer.MAX_VALUE)
.sortOrder(SortOrder.DESC)
.build())
.getResults()
.collect{it.getDisplayName().split("_")[0]}
.findAll{ it.isNumber() }
.collect{ it.toInteger() }
)
Unicity is not maintained again, if more than 1 user clicks on “Submit” at the same time.
Regards,
Ravi
0
0
arthurburkhardt
1.2K Messages
2 years ago
There might still be a race condition lasting a few milliseconds, which would result in an error message for one of the users => You can implement a retry mechanism (e.g. try the operation three times until you throw an exception)
So it is very possible to implement it correctly and get a robust sequence generator.
0
0
ravi
70 Messages
2 years ago
Hi Arthur,
Thank you again for your response. At this point of time, we would not want to change / replace the full name and display name. of assets
So, the above code, is not working in our scenario.
@pranay.mahendra @bineesh.babu.collibra.com Could you please suggest any solution.
Regards,
Ravi
1
0
daniel
19 Messages
2 years ago
If more than one user clicks submit, do you actually want to have two assets created or only the one?
Assuming this is a race condition, the simplest but not robust method would be to just add a random wait time (say between 1-2 seconds or better to use milliseconds if you can) at the top of the script. This way, if user 1 clicks the button, maybe they get a 1 second wait and then user 2 gets a 2 second wait, which should be plenty of time for the first script to execute and the asset created for the second script to run.
The tricky thing will be to get the right amount of wait time which doesn’t annoy users but gives your script enough time to complete.
0
0