Edge
Created Nov 18, 2023
1 member
19 discussions
The Edge topic in the Collibra Community is dedicated to supporting users working with Collibra Edge, the platform’s integration layer for harvesting metadata and lineage. It's a space to explore setup guidance, troubleshoot scaling issues, manage secure credential integration, and share best practices for running Edge in environments like Kubernetes and cloud platforms.
davidwright
9 months agoFellow Data Citizens, we are looking for help in moving forward an ideation. We have noticed an issue with technical lineage where relations are being deleted by the lineage harvesting process when the relationship type is the same as used by the lineage harvester.
The Issue: When ingesting lineage via the lineage harvester, any relation created via the UI, import file or via API is deleted by the lineage harvester if it is the same relationship type as one harvested by the lineage harvester, By example, your team imports source/target relations to document an ETL process from a lineage harvested database table to some target. The next time that the lineage harvester runs, that imported ETL lineage is deleted. This means that your team needs to replace that lineage whenever the harvester is run which is an onerous and possibly costly process. The product team offered a workaround by which the lineage is loaded via custom technical lineage. This is complex given the lack of an API as well as inflexible for those that may want to use the UI or upload file capability.
The Solution: Our team has proposed a solution to an ideation (https://productresources.collibra.com/ideation-platform/?id=DCC-I-196) to solve the issue. Our proposed solution would be an enhancement to limit the modification or deletion of relationships by technical lineage to the relations created by technical lineage.
There is a similar ideation out there that was recently posted as well: https://ideas.collibra.com/ideas/CDL-I-8 Given the existence of another ideation and comments on our ideation, we would anticipate that many customers are facing the same issue.
Please help us move this issue up the priority list for the Edge team by voting for both ideations. In that way, we can show the impact of this issue and need for its resolution.
Thank you!
Dave Wright
user_a0c573
1 year agoGreetings,
I am reaching out regarding the sizing of the Edge Site within a Google Compute Engine.
Referring to the URL provided (https://productresources.collibra.com/docs/collibra/latest/Content/Edge/EdgeSitesInstallation/ref_edge-site-system-requirements.htm?Highlight=edge%20%20system%20requirements), the minimum hardware requirement for Collibra Data Intelligence Cloud is stated as a 16-core CPU with x86 architecture and 64 GB of memory. However, considering that the current utilization is less than 2%, with expectations of enhancement later on, my client is inquiring about the possibility of downsizing to a 4 CPU and 16 GB RAM configuration, essentially reducing to 1/4th of the current specifications.
Additionally, there's a plan to resize the VM as the load increases in the future. My query pertains to the feasibility of this option and whether it might cause any issues with the existing installation.
celenemcfall
1 year agoEdge now supports integrating with your Vault applications and services for easier credential access and management policy enforcement. By removing sensitive information from your edge site, you reduce security risks. Edge supports integration with CyberArk, HashiCorp, Azure Key Vault, AWS Secrets Manager, and Google Secret Manager. To learn more and get started, read our documentation.
