Unauthorized heartbeat API

Question

Hello,
in our log files I keep finding a recurring log related to a ‘heartbeat’ API.
Has anyone seen this before? I’v e seen the heartbeat implementation in some other apps / sites and I think it’s a useful way to be able to alert for any outages.
However, in our case it seems the place where it’s being called for is not using correct authorization.

These are the logs that keep popping up :

2022-12-21 06:40:31.643 [http-nio-0.0.0.0-4400-exec-8] INFO c.c.i.adapters.web.util.FilterUtil - Returning ‘HTTP 401 Unauthorized’ on request ‘GET /rest/2.0/auth/sessions/heartbeat’ []

If anyone has seen this implemented and is using it , I’d like to hear more about it!
Thijs

daverichard · Answer

Interesting, the setup and configuration happened a few months before I was brought on, but I do not recall changing that default in any of the meetings I watched. Of course there was a lot of information overload. It is nice to be able to reset the level on that.

Also, that is funny, this morning there were no heartbeat errors (and still are none in our dev logs), but at some point today it started in Prod. I am adding that back to WARN.

Thank you for all of the assistance @arthur.burkhardt !

daverichard · Answer

Fortunately, we are not getting these again, I am curious what triggers them to start as we did not modify the log settings, just restarted our instance.

That said I would say a good 80% of our log files are constant

WARN org.hibernate.orm.deprecation - HHH90000022: Hibernate’s legacy org.hibernate.Criteria API is deprecated; use the JPA javax.persistence.criteria.CriteriaQuery instead

I would love to turn that off as most times I am just simply looking for an error related to a workflow, and before I am even able to get to the log the error is more than 200 lines off screen. It has meant that most times I am looking to review that error I just have the console open on auto-refresh.

daverichard · Answer

Fortunately, we are not getting these again, I am curious what triggers them to start as we did not modify the log settings, just restarted our instance.

That said I would say a good 80% of our log files are constant

WARN org.hibernate.orm.deprecation - HHH90000022: Hibernate’s legacy org.hibernate.Criteria API is deprecated; use the JPA javax.persistence.criteria.CriteriaQuery instead

I would love to turn that off as most times I am just simply looking for an error related to a workflow, and before I am even able to get to the log the error is more than 200 lines off screen. It has meant that most times I am looking to review that error I just have the console open on auto-refresh.

arthurburkhardt · Answer

@tomas.kissik.merck.com @thijs.waleson.heineken.com
We have the same issue, it is VERY annoying.
I opened a support ticket, but they were unable to help me. Here are some extracts of my messages:

it seems collibra uses the “Conversion Word” feature of logback - https://logback.qos.ch/manual/layouts.html#conversionWord
Can you please explain how to find out the full class path required to update logging levels?

I reverse decompiled the war file for Collibra 5.8 […] to find the full class path for “c.c.i.adapters.web.util.FilterUtil”: “com.collibra.identity.adapters.web.util.FilterUtil”. For reference, this class + AbstractAuthorizedQuartzJob accounted for 62.15% of meaningful log lines today, with little to no useful information.
(meaningful: what is not stack trace. Stack traces account for 49.25% of total lines)
e.g. at least 81.83% of log lines provide no meaningful value.
At this point, I’m able to guess the full class path from the decompiled code, but I don’t think this is a good way to go. What would be an acceptable way to configure log lines? Maybe a full inventory of classpaths would help.
I’m closing this ticket because my immediate problem is solved, I’ll raise to the CSM the overall issue with log management.

You can get rid of the messages by raising those two classes logging level to WARN

Class	Level
com.collibra.dgc.core.util.quartz.AbstractAuthorizedQuartzJob	WARN
com.collibra.identity.adapters.web.util.FilterUtil	WARN

arthurburkhardt · Answer

@tomas.kissik.merck.com @thijs.waleson.heineken.com
We have the same issue, it is VERY annoying.
I opened a support ticket, but they were unable to help me. Here are some extracts of my messages:

it seems collibra uses the “Conversion Word” feature of logback - https://logback.qos.ch/manual/layouts.html#conversionWord
Can you please explain how to find out the full class path required to update logging levels?

I reverse decompiled the war file for Collibra 5.8 […] to find the full class path for “c.c.i.adapters.web.util.FilterUtil”: “com.collibra.identity.adapters.web.util.FilterUtil”. For reference, this class + AbstractAuthorizedQuartzJob accounted for 62.15% of meaningful log lines today, with little to no useful information.
(meaningful: what is not stack trace. Stack traces account for 49.25% of total lines)
e.g. at least 81.83% of log lines provide no meaningful value.
At this point, I’m able to guess the full class path from the decompiled code, but I don’t think this is a good way to go. What would be an acceptable way to configure log lines? Maybe a full inventory of classpaths would help.
I’m closing this ticket because my immediate problem is solved, I’ll raise to the CSM the overall issue with log management.

You can get rid of the messages by raising those two classes logging level to WARN

Class	Level
com.collibra.dgc.core.util.quartz.AbstractAuthorizedQuartzJob	WARN
com.collibra.identity.adapters.web.util.FilterUtil	WARN

Ann Wuyts · Answer

Hi Thijs,

The heartbeat references you see in the logs are related to authentication, and not to application moninitoring (confirmed by our SRE team).

GET /rest/2.0/auth/session/heartbeat is used by the UI to poll the backend to check the liveness of the user’s session. It returns 200 if the user session is active. and 401 if the session is expired or doesn’t exist.

https://developer.collibra.com/rest/data-governance#/operations/heartbeat

date	total	of_which_web_util_filters
2023-06-20	33,591	29,211
2023-06-19	30,566	25,126
2023-06-18	6,344	3,766
2023-06-17	4,878	2,114
2023-06-16	18,582	13,277
2023-06-15	23,397	17,882
2023-06-14	20,766	15,786
2023-06-13	20,173	14,110
2023-06-12	23,107	17,148
2023-06-11	7,515	3,744
2023-06-10	6,944	4,443
2023-06-09	19,901	15,436
2023-06-08	21,511	14,820
2023-06-07	21,552	15,832
2023-06-06	23,381	17,268
2023-06-05	20,109	15,358
2023-06-04	8,424	2,280
2023-06-03	2,317	0
2023-06-02	4,663	0
2023-06-01	4,960	0

1	Grant Rollerson SUNCORP CORPORATE SERVICES PTY LTD
2	Tom Friesen Algonquin College
3	SB S Benson Plus3 IT
4	S stefan_busch
5	Brian Lloyd USA TODAY Co.
6	D daverichard