feat(observability): OpenTelemetry logs #13291
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
samugi
changed the title
github-actions
bot
added
core/proxy
core/pdk
core/clustering
plugins/aws-lambda
core/db
core/configuration
core/logs
chore
samugi
changed the title
samugi
force-pushed
the
feat/otel-formatted-logs
branch
2 times, most recently
from
724a84a
to
a775404
Compare
samugi
force-pushed
the
feat/otel-formatted-logs
branch
3 times, most recently
from
5e5257d
to
3621b8f
Compare
ADD-SP
requested changes
Jun 25, 2024
ADD-SP
reviewed
Jun 25, 2024
yes absolutely. I wasn't planning to rebase (for now), which will make syncing (via cherry-pick) with the existing EE PR easier as well. |
samugi
force-pushed
the
feat/otel-formatted-logs
branch
from
d75b19b
to
522e3ba
Compare
ADD-SP
reviewed
Jun 26, 2024
|
|
||
|
|
||
| function _M.maybe_push(stack_level, log_level, ...) | ||
| -- WARNING: do not yield in this function, as it is called from ngx.log |
There was a problem hiding this comment.
Yes, it is hard to detect yielding cheaply, let's keep this issue here for more discussion.
There was a problem hiding this comment.
I added a test: 5b6cef9
hanshuebner
approved these changes
Jun 26, 2024
samugi
force-pushed
the
feat/otel-formatted-logs
branch
from
9939120
to
5b6cef9
Compare
samugi
added a commit
to Kong/docs.konghq.com
that referenced
this pull request
Jun 26, 2024
samugi
added a commit
to Kong/docs.konghq.com
that referenced
this pull request
Jun 26, 2024
2 tasks
samugi
added a commit
to Kong/docs.konghq.com
that referenced
this pull request
Jun 26, 2024
samugi
force-pushed
the
feat/otel-formatted-logs
branch
from
92eb17a
to
b076790
Compare
Spans were not set as "active" during their lifetime. This commit addresses the issue by setting each span as the current "active" span in instrumentations. This is necessary in order to have the correct span_id information in the OTel-formatted log entries.
When a disabled and non-request scoped hook was called outside of a request scope, it was producing an error while trying to index ngx.ctx This commit adds a check to early return from is_group_enabled when outside of the request scope This allows for hooks to still work when used in other contexts where they can be enabled via the always_enable() function
This commit adds OpenTelemetry formatted logs as a new signal supported by the OpenTelemetry plugin. It patches the ngx.log function to collect the logged line and enrich it with additional context info: * severity * introspection details * timestamp And for request-scoped logs also: * Request ID * Trace ID * Span ID * Trace flags The log entries are protobuf-serialized and sent to the endpoint configured in the OpenTelemetry plugin's config.logs_endpoint field.
This commit collects an additional tracing information (trace flags) in the internal context and adds it to produced OTel-formatted log entries.
This commit moves the tracing and logging core code in the observability folder.
This commit adds unit and integration tests for the OTel-formatted logs feature.
feat(dynamic_hook): support non request-scoped hooks + disable When a disabled and non-request scoped hook was called outside of a request scope, it was producing an error while trying to index ngx.ctx This commit adds a check to early return from is_group_enabled when outside of the request scope This allows for hooks to still work when used in other contexts where they can be enabled via the always_enable() function This commit also adds support for disabling hooks.
samugi
force-pushed
the
feat/otel-formatted-logs
branch
2 times, most recently
from
95c7b8a
to
8bdb8ed
Compare
samugi
force-pushed
the
feat/otel-formatted-logs
branch
from
8bdb8ed
to
b33755b
Compare
|
no logic changes in recent commits, I've only fixed compat with older DPs and added tests. |
|
@ADD-SP if possible could we get this reviewed on Monday? We have a wonderful set of customers that I'd love to show this off to as a beta. Thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note for reviewers: commits are split and commented appropriately, I would recommend checking them one by one.
Summary
This PR adds OpenTelemetry formatted logs as a new
signal supported by the OpenTelemetry plugin.
It patches the
ngx.logfunction (and adapts kong PDK log) to collect the logged lines andenrich them with additional context info:
And for request-scoped logs also:
The log entries are protobuf-serialized and sent to the endpoint
configured in the OpenTelemetry plugin's
config.logs_endpointfield.Checklist
CHANGELOG/unreleased/kongor addingskip-changeloglabel on PR if unnecessary. README.mdIssue reference
KAG-4712