Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

timeout flush corrupts tags which contain colon #86

Open
shpatel2 opened this issue Feb 24, 2020 · 0 comments
Open

timeout flush corrupts tags which contain colon #86

shpatel2 opened this issue Feb 24, 2020 · 0 comments

Comments

@shpatel2
Copy link

shpatel2 commented Feb 24, 2020

Problem

When a colon (:) is in the tag, and a timeout flush occurs, then everything past the colon is lost.

Steps to replicate

  1. On windows, configure a in_tail, to tail a file like C:\temp\foo.log and set "tag *", this will cause the tag to be like "C:\temp\foo.log".

  2. Configure the concat plugin in a way which would cause a timeout flush

result: the timeout flush corrupts the tag...it eliminates everything after the colon, so it just says "C:"

Provide example config and message

  1. Create an empty C:\temp\foo.log file

  2. Create fluent.conf

########
# Source
########
<source>
  @type tail
  path C:/Temp/foo.log
  pos_file C:/Temp/foo.log.pos
  <parse>
	@type none
  </parse>
  tag *
</source>

################################
# Concatenate multiline messages
################################
<filter **>
  @type concat
  key message
  multiline_start_regexp /^(INFO)/
  continuous_line_regexp /[^(INFO)]/
  flush_interval 1
  use_first_timestamp true
  timeout_label @TIMEOUT_OUTPUT
</filter>

<label @TIMEOUT_OUTPUT>
    <match **>
        @type stdout
    </match>  
</label>

<match **>
	@type stdout
</match> 
  1. run 'fluentd -c fluent.conf'

  2. add the lines to foo.log and save it
    INFO - check the tag some other text which should have been concatenated.

Result:
The tag is corrupted:
2020-02-24 10:38:09.599785900 -0500 fluent.info: {"worker":0,"message":"fluentd worker is now running worker=0"}
2020-02-24 10:38:32.600000600 -0500 C:.Temp.foo.log: {"message":""}
2020-02-24 10:38:32.600013400 -0500 C: {"message":"INFO - check the tag"}
2020-02-24 10:38:33 -0500 [info]: #0 Timeout flush: C:.Temp.foo.log:default
2020-02-24 10:38:33.599202100 -0500 fluent.info: {"message":"Timeout flush: C:.Temp.foo.log:default"}

Expected Behavior

The tag should remain in-tact...it should say C:.temp.foo.log

Your environment

windows (this also occurs in docker)

  • OS version
    wntx64, but also occurs in linux/docker.

  • paste result of fluentd --version or td-agent --version
    fluentd 1.9.1

  • plugin version

    • paste boot log of fluentd or td-agent
    • paste result of fluent-gem list, td-agent-gem list or your Gemfile.lock
      fluent-plugin-concat (2.4.0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant