Docker logs: Distinction between stdout and stderr in correct order

[alfechner]

I need to stream the logs from my docker container for logging.

Currently I do this:

stream = container.logs(stdout=True, stderr=True, stream=True)

for line in stream:
    logger.info(line.decode("utf-8"))

Now I need to differentiate between stdout and stderr. The stdout is supposed to be logged to logger.info, the stderr to logger.error.

My approach is to setup two streams, one for stdout and one for stderr and read them in parallel with different threads.

def process_stream(stream, callback):
    for line in stream:
        callback(line.decode("utf-8").strip())

stdout_stream = container.logs(stdout=True, stderr=False, stream=True, timestamps=True)
stderr_stream = container.logs(stdout=False, stderr=True, stream=True, timestamps=True)

stdout_thread = Thread(
    target=process_stream,
    args=(
        stdout_stream,
        logger.info,
    ),
)
stderr_thread = Thread(
    target=process_stream,
    args=(
        stderr_stream,
        logger.error,
    ),
)

stdout_thread.start()
stderr_thread.start()

stdout_thread.join()
stderr_thread.join()

However, messages from stdout and stderr are not always logged in correct order.

I also looked into ways to access the log files from the Docker JSON File logging driver I'm using directly but couldn't find a way.

Is there a way stream the messages in correct order plus the information if the message came from stdout or stderr?

[Khushiyant]

A better approach would be to use a single stream with the demux parameter set to True, which allows you to differentiate between stdout and stderr logs.