Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: start & end times for consuming #3

Open
twmb opened this issue Oct 2, 2020 · 0 comments
Open

feature: start & end times for consuming #3

twmb opened this issue Oct 2, 2020 · 0 comments

Comments

@twmb
Copy link
Owner

twmb commented Oct 2, 2020

kcl can support consuming from specific timestamps.

The syntax for this should be s@<microsecond> and e@<microsecond> (where <microsecond> refers to a unix microsecond timestamp), and this can be pieced into the existing -o flag. Both or just one can be specified.

The logic for this is to basically first get the metadata for all consumed topics, and then issue a list offsets request with Timestamp being the start timestamp. While consuming, if the record timestamp for a partition surpasses the end timestamp, stop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant