Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log some filter exprs #17351

Merged
merged 1 commit into from
Jul 6, 2024
Merged

Conversation

XuPeng-SH
Copy link
Contributor

@XuPeng-SH XuPeng-SH commented Jul 6, 2024

User description

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

issue #14518

What this PR does / why we need it:

log some slow ranges expr
log some exprs


PR Type

Enhancement


Description

  • Introduced traceFilterExprInterval to manage logging intervals for filter expressions.
  • Added logic in Ranges method to conditionally log filter expressions based on the length of ranges.
  • Implemented logging for filter expressions in NewReader method when enableLogFilterExpr is set.
  • Added enableLogFilterExpr atomic boolean to txnTable struct to control logging behavior.

Changes walkthrough 📝

Relevant files
Enhancement
txn_table.go
Add conditional logging for filter expressions based on range length

pkg/vm/engine/disttae/txn_table.go

  • Introduced traceFilterExprInterval to manage logging intervals.
  • Added logic to conditionally log filter expressions based on range
    length.
  • Implemented logging for filter expressions in Ranges and NewReader
    methods.
  • +47/-0   
    types.go
    Add atomic boolean for enabling filter expression logging

    pkg/vm/engine/disttae/types.go

    • Added enableLogFilterExpr atomic boolean to txnTable struct.
    +1/-0     

    💡 PR-Agent usage:
    Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    Copy link

    PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

    PR Reviewer Guide 🔍

    ⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
    🧪 No relevant tests
    🔒 No security concerns identified
    ⚡ Key issues to review

    Logging Interval Logic:
    The logic to determine when to log based on traceFilterExprInterval and rangesLen seems complex and could be error-prone. Consider simplifying this logic or adding more comments to explain the thresholds and the step values.

    Performance Impact:
    Frequent logging based on the range length could potentially impact performance, especially for large datasets. It would be beneficial to evaluate the performance implications of this logging, particularly under high load.

    @matrix-meow matrix-meow added the size/S Denotes a PR that changes [10,99] lines label Jul 6, 2024
    Copy link

    PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

    PR Code Suggestions ✨

    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Ensure atomicity of traceFilterExprInterval updates to avoid race conditions

    The traceFilterExprInterval.Add(step) and traceFilterExprInterval.Store(0) operations
    should be atomic to avoid race conditions. Use CompareAndSwap to ensure atomicity.

    pkg/vm/engine/disttae/txn_table.go [601-604]

    -if traceFilterExprInterval.Add(step) >= 500000 {
    -    traceFilterExprInterval.Store(0)
    -    tbl.enableLogFilterExpr.Store(true)
    +for {
    +    current := traceFilterExprInterval.Load()
    +    if traceFilterExprInterval.CompareAndSwap(current, current+step) && current+step >= 500000 {
    +        traceFilterExprInterval.Store(0)
    +        tbl.enableLogFilterExpr.Store(true)
    +        break
    +    }
     }
     
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    Why: The suggestion correctly identifies a potential race condition in the handling of traceFilterExprInterval and proposes a safer atomic operation which is crucial for thread safety.

    8
    Ensure exprs is not nil before formatting to avoid nil pointer dereference

    Add a check to ensure exprs is not nil before calling plan2.FormatExprs(exprs) to avoid
    potential nil pointer dereference.

    pkg/vm/engine/disttae/txn_table.go [611-618]

    +exprsStr := "nil"
    +if exprs != nil {
    +    exprsStr = plan2.FormatExprs(exprs)
    +}
     logutil.Info(
         "TXN-FILTER-RANGE-LOG",
         zap.String("name", tbl.tableDef.Name),
    -    zap.String("exprs", plan2.FormatExprs(exprs)),
    +    zap.String("exprs", exprsStr),
         zap.Int("ranges-len", ranges.Len()),
         zap.Uint64("tbl-id", tbl.tableId),
         zap.String("txn", tbl.db.op.Txn().DebugString()),
     )
     
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    Why: This suggestion correctly addresses a potential nil pointer dereference which is a critical issue. However, the existing code does not directly show usage of exprs in a context that would lead to a nil dereference, but the precaution is valuable.

    7
    Enhancement
    Use a single call to ranges.Len() and a switch statement for better readability

    Avoid calling ranges.Len() multiple times by storing its result in a variable.

    pkg/vm/engine/disttae/txn_table.go [591-600]

     rangesLen := ranges.Len()
    -if rangesLen < 5 {
    +switch {
    +case rangesLen < 5:
         step = uint64(1)
    -} else if rangesLen < 10 {
    +case rangesLen < 10:
         step = uint64(5)
    -} else if rangesLen < 20 {
    +case rangesLen < 20:
         step = uint64(10)
    -} else {
    +default:
         step = uint64(0)
     }
     
    • Apply this suggestion
    Suggestion importance[1-10]: 6

    Why: The suggestion improves code readability by using a switch statement, which is a good practice for handling multiple conditions. However, the existing code already stores ranges.Len() in a variable, so the main improvement is the switch usage.

    6
    Simplify the conditions that set tbl.enableLogFilterExpr to true

    Combine the two conditions that set tbl.enableLogFilterExpr to true into a single
    condition to simplify the code.

    pkg/vm/engine/disttae/txn_table.go [606-608]

    -if rangesLen >= 20 {
    +if rangesLen >= 20 || traceFilterExprInterval.Add(step) >= 500000 {
    +    traceFilterExprInterval.Store(0)
         tbl.enableLogFilterExpr.Store(true)
     }
     
    • Apply this suggestion
    Suggestion importance[1-10]: 5

    Why: The suggestion to simplify the conditions is valid and improves code readability, but it's a moderate improvement and not critical for functionality or performance.

    5

    @mergify mergify bot merged commit 017c4d0 into matrixorigin:main Jul 6, 2024
    17 of 18 checks passed
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    None yet

    3 participants