Changed: logic in calculating evaluation metrics #476

iftwigs · 2024-04-03T14:13:47Z

Added ConfusionMatrix class.
Changed how TNs are calculated.
Added ground truth calculation as a property.

github-actions · 2024-04-24T17:15:19Z

Name                                               Stmts   Miss  Cover
----------------------------------------------------------------------
konfuzio_sdk/__init__.py                               8      1    88%
konfuzio_sdk/api.py                                  453     83    82%
konfuzio_sdk/cli.py                                   39      0   100%
konfuzio_sdk/data.py                                2313    261    89%
konfuzio_sdk/evaluate.py                             426     82    81%
konfuzio_sdk/extras.py                                73     17    77%
konfuzio_sdk/normalize.py                            380     27    93%
konfuzio_sdk/regex.py                                107      0   100%
konfuzio_sdk/samples.py                              149      0   100%
konfuzio_sdk/settings_importer.py                     35      1    97%
konfuzio_sdk/tokenizer/__init__.py                     0      0   100%
konfuzio_sdk/tokenizer/base.py                       173     29    83%
konfuzio_sdk/tokenizer/paragraph_and_sentence.py     174    151    13%
konfuzio_sdk/tokenizer/regex.py                       96      2    98%
konfuzio_sdk/trainer/__init__.py                       0      0   100%
konfuzio_sdk/trainer/base.py                         161     23    86%
konfuzio_sdk/trainer/document_categorization.py      784     74    91%
konfuzio_sdk/trainer/file_splitting.py               525     39    93%
konfuzio_sdk/trainer/image.py                         46     17    63%
konfuzio_sdk/trainer/information_extraction.py      1064    118    89%
konfuzio_sdk/trainer/tokenization.py                 136     97    29%
konfuzio_sdk/trainer/utils.py                         47      0   100%
konfuzio_sdk/urls.py                                 133     12    91%
konfuzio_sdk/utils.py                                415    137    67%
----------------------------------------------------------------------
TOTAL                                               7737   1171    85%

… Evaluation class, Deleted: code doubling

github-actions · 2024-05-09T19:13:31Z

Name                                               Stmts   Miss  Cover
----------------------------------------------------------------------
konfuzio_sdk/__init__.py                               8      1    88%
konfuzio_sdk/api.py                                  453     83    82%
konfuzio_sdk/cli.py                                   39      0   100%
konfuzio_sdk/data.py                                2340    262    89%
konfuzio_sdk/evaluate.py                             399     41    90%
konfuzio_sdk/extras.py                                73     17    77%
konfuzio_sdk/normalize.py                            380     27    93%
konfuzio_sdk/regex.py                                107      0   100%
konfuzio_sdk/samples.py                              149      0   100%
konfuzio_sdk/settings_importer.py                     35      1    97%
konfuzio_sdk/tokenizer/__init__.py                     0      0   100%
konfuzio_sdk/tokenizer/base.py                       173     29    83%
konfuzio_sdk/tokenizer/paragraph_and_sentence.py     174    151    13%
konfuzio_sdk/tokenizer/regex.py                       96      2    98%
konfuzio_sdk/trainer/__init__.py                       0      0   100%
konfuzio_sdk/trainer/base.py                         161     23    86%
konfuzio_sdk/trainer/document_categorization.py      784     74    91%
konfuzio_sdk/trainer/file_splitting.py               525     39    93%
konfuzio_sdk/trainer/image.py                         46     17    63%
konfuzio_sdk/trainer/information_extraction.py      1070    118    89%
konfuzio_sdk/trainer/tokenization.py                 136     97    29%
konfuzio_sdk/trainer/utils.py                         47      0   100%
konfuzio_sdk/urls.py                                 133     12    91%
konfuzio_sdk/utils.py                                415     86    79%
----------------------------------------------------------------------
TOTAL                                               7743   1080    86%

…bel-multiple

github-actions · 2024-06-06T13:11:31Z

Name                                               Stmts   Miss  Cover
----------------------------------------------------------------------
konfuzio_sdk/__init__.py                               8      1    88%
konfuzio_sdk/api.py                                  456     41    91%
konfuzio_sdk/cli.py                                   40      0   100%
konfuzio_sdk/data.py                                2361    253    89%
konfuzio_sdk/evaluate.py                             399     41    90%
konfuzio_sdk/extras.py                                73     17    77%
konfuzio_sdk/normalize.py                            386     25    94%
konfuzio_sdk/regex.py                                107      0   100%
konfuzio_sdk/samples.py                              149      0   100%
konfuzio_sdk/settings_importer.py                     35      1    97%
konfuzio_sdk/tokenizer/__init__.py                     0      0   100%
konfuzio_sdk/tokenizer/base.py                       173     29    83%
konfuzio_sdk/tokenizer/paragraph_and_sentence.py     174    151    13%
konfuzio_sdk/tokenizer/regex.py                       96      2    98%
konfuzio_sdk/trainer/__init__.py                       0      0   100%
konfuzio_sdk/trainer/base.py                         161     23    86%
konfuzio_sdk/trainer/document_categorization.py      786     74    91%
konfuzio_sdk/trainer/file_splitting.py               542     45    92%
konfuzio_sdk/trainer/image.py                         46     17    63%
konfuzio_sdk/trainer/information_extraction.py      1070    118    89%
konfuzio_sdk/trainer/tokenization.py                 136     97    29%
konfuzio_sdk/trainer/utils.py                         58      1    98%
konfuzio_sdk/urls.py                                 138      3    98%
konfuzio_sdk/utils.py                                420     86    80%
----------------------------------------------------------------------
TOTAL                                               7814   1025    87%

# Conflicts: # konfuzio_sdk/evaluate.py

…/konfuzio-sdk into 12397-evaluation-fix

github-actions · 2024-06-13T10:39:09Z

Name                                               Stmts   Miss  Cover
----------------------------------------------------------------------
konfuzio_sdk/__init__.py                               8      1    88%
konfuzio_sdk/api.py                                  456     41    91%
konfuzio_sdk/cli.py                                   40      0   100%
konfuzio_sdk/data.py                                2364    254    89%
konfuzio_sdk/evaluate.py                             443     35    92%
konfuzio_sdk/extras.py                                73     17    77%
konfuzio_sdk/normalize.py                            386     25    94%
konfuzio_sdk/regex.py                                107      0   100%
konfuzio_sdk/samples.py                              149      0   100%
konfuzio_sdk/settings_importer.py                     35      1    97%
konfuzio_sdk/tokenizer/__init__.py                     0      0   100%
konfuzio_sdk/tokenizer/base.py                       173     29    83%
konfuzio_sdk/tokenizer/paragraph_and_sentence.py     174    151    13%
konfuzio_sdk/tokenizer/regex.py                       96      2    98%
konfuzio_sdk/trainer/__init__.py                       0      0   100%
konfuzio_sdk/trainer/base.py                         161     23    86%
konfuzio_sdk/trainer/document_categorization.py      786     74    91%
konfuzio_sdk/trainer/file_splitting.py               543     45    92%
konfuzio_sdk/trainer/image.py                         46     17    63%
konfuzio_sdk/trainer/information_extraction.py      1072    118    89%
konfuzio_sdk/trainer/tokenization.py                 136     97    29%
konfuzio_sdk/trainer/utils.py                         58      1    98%
konfuzio_sdk/urls.py                                 138      3    98%
konfuzio_sdk/utils.py                                420     86    80%
----------------------------------------------------------------------
TOTAL                                               7864   1020    87%

…bel-multiple

…bel-multiple Fix: Evaluation all zero for label of type multiple

…/konfuzio-sdk into 12397-evaluation-fix

…s call

github-actions · 2024-06-14T06:38:12Z

Name                                               Stmts   Miss  Cover
----------------------------------------------------------------------
konfuzio_sdk/__init__.py                               8      1    88%
konfuzio_sdk/api.py                                  456     41    91%
konfuzio_sdk/cli.py                                   40      0   100%
konfuzio_sdk/data.py                                2364    254    89%
konfuzio_sdk/evaluate.py                             448     35    92%
konfuzio_sdk/extras.py                                73     17    77%
konfuzio_sdk/normalize.py                            386     25    94%
konfuzio_sdk/regex.py                                107      0   100%
konfuzio_sdk/samples.py                              149      0   100%
konfuzio_sdk/settings_importer.py                     35      1    97%
konfuzio_sdk/tokenizer/__init__.py                     0      0   100%
konfuzio_sdk/tokenizer/base.py                       173     29    83%
konfuzio_sdk/tokenizer/paragraph_and_sentence.py     174    151    13%
konfuzio_sdk/tokenizer/regex.py                       96      2    98%
konfuzio_sdk/trainer/__init__.py                       0      0   100%
konfuzio_sdk/trainer/base.py                         161     23    86%
konfuzio_sdk/trainer/document_categorization.py      786     74    91%
konfuzio_sdk/trainer/file_splitting.py               543     45    92%
konfuzio_sdk/trainer/image.py                         46     17    63%
konfuzio_sdk/trainer/information_extraction.py      1072    118    89%
konfuzio_sdk/trainer/tokenization.py                 136     97    29%
konfuzio_sdk/trainer/utils.py                         58      1    98%
konfuzio_sdk/urls.py                                 138      3    98%
konfuzio_sdk/utils.py                                420     86    80%
----------------------------------------------------------------------
TOTAL                                               7869   1020    87%

…uth/prediction matching

github-actions · 2024-06-20T09:45:37Z

Name                                               Stmts   Miss  Cover
----------------------------------------------------------------------
konfuzio_sdk/__init__.py                               8      1    88%
konfuzio_sdk/api.py                                  456     41    91%
konfuzio_sdk/cli.py                                   40      0   100%
konfuzio_sdk/data.py                                2364    253    89%
konfuzio_sdk/evaluate.py                             448     35    92%
konfuzio_sdk/extras.py                                73     17    77%
konfuzio_sdk/normalize.py                            386     25    94%
konfuzio_sdk/regex.py                                107      0   100%
konfuzio_sdk/samples.py                              149      0   100%
konfuzio_sdk/settings_importer.py                     35      1    97%
konfuzio_sdk/tokenizer/__init__.py                     0      0   100%
konfuzio_sdk/tokenizer/base.py                       173     29    83%
konfuzio_sdk/tokenizer/paragraph_and_sentence.py     174    151    13%
konfuzio_sdk/tokenizer/regex.py                       96      2    98%
konfuzio_sdk/trainer/__init__.py                       0      0   100%
konfuzio_sdk/trainer/base.py                         161     23    86%
konfuzio_sdk/trainer/document_categorization.py      786     74    91%
konfuzio_sdk/trainer/file_splitting.py               543     45    92%
konfuzio_sdk/trainer/image.py                         46     17    63%
konfuzio_sdk/trainer/information_extraction.py      1072    118    89%
konfuzio_sdk/trainer/tokenization.py                 136     97    29%
konfuzio_sdk/trainer/utils.py                         58      1    98%
konfuzio_sdk/urls.py                                 138      3    98%
konfuzio_sdk/utils.py                                420     86    80%
----------------------------------------------------------------------
TOTAL                                               7869   1019    87%

github-actions · 2024-06-20T11:31:37Z

Name                                               Stmts   Miss  Cover
----------------------------------------------------------------------
konfuzio_sdk/__init__.py                               8      1    88%
konfuzio_sdk/api.py                                  456     41    91%
konfuzio_sdk/cli.py                                   40      0   100%
konfuzio_sdk/data.py                                2364    254    89%
konfuzio_sdk/evaluate.py                             448     35    92%
konfuzio_sdk/extras.py                                73     17    77%
konfuzio_sdk/normalize.py                            386     25    94%
konfuzio_sdk/regex.py                                107      0   100%
konfuzio_sdk/samples.py                              149      0   100%
konfuzio_sdk/settings_importer.py                     35      1    97%
konfuzio_sdk/tokenizer/__init__.py                     0      0   100%
konfuzio_sdk/tokenizer/base.py                       173     29    83%
konfuzio_sdk/tokenizer/paragraph_and_sentence.py     174    151    13%
konfuzio_sdk/tokenizer/regex.py                       96      2    98%
konfuzio_sdk/trainer/__init__.py                       0      0   100%
konfuzio_sdk/trainer/base.py                         161     23    86%
konfuzio_sdk/trainer/document_categorization.py      786     74    91%
konfuzio_sdk/trainer/file_splitting.py               543     45    92%
konfuzio_sdk/trainer/image.py                         46     17    63%
konfuzio_sdk/trainer/information_extraction.py      1072    118    89%
konfuzio_sdk/trainer/tokenization.py                 136     97    29%
konfuzio_sdk/trainer/utils.py                         58      1    98%
konfuzio_sdk/urls.py                                 138      3    98%
konfuzio_sdk/utils.py                                420     86    80%
----------------------------------------------------------------------
TOTAL                                               7869   1020    87%

iftwigs added 3 commits March 25, 2024 17:47

Changed: logic in calculating evaluation metrics

30a77d7

Changed: confusion matrix calculator

47d7ab1

Added: EvaluationConfusionMatrix class

df33bcf

iftwigs added 7 commits April 25, 2024 14:52

Merge branch 'master' into 12397-evaluation-fix

e1f4ee3

Changed: class structure, method moved outside compare() function

838a03a

Changed: integrating the GT/TN into compare() and existing Extraction…

b54f66e

… Evaluation class, Deleted: code doubling

Added: Confusion matrix into the Extraction evaluation

7c13a39

Added: docstrings for confusion matrix class

4331405

Fixed: affected tests

39d9cc8

Fixed: affected tests

e056367

Merge branch 'master' into 12397-evaluation-fix

a755b87

iftwigs requested a review from MohamedAmineDHIAB May 21, 2024 12:45

nengelmann and others added 3 commits June 5, 2024 13:01

Fix: Evaluation all zero for label of type multiple

a49ea75

Merge branch 'master' into 12397-evaluation-fix

b19cb3f

Merge branch '12397-evaluation-fix' into 12529-fix-evavluation-for-la…

1fe3752

…bel-multiple

iftwigs closed this Jun 6, 2024

iftwigs reopened this Jun 6, 2024

nengelmann and others added 3 commits June 6, 2024 16:16

Fix: Assignment, Changed: Ordering Spans

01ae48e

Merge branch 'master' into 12397-evaluation-fix

4b2b8e1

# Conflicts: # konfuzio_sdk/evaluate.py

Merge branch '12397-evaluation-fix' of https://github.com/konfuzio-ai…

471da75

…/konfuzio-sdk into 12397-evaluation-fix

iftwigs and others added 5 commits June 13, 2024 19:47

Merge branch '12397-evaluation-fix' into 12529-fix-evavluation-for-la…

fdc660e

…bel-multiple

Merge pull request #501 from konfuzio-ai/12529-fix-evavluation-for-la…

007082e

…bel-multiple Fix: Evaluation all zero for label of type multiple

Fixed: conditions in compare()

93eb1ec

Merge branch '12397-evaluation-fix' of https://github.com/konfuzio-ai…

44a7a06

…/konfuzio-sdk into 12397-evaluation-fix

Fix: Reference before assignment, remove outdated span prioritize row…

ce4a7ab

…s call

Added: parametrized tests to demonstrate different cases of ground tr…

9513388

…uth/prediction matching

Fixed: numpy version for compatibility with different python versions

e576933

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changed: logic in calculating evaluation metrics #476

Changed: logic in calculating evaluation metrics #476

iftwigs commented Apr 3, 2024 •

edited

Loading

github-actions bot commented Apr 24, 2024

github-actions bot commented May 9, 2024

github-actions bot commented Jun 6, 2024

github-actions bot commented Jun 13, 2024

github-actions bot commented Jun 14, 2024

github-actions bot commented Jun 20, 2024

github-actions bot commented Jun 20, 2024

Changed: logic in calculating evaluation metrics #476

Are you sure you want to change the base?

Changed: logic in calculating evaluation metrics #476

Conversation

iftwigs commented Apr 3, 2024 • edited Loading

github-actions bot commented Apr 24, 2024

github-actions bot commented May 9, 2024

github-actions bot commented Jun 6, 2024

github-actions bot commented Jun 13, 2024

github-actions bot commented Jun 14, 2024

github-actions bot commented Jun 20, 2024

github-actions bot commented Jun 20, 2024

iftwigs commented Apr 3, 2024 •

edited

Loading