CoNLL-U metadata validation/cleansing

[chiarcos]

• version: Annatto 0.8.0 - 2024-06-17

• issue: After the conversion of (valid) CoNLL-U v1 data with non-standard metadata (see below), the output could be imported into ANNIS, but only partially visualized (no dependency view).

• suggestion

  • detect non-standard metadata during conversion, produce a warning and keep it as comment (not metadata)
  • add relevant test data to tests/data/import/conll

• background: In the CoNLL-U format, CoNLL comments before the sentence can be used to provide metadata, where a metadata attribute (e.g., text) is assigned a value (separated by =). In CoNLL-U v2, there are two obligatory metadata fields, text and sent_id, in CoNLL-U v1, metadata is optional, in CoNLL-X, metadata is treated as comment. In the following data snippet, an invalid separator is used, causing the ANNIS visualizer to break (p.c. by Thomas Krause). Apparently, this is because the converter tried to quietly recover the invalid metadata.

• example

    # text: 97 — Doch, was de Red von Ehrlichkeit, Von trüen Sinn in Freud un Leid, Kamm noch von't Voaderland derto — Denn flog åm ok de Schwanz mån so.
    1	97	_	NUM	NUM	_	3	compound	_	_
    2	-	_	PUNCT	$(	_	1	punct	_	_
    3	Doch	_	ADV	ADV	_	12	advmod	_	_
    4	,	_	PUNCT	$,	_	3	punct	_	_
    5	was	_	AUX	AUX.3.Ind.Prs.Sg	Number=Sing|Person=3|Tense=Pres	12	aux	_	_
    6	de	_	DET	DET.Def.Fem.Nom.Sg	Case=Nom|Definite=Def|Gender=Fem|Number=Sing	7	det	_	_
    7	Red	_	NOUN	NOUN.Fem.Nom	Case=Nom|Gender=Fem	12	obj	_	_
    8	von	_	ADP	ADP	_	9	case	_	_
    9	Ehrlichkeit	_	NOUN	NOUN.Dat.Neut	Case=Dat|Gender=Neut	7	nmod	_	_
    10	,	_	PUNCT	$,	_	7	punct	_	_
    11	Von	_	ADP	ADP	_	12	mark	_	_
    12	trüen	_	ADJ	ADJ.Inf	VerbForm=Inf	0	root	_	_
    13	Sinn	_	NOUN	NOUN.Nom.Pl	Case=Nom|Number=Plur	12	nsubj	_	_
    14	in	_	ADP	ADP	_	15	case	_	_
    15	Freud	_	NOUN	NOUN	_	13	nmod	_	_
    16	un	_	CCONJ	CCONJ	_	17	cc	_	_
    17	Leid	_	VERB	VERB	_	15	conj	_	_
    18	,	_	PUNCT	$,	_	12	punct	_	_
    19	Kamm	_	VERB	VERB.2.Imp.Sg	Mood=Imp|Number=Sing|Person=2	12	conj	_	_
    20	noch	_	ADV	ADV	_	19	advmod	_	_
    21	von't	_	ADP	ADP	_	22	case	_	_
    22	Voaderland	_	NOUN	NOUN	_	19	obl	_	_
    23	derto	_	ADV	ADV	_	25	advmod	_	_
    24	-	_	PUNCT	$(	_	23	punct	_	_
    25	Denn	_	ADV	ADV	_	26	advmod	_	_
    26	flog	_	VERB	VERB.3.Ind.Prt.Sg.st	Number=Sing|Person=3|Tense=Past	19	conj	_	_
    27	åm	_	PRON	PRON	_	30	case	_	_
    28	ok	_	ADV	ADV	_	30	advmod	_	_
    29	de	_	DET	DET.Def.Fem.Nom.Sg	Case=Nom|Definite=Def|Gender=Fem|Number=Sing	30	det	_	_
    30	Schwanz	_	NOUN	NOUN	_	26	obl	_	_
    31	mån	_	ADV	ADV	_	26	advmod	_	_
    32	so	_	ADV	ADV	_	26	compound:prt	_	_
    33	.	_	PUNCT	$.	_	12	punct	_	_
  

[MartinKl]

Thank you for the submission. Your request addresses several issues.

First, the dependency visualizer does not work, because graphml export sets the node key wrong. This is being fixed by #252

About supporting the data you provided and/or other versions of CoNLL: We suggest to stick to the notation using = as key-value delimiter for sentence annotations, since this seems easy to replace. We will nevertheless extend the conll module to import annotations that do not start with key = as bare values that will be added as a sentence annotation conll::comment holding said value. See #257 for more details. In case of your data this would lead to annotations conll::comment="text: ..." for each sentence.

Are there any other features of CoNLL-X that you consider necessary?

[chiarcos]

Thank you, #257 is the best way to deal with that IMHO.

As for other features of CoNLL-X, the last two columns have different functions (cf. https://aclanthology.org/W06-2920.pdf). I guess it's not worth supporting that because they were not widely used, in the first place and this pertains to legacy data, only, which does not seem to be publicly available anymore (at least not from https://ilk.uvt.nl/conll/post_task_data.html). It is still used by some older parsers, though, and sometimes required as input for downstream tasks. So, while I would not advise to go for full CoNLL-X support, I would suggest to be robust against CoNLL-X input, i.e., check whether CoNLL-X data with PHEAD (9th column) set to an integer would break the CoNLL-U conversion, because CoNLL-U expects pairs of IDs and dependency labels, there, and only these.

You can synthesize such data from CoNLL-U data by just copying the values from the HEAD column into the 9th column, and the values from the DEP column into the 10th column.