[Sponsor by MEL][New Feature] Build a quality metadata widget

[gkeimeHDF]

We want to add a widget that show the metadata quality.

This widget looks like the quality metadata that we can observe in data.gouv.fr (https://guides.etalab.gouv.fr/qualite/documenter-les-donnees/)

To make the scoring sortable and filtrable, we need integrate it directly in the back office in the geonetwork. We think we can add an index elastic search “scoring” that will give a quality indicator from 0 to 100. We need to look if this indicator will be customizable or generic.

Then to display the indicator, we could use the progressbar component available in geonetwork-ui.
We would like to integrate this widget in the datahub, in the search list and in the detail page.

A sample in the current search list:

A sample mobile version:

A sample in the detail page

A focus sample

Then when we focus, on the indicator, it will show a popup that’s show which criteria is filled or not.

This widget will be created inside libs > ui > elements > src > lib we will add a metadata-quality component that use the progressbar component to display the scoring.

To integrate it inside the datahub, we will add this component inside the existing geonetwork-ui components:

• “record-metadata” for the detail page
• “record-preview-row” for the search row.

Ideally if we don’t set the scoring index, the widget will not be shown (I don’t know if we need add a configuration to force to hide it in detail or/and in search part?)

We should allow which fields will be shown with configuration properties.

The list of fields displayed, and the scoring indicators can be uncoordinated (for example in our case l if the periodicity is outdated, we would like the indicator loses 50%).

In a first time we would like to check the presence about these fields:

• Title
• Description
• Category
• Key words
• Legal constraints
• Geographical influence
• Resource Contact
• Update frequency
• Modification date
• Provider

The field list should be customizable.

[jahow]

Thank you for the recap.

A few remarks:

• Agreed on putting the progress bar component in ui/elements; I think the popup could be a new generic ui/layout component
• I think the existing progress bar can be visually improved, it is honestly quite ugly IMO
• In terms of accessibility, we should take care of allowing opening the popup through keyboard navigation and on mobile
• how do you plan on checking whether the advertised periodicity is being respected?
• making the "formula" configurable will be quite complex I think; this is because we don't want Elasticsearch fields appearing in the app configuration as that might create a coupling with an implementation detail; on the other hand, this field list looks quite reasonable and could apply to other platforms
• about the checked fields:
  • What is "geographical influence"?
  • Is there a difference between "resource contact" and "provider"?

As for the implementation, you can see in #445 an example on how to use a runtime field to compute things dynamically in the index; this could be used for example to compute the various criteria.

[gkeimeHDF]

How do you plan on checking whether the advertised periodicity is being respected?

The periodicity will be checked in a futur version, we will study if is really possible on this next version.

What is "geographical influence"?

Maybe my translation was wrong, it's the geographic elements, in french "Emprise géographique"

Is there a difference between "resource contact" and "provider"?

I will check with MEL but i think the resource contact is like a mail reference... and the provider is more like the organization name.

[jahow]

Thanks for the followup! +1 for moving forward with the implementation, let's use this discussion if other questions arise

[jahow]

I would say that the best solution is probably to add the missing fields in

geonetwork-ui/libs/util/shared/src/lib/elasticsearch/constant.ts

Line 7 in c34b86e

export const ES_SOURCE_SUMMARY = [

But it also depends on how many fields are needed; one or two would be acceptable, but more might increase the response sizes too much. In that case we might have to look for another solution.

@gkeimeHDF what are the missing fields?

[gkeimeHDF]

1. The "keywords", "constraints", "contact.email", "updateFrequency" are missing in the search and present in the details.

The keywords regroup different fields like the "Thème" ? How can i check if one MetadataRecord has at least one topic?

3. I see there is a field "hasMaps" what is the meaning about this field? Is it the same about the filter "Données spatiales" ?

4. I see there is "dataUpdateOn" and "updateOn" what the difference and meaning about these two attributes?

[jahow]

1. The "keywords", "constraints", "contact.email", "updateFrequency" are missing in the search and present in the details.

Maybe aggregating all these fields into one in the search results would be a good solution? Not sure though, adding four new fields might not be a big deal either.

2. The keywords regroup different fields like the "Thème" ? How can i check if one MetadataRecord has at least one topic?

Topic and keywords are different; there are keywords of type "theme" though, but that is not what the "Thème" filter shows (which is topics).

3. I see there is a field "hasMaps" what is the meaning about this field? Is it the same about the filter "Données spatiales" ?

Both fields are computed differently but have the general same meaning: whether a geo-capable link is available in the metadata.

4. I see there is "dataUpdateOn" and "updateOn" what the difference and meaning about these two attributes?

dataUpdatedOn is not used as far as I can tell; updatedOn is the date of last modification of the metadata.

[gkeimeHDF]

4. If the data come from Harvesting, the date will be the last harwest ?

5. what the differences between :
   'MD_LegalConstraintsUseLimitationObject',
   'MD_LegalConstraintsOtherConstraintsObject',
   'MD_SecurityConstraintsUseLimitationObject',
   ?

6. Et que veut dire le flag "isOpenData" car j'ai vu des exemples avec :

• isOpenData: false
• constraints: "licence ouverte"
  

[jahow]

4. If the data come from Harvesting, the date will be the last harwest ?

In theory no it won't; the last change date is part of the metadata document.

5. what the differences between :
   'MD_LegalConstraintsUseLimitationObject',
   'MD_LegalConstraintsOtherConstraintsObject',
   'MD_SecurityConstraintsUseLimitationObject',
   ?

These are derived from ISO 19139/19115-3 fields with subtle differences which would be too long to explain here I think.

6. Et que veut dire le flag "isOpenData" car j'ai vu des exemples avec :

See how this field is generated: https://github.com/geonetwork/core-geonetwork/blob/58c831e3d1676e1c6299372fd8b5d0b3c9d48f75/web/src/main/webapp/xslt/common/index-utils.xsl#L376

Basically it's looking at whether the record has keywords flagged as "open data" ones.

[gkeimeHDF]

@jahow Hello i had created the pull request #497

Here the documentation related to this widget :
configuration-MEL.docx