Version 1.5.4

This release removes an outdated pathlib requirement and updates the workflows to run smoothly again

Version 1.5.3

Support rdflib version 6+

The rdflib upgrade from version 5.0.0 to 6+ introduced breaking changes
in the rdflib library. This update fixes these breaks without changing
functionality, unpins the rdflib version and defaults to the latest
version on install.

Dropping official support for Python 3.5 and 3.6

The introduction of the rdflib 6+ library restricts the
Python support for 3.7+.

It is still possible to install odml with a Python version <=3.6,
but these versions are no longer tested and officially supported.

Version 1.5.2

Pinning rdflib version to 5.0.0 until further notice

Due to major breaking changes introduced in the upgrade of rdflib 5.0.0 to 6.0.0, the rdflib version is pinned to 5.0.0 until the breaking code has been fixed.

Dropping Python 2 support

Python 2 has reached end of life. The codebase has been cleaned from Python 2 and Python 3 compatible code, all tests now run exclusively on Python 3 versions.

Features

• odml.save and odmlparser.ODMLWriter now support additional keywords. See issue #402 and PR #403 for details.

Version 1.5.1

RDF Subclassing feature

RDF subclasses are now properly added by default to any written RDF document. The RDF document will now also include RDF Subclass definitions in addition to the actual data to enable Subclass specific queries. See PR #400 and issue #397 for details.

Minor changes and updates

• Section properties can now be reordered. See PR #398 for details.
• Property values can now be inserted at a specified index. See PR #398 for details.
• Tuples can now be assigned using a list instead of the "(x;x;...)" syntax as well. See PR #393 and issue #392 for details.

Version 1.5.0

Python 2 deprecation warning

A Python 2 deprecation warning for August 2020 has been added. See issue #387 for details.

Validation feature update

See issues #377, #378 and #379 as well as Pull Request #389 for details.

An IssueID enum class as been added to provide identifiers to individual ValidationErrors. The Validation class itself has been refactored to provide the option to create standalone Validation instances with a different set of registered validations than the default library validation.
The Validation class now features the new register_custom_handler, run_validationand report methods to add custom validation handlers to an instance, re-run the validations of an existing Instance and provide a brief report of encountered errors and warnings. The general ValidationError.__repr__ string has been shortened to make the individual ValidationErrors more convenient to print and read. The default Validation is always run when a Document is saved or loaded via the ODMLParser and the Validation.report method is used to provide a warnings.warn message of the following format:

UserWarning: The saved Document contains formal issues. Run 'odml.validation.Validation(doc)' to resolve them.
Validation found 0 errors and 3 warnings in 1 Sections and 1 Properties.

Further changes to the Validation class and behavior include:

• an odml Document now provides a validate method that will run a default Validation and return the Validation instance to provide users with access to encountered issues.
• a validation_id field has been added to the ValidationError class.
• standalone Sections and Properties can now be validated.
• Sections and Properties are validated on init.
• the section_repository_present validation has been removed from the default validation list. Since Sections rarely have repositories set, this validation can lead to spam when validating a Document.

Cardinality feature

Property and Section now provide a cardinality feature. Users can now define a range how many Values a Property and how many Properties or Sections a Section should have. A cardinality can be set and read via its accessor method and can be set via an additional convenience method. Whenever a cardinality or an affected Value, Section or Property is set, a corresponding validation is triggered. If this a set cardinality for a Property or Section is violated, a message is printed to the command line directly and a warning is issued when a Document is saved or loaded. Every cardinality is saved to and loaded from all available file formats.
The full functionality of all cardinality features are documented in the tutorial and is available via readthedocs. For additional details see pull requests #374, #382, #383, #384 and issue #361.

Update in Section type default behavior

With recent updates the library now respects and enforces Section.type as a required attribute and allows save only with documents where this requirement is satisfied.
To allow backwards file compatibility and ease usage, Section.type is by default set to the string n.s. (not specified), which means files where no Section.type had been specified can be loaded and saved, but will contain n.s. as value for every Sections.type that was previously not specified.
Further the validation run before a document can be saved will issue a warning, if a Section.type with value n.s. is encountered and will still refuse to save with an error, if an empty Section.type is encountered. See PR #376 for details.

DictParser and ODMLParser fully support ignore errors

• the DictParser.DictReader is now able to ignore errors, collect warnings and print corresponding notifications and works now analogous to the xmlparser.XMLReader behaviour. See issue #367 for details.
• the ODMLParser.ODMLReader for JSON and YAML now uses ignore_errors by default e.g. when using the odml.load function for JSON and YAML odml files.

Fixes

• fixes an exception when trying to append or extend a Property with dtype tuple. See issue #364 for details.
• when trying to set the name attribute to None, it now silently sets the name to id instead, since name must not be empty. It would be set to id on load and can cause AttributeError exceptions with some methods if its not set.
• a bug was fixed in format.revmap where the reverse mapping of an odml attribute would always return the case that the attribute is part of the format, even if it was not.

Minor changes and updates

• all deprecation warnings now use the warnings module.
• the Property.value attribute deprecation warnings have been unified. See issue #360 for details.
• the base.Sectionable.create_section method has been updated to conform with Section.__init__. See issue #368 for details.
• all saved XML odML files now use the same XML header. See issue #339 for details.
• a function to manually refresh the terminology cache has been added. See issue #202 for details.
• a Validation to note non-human readable Property and Section names has been added. See issue #365 for details.
• getter and setter methods are added to the odml.Document.origin_file_name attribute. See issue #358 for details.
• the Exception type in odml.tools.converters.VersionConverter is changed to odml.tools.parser_utils.ParserException. See issue #359 for details.
• the odml.Property.export_leaf method now also includes sibling Properties on export.
• the rdf_converter has been cleaned up, see issues #211 and #345 for details.
• the test for the Section/Property order in documents obtained via the RDFReader has been expanded. See issue #265 for details.
• tests for Validation errors on Section or Property init have been added. See issue #369 for details.
• tests writing temporary files now properly clean up after themselves. See issue #381 for details.
• tests now use a common temporary directory to write files and use a constant for accessing the test/resources directory.
• the link to the odML tutorial in the README file now points to python-odml.readthedocs.org; the README file now also includes links to Travis and Coveralls.
• the tutorial now includes descriptions of the pprint method and a link to the odML templates hosting site. Further the tutorial has been updated to include descriptions of the cardinality feature and Validation usage.
• introduces major PEP8 fixes to basically all files of the library. See Pull Request #385 for details.
• the class reference now includes the Template, Terminology and Validation classes.

Version 1.4.5

Minor changes, updates and fixes.

• all usages of the unsafe yaml.load calls are replaced with yaml.save_load. This also prepares for Python 3.9 compatibility. See also issue #350 and pull request #356 for details.
• dtype tests now use both assertRegexpMatches and assertRegex depending on the Python version used to prepare for Python 3.9 compatibility while still keeping the Python 2 tests running.
• odml style tuple handling is refactored. Now lists of odml style tuples are properly saved to file and can be loaded again. If an invalid format is used to add an odml style tuple, more detailed exception messages are available. Also adds more odml style tuples tests. See issues #250, #353 and #354 for details.
• a deprecation warning is displayed when importing the odml module if a Python version <3.6 is used.
• introduces minor PEP8 fixes to all files and completes docstrings for full documentation.

Version 1.4.4

Introduction of inline style sheet

The XMLWriter can now be used to save an XML odML document with an inline XSL stylesheet for rendered viewing via a web browser.
The document can be saved with the current default G-Node odML document style or with a custom style. When a document containing such a stylesheet is loaded, the style is lost and needs to be provided again as described above. See issue #331 for details.

Export leaf feature

Subsets of odML documents can now be exported using the introduced export_leaf features for Section and Property.
When invoked on a Section or a Property, the method will return all direct ancestor Sections including all their direct Properties up to the root of the document and return a new, cloned document containing only this particular direct branch. See issue #340 for details.

Major pyyaml dependency fix

Since issue #291 the odml package had its pyyaml dependency fixed to version 4.2b4.
This release fixes the original problem with the pyyaml distribution and removes the fix to pyyaml version 4.2b4.
See issue #343 and pull request #344 for details.

odML package structure changes to reduce import cycles

The package structure was changed to reduce the number of import cycles and make various parser classes more easily available via the __init__ files. See issues #317 and #333 for details.

The changes to the package structure should not affect any of the main parsers and how they were previously imported nor affect any of the command line scripts provided with this package.

The following changes have been introduced to the package structure:

• a new subdirectory odml.tools.converters is added.
• format_converter and version_converter are moved into this directory.
• since odml-ui depends on the version_converter, a dummy file is left at its original location. It imports the VersionConverter class from its new location and prints a deprecation warning.
• the dict RDFConversionFormats from file tools.utils has been moved to RDF_CONVERSION_FORMATS in file tools.parser_utils and all usages have been switched to the new dict.
• the dict ConversionFormats from file tools.utils has been moved to the only file its using it, tools.converters.format_converter.
• the now unused file tools.utils has been removed.
• a new subdirectory odml.rdf was added and the files fuzzy_finder and query_creator were moved into this directory. Both files provide convenience and additional functions for odML specific RDF and are fairly independent from the rest of the library. Conceptually they are best kept separate from other convenience tools and parsers.

Minor changes, updates and fixes

• The section_subclasses.yaml file is moved from the doc folder to the new odml/resources folder. This allows usage of the file with all install options. See issue #212 for details.
• The RDF subclasses now also support DataCite Section types.
• The rdf_converter now features a new load_rdf_subclasses function, that either provides the content of the resources/subclasses.yml file or, if it cannot be accessed, deals with it without breaking the RDFWriter.
• Fixes calling the deprecated odml.Property.value attribute in the rdf_converter.
• The RDFWriter will now call every odml documents finalize method to ensure that all links and includes are resolved before exporting to RDF.
• The OWL odml ontology file is moved from doc/root-ontology.ttl to odml/resources/odml-ontology.ttl and included in Manifest.in.
• The RDF namespace is changed from https://g-node.org/projects/odml-rdf# to https://g-node.org/odml-rdf#. This step is taken since the odml-rdf ontology OWL file will be available under this URL. This change includes changes in all code and documentation occurrences in the project.
• Typos and minor inconsistencies where fixed in the odml - RDF subclasses mapping file resources/section_subclasses.yaml.
• The OWL odml ontology file resources/odml-ontology.ttl has been restructured:
  • all available subclasses where moved to their own file section.
  • all subclasses that are created via the resources/section_subclasses.yaml file have been added to the ontology file as well.
• Value errors concerning date, time and datetime now contain a message providing the required format. See issue #341 for details.

Version 1.4.3

Introduction of odML templates support and update in odML terminology handling

Support for importing and working with odML templates is added:

• the core library now features the TemplateHandler class to handle import and usage of odML templates.
• the default URL to fetch templates from has been set to https://templates.g-node.org.
• all terminology URLs are updated to the new terminology deployment at https://terminology.g-node.org.

Additional console script: 'odmlview'

Currently most web browsers no longer support viewing local files that include further local files like stylesheets; check here for additional details. The console script odmlview provides a local webserver that is able to properly serve odML XML files from a local directory and render them correctly, if the appropriate stylesheets are present in the same directory.

'odmlview' sets up a minimal webserver to view odml files saved in the
XML format via the webbrowser. After it is started, the webserver will
open a new tab in the default webbrowser and display the content of
the directory the server was started from. odML files can then be
viewed from there.
To properly render XML, an odML file may contain the element
'<?xml-stylesheet  type="text/xsl" href="odmlDocument.xsl"?>' where the
'odmlDocument.xsl' stylesheet should reside in the same directory as the
odML file to be rendered. By using the '--fetch' flag the latest version
of this stylesheet will be downloaded from `templates.g-node.org` to
the current directory when starting up the service.

Console script 'odmlconversion' is renamed

The console script odmlconversion is renamed to odmlconvert. For backwards compatibility the script will be available as odmlconversion with a deprecation notice.

'pyyaml' dependency update

The pyyaml dependency has been changed to the non-breaking beta version 4.2b4. See issue #291 for details.

Minor changes, updates and fixes

• odML entity IDs are automatically added when converting from odML version 1 to odMl version 1.1 for Document, Section and Property elements. If an ID already exists, it stays the same, if it is compatible with the Python UUID types. Otherwise (also if empty) a new ID is created for the odML entity.
• Property can now be set as int with value 0. See issue #314 for details.
• appending and extending of Property values of dtypes person, url and text is now possible. See issue #318 for details.
• the default RDFWriter format is set to turtle. See issue #214 for details.
• the RDFWriter now checks for a given file extension within the RDF document name if accidentally given there. See issue #213 for details.
• the RDFWriter now throws a ValueError if an unsupported RDF format is given. See issue #215 for details.
• the XMLReader now properly handles entity creation failures when started with option ignore_errors=True. See issue #276 for details.
• the pprint method has been added to Document to print while document section. See issue #319 for details.
• the README file has been changed from rst to the md format.
• the current tutorial has been updated to include latest changes.
• the automated builds have been updated to include Python versions 3.7 and 3.8; version 3.4 has been removed since it is no longer supported on travis.

Version 1.4.2

Print methods

pprint methods have been added to both Section and Property
to print whole Section trees with their child sections and properties.
The __repr__ style of Section and Property has been changed to
be more similar to the nixpy __repr__ style.
Printing a Section now also features the immediate Property child count
in addition to the immediate Section child count. See #309.

Deprecation of 'Property.value' in favour of 'Property.values'

To make working with odML more similar to working with the
metadata part of nixpy, the Property.value
attribute has been marked deprecated and the Property.values
attribute has been added. See #308.

Toggle warning messages when loading a document from file

A 'show_warnings' flag was added to all load from file methods. The flag is 'True' by
default and toggles whether warning messages are printed to the
command line when loading an odML document. See #311 for details.

Uncertainty changes

Uncertainty is now limited to float only. See #294.

Version converter changes

The VersionConverter dealt with an edge case of XML test files with opening tags
that were missing their closing tag rendering them broken. Catching this one edge case
circumvented opening XML files via lxml, leaving the resulting document open to various
encoding problems.

Support to resolve the specific tag edge cases is dropped in favour of properly opening
XML files via lxml. See #301.

A further change addressed, that previously all same named odml.Sections within a
document were indexed upon conversion. With these changes, only same named
odml.Sections that reside on the same level are indexed. See #312 for details.

Additional console script

The odmlconversion convenience console script has been added to convert multiple
previous odML version files to the latest odML version. See #302 for details.

Changes in cloning behaviour

When cloning a Section or a Property by default the id of any object is changed
to a new UUID. The cloning methods now feature a new keep_id attribute. If set to
True, the cloned object and any cloned children retain their original id. This
is meant to create exact copies of Section-Property trees in different documents. See #304 for details.

Additional validation

When a document is saved, a new validation check makes sure, that a document
contains only unique UUIDs this is required due to the introduction of creating
clones with identical ids. See #303 for details.

Version 1.4.1

Dependency changes

• pyyaml was version fixed on 3.13 to circumvent introduced breaking changes in the library. See #291, #292, #296, #298.
• docopt was added to support console scripts

Converter and Parser fixes

• Fixes that an XML file with an UTF-8 encoding file header was not being properly parsed by the VersionConverter XML parser. See #288, #296.
• Fixes the XMLParser that when reading a single string value from csv which contains commata, it now remains a single value and is not split up at each comma. See #295, #296.
• In the XMLParser any leading or trailing whitespaces are removed from any string values when it is written to csv. Along the same lines, multiple values that are saved to file via the VersionConverter do not contain leading whitespaces any longer. See #296.
• Thorough encoding and usage of unicode has been introduced to all Parsers and Converters to avoid encoding errors with Python 2 and Python 3. See #297.

Changes in Section and Property SmartList

• Adds SmartList.sort(). By default Document and Section child lists will retain the order in which child elements were added, but now a sort by name can be manually triggered. See #290.
• Adds SmartList comparison magic methods to partially address #265. The introduction of the RDF backend led to an issue when comparing odML entities. The used RDF library rdflib does not respect child order upon loading of a file, odML entities with children can not be compared without sorting the child elements. The added magic methods sort child elements by name before comparison without changing the actual order of the child elements. This only addresses the issue for Section and Property child lists, but does not solve the problem for the order of Property.values. See #290.

Document format update

• A new private attribute _origin_file_name is added to the Document entity. When an odML document is loaded from file, this attribute is now set with the file name from whence the document was loaded. See #297.

RDF format changes

• The RDF class Seq is now used instead of Bag to store odml.Property values to respect the order of values. See #292.
• Since rdflib currently does not support proper Seq behaviour with RDF li items, for now the index of the value items will be manually written as RDF properties, which rdflib supports when reading an RDF file. See #292.
• When writing an RDF file from an odML document that features an _origin_file_name, the value is exported as odml:hasFileName. See #297.
• xml is now the default ODMLWriter format when writing a document to RDF since the XML format of RDF is still the format with the broadest acceptance. See #297.

Addition of console scripts

• The odmltordf convenience console script has been added to convert multiple odML files to the RDF format from any odML format or version. See #298.