Migrating to gist v14.0 from v13.0

This file gives a quick overview of using the migration utilities provided by the gist team to help migrate from the previous version of gist.

See also additional documentation in the Major Version Migration.

Important Remarks

• These scripts are not cumulative; you must run the migrations for each version in sequence.

• These utilities are a starting point. You should review them before running them to make sure you approve of the actions prior to using them.

• These utilities do not handle everything, in some cases we can only warn you about changes that were made in gist and you will need to decide how you want to handle the changes.

Migration Queries

All of our migration tools are SPARQL queries.

The following directory structure holds the migration scripts:

.
└── queries/
    ├── action/
    │   ├── default/
    │   │   └── *.rq
    │   ├── local/
    │   │   └── *.rq
    │   └── ngraphs/
    │       └── *.rq
    └── report/
        ├── README
        ├── default/
        │   └── *.rq
        └── ngraphs/
            └── *.rq

The ./queries/ directory contains the following two directories:

• ./queries/action/ : Update queries for migrating from Gist v13.0 to v14.0

• ./queries/report/ : Validation queries that return SHACL validation reports for data that does not comply with Gist v14.0

Each of the above directories contains the following two directories:

• default/ : These queries only work on the default graph*

  * the handling of these queries will be dependent upon which Triplestore you are using and how it was configured. Some Triplestores will only use triples in the default graph, some will use all triples in all named graphs.

• ngraphs/ : These queries are used with migrate_endpoint.yaml and only work on named graphs

• local/ : These queries are used with migrate_local.yaml

Additionally, the ./queries/action/ directory contains a local/ directory for transforming RDF data in local files. The report queries do not require a local version.

All the action queries are provided in a safe form, with the update clauses commented out, e.g.

# test
select ?g ?s ?oldClass ?newClass

# update graph
# delete {graph ?g {?s rdf:type ?oldClass .}}
# insert {graph ?g {?s rdf:type ?newClass .}}

Once you have reviewed the queries, uncomment the DELETE/INSERT clauses and comment out the SELECT clause prior to execution.

Requirements

These migration scripts are meant to be run using the open source onto_tool program. Information about onto_tool and how to install it, is available at github.

Run on local files

Note: A known limitation of onto_tool is that it does not support named graphs when run against local files. It does support named graphs when run against a SPARQL endpoint.

1. Starting in this directory, put source data files in the ./input/ directory (the sample data can be removed). See the onto_tool documentation for which file formats are supported.

   The migrate_local.yaml file currently looks in the ./input/ directory for *.owl and *.ttl files. If other formats are supported and you plan to use them, you will need to update the yaml.

2. Execute the following command:

   onto_tool bundle migrate_local.yaml
   

3. onto_tool will output to STDOUT, you should see something like the following (this is the output from the sample data included in the ./input/ directory):

    INFO:root:Replace classes in default graph.
    INFO:root:Replace properties with inverses in default graph.
    INFO:root:Check for issues that should be reviewed.
    WARNING:root:Verification query ./queries/report/default/detect_removed_default_graph.rq produced non-empty results:
    Focus                     Path                            Value Severity   Message
    gist:GeoSegment           <urn:constraint:removed-entity>       sh:Warning Removed entity gist:GeoSegment referenced in da...
    gist:Obligation           <urn:constraint:removed-entity>       sh:Warning Removed entity gist:Obligation referenced in da...
   

   • Output files will be created in the ./output/ directory.

   • Report files will be created in the ./reports/ directory.

Run Against SPARQL Endpoint

You will need to tell onto_tool the URL of your SPARQL endpoint. You can do that by editing the migration_endpoint.yaml file, or you can put it into the command line execution like this:

Note: Use your own values in place of <...>

    onto_tool bundle -v user <USER> -v password <PWD>
                    -v endpoint <ENDPOINT-URI>
                    [ -v update_endpoint <UPDATE-URI> ]
                    -v report <REPORT-DIR> migrate_endpoint.yaml