Guide: Setting up RDF validation

Track	Test bed setup

This guide walks you through the points to consider when setting up validation for RDF and the steps to bring your validation service online. RDF content can be validated through various means; the current guide focuses on validation through SHACL shapes.

What you will achieve

At the end of this guide you will have understood what you need to consider when starting to implement validation services for your RDF vocabulary. You will also have gone through the steps to bring it online and make it available to your users.

An RDF validation service can be created using multiple approaches depending on your needs. You can have an on-premise (or local to your workstation) service through Docker or use the test bed’s resources and, with minimal configuration, bring online a public service that is automatically kept up-to-date.

For the purpose of this guide you will be presented the options to consider and start with a Docker-based instance that could be replaced (or complemented) by a setup through the test bed. Interestingly, the configuration relevant to the validator is the same regardless of the approach you choose to follow.

What you will need

• About 30 minutes.

• A text editor.

• A web browser.

• Access to the Internet.

• Docker installed on your machine (only if you want to run the validator as a Docker container).

• A basic understanding of RDF and SHACL shapes.

How to complete this guide

The steps described in this guide are for the most part hands-on, resulting in you creating a fully operational validation service. For these practical steps there are no prerequisites and the content for all files to be created are provided in each step. In addition, if you choose to try your setup as a Docker container you will also be issuing commands on a command line interface (all commands are provided and explained as you proceed).

Steps

You can complete this guide by following the steps described in this section. Not all steps are required, with certain ones being optional or complementary depending on your needs. The following diagram presents an overview of all steps highlighting the ones that apply in all cases (marked as mandatory):

When and why you should skip or consider certain steps depends on your testing needs. Each step’s description covers the options you should consider and the next step(s) to follow depending on your choice.

Step 1: Determine your testing needs

Before proceeding to setup your validator you need to clearly determine your testing needs. A first outline of the approach to follow would be provided by answering the following questions:

• Will the validator be available to your users as a tool to be used on an ad-hoc basis?

• Do you plan on measuring the conformance of your community’s members to the RDF-based specification?

• Is the validator expected to be used in a larger conformance testing context (e.g. during testing of a message exchange protocol)?

• Should the validator be publicly accessible?

• Should test data and validation reports be treated as confidential?

The first choice to make is on the type of solution that will be used to power your validation service:

• Standalone validator: A service allowing validation of individual RDF document instances based on a predefined configuration of SHACL shapes for business rule validation. The service supports fine-grained customisation and configuration of different validation types (e.g. specification versions) and supported communication channels. Importantly, use of the validator is anonymous and it is fully stateless in that none of the test data or validation reports are stored after validation completes.

• Complete test bed: The test bed is used to realise a full conformance testing campaign. It supports the definition of test scenarios as test cases, organised in test suites that are linked to specifications. Access is account-based allowing users to claim conformance to specifications and execute in a self-service manner their defined test cases. All results are recorded to allow detailed reporting, monitoring and eventually certification. Test cases can address RDF validation but are not limited to that, allowing validation of any complex exchange of information.

It is important to note that these two approaches are by no means exclusive. It is often the case that a standalone validator is defined as a first step that is subsequently used from within test cases in the test bed. The former solution offers a community tool to facilitate work towards compliance supporting ad-hoc data validation, whereas the latter allows for rigorous conformance testing to take place where proof of conformance is required. This could apply in cases where conformance is a qualification criterion before receiving funding or before being accepted as a partner in a distributed system.

Regardless of the choice of solution, the next point to consider will be the type of access. If public access is important then the obvious choice is to allow access over the Internet. An alternative would be an installation that allows access only through a restricted network, be it an organisation’s internal network or a virtual private network accessible only by your community’s members. Finally, an extreme case would be access limited to individual workstations where each community member would be expected to run the service locally (albeit of course without the expectation to test message exchanges with remote parties).

If access to your validation services over the Internet is preferred or at least acceptable, the simplest case is to opt for using the shared ISA² test bed resources, both regarding the standalone validator and the test bed itself. If such access is not acceptable or is technically not possible (e.g. access to private resources is needed), the proposed approach would be to go for a Docker-based on-premise installation of all components.

Summarising the options laid out in this section, you will first want to choose:

• Whether you will be needing a standalone validator, a complete test bed or both.

• Whether the validator and/or test bed will be accessible over the Internet or not.

Your choices here can help you better navigate the remaining steps of this guide. Specifically:

• Step 4: Setup validator as Docker container can be skipped if you are interested only in a public service or if you plan to only use the validator as part of conformance testing scenarios (i.e. within the test bed).

• Step 5: Setup validator on test bed can be skipped if a publicly accessible service is not an option for you.

• Step 7: Use the validator in GITB TDL test cases can be skipped if you only want data validation without additional conformance testing scenarios.

Step 2: Prepare validation artefacts

As an example case for RDF validation we will consider a variation of the EU purchase order case first seen in Guide: Creating a test suite. In short, for the purposes of this guide you are considered to be leading an EU cross-border initiative to define a new common specification for the exchange of purchase orders between retailers.

To specify the content of purchase orders your experts have created the following vocabulary:

@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix dc: <http://purl.org/dc/terms/> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

<http://itb.ec.europa.eu/sample/po#>
  a owl:Ontology ;
  dc:title "EU Purchase Orders" ;
  dc:description "An example ontology for EU Purchase Orders" .

#
# Classes
#

<http://itb.ec.europa.eu/sample/po#PurchaseOrder>
  a rdfs:Class ;
  rdfs:label "PurchaseOrder"@en ;
  rdfs:isDefinedBy <http://itb.ec.europa.eu/sample/po#> ;
  rdfs:subClassOf rdfs:Resource ;
  dc:identifier "po:PurchaseOrder" .

<http://itb.ec.europa.eu/sample/po#Item>
  a rdfs:Class ;
  rdfs:label "Item"@en ;
  rdfs:isDefinedBy <http://itb.ec.europa.eu/sample/po#> ;
  rdfs:subClassOf rdfs:Resource ;
  dc:identifier "po:Item" .

#
# PurchaseOrder properties
#

<http://itb.ec.europa.eu/sample/po#shipTo>
  a rdf:Property ;
  rdfs:label "shipTo"@en ;
  rdfs:isDefinedBy <http://itb.ec.europa.eu/sample/po#> ;
  rdfs:domain <http://itb.ec.europa.eu/sample/po#PurchaseOrder> ;
  rdfs:range <http://www.w3.org/ns/locn#Address> ;
  rdfs:subPropertyOf <http://www.w3.org/ns/locn#address> ;
  dc:identifier "po:shipTo" .

<http://itb.ec.europa.eu/sample/po#billTo>
  a rdf:Property ;
  rdfs:label "billTo"@en ;
  rdfs:isDefinedBy <http://itb.ec.europa.eu/sample/po#> ;
  rdfs:domain <http://itb.ec.europa.eu/sample/po#PurchaseOrder> ;
  rdfs:range <http://www.w3.org/ns/locn#Address> ;
  rdfs:subPropertyOf <http://www.w3.org/ns/locn#address> ;
  dc:identifier "po:billTo" .

<http://itb.ec.europa.eu/sample/po#hasItem>
  a rdf:Property ;
  rdfs:label "hasItems"@en ;
  rdfs:isDefinedBy <http://itb.ec.europa.eu/sample/po#> ;
  rdfs:domain <http://itb.ec.europa.eu/sample/po#PurchaseOrder> ;
  rdfs:range <http://itb.ec.europa.eu/sample/po#Item> ;
  dc:identifier "po:hasItem" .

#
# Item properties
#

<http://itb.ec.europa.eu/sample/po#productName>
  a rdf:Property ;
  rdfs:label "productName"@en ;
  rdfs:isDefinedBy <http://itb.ec.europa.eu/sample/po#> ;
  rdfs:domain <http://itb.ec.europa.eu/sample/po#Item> ;
  rdfs:range rdfs:Literal ;
  dc:identifier "po:productName" .

<http://itb.ec.europa.eu/sample/po#quantity>
  a rdf:Property ;
  rdfs:label "quantity"@en ;
  rdfs:isDefinedBy <http://itb.ec.europa.eu/sample/po#> ;
  rdfs:domain <http://itb.ec.europa.eu/sample/po#Item> ;
  rdfs:range xsd:positiveInteger ;
  dc:identifier "po:quantity" .

<http://itb.ec.europa.eu/sample/po#price>
  a rdf:Property ;
  rdfs:label "price"@en ;
  rdfs:isDefinedBy <http://itb.ec.europa.eu/sample/po#> ;
  rdfs:domain <http://itb.ec.europa.eu/sample/po#Item> ;
  rdfs:range xsd:decimal ;
  dc:identifier "po:price" .

This vocabulary defines the basic properties for an order’s items (product name, quantity and price in euros) and also a shipping and billing address for which the Core Location Vocabulary has been reused. The syntax used to express the vocabulary in this example is Turtle. You can download this here in the displayed Turtle format or in RDF/XML.

Based on this, a sample purchase order could be expressed in Turtle as follows:

@prefix ns0: <http://www.w3.org/ns/locn#> .
@prefix ns1: <http://itb.ec.europa.eu/sample/po#> .

<http://my.sample.po/po#purchaseOrder>
  a <http://itb.ec.europa.eu/sample/po#PurchaseOrder> ;
  ns1:shipTo <http://my.sample.po/po#home> ;
  ns1:billTo <http://my.sample.po/po#home> ;
  ns1:hasItem <http://my.sample.po/po#item1>;
  ns1:hasItem  <http://my.sample.po/po#item2> .

<http://my.sample.po/po#home>
  a <http://www.w3.org/ns/locn#Address> ;
  ns0:fullAddress "Rue du Test 123, 1000 - Brussels, Belgium" ;
  ns0:thoroughfare "Rue du Test" ;
  ns0:locatorDesignator "123" ;
  ns0:postCode "1000" ;
  ns0:postName "Brussels" ;
  ns0:adminUnitL1 "BE" .

<http://my.sample.po/po#item1>
  a ns1:Item ;
  ns1:productName "Mouse" ;
  ns1:quantity 20 ;
  ns1:priceEUR 15.99 .

<http://my.sample.po/po#item2>
  a ns1:Item ;
  ns1:productName "Keyboard" ;
  ns1:quantity 15 ;
  ns1:priceEUR 25.50 .

Given the inherent flexibility of RDF, your experts have chosen to add certain business rule restrictions as SHACL shapes that would apply to all purchase orders. The implemented rules ensure that:

• Only a single shipping address is defined.

• An optional billing address can be defined (the shipping address is considered as the default).

• An order includes at least one item.

• All items define their product name, quantity and price.

@prefix po:  <http://itb.ec.europa.eu/sample/po#> .
@prefix locn: <http://www.w3.org/ns/locn#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix sh:  <http://www.w3.org/ns/shacl#> .

#
# Rules for purchase orders
#

po:PurchaseOrderShape
	a sh:NodeShape ;
	sh:targetClass po:PurchaseOrder ;
	sh:property po:shipToShape ;
	sh:property po:billToShape ;
	sh:property po:hasItemsShape .

po:shipToShape
	a sh:PropertyShape ;
	sh:path po:shipTo ;
	sh:minCount 1 ;
	sh:maxCount 1 ;
	sh:class locn:Address ;
	sh:severity sh:Violation .

po:billToShape
	a sh:PropertyShape ;
	sh:path po:billTo ;
	sh:class locn:Address ;
	sh:maxCount 1 ;
	sh:severity sh:Violation .

 po:hasItemsShape
 	a sh:PropertyShape ;
	sh:path po:hasItem ;
	sh:minCount 1 ;
	sh:class po:Item ;
	sh:severity sh:Violation .

#
# Rules for purchase order items
#

po:ItemShape
	a sh:NodeShape ;
	sh:targetClass po:Item ;
	sh:property po:productNameShape ;
	sh:property po:quantityShape ;
	sh:property po:priceShape .

po:productNameShape
	a sh:PropertyShape ;
	sh:path po:productName ;
	sh:minCount 1 ;
	sh:maxCount 1 ;
	sh:nodeKind sh:Literal ;
	sh:severity sh:Violation .

po:quantityShape
	a sh:PropertyShape ;
	sh:path po:quantity ;
	sh:minCount 1 ;
	sh:maxCount 1 ;
	sh:datatype xsd:integer ;
	sh:minExclusive 0 ;
	sh:severity sh:Violation .

po:priceShape
	a sh:PropertyShape ;
	sh:path po:priceEUR ;
	sh:minCount 1 ;
	sh:maxCount 1 ;
	sh:datatype xsd:decimal ;
	sh:severity sh:Violation .

All purchase orders will need to validate against the above SHACL shapes. However, your business requirements also define the concept of a large purchase order which is one that includes more than 10 of each ordered item. To implement this restriction an additional SHACL shape is defined to be considered in addition to the common ones for orders that are supposed to be “large”. Such a SHACL shape file would be as follows:

@prefix po:  <http://itb.ec.europa.eu/sample/po#> .
@prefix sh:  <http://www.w3.org/ns/shacl#> .

#
# Rule for large purchase orders
#

po:LargeItemShape
	a sh:NodeShape ;
	sh:targetClass po:Item ;
	sh:property po:minimumItemsForLargeOrderShape .

po:minimumItemsForLargeOrderShape
	a sh:PropertyShape ;
	sh:path po:quantity ;
	sh:minExclusive 10 ;
	sh:severity sh:Violation .

Given these requirements and validation artefacts we want to support two types of validation (or profiles):

• basic: For all purchase orders acting as a common base. This is realised by validating against the common SHACL shapes.

• large: For large purchase orders. This includes validation against the common SHACL shapes and the additional one defined for large orders.

As the first configuration step for the validator we will prepare a folder with the required resources. For this purpose create a root folder named validator with the following subfolders and files:

validator
 |
 +-- resources
      |
      +-- order
           |
           +-- shapes
                |
                +-- PurchaseOrder-common-shapes.ttl
                +-- PurchaseOrder-large-shapes.ttl

You will likely note that we are creating several folders of no obvious use. Nonetheless please follow this structure as it will facilitate subsequent steps where we add resources depending on our needs. In terms of meaning of these folders consider the following:

• validator is the root folder for all files.

• resources is the root folder for all files that will be considered by the validator.

• order is the root folder for all files pertinent to purchase order validation. We separate this as the validator could be used to also validate completely different content.

• shapes is the folder containing all SHACL shape files.

Regarding the PurchaseOrder-common-shapes.ttl and PurchaseOrder-large-shapes.ttl files you can create them from the above content or download them (here: PurchaseOrder-common-shapes.ttl and PurchaseOrder-large-shapes.ttl). Finally, note that you are free to use any names for the files and folders; the ones used here will however be the ones considered in this guide’s subsequent steps.

Note

Validate your shapes: When authoring your SHACL shapes you should validate them against best practices and the SHACL specification. This adds a quality check over the shapes before you start using them for validation.

The Test Bed offers a SHACL shape validator for this purpose at https://www.itb.ec.europa.eu/shacl/shacl/upload.

Step 3: Prepare validator configuration

After having defined your testing needs and the validation artefacts for your specific case, the next step will be to configure the validator. The validator is defined by a core engine maintained by the test bed team and a layer of configuration, provided by you, that defines its use for a specific scenario. In terms of features the validator supports the following:

• Validation channels including a REST web service API, a SOAP web service API, a web user interface and a command-line tool.

• Configuration of SHACL shapes to drive the validation that can be local or remote.

• Definition of different validation types as logically-related sets of validation artefacts.

• Support per validation type allowing user-provided SHACL shape extensions.

• Support per validation type of loading the imports defined in the input that can also be set by users at validation time.

• Support for querying SPARQL endpoints to retrieve the content to validate.

• Definition of separate validator configurations that are logically split but run as part of a single validator instance. Such configurations are termed “validation domains”.

• Customisation of all texts presented to users.

Configuration is provided by means of key-value pairs in a property file. This file can be named as you want but needs to end with the .properties extension. In our case we will name this config.properties and place it within the order folder. Recall that the purpose of this folder is to store all resources relevant to purchase order validation. These are the validation artefacts themselves (PurchaseOrder-common-shapes.ttl and PurchaseOrder-large-shapes.ttl) and the configuration file (config.properties).

Define the content of the config.properties file as follows:

# The different types of validation to support. These values are reflected in other properties.
validator.type = basic, large
# Labels to describe the defined types.
validator.typeLabel.basic = Basic purchase order
validator.typeLabel.large = Large purchase order
# Validation artefacts (SHACL shapes) to consider for the "basic" type.
validator.shaclFile.basic = shapes/PurchaseOrder-common-shapes.ttl
# Validation artefacts (SHACL shapes) to consider for the "large" type.
validator.shaclFile.large = shapes/PurchaseOrder-common-shapes.ttl, shapes/PurchaseOrder-large-shapes.ttl
# The title to display for the validator's user interface.
validator.uploadTitle = Purchase Order Validator

All validator properties share a validator. prefix. The validator.type property is key as it defines one or more types of validation that will be supported (multiple are provided as a comma-separated list of values). The values provided here are important not only because they define the available validation types but also because they drive most other configuration properties. Regarding the validation artefacts themselves, these are provided by means of the validator.shaclFile property, specifically:

• validator.shaclFile.TYPE defines one or more (comma-separated) file paths (relative to the configuration file) to lookup SHACL shape files.

• validator.shaclFile.TYPE.remote.N can be defined multiple times for SHACL files that are to be loaded remotely (e.g. from a GitHub repository).

Regarding local files (i.e. using the validator.shaclFile.TYPE property) each provided path can be for a file or a folder. If a folder is referenced it will load all contained top-level files (i.e. ignoring subfolders). Regarding remotely loaded files (i.e. the validator.shaclFile.TYPE.remote.N) you may define as many of these as the number of files to load (the N placeholder is a zero-based integer). In addition, for each remotely loaded file we need to also specify its syntax, provided as its mime type. This can be omitted if the syntax can be derived by the file’s extension (e.g. Turtle for .ttl), but it is a good practice to always define this for consistency.

The example that follows illustrates the loading of two remote SHACL shape files for a validation type named v2.2.1 from a remote location:

validator.type = v2.2.1
...
validator.shaclFile.v2.2.1.remote.0.url = https://my.server.com/my_rules_1.ttl
validator.shaclFile.v2.2.1.remote.0.type = text/turtle
validator.shaclFile.v2.2.1.remote.1.url = https://my.server.com/my_rules_2.rdf
validator.shaclFile.v2.2.1.remote.1.type = application/rdf+xml

You may also combine local and remote SHACL shape files by defining a validator.shaclFile.TYPE property and one or more validator.shaclFile.TYPE.remote.N properties. In all cases, the shapes from all sources will be aggregated into a single model for the validation.

Note

Remote SHACL shape caching: Caching is used to avoid constant lookups of remote SHACL shape files. Once loaded, remote SHACL shape files will be automatically refreshed every hour.

In addition, apart from defining the SHACL shapes to apply as local and/or remote files, you may also define for a given validation type whether or not you allow user-provided SHACL shape extensions. This is achieved through the validator.externalShapes property:

...
validator.externalShapes.TYPE = optional

Specifying that for a given validation type you allow users to provide SHACL shape extensions will result in any such extensions being combined with your pre-defined SHACL shapes. This could be useful in scenarios where you want to define a common validation base but allow also ad-hoc extensions for e.g. restrictions defined at user-level (e.g. National validation rules to consider in addition to a common set of EU rules).

Note

Generic validator: It is possible to not predefine any SHACL shapes resulting in a validator that is truly generic, expecting all shapes to be provided by users. Such a generic instance actually exists at https://www.itb.ec.europa.eu/shacl/any/upload.

The purpose of the remaining properties is to customise the text descriptions presented to users:

• validator.typeLabel defines a label to present to users on the validator’s user interface for the type in question.

• validator.uploadTitle defines the title label to present to users on the validator’s user interface.

Once you have created the config.properties file, the validator folder should be as follows:

validator
 |
 +-- resources
      |
      +-- order
           |
           +-- config.properties
           +-- shapes
                |
                +-- PurchaseOrder-common-shapes.ttl
                +-- PurchaseOrder-large-shapes.ttl

This limited configuration file assumes numerous default configuration properties. An important example is that by default, the validator will expose a web user interface, a SOAP web service API and a REST API. This configuration is driven through the validator.channels property that by default is set to form, rest_api, soap_api (for a user form, REST API and SOAP web service respectively). All configuration properties provided in config.properties relate to the specific domain in question, notably purchase orders, reflected in the validator’s resources as the order folder. Although rarely needed, you may define additional validation domains each with its own set of validation artefacts and configuration file (see Configuring additional validation domains for details on this). Finally, if you are planning to host your own validator instance you can also define configuration at the level of the complete validator (see Additional configuration options regarding application-level configuration options).

For the complete reference of all available configuration properties and their default values refer to section Validator configuration properties.

Configuring SPARQL query support

The way in which the validator receives the content to validate depends on the specific channel you are using (web UI, REST API, SOAP API or command-line tool). Regardless of how you use the validator, the content to validate can provided:

• As an RDF file.

• As a URI that will provide or generate the RDF content.

• As direct input of RDF data.

A further option for input, that is not enabled by default, is to have the validator query a triple store through SPARQL queries. This approach allows you to store RDF data and dynamically select the parts to validate by passing to the validator the queries to execute. The queries used must be CONSTRUCT queries, which can return a complete data graph of data subset based on the query’s definition.

When configuring SPARQL query support for your validator you have several configuration options. You would typically preconfigure the SPARQL endpoint for your queries along with any credentials needed for authentication. Alternatively you could leave this information unspecified, allowing your users to define themselves the endpoint and the credentials to use. You could also only configure the endpoint, and expect users to provide personal authentication credentials if this matches your needs. Query-related configuration options are defined as part of a domain’s definition (see Domain-level configuration). The following sample configurations highlight how you would adapt your domain configuration file (config.properties) to use the relevant properties for different scenarios.

Example 1: Configure a SPARQL endpoint and require the user to provide personal credentials.

...
validator.queryEndpoint = http://host/sparql
# We don't specify credentials but mark them as required. Not doing so would default to them
# being considered as optional.
validator.queryAuthenticationInput = required

Example 2: Configure a SPARQL endpoint and the credentials to use (password provided via environment variable).

...
validator.queryEndpoint = http://host/sparql
# Configuring credentials will not allow users to provide their own.
validator.queryUsername = sparqlUser
validator.queryPassword = ${env:SPARQL_PASSWORD}

Example 3: Allow users to define any SPARQL endpoint and credentials.

...
# If this flag is not set to true the validator will not support queries.
# We didn't need to set this explicitly in the other examples as we already
# defined other query-related properties.
validator.supportsQueries = true

Note that the configuration discussed here only refers to you enabling SPARQL queries as an input to the validator. To see how these can be passed to the validator check the validator usage documentation.

Step 4: Setup validator as Docker container

Note

When to setup a Docker container: The purpose of setting up your validator as a Docker container is to host it yourself or run it locally on workstations. If you prefer or don’t mind the validator being accessible over the Internet it is simpler to delegate hosting to the test bed team by reusing the test bed’s infrastructure. If this is the case skip this section and go directly to Step 5: Setup validator on test bed. Note however that even if you opt for a validator managed by the test bed, it may still be interesting to create a Docker image for development purposes (e.g. to test new validation artefact versions) or to make available to your users as a complementary service (i.e. use online or download and run locally).

Once the validator’s configuration is ready (configuration file and validation artefacts) you can proceed to create a Docker image.

The configuration for your image is driven by means of a Dockerfile. Create this file in the validator folder with the following contents:

FROM isaitb/shacl-validator:latest
COPY resources /validator/resources/
ENV validator.resourceRoot /validator/resources/

This Dockerfile represents the most simple Docker configuration you can provide for the validator. Let’s analyse each line:

FROM isaitb/shacl-validator:latest	This tells Docker that your image will be built over the latest version of the test bed’s isaitb/shacl-validator image. This represents the validator’s core that expects configuration to drive the validation. It is available on the public Docker Hub and as such can be directly used through any Docker installation with Internet access.
COPY resources /validator/resources/	This copies your resources folder to the image under path /validator/resources/.
ENV validator.resourceRoot /validator/resources/	This instructs the validator that it should consider as the root of all its configuration resources the /validator/resources/ folder (which was just copied into it).

The contents of the validator folder should now be as follows:

validator
 |
 +-- Dockerfile
 +-- resources
      |
      +-- order
           |
           +-- config.properties
           +-- shapes
                |
                +-- PurchaseOrder-common-shapes.ttl
                +-- PurchaseOrder-large-shapes.ttl

That’s it. To build the Docker image open a command prompt to the validator folder and issue:

docker build -t po-validator .

This command will create a new local Docker image named po-validator based on the Dockerfile it finds in the current directory. It will proceed to download missing images (e.g. the isaitb/shacl-validator:latest image) and eventually print the following output:

Sending build context to Docker daemon  32.77kB
Step 1/3 : FROM isaitb/shacl-validator:latest
---> 39ccf8d64a50
Step 2/3 : COPY resources /validator/resources/
---> 66b718872b8e
Step 3/3 : ENV validator.resourceRoot /validator/resources/
---> Running in d80d38531e11
Removing intermediate container d80d38531e11
---> 175eebf4f59c
Successfully built 175eebf4f59c
Successfully tagged po-validator:latest

The new po-validator:latest image can now be pushed to a local Docker registry or to the Docker Hub. In our case we will proceed directly to run this as follows:

docker run -d --name po-validator -p 8080:8080 po-validator:latest

This command will create a new container named po-validator based on the po-validator:latest image you just built. It is set to run in the background (-d) and expose its internal listen port through the Docker machine (-p 8080:8080). Note that by default the listen port of the container (which you can map to any available host port) is 8080.

Your validator is now online and ready to validate RDF content. If you want to try it out immediately skip to Step 6: Use the validator. Otherwise, read on to see additional configuration options for the image.

Running without a custom Docker image

The discussed approach involved building a custom Docker image for your validator. Doing so allows you to run the validator yourself but also potentially push it to a public registry such as the Docker Hub. This would then allow anyone else to pull it and run a self-contained copy of your validator.

If such a use case is not important for you, or if you want to only use Docker for your local artefact development, you could also skip creating a custom image and use the base isaitb/shacl-validator image directly. To do so you would need to:

• Define a named or unnamed Docker volume pointing to your configuration files.

• Run your container by configuring it with the volume.

Considering the same file structure of the /validator folder you can launch your validator using an unnamed volume as follows:

docker run -d --name po-validator -p 8080:8080 \
           -v /validator/resources:/validator/resources/ \
           -e validator.resourceRoot=/validator/resources/  \
           isaitb/shacl-validator

As you see here we create the validator directly from the base image and pass it as a volume our resource root folder. When doing so you need to also make sure that the validator.resourceRoot environment variable is set to the path within the container.

Using this approach to run your validator has the drawback of being unable to share it as-is with other users. The benefit however is one of simplicity given that there is no need to build intermediate images. As such, updating the validator for configuration changes means that you only need to restart your container.

Configuring additional validation domains

Up to this point you have configured validation for purchase orders which defines one or more validation types (basic and large). This configuration can be extended by providing additional types to reflect:

• Additional profiles with different business rules (e.g. minimal).

• Specification versions (e.g. basic_v1.0, large_v1.0, basic_v1.1_beta).

• Other types of content that are linked to purchase orders (e.g. purchase_order_basic_v1.0 and order_receipt_v1.0).

All such extensions would involve defining potentially additional validation artefacts and updating the config.properties file accordingly.

Apart from extending the validation possibilities linked to purchase orders you may want to configure a completely separate validator to address an unrelated specification that would most likely not be aimed to the same user community. To do so you have two options:

• Repeat the previous steps to define a separate configuration and a separate Docker image. In this case you would be running two separate containers that are fully independent.

• Reuse your existing validator instance to define a new validation domain. The result will be two validation services that are logically separate but are running as part of a single validator instance.

The rationale behind the second option is simply one of required resources. If you are part of an organisation that needs to support validation for dozens of different RDF-based specifications that are unrelated, it would probably be preferable to have a single application to host rather than one per specification.

Note

Sharing artefact files accross domains: Setting the application property validator.restrictResourcesToDomain to false allows to add paths of validation artefacts that are outside of the domain root folder. This enables sharing artefacts between different domains.

In your current single domain setup, the purchase order configuration is reflected through folder order. The name of this folder is also by default assumed to match the name of the domain. A new domain could be named invoice that is linked to RDF-based invoices. This is represented by an invoice folder next to order that contains similarly its validation artefacts and domain-level configuration property file. Considering this new domain, the contents of the validator folder would be as follows:

validator
 |
 +-- Dockerfile
 +-- resources
      |
      +-- invoice
        (Further contents skipped)
      +-- order
        (Further contents skipped)

If you were now to rebuild the validator’s Docker image this would setup two logically-separate validation domains (invoice and order).

Note

Validation domains vs types: In almost all scenarios you should be able to address your validation needs by having a single validation domain with multiple validation types. Validation types under the same domain will all be presented as options for users. Splitting in domains would make sense if you don’t want the users of one domain to see the supported validation types of other domains.

Important: Support for such configuration is only possible if you are defining your own validator as a Docker image. If you plan to use the test bed’s shared validator instance (see Step 5: Setup validator on test bed), your configuration needs to be limited to a single domain. Note of course that if you need additional domains you can in this case simply repeat the configuration process multiple times.

Additional configuration options

We have seen up to now that configuring how validation takes place is achieved through domain-level configuration properties provided in the domain configuration file (file config.properties in our example). When setting up the validator as a Docker image you may also make use of application-level configuration properties to adapt the overall validator’s operation. Such configuration properties are provided as environment variables through ENV directives in the Dockerfile.

We already saw this when defining the validator.resourceRoot property that is the only mandatory property for which no default exists. Other such properties that you may choose to override are:

• validator.domain: A comma-separated list of names that are to be loaded as the validator’s domains. By default the validator scans the provided validator.resourceRoot folder and selects as domains all subfolders that contain a configuration property file (folder order in our case). You may want to configure the list of folder names to consider if you want to ensure that other folders get ignored.

• validator.domainName.DOMAIN: A mapping for a domain (replacing the DOMAIN placeholder) that defines the name that should be presented to users. This would be useful if the folder name itself (order in our example) is not appropriate (e.g. if the folder was named files).

The following example Dockerfile illustrates use of these properties. The values set correspond to the applied defaults so the resulting Docker images from this Dockerfile and the original one (see Step 4: Setup validator as Docker container) are in fact identical:

FROM isaitb/shacl-validator:latest
COPY resources /validator/resources/
ENV validator.resourceRoot /validator/resources/
ENV validator.domain order
ENV validator.domainName.order order

See Application-level configuration for the full list of supported application-level properties.

Finally, it may be the case that you need to adapt further configuration properties that relate to how the validator’s application is ran. The validator is built as a Spring Boot application which means that you can override all configuration properties by means of environment variables. This is rarely needed as you can achieve most important configuration through the way you run the Docker container (e.g. defining port mappings). Nonetheless the following adapted Dockerfile shows how you could ensure the validator’s application starts up on another port (9090) and uses a specific context path (/ctx).

FROM isaitb/shacl-validator:latest
COPY resources /validator/resources/
ENV validator.resourceRoot /validator/resources/
ENV server.servlet.context-path /ctx
ENV server.port 9090

Note

Custom port: Even if you define the server.port property to a different value other than the default 8080 this remains internal to the Docker container. The port through which you access the validator will be the one you map on your host through the -p flag of the docker run command.

The full list of such application configuration properties, as well as their default values, are listed in the Spring Boot configuration property documentation.

Environment-specific domain configuration

In the previous section you saw how could can configure the validator’s application by means of environment properties. Use of environment properties is also possible for specific validator domains, allowing you to externalise and override their configuration aspects. Typical cases where you may want to do this are:

• To adapt the configuration for different instances of the same validator.

• To hide sensitive properties such as internal URLs or passwords.

External configuration properties can be provided through environment variables or system properties (the latter being interesting when using a command line validator). These can contribute to the configuration:

• Complete properties, by defining a variable or property with the same name as a configuration property.

• Values, by referring to arbitrary variables or properties within a domain property file. This is done by defining a placeholder ${} and using within it the prefixes env: for environment variables or sys: for system properties (e.g. ${env:MY_VARIABLE}).

As a simple example of this, consider the definition of the title of the validator’s web UI. We want to adapt this title depending on the purpose of the specific validator instance. To begin with we can define the title via the validator.uploadTitle property in the config.properties file as follows:

validator.uploadTitle = Purchase Order Validator

The problem with this approach is that the upload title remains fixed across all instances of the validator. Alternatively we can define the value for the title as an environment variable named VALIDATOR_TITLE. To use this we can adapt config.properties to reference it as follows:

validator.uploadTitle = ${env:VALIDATOR_TITLE}

We can then adapt this for each Docker container we create by defining a specific value for the title:

docker run -e VALIDATOR_TITLE="Purchase Order Validator (Demo)" ...

A further alternative would be to externalise the complete validator.uploadTitle property by removing it from config.properties and specifying it as an environment variable:

docker run -e validator.uploadTitle="Purchase Order Validator (Demo)" ...

Note that such external configuration can also be used as a partial value. If we define an environment variable named VALIDATOR_ENV we could also use it within config.properties as follows:

validator.uploadTitle = Purchase Order Validator (${env:VALIDATOR_ENV})

Finally, you can use environment or system variables to override properties that are already defined in config.properties. This is useful if you use the file’s properties as default values and selectively override certain of them as needed. A property’s value is looked up in sequence as follows:

1. Look in environment variables.

2. If not found look in system properties.

3. If not found look in the domain configuration file.

4. If not found consider the property’s overall default value (see Domain-level configuration).

Step 5: Setup validator on test bed

Note

When to setup on test bed resources: Setting up your validator on the test bed’s resources removes hosting concerns and allows you to benefit from automatic service reloads for configuration changes. In doing so however you need to keep in mind that the validator will be exposed over the Internet. If this approach is not suitable for you (e.g. you want to expose the validator within a restricted network) you should consider setting up the validator as a Docker container (see Step 4: Setup validator as Docker container) that you can then host as you see fit.

To configure a validator using the test bed’s resources all you need to do is get in touch with the test bed team and provide the validator’s configuration. Specifically:

1. Send an email to DIGIT-ITB@ec.europa.eu describing your case: This step is needed for two reasons. Firstly you may want to have a further discussion and potentially a demo to better understand the available options. Secondly the test bed’s team would need to ensure that you qualify to use its resources (to e.g. avoid that you are a private company planning to offer commercial validation services).

2. Share the configuration for the validator: Once contact has been established you need to provide the initial configuration for the validator.

Regarding the second step, the validator’s configuration to be shared is the contents of the validator folder as described in Step 3: Prepare validator configuration. The eventual goal here will be to have the configuration available through an accessible Git repository. This can be done in a number of ways listed below in decreasing order of preference:

• Create a new Git repository: You can push all resources (the validator folder) to a new Git repository (e.g. on GitHub or the European Commission’s CITNet Bitbucket server). You can of course add any other resources to this repository as you see fit (e.g. a README file). Once done provide the repository’s URL to the test bed team.

• Provide the resources to the test bed team: You can send the configuration files themselves to the test bed’s team (e.g. make an archive of the validator folder). Ideally you should define the configuration file but if in doubt you can simply describe the resources and the test bed team will prepare the initial configuration for you. When following this approach a new Git repository will be created for you on the European Commission’s CITNet Bitbucket server for which you will be assigned write access (assuming you have a CITNet user account).

• Update an existing Git repository: If you already have a Git repository to maintain the validation artefacts you can reuse this by adding to it the required configuration file (config.properties in our case). When ready you will need to provide the test bed team with the URL to the repository and the location of the configuration file.

Following the initial configuration, the resulting Git repository will be monitored to detect any changes to the validation artefacts or the configuration file. If such a change is detected, the validation service will be automatically updated within a few minutes.

Note

Using a dedicated Git repository for the validator: Whether you define a new Git repository yourself or the test bed team creates one for you, the result is a repository that is dedicated to the validator. This approach is preferable to reusing an existing Git repository to avoid unwanted changes to the validator. whether or not this is done through GitHub, CITNet’s Bitbucket or another service depends on what best suits your needs.

As part of the initial setup for the validator the test bed team will also configure how it is accessed. The name used will match the name of the folder that contains your configuration file (order in the considered example), but this can differ according to your preferences. If this is the case make sure to inform the test bed team of your preferred naming.

Considering our example, for a name of order, the resulting root URL through which the validator will be accessed is https://www.itb.ec.europa.eu/shacl/order. The specific paths will depend on the supported validation channels as described in Step 6: Use the validator.

Offline validator for command-line use

Each validator set up on test bed resources is also made available as a command-line tool that can be downloaded and used in a standalone manner. This command-line tool is a Java-based application packaged as an “all-in-one” executable JAR file. For each supported DOMAIN the command-line validator is published at:

https://www.itb.ec.europa.eu/shacl-offline/DOMAIN/validator.zip

It is important to note that no additional configuration is needed to create such command-line tools. In addition, any changes in the validator’s configuration will result in its command-line tool being updated automatically.

Check section Validation via command-line tool for instructions on how to use it.

Step 6: Use the validator

Well done! At this step your validator has been successfully configured and is ready to use. Depending on which approach was followed, this may have been done either:

• As a Docker container (described in Step 4: Setup validator as Docker container).

• Through the test bed’s resources (described in Step 5: Setup validator on test bed).

The validation channels that are supported depend on the configuration you have supplied. This is done through the validator.channels property of your configuration file (config.properties) that defaults to form, rest_api, soap_api. The supported channels are as follows:

• form: A web user interface allowing a user to provide the RDF content to validate.

• rest_api: A REST API allowing machine-to-machine integration using HTTP calls.

• soap_api: A SOAP API allowing contract-based machine-to-machine integration using SOAP web service calls.

The following sub-sections describe how each channel can be used considering the example EU purchase order specification.

Validation via user interface

The validator’s user interface is available at the /shacl/DOMAIN/upload path. The exact path depends on how this is deployed:

• Via Docker: http://DOCKER_MACHINE:8080/shacl/order/upload

• On the test bed: https://www.itb.ec.europa.eu/shacl/order/upload

The first page that you see is a simple form to provide the RDF content to validate.

This form expects the following input:

• Content to validate: The RDF content that will be submitted for validation. The preceding dropdown selection determines how this will be provided, specifically as a file input (pre-selected), as a URI to be loaded remotely or as content to be provided using an editor. In case you have enabled SPARQL queries you will also see here the option to provide input via SPARQL query.

• Validate as: The type of validation to apply.

• Content syntax: The syntax to consider for the provided content. This may be omitted in case of file upload or URI input if the file’s extension can determine the syntax. Note that this is hidden in case you provide input via SPARQL query.

Note that all displayed labels can be adapted through the config.properties configuration file (see Properties related to UI labels). The available validation types match the ones defined in the validator.type property, displayed using the validator.typeLabel.TYPE labels. Moreover, the text title could be replaced by a configurable HTML banner, and further complemented with a HTML footer (see Domain-level configuration).

It is worth noting also that if your configuration defined only a single validation type, the user interface would be simplified by presenting only the content input controls (i.e. considering the single validation type as pre-selected).

In addition, if your configuration for the selected validation type allows for user-provided SHACL shapes, the form also includes the controls to manage the shape files you provide. Files can be defined via file upload or remote URI, specifying at the same time the syntax to consider if this cannot be determined by its extension (similar to the content to validate).

Finally, if you also specified in your configuration that the user should choose whether imports are to be loaded when forming the data graph to validate, this will also be reflected in the user interface.

Once you have provided your input click the Validate button to trigger the validation. Upon completion you will be presented with the validation results:

This screen includes an overview of the result listing:

• The validation timestamp (in UTC), the name of the validated file and the applied validation type (if more than one are configured).

• The overall result (SUCCESS or FAILURE).

• The number of errors, warnings and information messages.

This section is followed by the Details panel, where the details of each report item are listed:

• It’s type (whether this is an error, warning or information message).

• It’s description.

• It’s location in the provided input in terms of the focus node and its specific path.

• The test performed in terms of the name of the SHACL rule applied and the value that was tested.

With respect to reporting, apart from the on-screen display, you are also presented with flexible download controls to:

• Download the validation report in PDF format (selected by default - sample here). You may also select to download the report as a SHACL Validation Report in any of the provided syntaxes (sample in Turtle here).

• Download the validated content in any of the provided syntaxes.

• Download the SHACL shapes used for the validation in any of the provided syntaxes. Note that the shapes in this case will be the aggregated shapes from predefined files and user-provided ones (if supported and provided).

Finally, to trigger a new validation you may either use the form from the top of the result screen or click on the form’s title that will take you back to the previous page.

Validation via minimal user interface

If you are exposing a web user interface (see Validation via user interface) for your validator you also have the option of enabling an alternative minimal interface that could be used as an embedded component in another web page (e.g. via an iframe). This is enabled through the validator.supportMinimalUserInterface property in your domain configuration (file config.properties).

...
validator.supportMinimalUserInterface = true

The result of this is to expose a /shacl/DOMAIN/uploadm path. The path depends on how this is deployed:

• Via Docker: http://DOCKER_MACHINE:8080/shacl/order/uploadm

• On the test bed: https://www.itb.ec.europa.eu/shacl/order/uploadm

The minimal interface offers largely the same functionality as the complete one but with a more condensed layout and minimal styling. The initial input page you see for the validator is as follows:

The most significant difference is the result page which provides only an overview and the relevant download controls:

The meaning of the provided controls and displayed information for both input and result pages is identical to the complete user interface (see Validation via user interface).

Validation via REST web service API

The validator’s REST API is available under the /shacl/DOMAIN/api path. The exact path depends on how this is deployed:

• Via Docker: http://DOCKER_MACHINE:8080/shacl/order/api

• On the test bed: https://www.itb.ec.europa.eu/shacl/order/api

The operations that the REST API supports are the following:

Operation	Description	HTTP method	Request payload type
info	Retrieve the available validation types for a given domain.	GET	None
validate	Validate one RDF instance.	POST	application/json
validateMultiple	Validate multiple RDF instances.	POST	application/json

The supported operations as well their input and output are thoroughly documented:

• Using OpenAPI and Swagger, with the documentation available at the /shacl/swagger-ui.html path.

• Using Hydra where the API entry point is available at the /shacl/DOMAIN/api path.

The Swagger UI is notable as this provides rich, interactive documentation that can also be used to call the underlying operations. To access this navigate to:

• If running via Docker: http://DOCKER_MACHINE:8080/shacl/swagger-ui.html

• If running on the test bed: https://www.itb.ec.europa.eu/shacl

Note that before using the Swagger UI to execute any of the operations you will also need to specify the {domain} path parameter. In the example we have been following this would be order.

Coming back to the specific operations supported, the first one to address is the info operation. This can be useful if the validator is configured with multiple validation types in which case this service returns each type’s name and description.

Considering our order example making a GET request to http://DOCKER_MACHINE:8080/shacl/order/api/info (or https://www.itb.ec.europa.eu/shacl/order/api/info on the test bed), you receive a JSON response as follows:

{
    "domain": "order",
    "validationTypes": [
        {
            "type": "basic",
            "description": "Basic purchase order"
        },
        {
            "type": "large",
            "description": "Large purchase order"
        }
    ]
}

To trigger validation of a RDF instance you use the validate operation by making a POST request of type application/json to http://DOCKER_MACHINE:8080/shacl/order/api/validate (or https://www.itb.ec.europa.eu/shacl/order/api/validate on the test bed). The payload of the validate operation defines the following properties:

Property	Description	Required?	Type	Default value
contentToValidate	The content to validate.	Yes (optional if queries are supported)	A string that is interpreted based on the embeddingMethod input.
embeddingMethod	The way in which to interpret the contentToValidate value. It is advised to always provide this for best performance. If not provided the validator will attempt to infer the method from the content.	No	One of STRING (for content provided directly), BASE64 (for content embedded as a BASE64 encoded string) or URL (for a reference to remote content).
contentSyntax	The syntax of the provided content.	Yes, unless the content is provided as a file, or URI for a file with a well-known extension.	A mime type string (e.g. application/ld+json, application/rdf+xml, text/turtle, application/n-triples).
contentQuery	A SPARQL CONSTRUCT query to be used to get the content to validate.	No	String
contentQueryEndpoint	The SPARQL endpoint URL to query (if not preconfigured).	No	String
contentQueryUsername	The username to use for SPARQL endpoint authentication (if not preconfigured).	No	String
contentQueryPassword	The password to use for SPARQL endpoint authentication (if not preconfigured).	No	String
validationType	The type of validation to perform.	Yes, unless a single validation type is defined.	String	The single configured validation type (if defined).
externalRules	An array of user-provided SHACL shape extensions to be considered with any predefined ones. These are accepted only if explicitly allowed in the configuration for the validation type in question.	No	An array of RuleSet entries (see below for content).
loadImports	The choice of whether or not imports are to be loaded when forming the data graph to validate.	No	Boolean	false
reportSyntax	The syntax (provided as a mime type) for the resulting report (can be skipped for the default syntax).	No	A mime type string (e.g. application/ld+json, application/rdf+xml, text/turtle, application/n-triples).	The default configured for the validator (application/rdf+xml if not overriden).
reportQuery	A SPARQL CONSTRUCT query that will be executed on the resulting SHACL validation report as a post-processing step. If provided, the result of this query will replace the SHACL validation report in the service’s output.	No	String

In case user-provided SHACL shapes are supported, these are provided as RuleSet entries as elements of the externalRules array. The content of each RuleSet is as follows:

Property	Description	Required?	Type	Default value
ruleSet	The RDF containing the rules to apply (shapes).	Yes	A string that is interpreted based on the embeddingMethod input.
embeddingMethod	The way in which to interpret the value for ruleSet. It is advised to always provide this for best performance. If not provided the validator will attempt to infer the method from the ruleSet value.	No	One of STRING (for content provided directly), BASE64 (for content embedded as a BASE64 encoded string) or URL (for a reference to remote content).
ruleSyntax	The syntax of the provided ruleSet.	Yes, unless the content is provided as a file, or URI for a file with a well-known extension.	A mime type string (e.g. application/ld+json, application/rdf+xml, text/turtle, application/n-triples).

To illustrate how this can be used we will consider a purchase order that will fail validation when checked to be of large type due to it lacking the required quantities per item (you can download the sample here):

@prefix ns0: <http://www.w3.org/ns/locn#> .
@prefix ns1: <http://itb.ec.europa.eu/sample/po#> .

<http://my.sample.po/po#purchaseOrder>
  a <http://itb.ec.europa.eu/sample/po#PurchaseOrder> ;
  ns1:shipTo <http://my.sample.po/po#home> ;
  ns1:billTo <http://my.sample.po/po#home> ;
  ns1:hasItem <http://my.sample.po/po#item1>;
  ns1:hasItem  <http://my.sample.po/po#item2> .

<http://my.sample.po/po#home>
  a <http://www.w3.org/ns/locn#Address> ;
  ns0:fullAddress "Rue du Test 123, 1000 - Brussels, Belgium" ;
  ns0:thoroughfare "Rue du Test" ;
  ns0:locatorDesignator "123" ;
  ns0:postCode "1000" ;
  ns0:postName "Brussels" ;
  ns0:adminUnitL1 "BE" .

<http://my.sample.po/po#item1>
  a ns1:Item ;
  ns1:productName "Mouse" ;
  ns1:quantity 5 ;
  ns1:priceEUR 15.99 .

<http://my.sample.po/po#item2>
  a ns1:Item ;
  ns1:productName "Keyboard" ;
  ns1:quantity 15 ;
  ns1:priceEUR 25.50 .

As a first validation example we will provide the content to validate as a URI to be looked up:

{
    "contentToValidate": "https://www.itb.ec.europa.eu/files/samples/shacl/sample-invalid.ttl",
    "validationType": "large"
}

In the contentToValidate parameter we include the URI to the file whereas in the validationType parameter we select the desired validation type. Note that we need to define the validation type as we have more than one configured for purchase orders. If there was only one this parameter would be optional.

The response returned for this call will be the SHACL validation report in application/rdf+xml syntax:

<rdf:RDF
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:po="http://itb.ec.europa.eu/sample/po#"
    xmlns:locn="http://www.w3.org/ns/locn#"
    xmlns:sh="http://www.w3.org/ns/shacl#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#">
    <sh:ValidationReport>
        <sh:conforms rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">false</sh:conforms>
        <sh:result>
            <sh:ValidationResult>
                <sh:value rdf:datatype="http://www.w3.org/2001/XMLSchema#integer">3</sh:value>
                <sh:resultPath rdf:resource="http://itb.ec.europa.eu/sample/po#quantity"/>
                <sh:resultMessage>Value is not &gt; 10^^http://www.w3.org/2001/XMLSchema#integer</sh:resultMessage>
                <sh:focusNode rdf:resource="http://my.sample.po/po#item2"/>
                <sh:sourceShape rdf:resource="http://itb.ec.europa.eu/sample/po#minimumItemsForLargeOrderShape"/>
                <sh:sourceConstraintComponent rdf:resource="http://www.w3.org/ns/shacl#MinExclusiveConstraintComponent"/>
                <sh:resultSeverity rdf:resource="http://www.w3.org/ns/shacl#Violation"/>
            </sh:ValidationResult>
        </sh:result>
    </sh:ValidationReport>
</rdf:RDF>

In this report we can see the overall validation result (false in sh:conforms), as well as the individual report items (one in this case). Each such item includes:

• The severity of the item (sh:resultSeverity).

• The message for the item (sh:resultMessage).

• The location in the validated content that triggered the failure (sh:focusNode and sh:resultPath).

• The test that was made (the failing shape sh:sourceShape, the constraint type sh:sourceConstraintComponent and the tested value sh:value).

You may request the validation report in different syntaxes by providing as part of the input the reportSyntax property with the desired report mime type. For example, to request the report in Turtle format you would provide:

{
    "contentToValidate": "https://www.itb.ec.europa.eu/files/samples/shacl/sample-invalid.ttl",
    "validationType": "large",
    "reportSyntax": "text/turtle"
}

Resulting in the same report but this time in Turtle syntax:

@prefix sh:    <http://www.w3.org/ns/shacl#> .
@prefix xsd:   <http://www.w3.org/2001/XMLSchema#> .
@prefix locn:  <http://www.w3.org/ns/locn#> .
@prefix po:    <http://itb.ec.europa.eu/sample/po#> .

[ a            sh:ValidationReport ;
sh:conforms  false ;
sh:result    [ a                             sh:ValidationResult ;
                sh:focusNode                  <http://my.sample.po/po#item2> ;
                sh:resultMessage              "Value is not > 10^^http://www.w3.org/2001/XMLSchema#integer" ;
                sh:resultPath                 po:quantity ;
                sh:resultSeverity             sh:Violation ;
                sh:sourceConstraintComponent  sh:MinExclusiveConstraintComponent ;
                sh:sourceShape                po:minimumItemsForLargeOrderShape ;
                sh:value                      3
            ]
] .

This could also have been achieved by means of an HTTP Accept header sent with the POST request with a value of text/turtle.

Note

Default report syntax: The validation report is returned by default in application/rdf+xml syntax. You may override this as part of your domain configuration (in file config.properties) by setting the validator.defaultReportSyntax property to the desired mime type.

In the previous example we saw validation by passing a reference to a remote resource as a URI. You may also choose to embed the content to be validated within the request itself as a BASE64 encoded string. To do so you set the value of the contentToValidate property to the BASE64 string but also specify the contentSyntax property with the input’s mime type (text/turtle in our case) as this can no longer be determined from the content’s file extension:

{
    "contentToValidate": "QHByZWZpeCBuczA6IDxodHRwOi8vd3d3LnczLm9yZy9ucy9sb2NuIz4gLgpAcHJlZml4IG5zMTogPGh0dHA6Ly9pdGIuZWMuZXVyb3BhLmV1L3NhbXBsZS9wbyM+IC4KCjxodHRwOi8vbXkuc2FtcGxlLnBvL3BvI2hvbWU+CiAgYSA8aHR0cDovL3d3dy53My5vcmcvbnMvbG9jbiNBZGRyZXNzPiA7CiAgbnMwOmZ1bGxBZGRyZXNzICJSdWUgZHUgVGVzdCAxMjMsIDEwMDAgLSBCcnVzc2VscywgQmVsZ2l1bSIgOwogIG5zMDp0aG9yb3VnaGZhcmUgIlJ1ZSBkdSBUZXN0IiA7CiAgbnMwOmxvY2F0b3JEZXNpZ25hdG9yICIxMjMiIDsKICBuczA6cG9zdENvZGUgIjEwMDAiIDsKICBuczA6cG9zdE5hbWUgIkJydXNzZWxzIiA7CiAgbnMwOmFkbWluVW5pdEwxICJCRSIgLgoKPGh0dHA6Ly9teS5zYW1wbGUucG8vcG8jcHVyY2hhc2VPcmRlcj4KICBhIDxodHRwOi8vaXRiLmVjLmV1cm9wYS5ldS9zYW1wbGUvcG8jUHVyY2hhc2VPcmRlcj4gOwogIG5zMTpzaGlwVG8gPGh0dHA6Ly9teS5zYW1wbGUucG8vcG8jaG9tZT4gOwogIG5zMTpoYXNJdGVtIDxodHRwOi8vbXkuc2FtcGxlLnBvL3BvI2l0ZW0xPiA7CiAgbnMxOmhhc0l0ZW0gPGh0dHA6Ly9teS5zYW1wbGUucG8vcG8jaXRlbTI+IC4KCjxodHRwOi8vbXkuc2FtcGxlLnBvL3BvI2l0ZW0xPgogIGEgbnMxOkl0ZW0gOwogIG5zMTpwcm9kdWN0TmFtZSAiTW91c2UiIDsKICBuczE6cXVhbnRpdHkgMjAgOwogIG5zMTpwcmljZUVVUiAxNS45OSAuCgo8aHR0cDovL215LnNhbXBsZS5wby9wbyNpdGVtMj4KICBhIG5zMTpJdGVtIDsKICBuczE6cHJvZHVjdE5hbWUgIktleWJvYXJkIiA7CiAgbnMxOnF1YW50aXR5IDMgOwogIG5zMTpwcmljZUVVUiAyNS41MCAu",
    "contentSyntax": "text/turtle",
    "validationType": "large"
}

The contentSyntax property can also be provided when the input is a URI although this can be skipped if it can be determined from the referenced file’s extension. Note that in terms of the content to validate you may also specify the embeddingMethod property with a value of either STRING, URL or BASE64. The purpose of this is to explicitly define how the contentToValidate value should be interpreted. If not specified, the content will be checked to determine the method. The following example defines the embeddingMethod property and is equivalent to the previous one:

{
    "contentToValidate": "QHByZWZpeCBuczA6IDxodHRwOi8vd3d3LnczLm9yZy9ucy9sb2NuIz4gLgpAcHJlZml4IG5zMTogPGh0dHA6Ly9pdGIuZWMuZXVyb3BhLmV1L3NhbXBsZS9wbyM+IC4KCjxodHRwOi8vbXkuc2FtcGxlLnBvL3BvI2hvbWU+CiAgYSA8aHR0cDovL3d3dy53My5vcmcvbnMvbG9jbiNBZGRyZXNzPiA7CiAgbnMwOmZ1bGxBZGRyZXNzICJSdWUgZHUgVGVzdCAxMjMsIDEwMDAgLSBCcnVzc2VscywgQmVsZ2l1bSIgOwogIG5zMDp0aG9yb3VnaGZhcmUgIlJ1ZSBkdSBUZXN0IiA7CiAgbnMwOmxvY2F0b3JEZXNpZ25hdG9yICIxMjMiIDsKICBuczA6cG9zdENvZGUgIjEwMDAiIDsKICBuczA6cG9zdE5hbWUgIkJydXNzZWxzIiA7CiAgbnMwOmFkbWluVW5pdEwxICJCRSIgLgoKPGh0dHA6Ly9teS5zYW1wbGUucG8vcG8jcHVyY2hhc2VPcmRlcj4KICBhIDxodHRwOi8vaXRiLmVjLmV1cm9wYS5ldS9zYW1wbGUvcG8jUHVyY2hhc2VPcmRlcj4gOwogIG5zMTpzaGlwVG8gPGh0dHA6Ly9teS5zYW1wbGUucG8vcG8jaG9tZT4gOwogIG5zMTpoYXNJdGVtIDxodHRwOi8vbXkuc2FtcGxlLnBvL3BvI2l0ZW0xPiA7CiAgbnMxOmhhc0l0ZW0gPGh0dHA6Ly9teS5zYW1wbGUucG8vcG8jaXRlbTI+IC4KCjxodHRwOi8vbXkuc2FtcGxlLnBvL3BvI2l0ZW0xPgogIGEgbnMxOkl0ZW0gOwogIG5zMTpwcm9kdWN0TmFtZSAiTW91c2UiIDsKICBuczE6cXVhbnRpdHkgMjAgOwogIG5zMTpwcmljZUVVUiAxNS45OSAuCgo8aHR0cDovL215LnNhbXBsZS5wby9wbyNpdGVtMj4KICBhIG5zMTpJdGVtIDsKICBuczE6cHJvZHVjdE5hbWUgIktleWJvYXJkIiA7CiAgbnMxOnF1YW50aXR5IDMgOwogIG5zMTpwcmljZUVVUiAyNS41MCAu",
    "embeddingMethod": "BASE64",
    "contentSyntax": "text/turtle",
    "validationType": "large"
}

In case SPARQL queries are supported, the contentToValidate input property becomes optional, given that you can provide input via query. To do this use property contentQuery instead. In case the validator setup expects you to provide credentials or the SPARQL endpoint itself, this is done via the contentQueryUsername, contentQueryPassword and contentQueryEndpoint properties. Taking as an example a case where the user is expected to provide only the query, the request would be as follows:

{
    "contentQuery": "CONSTRUCT { ?s ?p ?o } WHERE { GRAPH <urn:sparql:tests:uri> { ?s ?p ?o } . }",
    "validationType": "large"
}

To define whether imports are to be loaded when forming the data graph to validate, you would provide the loadImports property with a Boolean value (true or false). An example is provided below:

{
    "contentToValidate": "https://www.itb.ec.europa.eu/files/samples/shacl/sample-invalid.ttl",
    "validationType": "large",
    "reportSyntax": "text/turtle",
    "loadImports": true
}

Finally, recall that in Step 3: Prepare validator configuration the possibility was mentioned to allow for a given validation type, external SHACL shapes to be provided as part of the validator’s input. To do this through the REST API you provide the externalRules array containing objects with three properties:

• ruleSet for the RDF content containing the SHACL shapes to consider.

• ruleSyntax for the syntax of the ruleSet RDF content.

• embeddingMethod (STRING, URL or BASE64) to determine how the ruleSet value is to be considered.

Similar to the input content, the ruleSyntax can be skipped if the ruleSet value is a URL for a file with a known file extension. Also the embeddingMethod can be skipped to allow the validator to determine it itself, although it is best to define this to improve performance.

A sample request specifying an additional set of SHACL shapes is provided below:

{
    "contentToValidate": "https://www.itb.ec.europa.eu/files/samples/shacl/sample-invalid.ttl",
    "validationType": "large",
    "reportSyntax": "text/turtle",
    "externalRules": [
        {
            "ruleSet": "https://path.to.file/shape_extensions.ttl"
        }
    ]
}

Note

Blocking user-provided extensions and choose to load imports: If you provide SHACL shape extensions or choose to load imports for a validation type where this has not been explicitly allowed the call will fail.

The remaining operation that is available is validateMultiple that can be used for batch validation. In this case the input to the service uses the same JSON structure but this time as an array. To use the operation submit a POST request of type application/json to http://DOCKER_MACHINE:8080/shacl/order/api/validateMultiple (or https://www.itb.ec.europa.eu/shacl/order/api/validateMultiple on the test bed).

In the following example two distinct validations are requested:

[
    {
        "contentToValidate": "https://www.itb.ec.europa.eu/files/samples/shacl/sample.ttl",
        "validationType": "basic",
        "reportSyntax": "text/turtle"
    },
    {
        "contentToValidate": "https://www.itb.ec.europa.eu/files/samples/shacl/sample-invalid.ttl",
        "validationType": "large",
        "reportSyntax": "text/turtle"
    }
]

The resulting response in this case always includes the reports as BASE64 encoded strings, specifying the syntax to consider:

[
    {
        "report": "QHByZWZpeCBzaDogICAgPGh0dHA6Ly93d3cudzMub3JnL25zL3NoYWNsIz4gLgpAcHJlZml4IHhzZDogICA8aHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEjPiAuCkBwcmVmaXggbnMwOiAgIDxodHRwOi8vd3d3LnczLm9yZy9ucy9sb2NuIz4gLgpAcHJlZml4IGxvY246ICA8aHR0cDovL3d3dy53My5vcmcvbnMvbG9jbiM+IC4KQHByZWZpeCBuczE6ICAgPGh0dHA6Ly9pdGIuZWMuZXVyb3BhLmV1L3NhbXBsZS9wbyM+IC4KQHByZWZpeCBwbzogICAgPGh0dHA6Ly9pdGIuZWMuZXVyb3BhLmV1L3NhbXBsZS9wbyM+IC4KClsgYSAgICAgICAgICAgIHNoOlZhbGlkYXRpb25SZXBvcnQgOwogIHNoOmNvbmZvcm1zICB0cnVlCl0gLgo=",
        "reportSyntax": "text/turtle"
    },
    {
        "report": "QHByZWZpeCBzaDogICAgPGh0dHA6Ly93d3cudzMub3JnL25zL3NoYWNsIz4gLgpAcHJlZml4IHhzZDogICA8aHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEjPiAuCkBwcmVmaXggbnMwOiAgIDxodHRwOi8vd3d3LnczLm9yZy9ucy9sb2NuIz4gLgpAcHJlZml4IGxvY246ICA8aHR0cDovL3d3dy53My5vcmcvbnMvbG9jbiM+IC4KQHByZWZpeCBuczE6ICAgPGh0dHA6Ly9pdGIuZWMuZXVyb3BhLmV1L3NhbXBsZS9wbyM+IC4KQHByZWZpeCBwbzogICAgPGh0dHA6Ly9pdGIuZWMuZXVyb3BhLmV1L3NhbXBsZS9wbyM+IC4KClsgYSAgICAgICAgICAgIHNoOlZhbGlkYXRpb25SZXBvcnQgOwogIHNoOmNvbmZvcm1zICBmYWxzZSA7CiAgc2g6cmVzdWx0ICAgIFsgYSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc2g6VmFsaWRhdGlvblJlc3VsdCA7CiAgICAgICAgICAgICAgICAgc2g6Zm9jdXNOb2RlICAgICAgICAgICAgICAgICAgPGh0dHA6Ly9teS5zYW1wbGUucG8vcG8jaXRlbTI+IDsKICAgICAgICAgICAgICAgICBzaDpyZXN1bHRNZXNzYWdlICAgICAgICAgICAgICAiVmFsdWUgaXMgbm90ID4gMTBeXmh0dHA6Ly93d3cudzMub3JnLzIwMDEvWE1MU2NoZW1hI2ludGVnZXIiIDsKICAgICAgICAgICAgICAgICBzaDpyZXN1bHRQYXRoICAgICAgICAgICAgICAgICBwbzpxdWFudGl0eSA7CiAgICAgICAgICAgICAgICAgc2g6cmVzdWx0U2V2ZXJpdHkgICAgICAgICAgICAgc2g6VmlvbGF0aW9uIDsKICAgICAgICAgICAgICAgICBzaDpzb3VyY2VDb25zdHJhaW50Q29tcG9uZW50ICBzaDpNaW5FeGNsdXNpdmVDb25zdHJhaW50Q29tcG9uZW50IDsKICAgICAgICAgICAgICAgICBzaDpzb3VyY2VTaGFwZSAgICAgICAgICAgICAgICBwbzptaW5pbXVtSXRlbXNGb3JMYXJnZU9yZGVyU2hhcGUgOwogICAgICAgICAgICAgICAgIHNoOnZhbHVlICAgICAgICAgICAgICAgICAgICAgIDMKICAgICAgICAgICAgICAgXQpdIC4K",
        "reportSyntax": "text/turtle"
    }
]

Validation via SOAP web service API

The validator’s SOAP API is available under the /shacl/soap path. The exact path depends on how this is deployed (path to WSDL provided):

• Via Docker: http://DOCKER_MACHINE:8080/shacl/soap/order/validation?wsdl

• On the test bed: https://www.itb.ec.europa.eu/shacl/soap/order/validation?wsdl

The SOAP API used is the GITB validation service API, meaning that the validator is a GITB-compliant validation service. The importance of this is that apart from using it directly, this SOAP API allows integration of the validator in more complex conformance testing scenarios as a validation step in GITB TDL test cases. This potential is covered further in Step 7: Use the validator in GITB TDL test cases.

The operations supported are as follows:

• getModuleDefinition: Called to return information on how to call the service (i.e. what inputs are expected).

• validate: Called to trigger validation for provided content.

You can download this SOAP UI project that includes sample calls of these operations (make sure to change the service URL to match your setup).

Regarding the getModuleDefinition operation, a request of:

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:v1="http://www.gitb.com/vs/v1/">
   <soapenv:Header/>
   <soapenv:Body>
      <v1:GetModuleDefinitionRequest/>
   </soapenv:Body>
</soapenv:Envelope>

Will produce a response as follows:

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Body>
      <ns4:GetModuleDefinitionResponse xmlns:ns2="http://www.gitb.com/core/v1/" xmlns:ns3="http://www.gitb.com/tr/v1/" xmlns:ns4="http://www.gitb.com/vs/v1/">
         <module operation="V" id="ValidatorService">
            <ns2:metadata>
               <ns2:name>ValidatorService</ns2:name>
               <ns2:version>1.0.0</ns2:version>
            </ns2:metadata>
            <ns2:inputs>
               <ns2:param type="binary" name="contentToValidate" use="R" kind="SIMPLE" desc="The content to validate, provided as a string, BASE64 or a URI."/>
               <ns2:param type="string" name="embeddingMethod" use="O" kind="SIMPLE" desc="The embedding method to consider for the 'contentToValidate' input ('BASE64', 'URL' or 'STRING')."/>
               <ns2:param type="string" name="contentSyntax" use="O" kind="SIMPLE" desc="The mime type of the provided content."/>
               <ns2:param type="string" name="validationType" use="O" kind="SIMPLE" desc="The type of validation to perform (if multiple types are supported)."/>
               <ns2:param type="list[map]" name="externalRules" use="O" kind="SIMPLE" desc="A list of maps that defines external SHACL shapes to consider in addition to any preconfigured ones. Each map item corresponds to a SHACL file and defines the following keys: 'ruleSet' (the rules to consider, see 'contentToValidate' for its semantics), 'ruleSyntax' (the syntax of the 'ruleSet' content, see 'contentSyntax' for its semantics), 'embeddingMethod' (the way to consider the 'ruleSet' value)."/>
            </ns2:inputs>
         </module>
      </ns4:GetModuleDefinitionResponse>
   </soap:Body>
</soap:Envelope>

This response can be customised through configuration properties in config.properties to provide descriptions specific to your setup. For example, extending config.properties with the following:

...
validator.webServiceId = PurchaseOrderValidator
validator.webServiceDescription.contentToValidate = The purchase order content to validate
validator.webServiceDescription.contentSyntax = The syntax (mime type) of the purchase order
validator.webServiceDescription.validationType = The type of validation to perform ('basic' or 'large')
validator.webServiceDescription.externalRules = Skip this as externally provided SHACL shapes are not supported

Will produce a response as follows:

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Body>
      <ns4:GetModuleDefinitionResponse xmlns:ns2="http://www.gitb.com/core/v1/" xmlns:ns3="http://www.gitb.com/tr/v1/" xmlns:ns4="http://www.gitb.com/vs/v1/">
         <module operation="V" id="PurchaseOrderValidator">
            <ns2:metadata>
               <ns2:name>PurchaseOrderValidator</ns2:name>
               <ns2:version>1.0.0</ns2:version>
            </ns2:metadata>
            <ns2:inputs>
               <ns2:param type="binary" name="contentToValidate" use="R" kind="SIMPLE" desc="The purchase order content to validate"/>
               <ns2:param type="string" name="embeddingMethod" use="O" kind="SIMPLE" desc="The embedding method to consider for the 'contentToValidate' input ('BASE64', 'URL' or 'STRING')."/>
               <ns2:param type="string" name="contentSyntax" use="O" kind="SIMPLE" desc="The syntax (mime type) of the purchase order"/>
               <ns2:param type="string" name="validationType" use="O" kind="SIMPLE" desc="The type of validation to perform ('basic' or 'large')"/>
               <ns2:param type="list[map]" name="externalRules" use="O" kind="SIMPLE" desc="Skip this as externally provided SHACL shapes are not supported"/>
            </ns2:inputs>
         </module>
      </ns4:GetModuleDefinitionResponse>
   </soap:Body>
</soap:Envelope>

Running the validation itself is done through the validate operation. This expects the following inputs:

Input	Description	Required?	Type	Default value
contentToValidate	The content to validate.	Yes (optional if queries are supported)	A string that is interpreted based on the embeddingMethod input.
embeddingMethod	The way in which to interpret the contentToValidate value.	Yes, but should be skipped in favour of the embeddingMethod attribute - see below.	One of BASE64 (for content embedded as a BASE64 encoded string), URI (for a reference to remote content) or STRING (for content embedded as-is).
contentSyntax	The syntax of the provided content.	Yes, unless the content is provided as a URI for a file with a well-known extension.	A mime type string (e.g. text/turtle).
contentQuery	A SPARQL CONSTRUCT query to be used to get the content to validate.	No	String
contentQueryEndpoint	The SPARQL endpoint URL to query (if not preconfigured).	No	String
contentQueryUsername	The username to use for SPARQL endpoint authentication (if not preconfigured).	No	String
contentQueryPassword	The password to use for SPARQL endpoint authentication (if not preconfigured).	No	String
validationType	The type of validation to perform.	Yes, unless a single validation type is defined.	String	The single configured validation type (if defined).
externalRules	A list of user-provided SHACL shape extensions to be considered with any predefined ones. These are accepted only if explicitly allowed in the configuration for the validation type in question.	No	A list of map entries (see below for content).
loadImports	The choice of whether or not imports are to be loaded when forming the data graph to validate.	No	Boolean	false
reportSyntax	The syntax (provided as a mime type) for the resulting report (can be skipped for the default syntax).	No	A mime type string (e.g. text/turtle).	The default configured for the validator (application/rdf+xml if not overriden).
addInputToReport	Whether or not the output report should include as context the validated input.	No	Boolean	false
addRulesToReport	Whether or not the output report should include as context the SHACL shapes used for validation.	No	Boolean	false
addRdfReportToReport	Whether or not the raw RDF report should be returned as part of the resulting context information.	No	Boolean	false
rdfReportSyntax	The mime type for the raw RDF report returned as part of the resulting context (if included).	No	A mime type string (e.g. text/turtle).	The default configured for the validator (application/rdf+xml if not overriden).
rdfReportQuery	A post-processing SPARQL CONSTRUCT query to run on the RDF report before returning it in the resulting context. This is considered only if addRdfReportToReport is set to true.	No	String

Regarding the externalRules, this is a list of one of more entries. For each such entry the following input properties are expected:

Input	Description	Required?	Type
ruleSet	The RDF content with the SHACL shape extensions.	Yes	A string that is interpreted based on the embeddingMethod input.
embeddingMethod	The way in which to interpret the ruleSet input value.	Yes, but should be skipped in favour of the embeddingMethod attribute - see below.	One of BASE64 (for content embedded as a BASE64 encoded string), URI (for a reference to remote content) or STRING (for content embedded as-is).
ruleSyntax	The syntax to consider for the provided RDF content.	Yes, unless the content is provided as a URI for a file with a well-known extension.	A mime type string (e.g. text/turtle).

The GITB SOAP API that the validator realises foresees flexibility in how each provided value is interpreted. This is done via the embeddingMethod attribute that is defined on each input element. The values it supports are:

Value	Description
STRING	The value is interpreted as-is as an embedded text.
BASE64	The value is interpreted as an embedded BASE64 string that will need to be decoded before processing.
URI	The value is interpreted as a remote URI reference that will be looked up before processing.

Note

embeddingMethod values: The two approaches to provide the embeddingMethod value (as an input or an attribute) is due to a known issue in the GITB software where not all embedding methods can be leveraged within test cases (see Step 7: Use the validator in GITB TDL test cases).

The sample SOAP UI project includes sample requests per case. As an example, validating via URI would be done using the following envelope:

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:v1="http://www.gitb.com/vs/v1/" xmlns:v11="http://www.gitb.com/core/v1/">
   <soapenv:Header/>
   <soapenv:Body>
      <v1:ValidateRequest>
         <sessionId>?</sessionId>
         <input name="contentToValidate" embeddingMethod="URI">
            <v11:value>https://www.itb.ec.europa.eu/files/samples/shacl/sample-invalid.ttl</v11:value>
         </input>
         <input name="validationType" embeddingMethod="STRING">
            <v11:value>large</v11:value>
         </input>
      </v1:ValidateRequest>
   </soapenv:Body>
</soapenv:Envelope>

A more complex case would be where user-provided SHACL shape extensions are provided (if allowed by the configuration). The following example submits content for validation via URI and provides two SHACL extension files, both provided as remote URI references:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:v1="http://www.gitb.com/vs/v1/" xmlns:v11="http://www.gitb.com/core/v1/">
<soapenv:Header/>
<soapenv:Body>
    <v1:ValidateRequest>
        <sessionId>123</sessionId>
        <input name="contentToValidate" embeddingMethod="URI">
            <v11:value>https://www.itb.ec.europa.eu/files/samples/shacl/sample-invalid.ttl</v11:value>
        </input>
        <input name="validationType" embeddingMethod="STRING">
            <v11:value>large</v11:value>
        </input>
        <input name="externalRules">
            <v11:item>
                <v11:item name="ruleSet" embeddingMethod="URI">
                    <v11:value>https://path.to.rules/shape_extensions_1.ttl</v11:value>
                </v11:item>
            </v11:item>
            <v11:item>
                <v11:item name="ruleSet" embeddingMethod="URI">
                    <v11:value>https://path.to.rules/shape_extensions_2.ttl</v11:value>
                </v11:item>
            </v11:item>
        </input>
    </v1:ValidateRequest>
</soapenv:Body>
</soapenv:Envelope>

The resulting report in all cases is XML-based and uses the GITB TRL syntax:

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Body>
      <ns4:ValidationResponse xmlns:ns2="http://www.gitb.com/core/v1/" xmlns:ns3="http://www.gitb.com/tr/v1/" xmlns:ns4="http://www.gitb.com/vs/v1/">
         <report name="SHACL Validation">
            <ns3:date>2019-06-24T15:42:45.959Z</ns3:date>
            <ns3:result>FAILURE</ns3:result>
            <ns3:counters>
               <ns3:nrOfAssertions>0</ns3:nrOfAssertions>
               <ns3:nrOfErrors>1</ns3:nrOfErrors>
               <ns3:nrOfWarnings>0</ns3:nrOfWarnings>
            </ns3:counters>
            <ns3:context type="map">
               <ns2:item name="input" embeddingMethod="STRING" type="string">
                  <ns2:value><![CDATA[@prefix ns0: <http://www.w3.org/ns/locn#> .
@prefix ns1: <http://itb.ec.europa.eu/sample/po#> .

<http://my.sample.po/po#home>
  a <http://www.w3.org/ns/locn#Address> ;
  ns0:fullAddress "Rue du Test 123, 1000 - Brussels, Belgium" ;
  ns0:thoroughfare "Rue du Test" ;
  ns0:locatorDesignator "123" ;
  ns0:postCode "1000" ;
  ns0:postName "Brussels" ;
  ns0:adminUnitL1 "BE" .

<http://my.sample.po/po#purchaseOrder>
  a <http://itb.ec.europa.eu/sample/po#PurchaseOrder> ;
  ns1:shipTo <http://my.sample.po/po#home> ;
  ns1:hasItem <http://my.sample.po/po#item1> ;
  ns1:hasItem <http://my.sample.po/po#item2> .

<http://my.sample.po/po#item1>
  a ns1:Item ;
  ns1:productName "Mouse" ;
  ns1:quantity 20 ;
  ns1:priceEUR 15.99 .

<http://my.sample.po/po#item2>
  a ns1:Item ;
  ns1:productName "Keyboard" ;
  ns1:quantity 3 ;
  ns1:priceEUR 25.50 .]]></ns2:value>
               </ns2:item>
               <ns2:item name="shapes" embeddingMethod="STRING" type="string">
                  <ns2:value><![CDATA[<rdf:RDF
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:po="http://itb.ec.europa.eu/sample/po#"
    xmlns:locn="http://www.w3.org/ns/locn#"
    xmlns:sh="http://www.w3.org/ns/shacl#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#">
  <sh:NodeShape rdf:about="http://itb.ec.europa.eu/sample/po#LargeItemShape">
    <sh:targetClass rdf:resource="http://itb.ec.europa.eu/sample/po#Item"/>
    <sh:property>
      <sh:PropertyShape rdf:about="http://itb.ec.europa.eu/sample/po#minimumItemsForLargeOrderShape">
        <sh:path rdf:resource="http://itb.ec.europa.eu/sample/po#quantity"/>
        <sh:minExclusive rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >10</sh:minExclusive>
        <sh:severity rdf:resource="http://www.w3.org/ns/shacl#Violation"/>
      </sh:PropertyShape>
    </sh:property>
  </sh:NodeShape>
  <sh:NodeShape rdf:about="http://itb.ec.europa.eu/sample/po#ItemShape">
    <sh:targetClass rdf:resource="http://itb.ec.europa.eu/sample/po#Item"/>
    <sh:property>
      <sh:PropertyShape rdf:about="http://itb.ec.europa.eu/sample/po#productNameShape">
        <sh:path rdf:resource="http://itb.ec.europa.eu/sample/po#productName"/>
        <sh:minCount rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >1</sh:minCount>
        <sh:maxCount rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >1</sh:maxCount>
        <sh:nodeKind rdf:resource="http://www.w3.org/ns/shacl#Literal"/>
        <sh:severity rdf:resource="http://www.w3.org/ns/shacl#Violation"/>
      </sh:PropertyShape>
    </sh:property>
    <sh:property>
      <sh:PropertyShape rdf:about="http://itb.ec.europa.eu/sample/po#quantityShape">
        <sh:path rdf:resource="http://itb.ec.europa.eu/sample/po#quantity"/>
        <sh:minCount rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >1</sh:minCount>
        <sh:maxCount rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >1</sh:maxCount>
        <sh:datatype rdf:resource="http://www.w3.org/2001/XMLSchema#integer"/>
        <sh:minExclusive rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >0</sh:minExclusive>
        <sh:severity rdf:resource="http://www.w3.org/ns/shacl#Violation"/>
      </sh:PropertyShape>
    </sh:property>
    <sh:property>
      <sh:PropertyShape rdf:about="http://itb.ec.europa.eu/sample/po#priceShape">
        <sh:path rdf:resource="http://itb.ec.europa.eu/sample/po#priceEUR"/>
        <sh:minCount rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >1</sh:minCount>
        <sh:maxCount rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >1</sh:maxCount>
        <sh:datatype rdf:resource="http://www.w3.org/2001/XMLSchema#decimal"/>
        <sh:severity rdf:resource="http://www.w3.org/ns/shacl#Violation"/>
      </sh:PropertyShape>
    </sh:property>
  </sh:NodeShape>
  <sh:NodeShape rdf:about="http://itb.ec.europa.eu/sample/po#PurchaseOrderShape">
    <sh:targetClass rdf:resource="http://itb.ec.europa.eu/sample/po#PurchaseOrder"/>
    <sh:property>
      <sh:PropertyShape rdf:about="http://itb.ec.europa.eu/sample/po#shipToShape">
        <sh:path rdf:resource="http://itb.ec.europa.eu/sample/po#shipTo"/>
        <sh:minCount rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >1</sh:minCount>
        <sh:maxCount rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >1</sh:maxCount>
        <sh:class rdf:resource="http://www.w3.org/ns/locn#Address"/>
        <sh:severity rdf:resource="http://www.w3.org/ns/shacl#Violation"/>
      </sh:PropertyShape>
    </sh:property>
    <sh:property>
      <sh:PropertyShape rdf:about="http://itb.ec.europa.eu/sample/po#billToShape">
        <sh:path rdf:resource="http://itb.ec.europa.eu/sample/po#billTo"/>
        <sh:class rdf:resource="http://www.w3.org/ns/locn#Address"/>
        <sh:maxCount rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >1</sh:maxCount>
        <sh:severity rdf:resource="http://www.w3.org/ns/shacl#Violation"/>
      </sh:PropertyShape>
    </sh:property>
    <sh:property>
      <sh:PropertyShape rdf:about="http://itb.ec.europa.eu/sample/po#hasItemsShape">
        <sh:path rdf:resource="http://itb.ec.europa.eu/sample/po#hasItem"/>
        <sh:minCount rdf:datatype="http://www.w3.org/2001/XMLSchema#integer"
        >1</sh:minCount>
        <sh:class rdf:resource="http://itb.ec.europa.eu/sample/po#Item"/>
        <sh:severity rdf:resource="http://www.w3.org/ns/shacl#Violation"/>
      </sh:PropertyShape>
    </sh:property>
  </sh:NodeShape>
</rdf:RDF>]]></ns2:value>
               </ns2:item>
            </ns3:context>
            <ns3:reports>
               <ns3:error xsi:type="ns3:BAR" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
                  <ns3:description>Value is not > 10^^http://www.w3.org/2001/XMLSchema#integer</ns3:description>
                  <ns3:location>Focus node [http://my.sample.po/po#item2] - Result path [http://itb.ec.europa.eu/sample/po#quantity]</ns3:location>
                  <ns3:test>Shape [http://itb.ec.europa.eu/sample/po#minimumItemsForLargeOrderShape] - Value [3]</ns3:test>
               </ns3:error>
            </ns3:reports>
         </report>
      </ns4:ValidationResponse>
   </soap:Body>
</soap:Envelope>

The report includes:

• The validation timestamp (in UTC).

• The overall result (SUCCESS or FAILURE).

• The count of errors, warnings and information messages.

• The context for the validation, i.e. the content that was validated and the applied SHACL shapes, depending on the addInputToReport and addRulesToReport input values.

• The list of report items displaying per item its description, location in the validated content (using the focus node and path) and performed test (using the applied shape’s name and the tested value).

Validation via command-line tool

Note

Command-line tool availability: Command-line tools are supported only for validators hosted on test bed resources and if generation of such a tool has been requested (see Step 5: Setup validator on test bed).

When a command line tool is set up for your validator it is available as an executable JAR file, packaged alongside a README file in a ZIP archive. This ZIP archive is downloadable from a URL of the form https://www.itb.ec.europa.eu/shacl-offline/DOMAIN/validator.zip, where DOMAIN is the name of the validator’s domain. Considering our purchase order example, the command-line validator would be available at https://www.itb.ec.europa.eu/shacl-offline/order/validator.zip.

To use it you need to:

1. Ensure you have Java running on your workstation (minimum version 11).

2. Download and extract the validator’s ZIP archive.

3. Open a command prompt and change to the directory in which you extracted the JAR file.

4. View the validator’s help message by issuing java -jar validator.jar

> java -jar validator.jar

Expected usage: java -jar validator.jar [-noreports] [-reportSyntax REPORT_SYNTAX] -contentToValidate FILE_1/URI_1 CONTENT_SYNTAX_1 ... [-contentToValidate FILE_N/URI_N CONTENT_SYNTAX_N] [-externalShapes SHAPE_FILE_1/SHAPE_URI_1 CONTENT_SYNTAX_1] ... [-externalShapes SHAPE_FILE_N/SHAPE_URI_N CONTENT_SYNTAX_N]
Where:
    - REPORT_SYNTAX is the mime type for the validation report(s).
    - FILE_X or URI_X is the full file path or URI to the content to validate, optionally followed by CONTENT_SYNTAX_X as the content's mime type.
    - SHAPE_FILE_X or SHAPE_URI_X is the full file path or URI to additional shapes to consider, optionally followed by CONTENT_SYNTAX_X as the shapes' mime type.
    - LOAD_IMPORTS is a boolean indicating whether the owl:Imports should be loaded (true) or not (false).

The summary of each validation will be printed and the detailed report produced in the current directory (as "report.X.SUFFIX").

In short, the validator can be used to validate one or more local or remote RDF files. If not possible to determine automatically, you may also specify the content type of each input, and also specify the content type for the output reports (reverting to the configured default if unspecified). Alternatively, if SPARQL queries are supported, you may also provide as input the SPARQL query to execute via the relevant flags (-contentQuery, -contentQueryEndpoint, -contentQueryUsername, -contentQueryPassword).

Running the validator will produce a summary output on the command console as well as the detailed validation report(s) (unless flag -noreports has been specified). To resolve potential problems during execution, an output log is also generated with a detailed log trace.

> java -jar validator.jar -contentToValidate https://www.itb.ec.europa.eu/files/samples/shacl/sample.ttl

Validating [https://www.itb.ec.europa.eu/files/samples/shacl/sample.ttl]... Done.


Validation report summary [https://www.itb.ec.europa.eu/files/samples/shacl/sample.ttl]:
- Date: 2019-09-03T08:41:24.802+02:00
- Result: SUCCESS
- Errors: 0
- Warnings: 0
- Messages: 0
- Detailed report in: [D:\temp\validator\order\report.0.jsonld]

Note

Offline validator use and remote resources: Depending on the validator’s configuration, one or more of its SHACL files may be defined as URIs (see Step 3: Prepare validator configuration). In addition, you may also provide the content to validate as a reference to a remote file or as a query to be executed against a remote SPARQL endpoint. In these cases the workstation running the validator would need access to the remote resources. Any proxy settings applicable for the workstation will automatically be used for the connections.

Step 7: Use the validator in GITB TDL test cases

As a next step over the standalone validator you may consider using it from within GITB TDL test cases running in the test bed. You would typically do this for the following reasons:

• You want to control access to the validation service based on user accounts.

• You prefer to record all data linked to validations (for e.g. subsequent inspection).

• You want to build complete conformance testing scenarios that are either focused on the validator or that use it as part of validation steps.

As described in Validation via SOAP web service API, the standalone validator offers by default a SOAP API for machine-to-machine integration that realises the GITB validation service specification. In short this means that the service can be easily included in any GITB TDL test case as the handler of a verify step by supplying as its value the full URL to the service’s WSDL.

The following example is a test case that validates purchase orders as being large. The input content is requested as a file from the user as well as the syntax to consider (based on a predefined set of options):

<?xml version="1.0" encoding="UTF-8"?>
<testcase id="testCase1_upload" xmlns="http://www.gitb.com/tdl/v1/" xmlns:gitb="http://www.gitb.com/core/v1/">
    <metadata>
        <gitb:name>testCase1_upload</gitb:name>
        <gitb:version>1.0</gitb:version>
        <gitb:description>Test case that allows the developer of an EU retailer system to upload a purchase order for validation as being considered large.</gitb:description>
    </metadata>
    <actors>
        <gitb:actor id="Retailer" name="Retailer" role="SUT"/>
    </actors>
    <steps>
        <!--
            Request from the user the content to validate as a file and its syntax.
        -->
        <interact id="userData" desc="Upload content">
            <request name="content" desc="Purchase order to validate:" contentType="BASE64"/>
            <request name="syntax" desc="Content syntax:" options="application/json+ld, application/rdf+xml, text/turtle" optionLabels="JSON-LD, RDF/XML, Turtle"/>
        </interact>
        <!--
            Trigger the validation.
        -->
        <verify handler="https://www.itb.ec.europa.eu/shacl/soap/order/validation?wsdl" desc="Validate purchase order">
            <input name="contentToValidate">$userData{content}</input>
            <input name="contentSyntax">$userData{syntax}</input>
            <input name="validationType">"large"</input>
        </verify>
    </steps>
</testcase>

Note that the input names (contentToValidate, contentSyntax and validationType) correspond to those expected by the SOAP API (see Validation via SOAP web service API). For string or binary content you typically don’t need to provide the embeddingMethod as this is determined automatically by the test bed. If however you are using string variables that contain BASE64 or URI references you would need to define this explicitly.

An example of this is a test case that requests the content from the user as a URI:

<testcase id="testCase1_upload" xmlns="http://www.gitb.com/tdl/v1/" xmlns:gitb="http://www.gitb.com/core/v1/">
    ...
    <steps>
        <!--
            Request from the user the content to validate as a URI.
            The syntax will be determined from the file's extension.
        -->
        <interact id="userData" desc="Provide input">
            <request name="content" desc="URI of the purchase order to validate:"/>
        </interact>
        <!--
            Trigger the validation.
        -->
        <verify handler="https://www.itb.ec.europa.eu/shacl/soap/order/validation?wsdl" desc="Validate purchase order">
            <input name="contentToValidate">$userData{content}</input>
            <!--
                Explicitly define the embeddingMethod to consider.
            -->
            <input name="embeddingMethod">"URL"</input>
            <input name="validationType">"large"</input>
        </verify>
    </steps>
</testcase>

A more complicated example is when external SHACL shapes are to be provided (if supported for the validation type in question). The following example shows two such extensions being provided, one as a URI and one from a file included in the test suite itself:

<?xml version="1.0" encoding="UTF-8"?>
<testcase id="testCase1_upload" xmlns="http://www.gitb.com/tdl/v1/" xmlns:gitb="http://www.gitb.com/core/v1/">
    ...
    <imports>
        <!--
            Import the additional shapes to consider from the test suite.
        -->
        <artifact type="binary" name="additionalShapes">resources/additional-shapes.ttl</artifact>
    </imports>
    <variables>
        <!--
            Define variables for the SHACL shape extensions.
        -->
        <var name="externalRules" type="list[map]"/>
        <!--
            First set of rules defined through a URL reference.
            The "ruleSyntax" is not needed as it can be determined from the referenced file.
        -->
        <var name="ruleSet1" type="map">
            <value name="ruleSet" type="string">https://path.to.rules/shape_extensions.ttl</value>
            <value name="embeddingMethod" type="string">URL</value>
        </var>
        <!--
            Second set of rules defined as BASE64 (to be assigned once the test starts).
            The "embeddingMethod" will be automatically set given that a file will be used.
        -->
        <var name="ruleSet2" type="map">
            <value name="ruleSyntax" type="string">text/turtle</value>
        </var>
    </variables>
    ...
    <steps>
        <!--
            Add as the "ruleSet" value the imported shapes from the test suite.
            This will also automatically set the "embeddingMethod" to "BASE64".
        -->
        <assign to="$ruleSet2{ruleSet}" type="binary">$additionalShapes</assign>
        <!--
            Add both additional sets of rules to the input list.
        -->
        <assign to="$externalRules" append="true">$ruleSet1</assign>
        <assign to="$externalRules" append="true">$ruleSet2</assign>
        <interact id="userData" desc="Upload content">
            <request name="content" desc="Purchase order to validate:" contentType="BASE64"/>
            <request name="syntax" desc="Content syntax:" options="application/json+ld, application/rdf+xml, text/turtle" optionLabels="JSON-LD, RDF/XML, Turtle"/>
        </interact>
        <verify handler="https://www.itb.ec.europa.eu/shacl/soap/order/validation?wsdl" desc="Validate purchase order">
            <input name="contentToValidate">$userData{content}</input>
            <input name="contentSyntax">$userData{syntax}</input>
            <input name="validationType">"large"</input>
            <!--
                Pass the list of additional shapes to consider.
            -->
            <input name="externalRules">$externalRules</input>
        </verify>
    </steps>
</testcase>

One additional point to make is on the definition of the service’s WSDL (i.e. the handler value). Although you can define this directly as in the previous examples, a better approach to improve portability is to define this in the test bed’s domain configuration as a domain parameter. Defining a validationService parameter in the domain you could thus redefine the verify step as:

...
<verify handler="$DOMAIN{validationService}" desc="Validate purchase order">
    ...
</verify>
...

Summary

Congratulations! You have just setup a validation service for your RDF specification. In doing so you considered your needs and defined your service through configuration on the ISA² test bed or as a Docker container. In addition, you used this service via its different APIs and considered how this could be used as part of complete conformance testing scenarios.

See also

In Step 7: Use the validator in GITB TDL test cases we briefly touched upon using the test bed for complete conformance testing scenarios. If this interests you, several additional guides are available that can provide you with further information:

• Guide: Creating a test suite on how to create a simple GITB TDL test suite.

• Guide: Defining your test configuration on how to configure a GITB TDL test suite in the test bed as part of your overall test setup.

• Guide: Executing a test case on how to execute tests and monitor results.

• Guide: Installing the test bed for development use on how to install your own test bed instance to test with.

For the full information on GITB TDL test cases check out the GITB TDL documentation, the reference for all test step constructs as well as a source of numerous complete examples.

In case you need to consider validation of further content types, be aware that the test bed provides similar support for:

• XML validation, to validate XML content using XML Schema and Schematron (see Guide: Setting up XML validation).

• JSON validation, to validate JSON content using JSON schema (see Guide: Setting up JSON validation).

• CSV validation, to validate CSV content using Table Schema (see Guide: Setting up CSV validation).

If you are planning on operating a validator on your own-premises for production use check the validator production installation guide for the steps to follow and guidelines to consider.

Finally, for more information on Docker and the commands used in this guide, check out the Docker online documentation.

References

This section contains additional references linked to this guide.

Validator configuration properties

The following sections list the configuration properties you can use to customise the operation of your validation service.

Domain-level configuration

The properties in this section are to be provided in the configuration property file (one per configured validation domain) you define as part of your validator configuration.

Note

Property placeholders: Numerous configuration properties listed below are presented with placeholders. These are marked in uppercase and have the following meaning:

• TYPE: The validation type to perform.

• OPTION: An option for a given validation type (e.g. a specific version).

• FULL_TYPE: For a validation type with options this is equals to TYPE.OPTION, otherwise it is the TYPE value itself.

• N: A zero-based integer (used as a counter).

Property	Description	Type	Default value
validator.channels	Comma-separated list of validation channels to have enabled. Possible values are (form, rest_api, soap_api).	Comma-separated Strings	form, rest_api, soap_api
validator.type	Comma-separated list of supported validation types. Values need to be reflected in the other properties’ TYPE and FULL_TYPE placeholders. FULL_TYPE equals TYPE.OPTION (in case type options are defined) or simply TYPE if there are no options (see also validator.typeOptions).	Comma-separated Strings
validator.typeOptions.TYPE	Comma-separated list of options defined for a given validation type (added as a postfix). Values need to be reflects in the other properties’ OPTION and FULL_TYPE placeholders. FULL_TYPE equals TYPE.OPTION (in case type options are defined) or simply TYPE if there are no options (see also validator.type).	Comma-separated Strings
validator.typeLabel.TYPE	Label to display in the web form for a given validation type (added as a postfix of validator.typeLabel). Only displayed if there are multiple types.	String
validator.optionLabel.OPTION	Label to display in the web form for an option across all validation types (added as a postfix of validator.optionLabel). Only displayed if there are options defined.	String
validator.typeOptionLabel.TYPE.OPTION	Label to display for an option for a specific validation type.	String
validator.completeTypeOptionLabel.TYPE.OPTION	Label to display for the full validation type and option combination (visible in the validator’s result screen).	String
validator.webServiceId	The ID of the SOAP web service.	String	ValidatorService
validator.webServiceDescription.contentToValidate	The description of the SOAP web service for element “contentToValidate”.	String	The content to validate, provided as a string, BASE64 or a URI.
validator.webServiceDescription.contentSyntax	The description of the SOAP web service for element “contentSyntax”.	String	The mime type of the provided content.
validator.webServiceDescription.validationType	The description of the SOAP web service for element “validationType”. Only displayed if there are multiple types.	String	The type of validation to perform (if multiple types are supported).
validator.webServiceDescription.externalRules	The description of the SOAP web service for element “externalRules”.	String	A list of maps that defines external SHACL shapes to consider in addition to any preconfigured ones. Each map item corresponds to a SHACL file and defines the following keys: 'ruleSet' (the rules to consider, see 'contentToValidate' for its semantics), 'ruleSyntax' (the syntax of the 'ruleSet' content, see 'contentSyntax' for its semantics), 'embeddingMethod' (the way to consider the 'ruleSet' value).
validator.webServiceDescription.embeddingMethod	The description of the SOAP web service for element “embeddingMethod”.	String	The embedding method to consider for the 'contentToValidate' input ('BASE64', 'URL' or 'STRING').
validator.shaclFile.FULL_TYPE	Comma-separated list of SHACL shape files loaded for a given validation type (added as a postfix). These can be a files or folders.	Comma-separated Strings
validator.shaclFile.FULL_TYPE.remote.N.url	Reference for a remotely loaded SHACL shape file for a given validation type (added as the FULL_TYPE placeholder). One or more such entries can be defined by incrementing the zero-based N counter.	String
validator.shaclFile.FULL_TYPE.remote.N.type	The content type for the remotely loaded SHACL shape file defined in the corresponding validator.shaclFile.FULL_TYPE.remote.N.url property. This can be skipped if the type can be determined from the referenced file name.	String (as mime type)
validator.externalShapes.FULL_TYPE	Whether or not user-provided shapes are allowed for the given validation type (added as a postfix). Possible values are (required, optional, none).	String	none
validator.defaultReportSyntax	The default report syntax to be returned if none is explicitly requested.	String (as mime type)	application/rdf+xml
validator.contentSyntax	The RDF content syntaxes (as mime types) that the web user interface will support for input and output.	Comma-separated Strings	application/ld+json, application/rdf+xml, text/turtle, application/n-triples
validator.reportsOrdered	Whether the report items are to be ordered (errors first, then warnings, then messages). Otherwise the items will appear based on where they were raised in the validated content.	Boolean	false
validator.showAbout	Whether or not to show the about panel on the web UI.	Boolean	true
validator.supportMinimalUserInterface	Enable a minimal user interface useful for embedding in other UIs or portals (applies only if the form validation channel is enabled).	Boolean	false
validator.bannerHtml	Configurable HTML banner replacing the text title.	String
validator.footerHtml	Configurable HTML banner for the footer.	String
validator.mergeModelsBeforeValidation	Before validating merge the data graph with the shape graph. This is done to ensure that any vocabularies defines as part of the test configuration also provide context to the considered data graph.	Boolean	true
validator.defaultPlugins.N.jar	Configuration for custom plugins that can be used to extend the validation. This is a default plugin definition that applies to all validation types. This property is the relative path pointing to the all-in-one JAR file that contains the plugin implementation(s).	String
validator.defaultPlugins.N.class	Paired with the its corresponding jar property (see above), this defines the fully qualified names of the classes to be loaded as plugin implementations.	Comma-separated Strings.
validator.plugins.FULL_TYPE.N.jar	Configuration for plugins that can be used to extend the validation specific to a given type (FULL_TYPE) and extending any default plugins (see above). This property is the relative path pointing to the all-in-one JAR file that contains the plugin implementation(s).	String
validator.plugins.FULL_TYPE.N.class	Paired with the its corresponding jar property (see above), this defines the fully qualified names of the classes to be loaded as plugin implementations.	Comma-separated Strings.
validator.loadImports	Whether or not imports should be loaded when loading the data graph to validate.	Boolean	false
validator.input.loadImports	Whether or not the user can specify if imports should be loaded when loading the data graph to validate. Possible values are (required, optional, none).	String	none
validator.loadImports.FULL_TYPE	Whether or not imports should be loaded when loading the data graph to validate for the given validation type (added as a postfix).	Boolean	false
validator.input.loadImports.FULL_TYPE	Whether or not the user can specify if imports should be loaded when loading the data graph to validate for a given validation type (added as a postfix). Possible values are (required, optional, none).	String	none
validator.maximumReportsForDetailedOutput	The maximum number of report items for which a PDF validation report will be generated (no report will be preoduced if exceeded).	Integer	5000
validator.maximumReportsForXmlOutput	The maximum number of report items to include in the XML validation report.	Integer	50000
validator.queryEndpoint	The SPARQL endpoint to execute queries against (when query input is supported). If queries are supported and no endpoint is configured, this will be expected to be provided as input.	String
validator.queryUsername	The username to use for authentication against the configured SPARQL endpoint (when query input is supported). If not configured this will be expected as an optional input (optional for anonymous usage).	String
validator.queryPassword	The password to use for authentication against the configured SPARQL endpoint (when query input is supported). If not configured this will be expected as an optional input (optional for anonymous usage).	String
validator.queryAuthenticationInput	If queries are supported and no username or password are configured, this can be used to set whether authentication is required, optional or never expected. Possible values are required, optional and none.	String	none when credentials are preconfigured and optional otherwise.
validator.queryPreferredContentType	The RDF syntax (provided as a mime type string) to request as a result from the SPARQL endpoint. Possible values are application/ld+json, application/rdf+xml, text/turtle and application/n-triples	String	application/rdf+xml
validator.supportsQueries	Whether or not SPARQL queries are supported to generate the input. If any other query-related properties are defined this is assume to be true.	Boolean	false
validator.importProperties	The path of a file declaring additional domain properties. If the application property validator.restrictResourcesToDomain is set to true or omitted, only relative paths of files under the domain root folder are accepted.	String

Note

Configuration of sensitive values: If you are enabling SPARQL queries for your validator and are preconfiguring the endpoint’s password you will likely want to not include its value in the configuration file. For this and other sensitive properties (e.g. internal URLs) check Environment-specific domain configuration to see how they can be externalised.

Properties related to UI labels

To override labels on the web UI you can use the following properties:

Property	Description	Default value
validator.uploadTitle	Title for the validator web form.	SHACL Validator
validator.reportTitle	The title for the produced PDF validation report.	Validation report
validator.label.fileInputLabel	Label for the file input.	Content to validate
validator.label.fileInputPlaceholder	Placeholder text displayed within the file input’s text field.	Select file...
validator.label.typeLabel	The label for the selection of the target validation type.	Validate as
validator.label.contentSyntaxLabel	The label for the selection of the content syntax type.	Content syntax
validator.label.contentSyntaxTooltip	The tooltip text for the content syntax selection.	Optional for content provided as a file or a URI if a known file extension is detected
validator.label.includeExternalShapes	Text appearing next to the option to include external shapes.	Include external shapes
validator.label.externalRulesTooltip	The tooltip text for the external shapes option.	Additional shapes that will be considered for the validation
validator.label.externalShapesLabel	The label displayed next to included external shapes.	External shapes
validator.label.optionContentFile	The text for the file content option.	File
validator.label.optionContentURI	The text for the URI content option.	URI
validator.label.optionContentDirectInput	The text for the direct input content option.	Direct input
validator.label.optionContentQuery	The text for the query content option.	Query
validator.label.uploadButton	The upload button’s text.	Upload
validator.label.resultSectionTitle	The title for the validation result panel.	Validation result
validator.label.resultSubSectionOverviewTitle	The title for the overview section of the results.	Overview
validator.label.resultDateLabel	The label for the result panel’s date.	Date:
validator.label.resultFileNameLabel	The label for the result panel’s file name.	File name:
validator.label.resultValidationTypeLabel	The label for the result panel’s validation type.	Validation type:
validator.label.resultResultLabel	The label for the result panel’s overall validation result.	Result:
validator.label.resultErrorsLabel	The label preceding the total error count.	Errors:
validator.label.resultWarningsLabel	The label preceding the total warning count.	Warnings:
validator.label.resultMessagesLabel	The label preceding the total message count.	Messages:
validator.label.resultSubSectionDetailsTitle	The title of the panel displaying the validation report’s details.	Details
validator.label.resultTestLabel	The label preceding a report item’s test description.	Test:
validator.label.resultLocationLabel	The label preceding a report item’s location description.	Location:
validator.label.reportItemFocusNode	The text displayed in a report item to identify the focus node of a shape.	Focus node
validator.label.reportItemResultPath	The text displayed in a report item to identify the relevant result path.	Result path
validator.label.reportItemShape	The text displayed in report item to identify the shape that produced it.	Shape
validator.label.reportItemValue	The text displayed in report items to highlight the value of the relevant target node.	Value
validator.label.optionDownloadReport	The text for the download option linked to the validation report.	Validation report
validator.label.optionDownloadContent	The text for the download option linked to the validated content.	Validated content
validator.label.optionDownloadShapes	The text for the download option linked to the shapes.	Shapes
validator.label.saveDownload	The label preceding the download controls.	Download
validator.label.saveAs	The text between the download controls for the content and syntax.	as
validator.label.loadImports	The label displayed next to the load imports checkbox.	Load imports defined in the input?
validator.label.loadImportsTooltip	The tooltip text for the load imports option.	Load imported resources defined via owl:imports when defining the data graph to validate.
validator.label.maximumReportsExceededForDetailedOutputMessage	Message to display when a PDF report is not generated due to the number of report items.	Findings are not listed here due to their large number. Download the validation report to view further details.
validator.label.queryEndpointInputPlaceholder	Placeholder text displayed within the SPARQL endpoint input’s text field.	SPARQL endpoint URL
validator.label.queryUsernameInputPlaceholder	Placeholder text displayed within the SPARQL endpoint username input’s text field.	Username
validator.label.queryPasswordInputPlaceholder	Placeholder text displayed within the SPARQL endpoint password input’s text field.	Password
validator.label.queryAuthenticateLabel	Label displayed in front of the checkbox to provide credentials for the SPARQL endpoint.	Authenticate?

Application-level configuration

These properties govern the validator’s application instance itself. They apply only when you are defining your own validator as a Docker image in which case they are supplied as environment variables (ENV directives in a Dockerfile). Note that apart from these properties any Spring Boot configuration property can also be supplied.

Note

Mandatory property: The only mandatory property that needs to be defined is validator.resourceRoot.

Property	Description	Type	Default value
validator.resourceRoot	The root folder under which domain subfolders will be loaded from.	String
validator.domain	The names of the domain subfolders to consider. By default all folders under validator.resourceRoot will be considered.	Comma-separated Strings
validator.domainName.DOMAIN	The name to display for a given domain folder (the folder name replacing the DOMAIN placeholder). This value will also be used in request paths.	String	The folder name is used.
logging.file.path	Path to a folder that will hold the validator’s log output.	String	/validator/logs
validator.tmpFolder	Path to a folder that contains temporary data and reports.	String	/validator/tmp
validator.acceptedShaclExtensions	Accepted local SHACL file extensions. All other files found in validator.shaclFile.FULL_TYPE (when folders are defined) are ignored.	Comma-separated Strings	rdf, ttl, jsonld
validator.acceptedHeaderAcceptTypes	Accepted Accept header syntaxes for the report type (used in the REST API).	Comma-separated Strings	application/rdf+xml, application/ld+json, text/turtle, , application/n-triples
validator.defaultReportSyntax	Global default report syntax, applied if not overriden at domain level or by the user’s request.	String (as mime type)	application/rdf+xml
validator.contentSyntax	Global default supported content syntaxes for the web upload form if not overriden at domain level.	Comma-separated Strings	application/ld+json, application/rdf+xml, text/turtle, application/n-triples
validator.cleanupRate	The rate at which the external file cache is refreshed (in milliseconds).	Long	3600000
validator.cleanupWebRate	The rate at which temporary files linked to the web form are cleaned (in milliseconds).	Long	600000
validator.docs.licence.description	Description of the licence in the Swagger UI.	String,	European Union Public Licence (EUPL) 1.2
validator.docs.licence.url	URL to the licence for the Swagger UI.	String,	https://eupl.eu/1.2/en/
validator.docs.version	Version number to display in the Swagger UI.	String,	1.0.0
validator.docs.title	Title to display in the Swagger UI.	String,	SHACL Validator REST API
validator.docs.host	The host to display as the root for the REST API Swagger documentation.	String	localhost:8080
validator.docs.schemes	Comma-separated scheme values for the Swagger documentation	Comma-separated Strings	http
validator.hydraServer	The server part to add as of the REST API’s vocabulary’s URL for the Hydra documentation.	String	http://localhost:${server.port}
validator.hydraRootPath	The root path to append to the validator.hydraServer for the Hydra documentation of the REST API.	String	${server.servlet.context-path}
validator.queryPreferredContentType	In case SPARQL queries are supported, this is the default content type requested as queries’ output.	String	application/rdf+xml
validator.restrictResourcesToDomain	Whether local validation artefacts can be loaded from outside the domain root folder. If false, both relative and absolute paths are accepted.	Boolean	true
validator.webhook.statistics	The URL of the backend service that will collect usage statistics. This property is optional and, if (and only if) present, the validator will report usage statistics.	String
validator.webhook.statisticsSecret	The validator client secret to be passed to the backend service for usage statistics reporting.	String
validator.identifier	The validator’s identifier to be sent for usage statistics reporting.	String	rdf
validator.webhook.statisticsEnableCountryDetection	Whether the usage statistics reporting service can detect users’ countries from their IP addresses. If false no information about the user’s country or IP address will be reported.	Boolean	false
validator.webhook.statisticsCountryDetectionDbFile	The path to the .mmdb file with the geolocation database that is used to resolve the user’s country from an IP address. This database will only be used when the property validator.webhook.statisticsEnableCountryDetection takes the value true.	String
validator.webhook.ipheader	The HTTP header to use to retrieve the user’s IP address for usage statistics if the validator is behind a proxy. This property is only used to detect the user’s country, when such feature is enabled (see validator.webhook.statisticsEnableCountryDetection).	String	X-Real-IP

Change history

This section lists the change history for the RDF validator core component, applicable to all validator instances built over it. The core validator is provided through a latest release that is updated for every change, as well as milestone releases for deployments where ensuring stability is more important than rapid updates. Note that all updates, be it on the release milestones or the latest snapshot, are guaranteed to always be backwards compatible with configurations defined for earlier builds.

The table that follows lists all changes per build date and release milestones. Users operating a snapshot “latest” release can use the build dates to determine what updates are available. When a release number is listed next to a change, this includes all changes leading up to and including it.

Note

All validators managed by the Test Bed are always operating with the very latest build. In addition, release milestones are currently only offered for Docker images.

Release	Date	Type	Description
	2021/06/29	Feature	Allow validator usage statistics reporting via webhook.
	2021/05/21	Improvement	Context data is no longer included in the PDF validation report.
	2021/05/03	Improvement	Add previous page button in error page and specific error handling for document downloads.
1.0.0	2021/04/21	Improvement	Support for publishing of milestone releases.
	2021/04/19	Improvement	Support for unicode properties in configuration files.
	2021/04/07	Feature	Add build timestamp in all JAR files’ manifests.
	2021/03/11	Feature	Allow the reuse of validation artefacts across domains and the split of domain configuration in several files.
	2021/02/02	Bug	When running as a CLI tool, correct the handling of SPARQL arguments when the validator has a single validation type.
	2021/01/27	Improvement	Improve the detection of user-provided validation artefacts based on their embedding method.
	2021/01/27	Bug	Correct RDF/XML exports that fail due to malformed URIs.
	2021/01/25	Feature	Support post-processing of the SHACL validation report via SPARQL queries.
	2021/01/21	Improvement	Consider content as plain-text if not possible to parse as a URL or BASE64.
	2021/01/21	Bug	Correct the processing of input content or SHACL shapes provided as URIs when no file extension is present.
	2021/01/21	Improvement	Improve REST API documentation.
	2021/01/20	Improvement	Adjust the size of the SPARQL query editor.
	2021/01/20	Improvement	Ensure that when providing input via SPARQL query, the query is always provided as part of the input.
	2021/01/19	Feature	Allow any configuration property to be provided via environment variable or system property.
	2020/01/19	Feature	Allow the validator’s input to be provided via SPARQL queries.
	2020/01/18	Feature	Skip PDF report generation for large validation reports.
	2020/01/18	Feature	When used via the SOAP API allow the XML validation report’s additional information (input, rules) to be included if the caller requests them.
	2020/01/18	Bug	When used via CLI tool, correct the report’s encoding if unicode characters are included.
	2020/01/18	Improvement	Include startup and teardown messages when using the validator via CLI tool.
	2021/01/18	Bug	Correctly define and clean up log and work folders when running via CLI tool.
	2020/12/07	Feature	Set a maximum threshold for the items included in an XML validation report.
	2020/12/04	Improvement	Improve the internal definition of common dependencies.
	2020/12/04	Improvement	Allow PDF validation reports to be produced by command line tools.
	2020/12/04	Bug	Correctly clean up temporary files on CLI tool exit.
	2020/11/11	Feature	Support the configurable loading of OWL imports as part of the provided input when forming the complete graph to validate.
	2020/10/23	Improvement	Display error messages from custom plugins.
	2020/10/08	Improvement	Adapt REST API documentation via Swagger and Hydra.
	2020/10/07	Feature	Add REST API operation to retrieve the information on the validator’s all available validation domains.
	2020/10/07	Bug	Correct the name of the configuration property used to define a specific log output folder.
	2020/09/30	Feature	Set a maximum threshold for report items displayed on the web user interface.
	2020/09/30	Improvement	Add spinners on report download buttons of the web user interface to indicate a pending download.
	2020/09/28	Feature	Support validation type options as an extra configuration level over validation types.
	2020/09/28	Improvement	Always display on the web user interface external artefact inputs if supported the same way by all validation types.
	2020/09/24	Improvement	Improved reporting for unexpected errors.
	2020/09/23	Bug	Validation report may fail to open when accessing via the minimal web user interface.
	2020/09/21	Bug	The web interface’s code editor may fail to correctly highlight the offending line in the validated content.
	2020/09/21	Bug	The web interface should not display a report item as clickable if it cannot be highlighted in the validated content.
	2020/09/16	Improvement	Improve internal support for handling UI events and configuration loading.
	2020/09/15	Improvement	Upgrade library dependencies.
	2020/09/14	Bug	Replace the mention of “schemas” with “artefacts” in error messages.
	2020/07/13	Improvement	Improve the handling of user-provided validation artefacts.
	2020/07/02	Improvement	Improve support for custom validator plugins to harmonise approach with other validators.
	2020/06/30	Improvement	Extract common validator components to shared libraries.
	2020/06/16	Feature	Support custom validator plugins to extend the capabilities of the validator beyond the limitations of its validation artefacts.
	2020/06/16	Feature	Allow configuration of whether or not the input data graph should be merged with the shapes graph before validation to benefit from the shapes graph additional context.
	2020/05/19	Improvement	Improved the efficiency of the validation report’s generation.
	2020/05/18	Bug	Correct the cleanup of temporary resources.
	2020/05/15	Improvement	Upgrade of internal library dependencies.
	2020/04/03	Improvement	Allow report item labels to be customised via configuration.
	2020/03/25	Bug	Correct the caching of remote validation artefacts.
	2020/03/04	Improvement	Increase the maximum file upload limit for the validator’s web application.
	2020/02/14	Improvement	Ensure that SHACL file imports are recursively included in the shapes graph.
	2020/02/13	Improvement	Include all imported SHACL files in the shape graph to use for the validation.
	2020/02/05	Improvement	Enable wildcard CORS for the validator to simplify its integration.
	2020/02/05	Improvement	Clean up the descriptions of report items to not include empty placeholders.
	2020/01/08	Bug	Report validation type errors consistently across the validator’s SOAP and REST APIs.
	2020/01/08	Bug	Correct the lookup of offending values for all shape target types.
	2019/11/29	Feature	Support configurable banner and footer HTML blocks for the validator’s web UI.
	2019/09/03	Improvement	Improved busy indicator for minimal web UI.
	2019/09/02	Improvement	Style improvements for the validator’s minimal web UI.
	2019/09/02	Feature	Support for the validator to be used as a CLI tool.
	2019/07/03	Improvement	Include the validation type in the web UI’s result screen and allow all UI labels to be configurable.
	2019/07/02	Bug	Error handling, reporting and styling fixes.
	2019/06/25	Improvement	Improve reporting for invalid user-provided shapes.
	2019/06/24	Feature	Support for ordering of report items.
	2019/06/21	Feature	Allow validation report to be downloaded in various syntaxes, including PDF.
	2019/06/21	Feature	Support for inputs to be provided via file, URI or via editor.
	2019/06/03	Improvement	Improved the fault tolerance for the generation of XML validation reports.
	2019/05/28	Improvement	Improved error reporting for unexpected errors.
	2019/05/27	Feature	Support web user interface for the validator.
	2019/05/13	Improvement	Migration to Java 11.
	2019/05/13	Feature	Support SOAP API for the validator.
	2019/04/23	Feature	Extend REST API to support batch validation and user-provided SHACL shapes.
	2019/04/08	Feature	Document validator REST API via Hydra.
	2019/04/08	Feature	Support configuration of remote SHACL files.
	2019/04/03	Feature	Support input to be provided as a BASE64-encoded string.
	2019/04/01	Bug	Correctly handle the configured default reporting syntax.
	2019/03/29	Improvement	Documentation extensions and improved Docker support.
	2019/03/27	Feature	Initial version.

How to determine my validator’s version or build timestamp?

The version information of your validator (be it a full web application instance or a command line tool), can be inspected in any of the following ways:

• If the validator UI is enabled, viewing the HTML source of the validator’s web page. This includes at the end hidden divs with ids build.version and build.timestamp.

• If using via Docker, inspecting the Docker image tag and history. For the build timestamp, consider the top line returned by docker history --format "{{.CreatedAt}}" isaitb/shacl-validator:latest.

• If using via JAR file, inspecting its manifest file. This can be found in the JAR at /META-INF/MANIFEST.MF and locating properties Implementation-Version and Build-Timestamp.

• If using via JAR file, inspecting the validator’s configuration file. This can be found in the JAR at /BOOT-INF/lib/shaclvalidator-common-1.0.0-SNAPSHOT.jar/application.properties and locating properties validator.buildVersion and validator.buildTimestamp.

In all cases, the timestamp is formatted as yyyy-MM-dd HH:mm:ss (YEAR-MONTH-DAY HOUR:MINUTE:SECOND) such as “2021-03-20 14:50:30”, followed by the relevant timezone (“Z” for UTC).