Guide: Creating a test suite

Track Test bed setup

This guide walks you through the process of creating a simple GITB TDL test suite.

What you will achieve

At the end of this guide you will have created a GITB TDL test case bundled in a test suite, ready to be uploaded to the test bed. The test case you will implement will include:

  • Requesting a user to upload a file.
  • Validating content.

The resulting test suite will be self-contained, requiring only minimal configuration on the side of the test bed to be used.

What you will need

  • About 15 minutes.
  • An XML or text editor.
  • A tool to create ZIP archives.
  • A basic understanding of XML and XML Schema (XSD).
  • The GITB TDL documentation (optionally, to lookup further documentation and examples).

How to complete this guide

This guide involves the creation of a test suite and its contained test cases. There are no prerequisite actions needed and the content of all the files you will create is provided in each step.

Steps

Carry out the following steps to complete this guide.

Step 1: Define your testing needs

The first thing to do is to define what you are testing. The fact that you are creating a test suite means that you are part of a project that is creating or reusing specifications that your eventual users will need to implement. These can relate to the communication protocol between remote parties or the content they exchange (or both). For the purpose of this guide we will consider that your project is an EU cross-border initiative to define a new common specification for the exchange of purchase orders between retailers.

To specify the content of purchase orders your experts have created the following XML Schema:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" targetNamespace="http://itb.ec.europa.eu/sample/po.xsd" xmlns="http://itb.ec.europa.eu/sample/po.xsd" elementFormDefault="qualified">

  <xs:element name="purchaseOrder" type="PurchaseOrderType"/>

  <xs:element name="comment" type="xs:string"/>

  <xs:complexType name="PurchaseOrderType">
    <xs:sequence>
      <xs:element name="shipTo" type="Address"/>
      <xs:element name="billTo" type="Address"/>
      <xs:element ref="comment" minOccurs="0"/>
      <xs:element name="items"  type="Items"/>
    </xs:sequence>
    <xs:attribute name="orderDate" type="xs:date"/>
  </xs:complexType>

  <xs:complexType name="Address">
    <xs:sequence>
      <xs:element name="name"   type="xs:string"/>
      <xs:element name="street" type="xs:string"/>
      <xs:element name="city"   type="xs:string"/>
      <xs:element name="zip"    type="xs:decimal"/>
    </xs:sequence>
    <xs:attribute name="country" type="CountryType" use="required"/>
  </xs:complexType>

  <xs:complexType name="Items">
    <xs:sequence>
      <xs:element name="item" minOccurs="0" maxOccurs="unbounded">
        <xs:complexType>
          <xs:sequence>
            <xs:element name="productName" type="xs:string"/>
            <xs:element name="quantity" type="xs:positiveInteger"/>
            <xs:element name="priceEUR"    type="xs:decimal"/>
            <xs:element ref="comment"   minOccurs="0"/>
          </xs:sequence>
          <xs:attribute name="partNum" type="xs:string" use="required"/>
        </xs:complexType>
      </xs:element>
    </xs:sequence>
  </xs:complexType>

  <xs:simpleType name="CountryType">
    <xs:restriction base="xs:string">
      <xs:pattern value="[A-Z]{2}"/>
    </xs:restriction>
  </xs:simpleType>

</xs:schema>

Based on this, a sample purchase order would be as follows:

<?xml version="1.0"?>
<purchaseOrder xmlns="http://itb.ec.europa.eu/sample/po.xsd" orderDate="2018-01-22">
  <shipTo country="BE">
    <name>John Doe</name>
    <street>Europa Avenue 123</street>
    <city>Brussels</city>
    <zip>1000</zip>
  </shipTo>
  <billTo country="BE">
    <name>Jane Doe</name>
    <street>Europa Avenue 210</street>
    <city>Brussels</city>
    <zip>1000</zip>
  </billTo>
  <comment>Send in one package please</comment>
  <items>
    <item partNum="XYZ-123876">
      <productName>Mouse</productName>
      <quantity>1</quantity>
      <priceEUR>15.99</priceEUR>
      <comment>Confirm this is wireless</comment>
    </item>
    <item partNum="ABC-32478">
      <productName>Keyboard</productName>
      <quantity>1</quantity>
      <priceEUR>25.50</priceEUR>
    </item>
  </items>
</purchaseOrder>

For the current guide we will consider that the communication protocol with which such purchase orders are exchanged is out of scope.

The requirement for your project is to provide a conformance testing service to the retailer systems’ developers to facilitate their implementation of the new specification. To help with this you need to foresee a service through which developers can upload a generated purchase order for validation based on the defined XML Schema.

Note

Further test capabilities: The test bed and the GITB TDL can be used to realise a multitude of testing needs involving complex flows and virtually any form of content or communication exchange. This tutorial focuses on XML and direct upload to keep things simple but you can always check out further possibilities in the documentation on GITB TDL and GITB Test Services.

Step 2: Design your test suite

Before you start creating the test suite you need to determine your specification’s actors. These are the parties that are directly or indirectly defined by your specification that will be involved in your test suite’s test cases. To address your current needs we will define a single actor named Retailer. This represents the retailer’s system that will need to conform to the specifications.

In terms of the structure of the test suite, we will create a single test case for the Retailer actor that will prompt the user to upload a purchase order for validation.

Step 3: Create the test suite

Start off by creating a folder in your workspace (assumed e.g. to be /workspace to contain your test suite resources:

/workspace/testSuite1

Within this folder create file testSuite1.xml. This is the file that identifies your test suite (you could have multiple for a given specification), defines the actors it involves and the test cases it contains (to distinguish them from other resources). The content of testSuite1.xml is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite id="testSuite1" xmlns="http://www.gitb.com/tdl/v1/" xmlns:gitb="http://www.gitb.com/core/v1/">
    <metadata>
        <gitb:name>testSuite1</gitb:name>
        <gitb:description>Test suite to validate the EU purchase order specification.</gitb:description>
        <gitb:version>1.0</gitb:version>
    </metadata>
    <actors>
        <gitb:actor id="Retailer">
            <gitb:name>Retailer</gitb:name>
            <gitb:desc>The EU retailer system that needs to be capable of producing and processing purchase orders.</gitb:desc>
        </gitb:actor>
    </actors>
    <testcase id="testCase1_upload"/>
</testsuite>

Lets see what this file defines:

  • It identifies the test suite as testSuite1 with a description to understand its purpose. This information will be used in the test bed interface when managing the test suite.
  • In the actors section there is a single actor defined named Retailer to match our design. The name and description of the actor are provided here to avoid configuring them in the test bed (for actors already defined in the test bed you could specify only the id).
  • A single test case entry is specified identified as testCase1_upload. This will be defined next.

Step 4: Create the test case

The next step is to create the test case to validate the uploaded purchase order (the one referred to as testCase1_upload in the test suite).

Within the test suite folder create file testCase1.xml with the following content:

<?xml version="1.0" encoding="UTF-8"?>
<testcase id="testCase1_upload" xmlns="http://www.gitb.com/tdl/v1/" xmlns:gitb="http://www.gitb.com/core/v1/">
    <metadata>
        <gitb:name>testCase1_upload</gitb:name>
        <gitb:version>1.0</gitb:version>
        <gitb:description>Test case that allows the developer of an EU retailer system to upload a purchase order for validation.</gitb:description>
    </metadata>
    <imports>
        <artifact type="schema" encoding="UTF-8" name="purchaseOrderXSD">testSuite1/artifacts/PurchaseOrder.xsd</artifact>
    </imports>
    <variables>
        <var name="purchaseOrderToValidate" type="binary"/>
    </variables>
    <actors>
        <gitb:actor id="Retailer" name="Retailer" role="SUT"/>
    </actors>
    <steps>
        <interact desc="Upload content">
            <request desc="Purchase order to validate:">$purchaseOrderToValidate</request>
        </interact>
        <verify handler="XSDValidator" desc="Validate purchase order">
            <input name="xmldocument">$purchaseOrderToValidate</input>
            <input name="xsddocument">$purchaseOrderXSD</input>
        </verify>
    </steps>
</testcase>

This test case specifies the following:

  • Its identifier, name and metadata. It is important that testCase1_upload is set for the id attribute as it is based on this that the file will be looked up and associated to the test suite. The other metadata is there to present an appropriate description to users in the test bed.
  • In the imports section we define the XML Schema to be used for the validation. We will store this in a purchaseOrderXSD variable of type schema for further referencing.
  • We define variable purchaseOrderToValidate that will contain the content the user will upload for validation.
  • In the actors section the Retailer actor is set as the test case’s SUT (System Under Test), i.e. what is being tested.
  • The steps section contains the test case’s individual steps.

The first step this test case will execute is an interact, specifically containing a request. This is the way in which input is requested from the user executing the test case. Considering that the input is a file (based on the the binary type of the referenced purchaseOrderToValidate variable), this will result in a popup with a file upload field. The uploaded file is stored in variable purchaseOrderToValidate. Note that the request element expects an expression for its value which is why it starts with the $ sign (to signify a variable reference).

The second and last step of this test case is the validation of the uploaded content through the verify step. The handler attribute identifies the implementation that will carry out the validation. This could be a remote service or an implementation provided by the test bed itself (also referred to as an embedded handler). Considering that we have only a single XSD file to validate against, we use the XSDValidator that expects to be provided the XML content to validate as well as the XSD to validate against. These are both supplied through the input parameters xmldocument and xsddocument that respectively receive the values of the purchaseOrderToValidate and purchaseOrderXSD variables (again referenced through expressions).

Note

Complex XML validation: XML validation often requires multiple XSD files to check syntax and numerous Schematron files to check content. In such cases it is advised to decouple validation to an external validation service. See also [REF] on the approach to set such a service up.

The last point to complete is to make available to the test case the PurchaseOrder.xsd file that will be passed to the XSDValidator. The test case defines this in its imports section as follows:

<imports>
    <artifact type="schema" encoding="UTF-8" name="purchaseOrderXSD">testSuite1/artifacts/PurchaseOrder.xsd</artifact>
</imports>

To make this available, create folder artifacts within the test suite folder and within this copy file PurchaseOrder.xsd (available [here]).

Note

Artifact path: Notice that the path of the artifact always starts with the identifier of the test suite, in this case testSuite1.

Step 5: Package the test suite

At this point you have created all resources needed for your test suite. To recap, the structure of your test suite folder needs to match the following:

testSuite1
 |
 +-- artifacts
 |    |
 |    +-- PurchaseOrder.xsd
 |
 +-- testCase1.xml
 +-- testSuite.xml

To complete the creation of your test suite, ZIP the contents of the testSuite1 folder, without including the testSuite1 folder itself. Name the ZIP archive testSuite1.zip (the resulting archive should match the one available here).

Your test suite is now ready to be uploaded to the test bed.

Summary

Congratulations! You have just created a GITB TDL test suite to validate user-uploaded content. In doing so you were introduced to the basic structure of a test suite as well as the content of the test suite and test case files. In the test case you learnt how to request input from users, store it and then reference it for further use. You also saw an example of importing a resource and of using one of the test bed’s embedded validation handlers to validate content.

See also

Check out the complete GITB TDL documentation for additional information on the GITB TDL, the reference of all GITB TDL constructs used as well as numerous samples and complete examples.

A good next step from here will be to extend the test suite you just built by including the handling of messaging to and from the test bed. This is provided in Guide: Messaging in test cases. Alternatively if you want to try out the test suite you built you can check out the following: