Guide: Creating a test suite

Track

Test bed setup

This guide walks you through the process of creating a simple GITB TDL test suite.

What you will achieve

At the end of this guide you will have created a GITB TDL test case bundled in a test suite, ready to be uploaded to the test bed. The test case you will implement will include:

  • Requesting a user to upload a file.

  • Validating content.

The resulting test suite will be self-contained, requiring only minimal configuration on the side of the test bed to be used.

What you will need

  • About 15 minutes.

  • An XML or text editor.

  • A tool to create ZIP archives.

  • A basic understanding of XML and XML Schema (XSD).

  • The GITB TDL documentation (optionally, to lookup further documentation and examples).

How to complete this guide

This guide involves the creation of a test suite and its contained test cases. There are no prerequisite actions needed and the content of all the files you will create is provided in each step.

Steps

Carry out the following steps to complete this guide.

Step 1: Define your testing needs

The first thing to do is to define what you are testing. The fact that you are creating a test suite means that you are part of a project that is creating or reusing specifications that your eventual users will need to implement. These can relate to the communication protocol between remote parties or the content they exchange (or both). For the purpose of this guide we will consider that your project is an EU cross-border initiative to define a new common specification for the exchange of purchase orders between retailers.

To specify the content of purchase orders your experts have created the following XML Schema:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" targetNamespace="http://itb.ec.europa.eu/sample/po.xsd" xmlns="http://itb.ec.europa.eu/sample/po.xsd" elementFormDefault="qualified">

  <xs:element name="purchaseOrder" type="PurchaseOrderType"/>

  <xs:element name="comment" type="xs:string"/>

  <xs:complexType name="PurchaseOrderType">
    <xs:sequence>
      <xs:element name="shipTo" type="Address"/>
      <xs:element name="billTo" type="Address"/>
      <xs:element ref="comment" minOccurs="0"/>
      <xs:element name="items"  type="Items"/>
    </xs:sequence>
    <xs:attribute name="orderDate" type="xs:date"/>
  </xs:complexType>

  <xs:complexType name="Address">
    <xs:sequence>
      <xs:element name="name"   type="xs:string"/>
      <xs:element name="street" type="xs:string"/>
      <xs:element name="city"   type="xs:string"/>
      <xs:element name="zip"    type="xs:decimal"/>
    </xs:sequence>
    <xs:attribute name="country" type="CountryType" use="required"/>
  </xs:complexType>

  <xs:complexType name="Items">
    <xs:sequence>
      <xs:element name="item" minOccurs="0" maxOccurs="unbounded">
        <xs:complexType>
          <xs:sequence>
            <xs:element name="productName" type="xs:string"/>
            <xs:element name="quantity" type="xs:positiveInteger"/>
            <xs:element name="priceEUR"    type="xs:decimal"/>
            <xs:element ref="comment"   minOccurs="0"/>
          </xs:sequence>
          <xs:attribute name="partNum" type="xs:string" use="required"/>
        </xs:complexType>
      </xs:element>
    </xs:sequence>
  </xs:complexType>

  <xs:simpleType name="CountryType">
    <xs:restriction base="xs:string">
      <xs:pattern value="[A-Z]{2}"/>
    </xs:restriction>
  </xs:simpleType>

</xs:schema>

Based on this, a sample purchase order would be as follows:

<?xml version="1.0"?>
<purchaseOrder xmlns="http://itb.ec.europa.eu/sample/po.xsd" orderDate="2018-01-22">
  <shipTo country="BE">
    <name>John Doe</name>
    <street>Europa Avenue 123</street>
    <city>Brussels</city>
    <zip>1000</zip>
  </shipTo>
  <billTo country="BE">
    <name>Jane Doe</name>
    <street>Europa Avenue 210</street>
    <city>Brussels</city>
    <zip>1000</zip>
  </billTo>
  <comment>Send in one package please</comment>
  <items>
    <item partNum="XYZ-123876">
      <productName>Mouse</productName>
      <quantity>1</quantity>
      <priceEUR>15.99</priceEUR>
      <comment>Confirm this is wireless</comment>
    </item>
    <item partNum="ABC-32478">
      <productName>Keyboard</productName>
      <quantity>1</quantity>
      <priceEUR>25.50</priceEUR>
    </item>
  </items>
</purchaseOrder>

For the current guide we will consider that the communication protocol with which such purchase orders are exchanged is out of scope.

The requirement for your project is to provide a conformance testing service to the retailer systems’ developers to facilitate their implementation of the new specification. To help with this you need to foresee a service through which developers can upload a generated purchase order for validation based on the defined XML Schema.

Note

Further test capabilities: The test bed and the GITB TDL can be used to realise a multitude of testing needs involving complex flows and virtually any form of content or communication exchange. This tutorial focuses on XML and direct upload to keep things simple but you can always check out further possibilities in the documentation on GITB TDL and GITB Test Services.

Step 2: Design your test suite

Before you start creating the test suite you need to determine your specification’s actors. These are the parties that are directly or indirectly defined by your specification that will be involved in your test suite’s test cases. To address your current needs we will define a single actor named Retailer. This represents the retailer’s system that will need to conform to the specifications.

In terms of the structure of the test suite, we will create a single test case for the Retailer actor that will prompt the user to upload a purchase order for validation.

Step 3: Create the test suite

Start off by creating a folder in your workspace (assumed e.g. to be /workspace to contain your test suite resources:

/workspace/testSuite1

Within this folder create file testSuite1.xml. This is the file that identifies your test suite (you could have multiple for a given specification), defines the actors it involves and the test cases it contains (to distinguish them from other resources). The content of testSuite1.xml is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite id="testSuite1" xmlns="http://www.gitb.com/tdl/v1/" xmlns:gitb="http://www.gitb.com/core/v1/">
    <metadata>
        <gitb:name>[TS1] Purchase order validation tests</gitb:name>
        <gitb:description>Test suite to validate the EU purchase order specification.</gitb:description>
        <gitb:version>1.0</gitb:version>
    </metadata>
    <actors>
        <gitb:actor id="Retailer">
            <gitb:name>Retailer</gitb:name>
            <gitb:desc>The EU retailer system that needs to be capable of producing and processing purchase orders.</gitb:desc>
        </gitb:actor>
    </actors>
    <testcase id="testCase1_upload"/>
</testsuite>

Lets see what this file defines:

  • It identifies the test suite as testSuite1 with a description to understand its purpose. This information will be used in the test bed interface when managing the test suite.

  • In the actors section there is a single actor defined named Retailer to match our design. The name and description of the actor are provided here to avoid configuring them in the test bed (for actors already defined in the test bed you could specify only the id).

  • A single test case entry is specified identified as testCase1_upload. This will be defined next.

Step 4: Create the test case

The next step is to create the test case to validate the uploaded purchase order (the one referred to as testCase1_upload in the test suite).

Within the test suite folder create a subfolder named tests and within it file testCase1.xml with the following content:

<?xml version="1.0" encoding="UTF-8"?>
<testcase id="testCase1_upload" xmlns="http://www.gitb.com/tdl/v1/" xmlns:gitb="http://www.gitb.com/core/v1/">
    <metadata>
        <gitb:name>[TC1] Upload purchase order</gitb:name>
        <gitb:version>1.0</gitb:version>
        <gitb:description>Test case that allows the developer of an EU retailer system to upload a purchase order for validation.</gitb:description>
    </metadata>
    <imports>
        <artifact type="schema" encoding="UTF-8" name="purchaseOrderXSD">resources/PurchaseOrder.xsd</artifact>
    </imports>
    <actors>
        <gitb:actor id="Retailer" name="Retailer" role="SUT"/>
    </actors>
    <steps>
        <interact id="userData" desc="Upload purchase order">
            <request desc="Purchase order to validate:" name="order" inputType="UPLOAD"/>
        </interact>
        <verify handler="XmlValidator" desc="Validate purchase order">
            <input name="xml">$userData{order}</input>
            <input name="xsd">$purchaseOrderXSD</input>
        </verify>
    </steps>
</testcase>

This test case specifies the following:

  • Its identifier, name and metadata. It is important that testCase1_upload is set for the id attribute as it is based on this that the file will be looked up and associated to the test suite. The other metadata is there to present an appropriate description to users in the test bed.

  • In the imports section we define the XML Schema to be used for the validation. We will store this in a purchaseOrderXSD variable of type schema for further referencing.

  • In the actors section the Retailer actor is set as the test case’s SUT (System Under Test), i.e. what is being tested.

  • The steps section contains the test case’s individual steps.

The first step this test case will execute is an interact, specifically containing a request. This is the way in which input is requested from the user executing the test case. Considering that the input is a file (based on the inputType being UPLOAD), this will result in a popup with a file upload field. The uploaded file is named order and is stored in the step’s output (named userData - the step’s id).

The second and last step of this test case is the validation of the uploaded content through the verify step. The handler attribute identifies the implementation that will carry out the validation. This could be a remote service or an implementation provided by the test bed itself (also referred to as an embedded handler). Considering that we have a simple XSD validation, we use the XmlValidator that expects to be provided the XML content to validate as well as the XSD to validate against. These are both supplied through the input parameters xml and xsd that respectively receive values from variables userData{order} and purchaseOrderXSD. Note that referring to existing variables is done by prefixing them with the $ sign (to signify a variable reference).

Note

Complex XML validation: XML validation often requires XSD files to check syntax and Schematron files to check content. It could be best in such cases to decouple validation to an external validation service. See our XML validation guide for detailed instructions on how to do this.

The last point to address is making available for the test case the PurchaseOrder.xsd file. The test case defines this in its imports section as follows:

<imports>
    <artifact type="schema" encoding="UTF-8" name="purchaseOrderXSD">resources/PurchaseOrder.xsd</artifact>
</imports>

To make this available, create folder resources within the test suite folder and within this copy file PurchaseOrder.xsd (available [here]).

Note

Artifact paths: The paths of imported resources are always provided as relative to the test suite root folder.

Step 5: Package the test suite

At this point you have created all resources needed for your test suite. To recap, the structure of your test suite folder needs to match the following:

testSuite1
├── resources
│   └── PurchaseOrder.xsd
├── tests
│   └── testCase1.xml
└── testSuite.xml

To complete the creation of your test suite, ZIP the contents of the testSuite1 folder, without including the testSuite1 folder itself. Name the ZIP archive testSuite1.zip (the resulting archive should match the one available here).

Your test suite is now ready to be uploaded to the test bed. Note how you name and organise the test suite’s files is something entirely up to you. Upon upload the test bed will inspect all ZIP archive entries and determine the purpose of each resource. You only need to follow the following basic rules:

  • There needs to be a single test suite definition file.

  • Any test cases referenced by the test suite must match (based on their ID) test case files present in the archive.

  • Reference paths for resources in e.g. test case imports, must be valid.

In any case the above, and multiple other checks, will be verified when you upload your test suite.

Summary

Congratulations! You have just created a GITB TDL test suite to validate user-uploaded content. In doing so you were introduced to the basic structure of a test suite as well as the content of the test suite and test case files. In the test case you learnt how to request input from users, store it and then reference it for further use. You also saw an example of importing a resource and of using one of the test bed’s embedded validation handlers to validate content.

See also

Check out the complete GITB TDL documentation for additional information on the GITB TDL, the reference of all GITB TDL constructs used as well as numerous samples and complete examples.

A good next step from here will be to extend the test suite you just built by including the handling of messaging to and from the test bed. This is provided in Guide: Developing complex tests that also goes into details on setting up an efficient development workflow. Alternatively if you want to try out the test suite you built you can check out the following: