Guide: Creating a test suite
Track |
---|
This guide walks you through the process of creating a simple GITB TDL test suite.
What you will achieve
At the end of this guide you will have created a GITB TDL test case bundled in a test suite, ready to be uploaded to the test bed. The test case you will implement will include:
Requesting a user to upload a file.
Validating content.
The resulting test suite will be self-contained, requiring only minimal configuration on the side of the test bed to be used.
What you will need
About 15 minutes.
An XML or text editor.
A tool to create ZIP archives.
A basic understanding of XML and XML Schema (XSD).
The GITB TDL documentation (optionally, to lookup further documentation and examples).
How to complete this guide
This guide involves the creation of a test suite and its contained test cases. There are no prerequisite actions needed and the content of all the files you will create is provided in each step.
Steps
Carry out the following steps to complete this guide.
Step 1: Define your testing needs
The first thing to do is to define what you are testing. The fact that you are creating a test suite means that you are part of a project that is creating or reusing specifications that your eventual users will need to implement. These can relate to the communication protocol between remote parties or the content they exchange (or both). For the purpose of this guide we will consider that your project is an EU cross-border initiative to define a new common specification for the exchange of purchase orders between retailers.
To specify the content of purchase orders your experts have created the following XML Schema:
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" targetNamespace="http://itb.ec.europa.eu/sample/po.xsd" xmlns="http://itb.ec.europa.eu/sample/po.xsd" elementFormDefault="qualified">
<xs:element name="purchaseOrder" type="PurchaseOrderType"/>
<xs:element name="comment" type="xs:string"/>
<xs:complexType name="PurchaseOrderType">
<xs:sequence>
<xs:element name="shipTo" type="Address"/>
<xs:element name="billTo" type="Address"/>
<xs:element ref="comment" minOccurs="0"/>
<xs:element name="items" type="Items"/>
</xs:sequence>
<xs:attribute name="orderDate" type="xs:date"/>
</xs:complexType>
<xs:complexType name="Address">
<xs:sequence>
<xs:element name="name" type="xs:string"/>
<xs:element name="street" type="xs:string"/>
<xs:element name="city" type="xs:string"/>
<xs:element name="zip" type="xs:decimal"/>
</xs:sequence>
<xs:attribute name="country" type="CountryType" use="required"/>
</xs:complexType>
<xs:complexType name="Items">
<xs:sequence>
<xs:element name="item" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="productName" type="xs:string"/>
<xs:element name="quantity" type="xs:positiveInteger"/>
<xs:element name="priceEUR" type="xs:decimal"/>
<xs:element ref="comment" minOccurs="0"/>
</xs:sequence>
<xs:attribute name="partNum" type="xs:string" use="required"/>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
<xs:simpleType name="CountryType">
<xs:restriction base="xs:string">
<xs:pattern value="[A-Z]{2}"/>
</xs:restriction>
</xs:simpleType>
</xs:schema>
Based on this, a sample purchase order would be as follows:
<?xml version="1.0"?>
<purchaseOrder xmlns="http://itb.ec.europa.eu/sample/po.xsd" orderDate="2018-01-22">
<shipTo country="BE">
<name>John Doe</name>
<street>Europa Avenue 123</street>
<city>Brussels</city>
<zip>1000</zip>
</shipTo>
<billTo country="BE">
<name>Jane Doe</name>
<street>Europa Avenue 210</street>
<city>Brussels</city>
<zip>1000</zip>
</billTo>
<comment>Send in one package please</comment>
<items>
<item partNum="XYZ-123876">
<productName>Mouse</productName>
<quantity>1</quantity>
<priceEUR>15.99</priceEUR>
<comment>Confirm this is wireless</comment>
</item>
<item partNum="ABC-32478">
<productName>Keyboard</productName>
<quantity>1</quantity>
<priceEUR>25.50</priceEUR>
</item>
</items>
</purchaseOrder>
For the current guide we will consider that the communication protocol with which such purchase orders are exchanged is out of scope.
The requirement for your project is to provide a conformance testing service to the retailer systems’ developers to facilitate their implementation of the new specification. To help with this you need to foresee a service through which developers can upload a generated purchase order for validation based on the defined XML Schema.
Note
Further test capabilities: The test bed and the GITB TDL can be used to realise a multitude of testing needs involving complex flows and virtually any form of content or communication exchange. This tutorial focuses on XML and direct upload to keep things simple but you can always check out further possibilities in the documentation on GITB TDL and GITB Test Services.
Step 2: Design your test suite
Before you start creating the test suite you need to determine your specification’s actors. These are the parties that are directly or indirectly defined by your specification that will be involved in your test suite’s test cases. To address your current needs we will define a single actor named Retailer. This represents the retailer’s system that will need to conform to the specifications.
In terms of the structure of the test suite, we will create a single test case for the Retailer actor that will prompt the user to upload a purchase order for validation.
Step 3: Create the test suite
Start off by creating a folder in your workspace (assumed e.g. to be /workspace
to contain your test suite resources:
/workspace/testSuite1
Within this folder create file testSuite1.xml
. This is the file that identifies your test suite (you could have multiple for a given
specification), defines the actors it involves and the test cases it contains (to distinguish them from other resources). The content
of testSuite1.xml
is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<testsuite id="testSuite1" xmlns="http://www.gitb.com/tdl/v1/" xmlns:gitb="http://www.gitb.com/core/v1/">
<metadata>
<gitb:name>[TS1] Purchase order validation tests</gitb:name>
<gitb:description>Test suite to validate the EU purchase order specification.</gitb:description>
<gitb:version>1.0</gitb:version>
</metadata>
<actors>
<gitb:actor id="Retailer">
<gitb:name>Retailer</gitb:name>
<gitb:desc>The EU retailer system that needs to be capable of producing and processing purchase orders.</gitb:desc>
</gitb:actor>
</actors>
<testcase id="testCase1_upload"/>
</testsuite>
Lets see what this file defines:
It identifies the test suite as
testSuite1
with a description to understand its purpose. This information will be used in the test bed interface when managing the test suite.In the
actors
section there is a single actor defined namedRetailer
to match our design. Thename
anddescription
of the actor are provided here to avoid configuring them in the test bed (for actors already defined in the test bed you could specify only theid
).A single test case entry is specified identified as
testCase1_upload
. This will be defined next.
Step 4: Create the test case
The next step is to create the test case to validate the uploaded purchase order (the one referred to as testCase1_upload
in the test suite).
Within the test suite folder create a subfolder named tests
and within it file testCase1.xml
with the following content:
<?xml version="1.0" encoding="UTF-8"?>
<testcase id="testCase1_upload" xmlns="http://www.gitb.com/tdl/v1/" xmlns:gitb="http://www.gitb.com/core/v1/">
<metadata>
<gitb:name>[TC1] Upload purchase order</gitb:name>
<gitb:version>1.0</gitb:version>
<gitb:description>Test case that allows the developer of an EU retailer system to upload a purchase order for validation.</gitb:description>
</metadata>
<imports>
<artifact type="schema" encoding="UTF-8" name="purchaseOrderXSD">resources/PurchaseOrder.xsd</artifact>
</imports>
<actors>
<gitb:actor id="Retailer" name="Retailer" role="SUT"/>
</actors>
<steps>
<interact id="userData" desc="Upload purchase order">
<request desc="Purchase order to validate:" name="order" inputType="UPLOAD"/>
</interact>
<verify handler="XmlValidator" desc="Validate purchase order">
<input name="xml">$userData{order}</input>
<input name="xsd">$purchaseOrderXSD</input>
</verify>
</steps>
<output>
<success>
<default>"Test session succeeded."</default>
</success>
<failure>
<default>"Test session failed. Refer to the failed step's report for more details."</default>
</failure>
</output>
</testcase>
This test case specifies the following:
Its identifier, name and metadata. It is important that
testCase1_upload
is set for theid
attribute as it is based on this that the file will be looked up and associated to the test suite. The other metadata is there to present an appropriate description to users in the test bed.In the
imports
section we define the XML Schema to be used for the validation. We will store this in apurchaseOrderXSD
variable of typeschema
for further referencing.In the
actors
section theRetailer
actor is set as the test case’sSUT
(System Under Test), i.e. what is being tested.The
steps
section contains the test case’s individual steps.The
output
section contains customisable user-friendly messages on the overall output.
The first step this test case will execute is an interact
, specifically containing a request
. This is the way in which input is requested
from the user executing the test case. Considering that the input is a file (based on the inputType
being UPLOAD
),
this will result in a popup with a file upload field. The uploaded file is named order
and is stored in the step’s output (named userData
- the step’s id
).
The second and last step of this test case is the validation of the uploaded content through the verify
step. The handler
attribute identifies the
implementation that will carry out the validation. This could be a remote service or an implementation provided by the test bed itself (also referred to as an
embedded handler). Considering that we have a simple XSD validation, we use the XmlValidator that expects to be provided the
XML content to validate as well as the XSD to validate against. These are both supplied through the input parameters xml
and xsd
that respectively receive values from variables userData{order}
and purchaseOrderXSD
. Note that referring to existing variables
is done by prefixing them with the $
sign (to signify a variable reference).
Note
Complex XML validation: XML validation often requires XSD files to check syntax and Schematron files to check content. It could be best in such cases to decouple validation to an external validation service. See our XML validation guide for detailed instructions on how to do this.
The last point to address is making available for the test case the PurchaseOrder.xsd
file. The test case defines this in its imports
section as follows:
<imports>
<artifact type="schema" encoding="UTF-8" name="purchaseOrderXSD">resources/PurchaseOrder.xsd</artifact>
</imports>
To make this available, create folder resources
within the test suite folder and within this copy file PurchaseOrder.xsd
(available
[here
]).
Note
Artifact paths: The paths of imported resources are always provided as relative to the test suite root folder.
Step 5: Package the test suite
At this point you have created all resources needed for your test suite. To recap, the structure of your test suite folder needs to match the following:
testSuite1
├── resources
│ └── PurchaseOrder.xsd
├── tests
│ └── testCase1.xml
└── testSuite.xml
To complete the creation of your test suite, ZIP the contents of the testSuite1
folder, without including the testSuite1
folder itself. Name the ZIP
archive testSuite1.zip
(the resulting archive should match the one available here
).
Your test suite is now ready to be uploaded to the test bed. Note how you name and organise the test suite’s files is something entirely up to you. Upon upload the test bed will inspect all ZIP archive entries and determine the purpose of each resource. You only need to follow the following basic rules:
There needs to be a single test suite definition file.
Any test cases referenced by the test suite must match (based on their ID) test case files present in the archive.
Reference paths for resources in e.g. test case imports, must be valid.
In any case the above, and multiple other checks, will be verified when you upload your test suite.
Summary
Congratulations! You have just created a GITB TDL test suite to validate user-uploaded content. In doing so you were introduced to the basic structure of a test suite as well as the content of the test suite and test case files. In the test case you learnt how to request input from users, store it and then reference it for further use. You also saw an example of importing a resource and of using one of the test bed’s embedded validation handlers to validate content.
See also
Check out the complete GITB TDL documentation for additional information on the GITB TDL, the reference of all GITB TDL constructs used as well as numerous samples and complete examples.
A good next step from here would be to extend the test suite you just built by including the handling of messaging to and from the test bed. This is covered in two further step-by-step tutorials:
Guide: Basic test case messaging covering simple message exchange tests.
Guide: Developing complex tests covering more complex needs and the setup of your development workflow.
Alternatively, if you want to immediately try out the test suite you built you can check out the following:
Guide: Installing the test bed for development use to install your own test bed instance to test with.
Guide: Defining your test configuration to complete the setup to start testing.
Guide: Executing a test case to execute the test case.