Guide: Installing the test bed for production use

Track

Test bed operation

This guide walks you through the process of installing a new test bed instance for production use.

What you will achieve

At the end of this guide you will have installed your production test bed instance, i.e. the instance that your users will use to carry out their conformance testing. If you choose to not carry out an actual installation you will at least be aware of the steps and settings to configure for such a setup.

This guide acts as a counterpart to the test bed’s basic installation guide for development instances.

Note

Using the ISA² test bed: The European Commission offers a public and shared test bed instance that you can reuse for your project. This instance, termed the ISA² test bed, is available at https://www.itb.ec.europa.eu/itb and is operated by the test bed’s support team. The current guide concerns the setup of your own instance in case you have specific requirements that prevent you from reusing the shared instance (e.g. internal network access, use of sensitive data) or if you simply want to manage all aspects of your testing services yourself.

What you will need

  • About 1 hour.

  • A text editor.

  • A web browser.

  • A machine with at least a dual core CPU, 4 GBs of RAM and 30 GBs of free storage. This machine should also be able to access the internet (for software installations).

  • Administrator privileges on the target machine.

  • An understanding of Docker and ICT concepts such as ports, firewalls and reverse proxies.

How to complete this guide

During the course of this guide you will build up your configuration that you will eventually execute to install your test bed instance. Certain steps are highlighted as optional when linked to features that you can skip if not applicable for your case.

The steps of this guide follow a logical sequence matching the actions you should take during your setup. Alternatively you may of course directly refer to individual steps as focused documentation on setting up specific features. Optional features may be enabled progressively at any point during the test bed’s operation and can likewise be adapted and disabled.

Steps

Carry out the following steps to complete this guide.

Step 1: Install Docker

The first step is to ensure that Docker is installed on the machine that will host the test bed. If you already have Docker ensure it is at least at version 17.06.0, otherwise you can follow Docker’s online instructions to install or upgrade it.

Once your installation is complete, you will be able to test it using the docker --version command. This should provide output as follows:

> docker --version

Docker version 18.03.0-ce, build 0520e24

To complete your Docker setup you will also install Docker Compose, a tool for defining and running multi-container Docker applications (such as the test bed). You can check whether your Docker installation already includes Docker Compose by issuing the docker-compose --version command that should give output as follows:

> docker-compose --version
docker-compose version 1.16.1, build 6d1ac219

The minimum version of Docker Compose you need is 1.10.1. If you don’t have Docker Compose already installed follow its installation guide to set it up.

Note

Use of Docker Compose: It is not strictly necessary to use Docker Compose for a test bed installation although it is highly advised given the simplifications it offers. The rest of this guide assumes the use of Docker Compose in provided samples.

Step 2: Determine the access URL

It is assumed that your organisation is using a reverse proxy that manages how your services and resources are exposed to the outside world. As a result of this guide, the test bed will be exposed via this proxy as an additional service. It is important to define the path under which the test bed will be exposed given that several configuration options depend on its final address.

For the purpose of this guide we will assume that your proxy is configured to use HTTPS and that its base address is https://www.my.org. Based on this, we define the following paths for our installation:

  • /itb, under which the test bed’s UI will be exposed (the full address being https://www.my.org/itb).

  • /itbsrv, under which access is provided to the test bed’s callback endpoint (the full address being https://www.my.org/itbsrv).

Use of these paths is not mandatory but if you change them you would need to reflect their values in the configuration that follows.

Overall, use of a proxy is not mandatory per se but it is strongly advised given the subsequent flexibility that it offers. This reverse proxy would at least be used to manage your server certificate to ensure that you connect to the test bed over HTTPS (which is considered mandatory for a production installation), and to hide your internal configuration and topology from the outside world.

Step 3: Prepare basic configuration

The installation of the test bed will be driven via Docker Compose. In this step we will define the basic Docker Compose script that any production installation would need. This configuration will be revisited in subsequent steps to adapt it for specific features.

To begin, create a folder on the machine you will install the test bed on. For the purpose of this guide lets assume this is:

/opt/testbed

Within the /opt/testbed folder create a docker-compose.yml file with the following contents:

version: '2.1'

volumes:
   gitb-repo:
   gitb-dbdata:

services:
   gitb-redis:
      image: redis:6.0.8
      container_name: itb-redis
      restart: unless-stopped
   gitb-mysql:
      image: isaitb/gitb-mysql
      container_name: itb-mysql
      restart: unless-stopped
      environment:
       - MYSQL_ROOT_PASSWORD=CHANGE_ME
       - MYSQL_PASSWORD=CHANGE_ME
      volumes:
       - gitb-dbdata:/var/lib/mysql
      healthcheck:
       test: "/usr/bin/mysql --user=root --password=$$MYSQL_ROOT_PASSWORD --execute \"SHOW DATABASES;\""
       interval: 3s
       retries: 10
   gitb-srv:
      image: isaitb/gitb-srv
      container_name: itb-srv
      restart: unless-stopped
      environment:
       - gitb.messaging.callbackURL=https://www.my.org/itbsrv/MessagingClient
       - HMAC_KEY=CHANGE_ME
      ports:
       - "8080:8080"
   gitb-ui:
      image: isaitb/gitb-ui
      container_name: itb-ui
      restart: unless-stopped
      ports:
       - "9000:9000"
      environment:
       - THEME=ec
       - TESTBED_MODE=production
       - TESTBED_HOME_LINK=https://www.my.org/itb
       - AUTHENTICATION_COOKIE_PATH=/itb
       - DB_DEFAULT_PASSWORD=CHANGE_ME
       - HMAC_KEY=CHANGE_ME
       - APPLICATION_SECRET=CHANGE_ME
       - MASTER_PASSWORD=CHANGE_ME
      volumes:
       - gitb-repo:/gitb-repository
      depends_on:
       gitb-redis:
         condition: service_started
       gitb-mysql:
         condition: service_healthy
       gitb-srv:
         condition: service_started 

This is a YAML file that defines the components you will install as well as their configuration. To avoid issues with whitespace (YAML can be picky about that) you can download a correct copy from here.

This configuration defines how the test bed’s components are to be setup, specifically:

  • Two volumes named gitb-repo and gitb-dbdata are defined to persist the test bed’s data. These are defined as such so that other components can be removed or replaced without data loss.

  • gitb-redis is a cache server used for user session management.

  • gitb-mysql is the test bed’s database.

  • gitb-srv is the test bed’s backend test engine that executes tests.

  • gitb-ui is the test bed’s frontend that users access.

../_images/DockerInstallation1.png

Figure 2: Interactions and functions of test bed containers

The docker-compose.yml file defines per component environment variables that customise the installation. Before proceeding you must edit the file to replace the values defined as CHANGE_ME. These are secrets and passwords that are critical for your security and must be set by you to appropriate values. They are summarised in the following table:

Container

Property

Description

gitb-mysql

MYSQL_ROOT_PASSWORD

The password for the root user of the test bed’s MySQL database. This is used for the initial database setup.

gitb-mysql

MYSQL_PASSWORD

The password for the DB user (named gitb) that the test bed uses to connect to its database.

gitb-srv

HMAC_KEY

A key used to secure internal API calls between gitb-ui and gitb-srv.

gitb-ui

DB_DEFAULT_PASSWORD

The password used to access the test bed’s database. This must be set to match the value provided for the MYSQL_PASSWORD variable of the gitb-mysql container.

gitb-ui

HMAC_KEY

You must set this with the value you defined for the HMAC_KEY variable of the gitb-rsv container. This is a shared key between these two containers to secure their internal exchanges.

gitb-ui

APPLICATION_SECRET

This is the key used by the test bed to encrypt and verify user session tokens. Set this to a random string of at least 32 characters’ length.

gitb-ui

MASTER_PASSWORD

A key used to encrypt any sensitive values recorded in the test bed’s database (e.g. custom organisation properties marked as “secrets”).

Warning

When running in production mode if any of the above properties are not correctly set the test bed will fail to start up.

Apart from the properties linked to secrets, you should also note the following:

  • Addresses and paths: The values for TESTBED_HOME_LINK, AUTHENTICATION_COOKIE_PATH and gitb.messaging.callbackURL depend on how you expose the test bed through your proxy. You should adapt these to match your eventual proxy setup.

  • Automatic restarts: All containers are set to restart unless stopped ensuring that they are automatically brought online.

  • DB health check: The test bed’s DB is configured with a health check to ensure that it starts being accessed only when ready.

  • Port mappings: The test bed’s containers will map to host ports 8080 and 9090. You can adapt these if these ports are not available (e.g. replace 9000:9000 to 9999:9000 to expose gitb-ui on port 9999).

  • Theme: The THEME=ec setting will apply a Commission theme to the UI. You can remove this entry to avoid this.

  • Mode: The TESTBED_MODE=production setting removes a warning that you are running in development mode and ensures sensitive values have been correctly configured for production use.

Updating configuration during operation

The test bed’s configuration properties can be adapted post-installation and after it has started being used. In all cases to do this you would edit your docker-compose.yml file and restart your service to take effect.

Certain care however is needed when updating the properties that refer to passwords and keys. Specifically:

  • The values for APPLICATION_SECRET and HMAC_KEY can be updated at any time without side-effects. Keep in mind only that APPLICATION_SECRET must be sufficiently long and that HMAC_KEY must be set to the same value in both gitb-srv and gitb-ui.

  • To change the database root password connect to the gitb-mysql container and update the password to the desired value. Once you have done this ensure to update the MYSQL_ROOT_PASSWORD value to match. No container restart is needed given that this is only used for the database’s initial health check.

  • To change the test bed’s database user password you need to first stop the gitb-ui container. Once this is done connect to gitb-mysql and update the user’s password. The new value will then need to be reflected in the MYSQL_PASSWORD and DB_DEFAULT_PASSWORD properties.

  • Changing the MASTER_PASSWORD will take immediate effect for the encryption of new texts. However, any existing encrypted values will become unusable and will need to be re-entered manually.

Note

Updating the master password: Support to change the MASTER_PASSWORD and automatically adapt existing encrypted values is expected to be available in the next test bed release.

Step 4: Enable direct messaging

Note

Optional step: This step is needed only if you use embedded messaging handlers in your test cases.

The test cases you will expose through your instance likely involve messaging to and from systems under test (via receive and send steps). How such messaging takes place is determined by the steps’ configured handlers. In case you use embedded messaging handlers you must be aware of how the test bed manages ports, and enable access through your configuration. Important: If you use custom messaging handlers (simply put, your steps’ handler attributes are set with service URLs) or no messaging at all, you can ignore this section.

When embedded messaging handlers are used, test sessions are isolated by means of the connecting system’s public IP address. This is provided by the user as configuration before launching a test session. In case multiple sessions are launched for the same IP address, the test bed will isolate communications by starting to listen on a new port per session. This necessitates that the external systems under test can make direct connections to such dynamically opened ports. To allow this you need to adapt Docker’s approach for port mappings and allow access to the opened ports. These points are covered in the following steps.

Adapt Docker port mappings

To enable access to dynamically opened ports, you need to foresee a wide range of internal ports mapped to your host. By default, mapping large port ranges can be time-consuming for a container’s startup. As such it is a good practice to run Docker with hairpin NAT enabled, rather than relying on a per-port userland proxy. This is done by setting the following value in Docker’s configuration file (located at e.g. /etc/default/docker for an Ubuntu distribution):

DOCKER_OPTS="--userland-proxy=false"

Once this setting is applied make sure to restart your Docker process to take effect.

Allow access to test engine ports

The range of ports that you will need to support depends on how many parallel test sessions you expect to have from the same IP address. In practice this would likely be limited so you can start with a limited port range (e.g. 10 ports). This can be adapted at any time depending on your use.

Edit your docker-compose.yml file to adapt the setup of your gitb-srv container as follows:

...
services:
  ...
  gitb-srv:
    ...
    environment:
     - gitb.messaging.server-ip-address=itbsrv.my.org
       ...
    ports:
     - "8080-8090:8080-8090"
  ...

The gitb.messaging.server-ip-address variable needs to be set with a hostname or IP address that maps directly to the test bed’s host machine (not the reverse proxy). The port range defined in the ports configuration foresees 10 ports for test sessions (8081 through 8090) in addition to the base port (8080). With such mappings in place, test sessions upon initialisation will inform the tester that the connection host and port for direct messaging will be e.g. itbsrv.my.org and 8081 respectively.

Note

Direct messaging update: The described approach of direct messaging for the test bed’s embedded messaging handlers is considered deprecated and is planned for a significant update in the near future. This update will not necessitate direct exposure of the test bed’s engine nor will require dynamic port assignments.

Step 5: Enable optional features

The following sections cover the optional features supported by the test bed. Follow each section’s instructions to enable the relevant feature for your instance.

Antivirus scanning

When using the test bed you and your users will typically be making several file uploads. These could range from community administrators uploading test suites to users uploading data to be used in test sessions. The test bed supports the use of an antivirus service to scan all user-provided files ensuring that they are virus-free before being stored and used. Enabling antivirus checking is strongly advised for any non-development test bed instance although not strictly mandatory. If you choose to not set up such a service ensure that this is an informed decision.

To enable antivirus scanning the test bed supports the configuration of a ClamAV server, a robust, popular and free antivirus solution. You can either reuse an existing ClamAV server for this purpose or set one up as a container within your overall test bed service.

In case you are reusing an existing ClamAV service this can be configured as follows:

...
services:
  ...
  gitb-ui:
    ...
    environment:
     - ANTIVIRUS_SERVER_ENABLED=true
     - ANTIVIRUS_SERVER_HOST=10.0.0.5
     - ANTIVIRUS_SERVER_PORT=3310
  ...

This configuration enables antivirus checking making use of a server on host 10.0.0.5 listening on port 3310 (you can change these values to match your setup).

In case you don’t have an existing antivirus server one can be easily included as container in the test bed’s service. A good Docker image for this is the docker-clamav image from the Docker hub that is also configured to automatically refresh its virus definitions. To enable this adapt the docker-compose.yml file as follows:

...
services:
  ...
  gitb-ui:
    ...
    environment:
     - ANTIVIRUS_SERVER_ENABLED=true
     - ANTIVIRUS_SERVER_HOST=gitb-antivirus
     - ANTIVIRUS_SERVER_PORT=3310
  ...
  gitb-antivirus:
      image: mkodockx/docker-clamav
      container_name: itb-antivirus
      restart: unless-stopped

Email notifications and support

You can configure the test bed with an SMTP service to enable the sending of emails. Emails are sent by the test bed in two cases:

  • When users select to contact support, in which case their request is sent to a configured default mailbox and/or a community’s support mailbox.

  • To send self-registration notifications to a community’s support mailbox.

The test bed foresees several environment variables to define how emails are to be sent. These are listed in the following table:

Property

Description

Default value

EMAIL_ENABLED

Set to true to enable emails to be sent to the support team from within the test bed.

false

EMAIL_FROM

The “FROM” address for support emails (e.g. Contact Form <contact@itb.ec.europa.eu>).

Contact Form <contact@itb.ec.europa.eu>

EMAIL_TO

The support team’s email address.

DIGIT-ITB@ec.europa.eu

EMAIL_SMTP_HOST

The SMTP server’s host.

EMAIL_SMTP_PORT

The SMTP server’s port.

EMAIL_SMTP_AUTH_ENABLED

Set to true if authentication is required by the SMTP server (if EMAIL_ENABLED is true).

false

EMAIL_SMTP_AUTH_USERNAME

The username to use when authenticating against the SMTP server.

EMAIL_SMTP_AUTH_PASSWORD

The password to use when authenticating against the SMTP server.

EMAIL_SMTP_SSL_ENABLED

Whether the connection to the SMTP server should use an SSL channel.

false

EMAIL_SMTP_STARTTLS_ENABLED

Whether the connection to the SMTP server should use the STARTTLS option (initial connection in cleartext and then upgrade to SSL).

false

EMAIL_ATTACHMENTS_MAX_COUNT

The maximum number of attachments allowed to include in contact form submissions. Setting this to 0 or negative disables contact form attachments.

5

EMAIL_ATTACHMENTS_MAX_SIZE

The maximum total size (in MBs) for attachments.

5

EMAIL_ATTACHMENTS_ALLOWED_TYPES

A comma-separated list of MIME types corresponding to the types of files that are allowed as attachments in contact form submissions.

text/plain,image/gif,image/png,image/jpeg,application/pdf,application/xml,text/xml

Considering the above variables, emails can be enabled by adapting docker-compose.yml as follows:

...
services:
  ...
  gitb-ui:
    ...
    environment:
     - EMAIL_ENABLED=true
     - EMAIL_SMTP_HOST=mail.my.org
     - EMAIL_SMTP_PORT=25
     - EMAIL_SMTP_AUTH_ENABLED=true
     - EMAIL_SMTP_AUTH_USERNAME=a_username
     - EMAIL_SMTP_AUTH_PASSWORD=a_password
     - EMAIL_FROM=contact@my.org
     - EMAIL_TO=support@my.org
  ...

Self-registration

The test bed by default allows users to register themselves for any community that is configured as being publicly accessible. In case your community (or communities) are only planned to be private, i.e. with community administrators managing organisations and users, you can disable user self-registration to remove relevant options and controls from the user interface. Such features include the self-registration link on the test bed’s welcome page but also relevant configuration options in the community management screens.

To disable self-registration adapt docker-compose.yml as follows:

...
services:
  ...
  gitb-ui:
    ...
    environment:
     - REGISTRATION_ENABLED=false
  ...

Demo test cases

The test bed can be configured with a set of publicly available demos that illustrate how you expect users to use your test bed instance. This is something that you would do after the test bed’s initial installation as it requires you to first use it to setup the demos.

Once you have made an initial installation of the test bed, follow these steps to configure your demos:

  1. Create an organisation that will be used as your “demo” organisation.

  2. For this organisation create the conformance statements that you want to share as demos.

  3. Create a basic user for the organisation (not an administrator) that will be used as the demo account to connect with.

The important thing to retain is the internal identifier for the created user, as this will be used in the subsequent configuration. If you select to view the user’s details, this identifier (an integer) is the last path element you would see in the address bar.

You can now proceed to adapt docker-compose.yml as follows (assuming 10 as an example of the demo user’s identifier):

...
services:
  ...
  gitb-ui:
    ...
    environment:
     - DEMOS_ENABLED=true
     - DEMOS_ACCOUNT=10
  ...

Once this is configured, users will see a link on the test bed’s welcome page to try out your demos. Doing so will connect them using the configured demo account with which they can execute and review tests but not make modifications.

Conformance certificate timestamps

Community administrators can issue conformance certificates to their members once they have successfully completing their testing. These are PDF documents that can be configured as part of a community’s settings, be signed by a provided digital certificate. The configuration of such digital certificates is managed through the test bed interface, however it is possible at the level of the overall test bed installation to enable trusted timestamps as part of these signatures. Trusted timestamps are provided by TSA (Time Stamp Authority) servers.

To enable such timestamps adapt docker-compose.yml as follows:

...
services:
  ...
  gitb-ui:
    ...
    environment:
     - TSA_SERVER_ENABLED=true
  ...

If timestamps are enabled, the test bed by default uses the Free Time Stamp Authority (freeTSA). This can be replaced however by means of the TSA_SERVER_URL property as follows:

...
services:
  ...
  gitb-ui:
    ...
    environment:
     - TSA_SERVER_ENABLED=true
     - TSA_SERVER_URL=https://freetsa.org/tsr
  ...

EU Login integration

Commission-hosted test bed instances are eligible for integration with EU Login, the European Commission’s central authentication service.

To integrate with EU Login you first need to register your test bed instance with the EU Login service. To do this you start by creating an internal support ticket and eventually completing a registration form. From the test bed’s perspective, the important points from this registration form are:

  • To provide the public address of your test bed instance.

  • To require EU Login to return the user’s email address upon authentication.

The registration form can be further customised to e.g. require multi-factor authentication or restrict usage only to specific user domains.

Once you have been onboarded to the EU Login service you need to configure your test bed instance to use it. To do so adapt docker-compose.yml as follows:

...
services:
  ...
  gitb-ui:
    ...
    environment:
     - AUTHENTICATION_SSO_ENABLED=true
     - AUTHENTICATION_SSO_LOGIN_URL=https://ecas.ec.europa.eu/cas/login
     - AUTHENTICATION_SSO_CALLBACK_URL=https://www.my.org/itb/callback
  ...

It is important to note here that the AUTHENTICATION_SSO_CALLBACK_URL is based on the public address you select for the test bed (see Step 2: Determine the access URL). In short, you need to set this property to match the full address of your test bed instance, followed by a /callback postfix.

It is possible to start using the test bed without EU Login integration and switch to it later on. To have a test bed instance use its own username and password accounts, it suffices to leave AUTHENTICATION_SSO_ENABLED as false or simply omit it (false is the default). Once you are ready to switch to EU Login you can then enable the above-described properties to make the switch. In this case however, you would most probably want any existing users to still be able to have access. This is possible by setting a test bed instance in EU Login migration mode, whereby they are given the option to link their EU Login account with one or more existing test bed accounts. To place your test bed instance in migration mode adapt docker-compose.yml as follows:

...
services:
  ...
  gitb-ui:
    ...
    environment:
     - AUTHENTICATION_SSO_ENABLED=true
     - AUTHENTICATION_SSO_LOGIN_URL=https://ecas.ec.europa.eu/cas/login
     - AUTHENTICATION_SSO_CALLBACK_URL=https://www.my.org/itb/callback
     - AUTHENTICATION_SSO_IN_MIGRATION_PERIOD=true
  ...

When in migration mode, users are presented in the welcome page with an option to migrate their existing account(s), and with a link to a step-by-step migration tutorial.

Step 6: Set up supporting services

To support your test bed instance you may require custom service extensions to implement the specific messaging, validation or processing needs of your test cases. In addition, further components may be needed to complete your setup such as databases or messaging gateways.

You are free to set up such components any way you see fit, the only requirement being that the services needed to be contacted by the test bed are indeed accessible. Nonetheless, and in view of simplifying operation and management, you may choose to include such components as part of the test bed’s overall dockerised service. This is something that you would likely do at least for your custom messaging, validation and processing services, given that they act as capability extensions for your test bed instance. Such services are built with containerisation in mind (the template services already define Docker configurations) and can be easily included.

For example, when defining a custom messaging service and validator, the only thing that would be needed apart from defining their Docker images is to adapt docker-compose.yml as follows:

...
services:
  ...
  gitb-srv:
    ...
  gitb-ui:
    ...
  my-messaging-service:
    image: myorg/my-messaging-service
    restart: unless-stopped
  my-validation-service:
    image: myorg/my-validation-service
    restart: unless-stopped

Specifying restart: unless-stopped allows these containers to benefit from automatic restarts. In terms of access, the other containers would be able to contact internally these services using their container names as host names and their exposed ports (8080 by default). In case you also need to expose these services directly to your users you can always map their internal ports to your host:

...
services:
  ...
  my-validation-service:
    image: myorg/my-validation-service
    restart: unless-stopped
    ports:
     - "9999:8080"

With your service’s port mapped to your host you would then need to configure your reverse proxy accordingly to allow external access.

Step 7: Configure reverse proxy

As discussed in Step 2: Determine the access URL, use of a reverse proxy for a production installation is strongly advised. A proxy allows you to decouple your internal configuration from the outside world, and acts as a central point to manage access and apply server certificates.

For the purposes of this section we consider the use of nginx as a proxy server although similar configuration could be applied to other servers. In order to illustrate the expected configuration we will assume the following sample settings and values:

  • The internal address of the host running the test bed is 10.0.0.4.

  • The public address of your proxy is https://www.my.org and the test bed’s paths are /itb and /itbsrv.

  • Your test bed containers are listening on the default ports (8080 for gitb-srv and 9000 for gitb-ui).

Taking into account the above points, adapt your server’s nginx.conf file to add the following location mappings:

http {
  ...
  # This is your HTTPS server.
  server {
    listen 443 ssl;
    ...
    location /itb/assets/javascripts/routes {
      proxy_pass http://10.0.0.4:9000/assets/javascripts/routes;
      sub_filter "10.0.0.4:9000" "www.my.org/itb";
      sub_filter "10.0.0.4:9000" "www.my.org/itb";
      # The following ensures websocket communications are set to be secure (wss).
      sub_filter "'ws'" "'wss'";
      sub_filter_types *
      sub_filter_once off;
    }
    location /itb/ {
      proxy_pass http://10.0.0.4:9000/;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "Upgrade";
      sub_filter "/webjars/" "/itb/webjars/";
      sub_filter "/assets/" "/itb/assets/";
      sub_filter_once off;
      add_header ITB-PATH /itb;
      add_header Strict-Transport-Security 'max-age=31536000; includeSubDomains';
    }
    location /itbsrv/ {
      proxy_pass http://10.0.0.4:8080/itbsrv/;
      sub_filter "http://10.0.0.4:8080/itbsrv/" "https://www.my.org/itbsrv/";
      sub_filter_once off;
    }
  }
}

The above configuration covers the test bed platform. If you want to expose additional services to your users (e.g. a validator), you would need further configuration such as the following:

http {
  ...
  # This is your HTTPS server.
  server {
    listen 443 ssl;
    ...
    location /validator/ {
      proxy_pass http://10.0.0.4:9999/;
      # Replace the address in the service's endpoint as listed in its WSDL.
      sub_filter "http://10.0.0.4:9999/services/validation" "https://www.my.org/validator/services/validation";
      sub_filter_once off;
      sub_filter_types application/wsdl+xml application/xml text/xml text/html;
    }
  }
}

Step 8: Firewall configuration

Adapting your organisation’s firewall is the final step to consider in your installation. Regarding the test bed, you need to consider the following points:

  • Access to the test bed should already be possible given that it is done via your reverse proxy (i.e. the firewall allows port 443).

  • In case you use direct messaging in your test cases you need to also allow incoming connections on the relevant port range of gitb-srv.

  • You must not block web socket communications.

The above points are the minimum considerations you need to take into account for your firewall setup.

Step 9: Install the test bed

Open a command prompt to the /opt/testbed folder (i.e. where you have your docker-compose.yml file). In this, issue the docker-compose up -d command from which you should eventually see the following output:

docker-compose up -d
...
Creating itb-srv ...
Creating itb-redis ...
Creating itb-mysql ...
Creating itb-srv
Creating itb-redis
Creating itb-srv ... done
Creating itb-ui ...
Creating itb-ui ... done

What this does is to first download all required test bed images (as well as the images they build upon) and then start up all services. The names you see (e.g. itb-srv) can be used to refer to individual containers (e.g. to inspect logs).

To ensure the test bed has completed its initialisation you should check the logs of itb-srv and itb-ui. For itb-srv issue docker logs -f itb-srv for which you should see output completing as follows:

> docker logs -f itb-srv

...
12-Sep-2018 12:52:41.124 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive /usr/local/tomcat/webapps/itbsrv.war has finished in 18,756 ms
12-Sep-2018 12:52:41.140 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
12-Sep-2018 12:52:41.151 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
12-Sep-2018 12:52:41.159 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 18918 ms

You can exit the log display by issuing a CTRL-C. To check itb-ui issue docker logs -f itb-ui for which the output should complete as follows:

> docker logs -f itb-ui

...
INFO  2018-09-12 12:52:32 [main] application - Application has started in production mode
INFO  2018-09-12 12:52:32 [main] play - Application started (Prod)
INFO  2018-09-12 12:52:33 [main] play - Listening for HTTP on /0.0.0.0:9000

Once again exit the log display by issuing a CTRL-C. You should now have a fully functioning test bed installation.

Step 10: Test your installation

To test your installation you can log into the test bed’s user interface using the default administrator account. Open a browser window and navigate to https://www.my.org/itb. If the installation was successful you should see the test bed’s welcome page:

../_images/welcome3.png

Now try to log into the test bed using the default administrator account. To do this click on the Click to log in button at which point you should see the test bed’s login page:

../_images/login3.png

Use here the default administrator account of test@test.com with password test. You will then be prompted to set a new password, following which you should be greeted with the test bed’s empty home page:

../_images/home1.png

From this point on you are free to set up and use the test bed as you please. You may now also add a new test bed administrator and, once you have logged in with the new account, delete the default one.

Note

First access with EU Login: In case you have enabled EU Login integration you will need to link your EU Login profile with the default test bed administrator account. To do this:

  1. Enable the EU Login migration mode for your test bed instance.

  2. From the welcome page select to migrate an account to EU Login.

  3. Authenticate against EU Login and then provide the credentials of the default administrator account.

Step 11: Manage the test bed

An optional additional step would be to experiment with managing your new test bed instance.

Managing the test bed is done as with any Docker container. In fact, as we are using Docker Compose, management is actually simpler as you don’t need to treat individual components. The following list provides a set of commands that you would eventually be using. Note that when using the docker-compose command you need to do so from the folder that contains the docker-compose.yml file):

Action

Command

Create and run the test bed

docker-compose up -d

View a component’s log file (e.g. itb-ui)

docker logs -f itb-ui

Stop the test bed

docker-compose stop

Start the test bed

docker-compose start

Delete the test bed (keeping all data)

docker-compose down

Delete the test bed (removing all data)

docker-compose down -v

Obtain the latest test bed version

docker-compose pull

If you have a running test bed and want to update it to the latest release you would use a combination of the above commands as follows:

  1. docker-compose pull to get any latest versions but not make any changes.

  2. docker-compose down to remove the previous containers but keeping your data.

  3. docker-compose up -d to create new containers based on the latest downloaded versions (reusing your previous data).

A detailed description on updating a test bed instance to the latest release is provided in Guide: Updating the test bed.

Summary

Congratulations! You have now prepared and set up a production test bed installation. In doing so you also installed Docker, Docker Compose and created a docker-compose.yml file to configure and automate your test bed’s installation and management. You also reviewed and enabled optional available features and configuring supporting services to enable access to your test bed instance for your users.

See also

With a running test bed you are now free to start using it to configure your testing strategy and execute tests. Concerning the test bed’s use make sure to check out the user guide (for test bed administrators). This contains detailed information on how to create your domain, specifications and test suites, as well as register the organisations and users that will be using them to test.

Given that this is a production test bed instance, it is likely that you have already configured your domain and community on a development instance. If this is the case keep in mind that you can export your full setup from your development instance and then import it to production. Details on how to do this are provided in the administrator user guide. In the unlikely case you don’t have such a development instance it is strongly advised to use one when building up your test configuration.

For more information on the Docker and Docker Compose tools, commands and properties used in this guide, check out the Docker online documentation.

Finally, in terms of further guides that would be of interest you should consider:

References

This section contains additional references linked to this guide.

Configuration properties

All configuration properties listed in this section are pertinent to the gitb-ui container and are defined as environment variables in your docker-compose.yml file. The following table summarises all properties, their purpose, supported values and defaults.

Property

Category

Description

Default

ANTIVIRUS_SERVER_ENABLED

Antivirus

Set to true to virus scan user-uploaded files using a ClamAV service.

false

ANTIVIRUS_SERVER_HOST

Antivirus

The ClamAV server’s hostname.

ANTIVIRUS_SERVER_PORT

Antivirus

The TCP port the ClamAV server is listening on for scan requests.

ANTIVIRUS_SERVER_TIMEOUT

Antivirus

The timeout (in milliseconds) after which to fail if the antivirus server fails to respond. This is disabled by using 0

0

APPLICATION_SECRET

Encryption

The key to use to encrypt user’s session tokens.

AUTHENTICATION_COOKIE_PATH

Authentication

The cookie path to set for generated session cookies (should be adapted to accommodate reverse proxy mappings).

/

AUTHENTICATION_SESSION_MAX_IDLE_TIME

Authentication

The maximum time (in seconds) that a session will be kept alive if there is no activity.

3600

AUTHENTICATION_SESSION_MAX_TOTAL_TIME

Authentication

The maximum overall time (in seconds) that a session is allowed to exist without re-authentication. A value of -1 disables this.

-1

AUTHENTICATION_SSO_CALLBACK_URL

Authentication

The complete URL of the test bed instance to redirect back to. This should be set to the test bed’s full address postfixed with /callback.

AUTHENTICATION_SSO_ENABLED

Authentication

Whether or not the test bed will use EU Login as an SSO solution.

false

AUTHENTICATION_SSO_IN_MIGRATION_PERIOD

Authentication

Whether or not features relevant to migrating from a non-SSO to an SSO-enabled environment should be activated.

false

AUTHENTICATION_SSO_LOGIN_URL

Authentication

The complete URL of the EU Login service to redirect to for authentication.

DATA_ARCHIVE_KEY

Sandbox setup

The password with which to open the data archive used to initialise the test bed’s configuration (used if setting up a test bed instance as a sandbox).

DATA_WEB_INIT_ENABLED

Sandbox setup

A boolean flag that instructs the test bed to prompt the user for a manual upload of a data archive at first connection to initialise its configuration (used if setting up a test bed instance as a sandbox).

false

DB_DEFAULT_PASSWORD

Database

A custom password for the DB user account that is used to connect to the database. This needs to match the database’s MYSQL_PASSWORD environment variable as defined in the docker-compose.yml file (if specified).

DB_DEFAULT_USER

Database

A custom username for the DB user account that is used to connect to the database. This needs to match the database’s MYSQL_USER environment variable as defined in the docker-compose.yml file (if specified).

DEMOS_ACCOUNT

Demos

The database user ID of a specific test bed user to be used as a demo user.

DEMOS_ENABLED

Demos

Whether links to demo scenarios should be displayed.

false

EMAIL_ATTACHMENTS_MAX_COUNT

Email

The maximum number of attachments allowed to include in contact form submissions. Setting this to 0 or negative disables contact form attachments.

5

EMAIL_ATTACHMENTS_MAX_SIZE

Email

The maximum total size (in MBs) for attachments.

5

EMAIL_ATTACHMENTS_ALLOWED_TYPES

Email

A comma-separated list of MIME types corresponding to the types of files that are allowed as attachments in contact form submissions.

text/plain,image/gif,image/png,image/jpeg,application/pdf,application/xml,text/xml

EMAIL_ENABLED

Email

Set to true to enable emails to be sent to the support team from within the test bed.

false

EMAIL_FROM

Email

The “FROM” address for support emails (e.g. Contact Form <contact@itb.ec.europa.eu>).

Contact Form <contact@itb.ec.europa.eu>

EMAIL_SMTP_AUTH_ENABLED

Email

Set to true if authentication is required by the SMTP server (if EMAIL_ENABLED is true).

false

EMAIL_SMTP_AUTH_PASSWORD

Email

The password to use when authenticating against the SMTP server.

EMAIL_SMTP_AUTH_USERNAME

Email

The username to use when authenticating against the SMTP server.

EMAIL_SMTP_HOST

Email

The SMTP server’s host.

EMAIL_SMTP_PORT

Email

The SMTP server’s port.

EMAIL_SMTP_SSL_ENABLED

Email

Whether the connection to the SMTP server should use an SSL channel.

false

EMAIL_SMTP_STARTTLS_ENABLED

Email

Whether the connection to the SMTP server should use the STARTTLS option (initial connection in cleartext and then upgrade to SSL).

false

EMAIL_TO

Email

The support team’s email address.

DIGIT-ITB@ec.europa.eu

GUIDES_EULOGIN_MIGRATION

Guides

The full address for the guide on how to migrate to EU Login.

https://www.itb.ec.europa.eu/docs/guides/latest/migratingToEULogin

GUIDES_EULOGIN_USE

Guides

The full address for the guide on how to use EU Login.

https://www.itb.ec.europa.eu/docs/guides/latest/usingEULogin

HMAC_KEY

Encryption

The key to use to sign request digests sent between the gitb-ui and gitb-srv components.

HMAC_WINDOW

Encryption

The maximum window (in milliseconds) in which to accept exchanged requests sent between gitb-ui and gitb-srv.

10000

MASTER_PASSWORD

Encryption

The master password to use to encrypt sensitive values in the test bed’s database.

PROXY_SERVER_ENABLED

Proxy

Set to true to define a proxy for outgoing connections from the test bed.

false

PROXY_SERVER_HOST

Proxy

The proxy server’s host.

PROXY_SERVER_PORT

Proxy

The proxy server’s port.

PROXY_SERVER_AUTH_ENABLED

Proxy

Set to true in case the proxy server requires authentication.

false

PROXY_SERVER_AUTH_PASSWORD

Proxy

The password to use when authenticating against the proxy server.

PROXY_SERVER_AUTH_USERNAME

Proxy

The username to use when authenticating against the proxy server.

REGISTRATION_ENABLED

General

Whether or not self-registration for public communities is allowed for this test bed instance.

true

SAVED_FILE_MAX_SIZE

General

The maximum size (in KBs) for uploaded files that are to be persisteb in the test bed’s database (not email attachments).

1024

SERVER_REQUEST_TIMEOUT_IN_SECONDS

General

The maximum time in seconds allowed for web requests before considering them invalid.

60

SURVEY_ADDRESS

Satisfaction survey

The address of the test bed’s satisfaction survey.

https://ec.europa.eu/eusurvey/runner/itb

SURVEY_ENABLED

Satisfaction survey

Set to false to disable the link to the test bed’s satisfaction survey.

true

TESTBED_HOME_LINK

General

The full link to use to return back to the welcome page if an error occurs (may need to be adapted to accomodate reverse proxy settings).

/

TESTBED_MODE

General

Whether or not the test bed is running in production or development mode.

development

TEST_SESSION_ARCHIVE_THRESHOLD

General

The number of days for which test session data will be maintained without being archived.

30

TSA_SERVER_ENABLED

Conformance certificates

Set to true to enable trusted timestamp generation using a TSA server for signatures of conformance certificates.

false

TSA_SERVER_URL

Conformance certificates

The URL of the TSA server for signature timestamp generation.

https://freetsa.org/tsr

USERGUIDE_CA

User guide

The address of the user guide for community administrators.

https://www.itb.ec.europa.eu/docs/itb-ca/latest

USERGUIDE_OA

User guide

The address of the user guide for organisation administrators.

https://www.itb.ec.europa.eu/docs/itb-oa/latest

USERGUIDE_OU

User guide

The address of the user guide for organisation users.

https://www.itb.ec.europa.eu/docs/itb-ou/latest

USERGUIDE_TA

User guide

The address of the user guide for test bed administrators.

https://www.itb.ec.europa.eu/docs/itb-ta/latest