NAV
json shell python

Interoperability Testing Tool

Introduction

This section describes:

  1. Testing Tool design and components description.
  2. Event bus API internal testing tool calls (between components)

This section does NOT cover:

  1. Event bus API between testing tool and other F-Interop components (GUI, results repository, etc)

General F-Interop even bus API

The general architecture and interfaces of F-Interop are described in the diagram

The detailed view of the exchanges using AMQP topics/routing keys, is described in: even bus api

Interoperability testing tool API (inter-component API)

The documentation here bellow details also the inter-components communication, this detailed level of the API is described in the zoomed-in box of this diagram:

Screenshot

Implementors of testing tools implementing a different one if they found this doesnt fit their needs.

For doing so, please contact the main F-Interop developers, architect or integrator for studying these needs.

Namespaces

For filtering/matching messages e.g. when subscribing to a bunch, we have defined namespaces. The testing tool defines some namespaces which are used for categorizing type of calls/events in the event bus.

The name spaces used in the API spec for the interoperability and conformance testing tools are:

Most relevant (intra) testing tool events :

event type requirements interaction(publisher/subscribers)
tun.start Testing Tool MAY implement (if IP tun needed) Event: Testing Tool -> Agent
tun.started - Event: Testing Tool -> Agent
testingtool.ready Testing Tool MUST publish event Event: -
testingtool.configured Testing Tool MUST publish event Event: Testing Tool -> Orchestrator, GUI
testsuite.start Testing Tool MUST listen to event Event: GUI -> Testing Tool
testsuite.testcase.ready Testing Tool MUST publish event Event: GUI -> Testing Tool
testsuite.testcase.configuration Testing Tool MAY publish event (if needed for executing the test case) Event: Testing Tool -> GUI & automated-iut
testsuite.testcase.start Testing Tool MUST listen to event Event: GUI -> Testing Tool
testsuite.step.stimuli.execute Testing Tool MUST publish event Event: Testing Tool -> GUI
testsuite.step.stimuli.executed Testing Tool MUST listen to event Event: GUI (or automated-IUT)-> Testing Tool
testsuite.step.verify.execute Testing Tool MUST publish event Event: Testing Tool -> GUI (or automated-IUT)
testsuite.step.verify.executed Testing Tool MUST listen to event Event: GUI (or automated-IUT)-> Testing Tool
testsuite.testcase.finished Testing Tool MUST publish event Event: Testing Tool -> GUI
testsuite.testcase.restart Testing Tool MUST listen to event Event: GUI -> Testing Tool
testsuite.testcase.select Testing Tool MUST listen to event Event: GUI (or automated-IUT)-> Testing Tool
testsuite.testcase.skip Testing Tool MUST listen to event Event: GUI (or automated-IUT)-> Testing Tool
testsuite.testcase.stop Testing Tool MUST listen to event Event: GUI & automated-iut -> Testing Tool
testsuite.testcase.verdict Testing Tool MUST publish event Event: Testing Tool -> GUI
testsuite.abort Testing Tool MUST listen to event Event: GUI (or automated-IUT)-> Testing Tool
testsuite.finish Testing Tool MUST listen to event Event: GUI -> Testing Tool
testsuite.report Testing Tool MUST publish event Event: Testing Tool -> GUI
testingtool.terminate Testing Tool MUST listen to event Event: GUI, (or Orchestrator) -> Testing Tool
dissection.autotriggered Testing Tool MUST publish event Event: Testing Tool -> GUI

Testing Tool Components

Brief description of components

Screenshot

Test Coordinator

Source Code:

https://gitlab.f-interop.eu/f-interop-contributors/ioppytest

Description

The coordinator is the component that glues all parts together. It’s purpose is to coordinate the test cases execution during a test session. It’s a generic executor of the test case, although at the bootstrapping phase it retrieves a test session context from the test orchestrator.

The base document on which the tests are executed are the test descriptions (extended version from ETSI’s)

TEDs example
--- !configuration
configuration_id: COAP_CFG_01_v01
uri: http://f-interop.paris.inria.fr/tests/CoAP_CFG_01
nodes:
  - coap_client
  - coap_server
topology:
  - link_id : link_01
    capture_filter: udp port 5683
    nodes:
      - coap_client
      - coap_server
description:
  - node: coap_server
    message:
      - CoAP servers are requested to offer the following resources
      - - /test
        - Default test resource
        - Should not exceed 64bytes
      - - /seg1/seg2/seg3
        - Long path ressource
        - Should not exceed 64bytes
      - - /query
        - Ressource accepting query parameters
        - Should not exceed 64bytes
        (...)

  - node: coap_client
    message:
      - None

--- !testcase
testcase_id: TD_COAP_CORE_01_v01
uri : http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01_v01
configuration: COAP_CFG_01_v01
objective: Perform GET transaction(CON mode)
pre_conditions: Server offers the resource /test with resource content is not empty that handles GET with an arbitrary payload
references: '[COAP] 5.8.1, 1.2, 2.1, 2.2, 3.1'
sequence:
  - step_id: 'TD_COAP_CORE_01_v01_step_01'
    type: stimuli
    iut : coap_client
    description:
      - Client is requested to send a GET request with
      - Type = 0(CON)
      - Code = 1(GET)

  - step_id: TD_COAP_CORE_01_v01_step_02
    type: check
    description:
      - The request sent by the client contains
      - Type=0 and Code=1
      - Client-generated Message ID(\u2794 CMID)
      - Client-generated Token(\u2794 CTOK)
      - Uri-Path option "test"

  - step_id: TD_COAP_CORE_01_v01_step_03
    type: check
    description:
        - Server sends response containing
        - Code = 2.05(Content)
        - Message ID = CMID, Token = CTOK
        - Content-format option
        - Non-empty Payload

  - step_id: TD_COAP_CORE_01_v01_step_04
    type: verify
    iut: coap_client
    description:
        - Client displays the received information

The most important events sent and actions triggered by the test coordinator are:

The component’s implementation provided by F-Interop is protocol agnostic, once instantiated it gets the specific context from the orchestrator and feeds from the test extended descriptions (TED yaml files). TEDs play a very important role in the architecture, they describe in detail each test case execution. TEDs are meant to be easy to write using YAML language, and they are written by the contributors. TEDs describe the setup of the test, they include a step by step description of each test case. See example for CoAP

API services

Coordinator provides the following services:

SERVICES DESCRIPTION
testsuite.getstatus Message for debugging purposes. The coordination component returns the status of the execution
testsuite.gettestcases Message for requesting the list of test cases included in the test suite.

API events

Coordinator listens and consumes the following messages from the bus:

MESSAGES CONSUMED DESCRIPTION
testsuite.start Message for triggering start of test suite. The command is given by one of the users of the session.
testsuite.finish TBD if needed
testsuite.abort Message for aborting the ongoing test session.
testsuite.testcase.skip Message for skipping a test case. Coordinator passes to the next test case if there is any left.
testsuite.testcase.select Message for selecting next test case to be executed. Allows the user to relaunch an already executed test case.
testsuite.testcase.start Message for triggering the start of the test case. The command is given by one of the users of the session.
testsuite.testcase.stop TBD if needed
testsuite.testcase.restart Message for triggering the restart of a test case (TBD if needed)
testsuite.step.stimuli.executed Message pushed by UI or agent indicating the stimuli was executed.
testsuite.step.check.response TBD (for step_by_step analysis)
testsuite.step.verify.response Message pushed by UI or agent providing the response to verify step.
testingtool.terminate Signal so testing tool can gracefully stop all its processes

Coordinator generates and publishes the following messages:

MESSAGES PUBLISHED DESCRIPTION
testingtool.ready Indicates that the testing tool is ready to receive the testsuite.start signal
testsuite.testcase.ready Indicates next testcase to be executed
testsuite.step.execute Provides info about next step to execute, message used by user interfaces and automated-IUTs
testsuite.testcase.configuration Provides information to the user about the current configuration that needs to be used for the test case. Pushed at the beginning of th testcase
testsuite.testcase.finished Indicates that the testcase has finished
testsuite.testcase.verdict Provides final verdict of the test case
testsuite.finished Indicates there’s no more test cases to execute
testsuite.report Provides final report of all the test cases of the test suite

Sniffer

Description

This is the component handling the sniffing of the traffic exchanged between the participants of the test session.

What this component is not: - doesnt use tcpdump - it doesn’t capture packets from a network interface

This component listens to the right AMQP topics of the AMQP event bus, and dumps all RAW IP data messages (assumably coming from the agents) into dump files (.pcap files).

The component can be controlled queried remotelly using the following Even Bus API calls.

API services

METHOD DESCRIPTION
sniffing.start Action for starting the capture.
sniffing.stop Action for finishing the capture.
sniffing.getcapture Service for retrieving a capture within the session execution scope.
sniffing.getlastcapture Service for retrieving the last capture within the session execution scope.

API events

None

Test Analyser (TAT)

Description

The TAT provides F-Interop with some protocol analysis services. The TAT is the component that performs the verification of traces during a test session based on the CHECKs defined in the Test Description.

F-Interop provides TATs for different protocols under test.

The TAT issues three types of verdicts:

The architecture supports TATs which perform

TATs are created both by the F-Interop core team and by external contributors. The F-Interop API specification defines the format of the messages a TAT will receive from the Event Bus, and the format of the messages it will produce.

TAT may implement it’s own dissector, and in that case they implement all the services described for the Dissector component (this is the case for the CoAP testing tool)

API services

Services provided by TAT:

METHOD DESCRIPTION
analysis.testcase.analyze Analyze frames from one or more PCAP files and returns the verdict of specified test case.
analysis.testsuite.gettestcases Return the implemented test cases by the testing analysis tool.

API events

None

Protocol Dissector

Description

Component for decoding network traces provided as a pcap file, and which returns human readable description of them including headers, fields, values etc.

API services

METHOD DESCRIPTION
dissection.dissectcapture Dissects frames provided as input in PCAP file, returns a json file containing the different fields of the specified protocol. Generates Token for future interactions.
dissection.getframes Return all the frames -or selection if specified- of a previously dissected file. Token for context identification needs to be provided.

API events

Dissector generates and publishes the following messages:

MESSAGES PUBLISHED DESCRIPTION
dissection.autotriggered A trace dissection of all the generated packets since the start of the testcase. This message is printed in the GUI.

Agent

Introduction

The “agent” is a program a user downloads gitlab or Pypi and which allows him/her to connect the IUT to the testing tool.

Through the agent, the testing tool can (remotely) interact with the IUT, for example by changing its configuration or injecting/receiveing packets in/from the interface the IUT is listening to.

Source code

N.B. this is work in progress

agent’s gitlab repo

Routing key / topics basics

see requirements coming from the F-Interop Platform

Embedded services and interfaces

Agent’s embeded services have two purposes, first, triggering actions in the IUTs, particularly interesting when automating an IUT, and secondly opening virtual interfaces on the machine which runs them which work as a VPN with testing tool, and other IUTs which are part of the session.

For IUTs which “talk” IP , agents open a virtual network interface on a PC, this builds the VPN-like setup where both IUTs are part of , and also the testing tool.

The contributor implementing an automated-IUT or bot-like IUT MAY use these agent hooks for automating the whole execution of a test. Note that this is the advance way of using F-Interop and is not mandatory to do so for using F-Interop.

Tun interface

Routing key: control.tun.toAgent.agent_name

Contents Type Description
_type String Services can be one of the following {“tun.start”, “tun.stop”,“tun.restart”} TBD
ipv4_host String e.g. xxx.xxx.xxx.xxx
ipv4_network String
ipv4_netmask String
ipv6_host String
ipv6_prefix String

Event for starting tun interface

 - - -
ROUTING_KEY: control.tun.toAgent.agent_coap_client
 - - -
PROPS: {"content_type": "application/json"}
 - - -
BODY {
    "_type": "tun.start",
    "ipv6_host": ":3",
    "ipv6_prefix": "bbbb",
    "name": "agent_coap_client"
    }
 - - -

Routing key: data.tun.fromAgent.agent_name

For the time being channel_id is used

Contents Type Description
data String packet bytes captured in tun

Data packet from agent tun to backend

 - - -
ROUTING_KEY: data.tun.fromAgent.coap_client_agent
 - - -
PROPS: { "content_type": "application/json" }
 - - -
BODY {
    "_type": "packet.sniffed.raw",
    "timestamp": "1492014786.91",
    "msg_id": "b53d840f-1f9d-11e7-a462-acbc32cdf38b",
    "routing_key": "data.tun.fromAgent.coap_client_agent",
    "interface_name": "tun0",
    "data": [96, 9, 36, 188, 0, 19, 17, 64, 187, 187, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 0, 0, 1, 187, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 3, 218, 110, 22, 51, 0, 19, 57, 80, 64, 1, 141, 176,
    180, 116, 101, 115, 116, 193, 2]}
 - - -

ZMQ interface

Routing key: control.agent.zmq.agent_id

An agent can open ZMQ socket and send messages over it.

Contents Type Description
zmq_socket_type String [REQ, PUB]
connect path Where to connect to
data String data to send through the ZMQ socket
{
    "_type" : "agent_hook.zmq", 
    "zmq_socket_type": "REQ",
    "connect": "localhost:50000",
    "data": {"sub": ["cmdToMote"]}
}

HTTP interface

Routing key: control.agent.http.agent_id

An agent can send an HTTP request and send back the answer.

Contents Type Description
verb String [GET, POST, PUT, DELETE]
URL String URL that the agent will target. It has to be accessible from the host where the agent is running.
data JSON JSON to send
{
    "_type" : "agent_hook.http",
    "url": "http://example.com",
    "verb": "GET",
    "data": {"data": "data"}
}

Ping interface

Routing key: control.agent.ping.agent_id

An agent can send an ICMP echo request (ping) to a given host. On the background, the agent is performing it through the shell command.

** Request format **

Contents Type Description
version String IPv4 is the default. IPv6 is also available
host String my.example.com
payload String payload to send on the ping request
count int Number of packet to send. Default 1
interface String Network interface to use
{
    "_type": "agent_hook.ping",
    "version": "IPv6",
    "host": "bbbb::1",
    "payload": "cafe",
    "count": 1,
    "interface": "tun0"
}

Agent: kick-off document

With this doc we expect to help people understand what the agent does, and how to deploy it, and also make a short mention of each agent mode of use.

Please read agent’s README file before starting with this one.

Use Case 1: IPv6 Tunnel using Agent’s TUN interface

The final objective is to create a sort of IPv6 based VPN between two computers.

What can I do with that? Send any IPv6 packets from one machine to the other (both ways), e.g. ping6, CoAP ping, CoAP get, etc

             +--------------------------------+                                                                        +--------------------------------+
             | +----------------------------+ |                                                                        | +----------------------------+ |
             | |         IPv6-based         | |                                                                        | |         IPv6-based         | |
             | |        communicating       | |                                                                        | |        communicating       | |
             | |      piece of software     | |                                                                        | |      piece of software     | |
             | |      (e.g. coap client)    | |                     +----------------------------+                     | |      (e.g. coap sever)     | |
             | |                            | |                     |                            |                     | |                            | |
             | +----------------------------+ |                     |      Packet Routerased     |                     | +----------------------------+ |
      PC     |                                |                     |    (routes AMQP packets)   |                     |                                | PC
      user 1 | +------tun interface--------+  |                     |                            |                     | +------tun interface--------+  | user 2
             |                                |                     |                            |                     |                                |
             |            Agent (agent_x)     |                     +----------------------------+                     |            Agent (agent_y)     |
             |                                |                                                                        |                                |
             |          (tun mode)            |                              ^    +                                    |          (tun mode)            |
             |                                |                              |    |                                    |                                |
             |                                |                              |    |                                    |                                |
             +--------------------------------+                              |    |                                    +--------------------------------+
                                                         *.fromAgent.agent_x |    |  *.toAgent.agent_y
                           +     ^                                           |    |                                                 +     ^
                           |     |                                           |    |                                                 |     |
data.tun.fromAgent.agent_x |     | data.tun.toAgent.agent_x                  |    |                      data.tun.fromAgent.agent_y |     | data.tun.toAgent.agent_y
                           |     |                                           |    |                                                 |     |
                           v     +                                           +    v                                                 v     +

             +------------------------------------------------------------------------------------------------------------------------------------------------>
                                                                              AMQP Event Bus
             <-------------------------------------------------------------------------------------------------------------------------------------------------+

Experiment setup

examples for the ipv6 communicating piece of software:

And for the lazy ones, just using ping6 would do..

supported OSs

Installation

  1. before anything make sure to have:

    i. if agent running on MacOs:

    install tuntap driver needed: brew install Caskroom/cask/tuntap

    ii. install python2 (needed for agent compoennt)

    iii. install python3 (needed for the packet-router component)

  2. agent:

    i. download agent code: https://gitlab.f-interop.eu/f-interop-contributors/agent

    ii. git checkout tags/agent-kickoff-tutorial

    iii. read README.md

  3. packet-router

    i. download packet-router code: https://gitlab.f-interop.eu/f-interop-contributors/coap_testing_tool

    ii. don’t read README.md, we just need the packet router component from this repo

  4. you need a rabbit mq session (which is basically a vhost, and users with some permissions on it).

    Either:

    i. ask a f-interop developer (federico.sismondi@inria.fr) to generate one

    ii. deploy your own rabbit message broker

    iii. use orchestrator API for automatically deploying one <- simpler option if API is up and running

  5. install requirements for agent and packet-router (see requirements.txt in respective dirs)

Running the experiment

0. On both sides:

export AMQP connection variables, the correct ones!! ( see installation section, step 3)

e.g.

export AMQP_EXCHANGE='amq.topic'

export AMQP_URL="amqp://someUser:somePassword@f-interop.rennes.inria.fr/sessionXX"

1. From agent1’s side :

sudo python -m agent connect --url $AMQP_URL --name coap_client_agent

2. From agent2’s side :

sudo python -m agent connect --url $AMQP_URL --name coap_server_agent

3. From any side:

python3 -m coap_testing_tool.packet_router.packet_router

Note: By default packet router routes messages from ‘coap_client_agent’ to 'coap_server_agent’.

4. Send message to agents to enable the tun (from inside agent’s source directory):

` python3 -m utils.bootstrap_agent coap_client_agent bbbb :1

python3 -m utils.bootstrap_agent coap_server_agent bbbb :2 `

NOTE: if warning printed on the output of last command, then something went wrong, probably one of the agents is either not running, or using the wrong amqp params

5. Test the tun!

from coap_client_agent

ping6 bbbb::02

from coap_server_agent

ping6 bbbb::01

you can test with a coap client-server interaction!

sniff the traffic with wireshark and verify all traffic exchanged is AMQP traffic

Use Case 2 : 802.15.4 Tunnel using Agent’s SERIAL interface

The final objective is to create a sort of 802.15.4 based VPN between WPAN devices.

What can I do with that? Send 802.15.4 frames, just as if both devices were sitting next to each other.

This mode of use can be used for example for testing 6LoWPAN interop tests between two devices.

IMPORTANT: This mode of functioning assumes the following IEEE802.15.4 settings:

  1. Channel, modulation, data-rate (Channels 11-26 at 2.4 GHz).
  2. MAC mode is beaconless.
  3. Security is off

for more see agent’s README.md file.



                                                                          +----------------------------+
                                                                          |                            |
                                                                          |      Packet Routerased     |
                                                                          |    (routes AMQP packets)   |
                                                                          |                            |
                                                                          |                            |
                                                                          +----------------------------+

                                                                                   ^    +
                                                                                   |    |
                                                               *.fromAgent.agent_x |    | *.toAgent.agent_y
                                                                                   |    |
                                                                                   |    |
                                                             (*.fromAgent.agent_y) |    | (*.fromAgent.agent_x)
                                                                                   |    |
                                                                                   |    |
                                                                                   |    |
                                                                                   +    v

                   +------------------------------------------------------------------------------------------------------------------------------------------------>
                                                                                    AMQP E^ent Bus
                   <-------------------------------------------------------------------------------------------------------------------------------------------------+

                                 ^     +                                                                        ^     +
                                 |     |                                                                        |     |
data.serial.fromAgent.agent_x    |     | data.serial.toAgent.agent_x           data.serial.fromAgent.agent_y    |     | data.serial.toAgent.agent_y
                                 |     |                                                                        |     |
                                 +     v                                                                        +     v

                           +----------------+                                                             +----------------+
                           |                |                                                             |                |
                           |                |                                                             |                |
                           |     Agent      |                                                             |     Agent      |
                           | (serial mode)  |                                                             | (serial mode)  |
                           |                |                                                             |                |
                           |                |                                                             |                |
                           +-------+--------+                                                             +-------+--------+
                                   | USB interface                                                                | USB interface
                                   | (SLIP protocol)                                                              | (SLIP protocol)
                           +-------+--------+                        +---------------+                    +-------+--------+                        +---------------+
                           |                |     802.15.4 frame     |               |                    |                |     802.15.4 frame     |               |
                           |                |  <-----------------+   |   802.15.4    |                    |                |  <-----------------+   |   802.15.4    |
                           |    probe mote  |                        |     user      |                    |    probe mote  |                        |     user      |
                           |  (acti^e mode) |  +----------------->   |    de^ice     |                    |  (acti^e mode) |  +----------------->   |    de^ice     |
                           |                |                        |               |                    |                |                        |               |
                           |                |                        |               |                    |                |                        |               |
                           +----------------+                        +---------------+                    +----------------+                        +---------------+

                        |___________________________ ___________________________________|                |___________________________ ___________________________________|
                                                    V                                                                                V
                                                  USER 1                                                                           USER 2

Experiment setup

supported OS for the agent

supported HW for the probes

Installation

  1. before anything make sure to have:

    i. install python2 (needed for agent compoennt)

    ii. install python3 (needed for the packet-router component)

  2. agent:

    i. download agent code: https://gitlab.f-interop.eu/f-interop-contributors/agent

    ii. git checkout tags/agent-kickoff-tutorial

    iii. read README.md

  3. packet-router

    i. download packet-router code: https://gitlab.f-interop.eu/f-interop-contributors/coap_testing_tool

    ii. don’t read README.md, we just need the packet router component from this repo

  4. you need a rabbit mq session (which is basically a vhost, and users with some permissions on it).

    Either:

    i. ask a f-interop developer (federico.sismondi@inria.fr) to generate one

    ii. deploy your own rabbit message broker

    iii. use orchestrator API for automatically deploying one <- simpler option if API is up and running

  5. install requirements for agent and packet-router (see requirements.txt in respective dirs)

  6. download and compile probe firmware for the correct platform (depends on which mote you own) (TBD)[TBD]

Running the experiment

0. On both sides:

export AMQP connection variables, and USB params for the serial connection

check usb port, with for example with ls /dev/tty*

export FINTEROP_CONNECTOR_SERIAL_PORT=/dev/tty.XXX export FINTEROP_CONNECTOR_BAUDRATE=xxxxx (e.g. 115200 for motesky)

env vars:

export AMQP_EXCHANGE='amq.topic' export AMQP_URL="amqp://someUser:somePassword@f-interop.rennes.inria.fr/sessionXX"

1. From agent1’s side :

python -m agent connect --url $AMQP_URL --name coap_client_agent --serial

2. From agent2’s side :

python -m agent connect --url $AMQP_URL --name coap_server_agent --serial

3. From any side:

python3 -m coap_testing_tool.packet_router.packet_router

Note: By default packet router routes messages from 'coap_client_agent’ to 'coap_server_agent’.

NOTE: if warning printed on the output of last command, then something went wrong, probably one of the agents is either not running, or using the wrong amqp params

4. Test the tun!

Contact me

Guided Tour: CoAP Test Suite

Terminology

CoAP Test Description

List of tests cases is available in : http://doc.f-interop.eu/testsuites/coap

An example of a test case : http://doc.f-interop.eu/tests/TD_COAP_CORE_01

Location Models

User to User session, each user drives one IUT

Screenshot

Single user session, user tests IUT vs reference implementation

Screenshot

Variation: Agent as router (WIP)

Screenshot

Guided Tour of a CoAP interop test

This subsection describes the whole process of executing an interop test using the single-user mode, for testing against an reference implemetation

1. Account and interop session creation

Create an acount on: [F-Interop portal][http://go.f-interop.eu/registration]

Account creation

Screenshot

CoAP Interop session creation

after creating account and logging in you should see:

Screenshot

Enter New Test Session, and select CoAP test suite (single-user)

Screenshot

CoAP Interop session configuration

Screenshot

Continue, and then click on start, testing tool container will be deployed right after, and also the reference implementation instance.

2. Preparing the testing environment

Here we show the steps needed to set up the test environment.

Agent running zoom-in

The test need to be executed using a VPN-like setup, this is why user needs to download the agent component which works as a VPN client. This will only open a virtual interface, assign a private IP, and configure some IP routes in the routing table.

Screenshot

Agent configured zoom-in

Screenshot

3. Consult the test doc before starting the test (recomended)

check out the test spec hosted online at:

http://doc.f-interop.eu/testsuites/coap

see example on the right ->

+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Tescase ID       | TD_COAP_CORE_01                                                                                                                                                                                          |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Testcase URL     | http://doc.f-interop.eu/tests/TD_COAP_CORE_01                                                                                                                                                            |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Test Purpose     | Perform GET transaction(CON mode)                                                                                                                                                                        |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Configuration ID | COAP_CFG_01                                                                                                                                                                                              |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| References       | [COAP] 5.8.1, 1.2, 2.1, 2.2, 3.1                                                                                                                                                                         |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Preconditions    | - Server offers the resource /test with resource content is not empty that handles GET with an arbitrary payload                                                                                         |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Notes            | We acknowledge the efforts made by ETSI CTI and ProbeIT who have contributed to the content of this document                                                                                             |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Test Sequence    |                                                                                                                                                                                                          |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                  | STIMULI - TD_COAP_CORE_01_step_01                                                                                                                                                                        |
|                  |                                                                                                                                                                                                          |
|                  |     - Client is requested to send a GET request with                                                                                                                                                     |
|                  |         - Type = 0(CON)                                                                                                                                                                                  |
|                  |         - Code = 1(GET)                                                                                                                                                                                  |
|                  |                                                                                                                                                                                                          |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                  | CHECK - TD_COAP_CORE_01_step_02                                                                                                                                                                          |
|                  |                                                                                                                                                                                                          |
|                  |     - The request sent by the client contains                                                                                                                                                            |
|                  |         - Type=0 and Code=1                                                                                                                                                                              |
|                  |         - Client-generated Message ID (* CMID)                                                                                                                                                           |
|                  |         - Client-generated Token (* CTOK)                                                                                                                                                                |
|                  |         - UTEST Uri-Path option "test"                                                                                                                                                                   |
|                  |                                                                                                                                                                                                          |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                  | CHECK - TD_COAP_CORE_01_step_03                                                                                                                                                                          |
|                  |                                                                                                                                                                                                          |
|                  |     - Server sends response containing                                                                                                                                                                   |
|                  |         - Code = 2.05(Content)                                                                                                                                                                           |
|                  |         - Message ID = CMID, Token = CTOK                                                                                                                                                                |
|                  |         - Content-format option                                                                                                                                                                          |
|                  |         - Non-empty Payload                                                                                                                                                                              |
|                  |                                                                                                                                                                                                          |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                  | VERIFY - TD_COAP_CORE_01_step_04                                                                                                                                                                         |
|                  |                                                                                                                                                                                                          |
|                  |     - Client displays the received information                                                                                                                                                           |
|                  |                                                                                                                                                                                                          |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+


"mantainer": "Federico Sismondi", "mantainer_email": "federico.sismondi@inria.fr" if you spotted any errors or you want to comment on sth don't hesitate to contact me.

4. Start running the interop tests

Start a testcase zoom-in

Start the first test case.

Start a testcase zoom-in

Start the first test case.

Screenshot

5. Filtering information displayed in GUI

HINT: messages displayed in GUI can be filtered by clicking on each tag in timeline on the top-left

Testcase information filtering: zoom-in

Screenshot

Raw byte representation of packets exchanged zoom-in

Screenshot

Packet dissection representation: zoom-in

Screenshot

6. Step Sequence Execution zoom-in

The information user needs in orded execute the test case’s step.

IMPORTANT: The information displayed is only relevant to the IUT role. If user runs a CoAP client, then only CoAP client test actions are going to be displayed.

Screenshot

7. Example of Execution: TD_COAP_CORE_01

In the following images we display how it would look like to execute completely a test case.

Here we use as CoAP client Copper plugin for mozilla firefox

CoAP client executes STIMULI step for TD_COAP_CORE_01 zoom-in

Screenshot

Dissection of CoAP GET REQUEST message zoom-in

Screenshot

Dissection of CoAP RESPONSE zoom-in

Screenshot

VERIFY step, did my IUT display correctly the received info? zoom-in

In here we use an example Copper coap client, of course each CoAP client implementation will have a implementation specific way of verifying this step.

Screenshot

8. Traces analysis and validation zoom-in

Screenshot

Screenshot

9. Final report, networks traces and sesion data download

The F-Interop platform keeps only a minimal set of results and data from the sessions, so don’t forget to download all your data before you close the session!

Test suite results summary

Screenshot

Test cases results

Screenshot

Download all generated network traces (PCAP files)

Oops, feature still not available :/

Download all generated data and logs

Oops, feature still not available :/

Contact me

Guided Tour: 6LoWPAN Test Suite

Terminology

Test Description

List of tests cases is available in :
http://doc.f-interop.eu/testsuites/6lowpan

An example of a test case :
http://doc.f-interop.eu/testsuites/6lowpan/TD_6LOWPAN_HC_01

Test Configurations

Most of scenarios involve only a 6lowpan link that interconnects two nodes or three nodes.
Screenshot

a. Scenario involving two LBR include two 6lowpan networks connected to F-Interop platform :
Screenshot

b. scenario involving LR and Hosts :
Screenshot

c. Others scenarios :
TBD

Location Models

EUT1 driven by UserA which is located in France, and EUT2 driven by UserB which is located in another country

Screenshot

EUT1 driven by UserA sitting next to UserB driving EUT2

a. EUT1 driven by UserA and EUT2 driven by UserB in same place - communicate through virtual tunnel.
EUT1 and EUT2 must not be close in order to avoid local loop

Screenshot

b. EUT1 driven by UserA and EUT2 driven by UserB in the same place and communicate in direct mode.
Screenshot

EUT1 and EUT2 sitting next to each other, both driven by the same User - communication in direct mode

Screenshot

Guided Tour of a 6LoWPAN interop test

This subsection describes the whole process of executing an interop test using the 6LoWPAN test suite (single user) mode and 6LoWPAN test suite (user to user) mode implemented on f-interop plarform.

1. Agent and Probe-mote general information

Agent

The agent is a program that the user downloads from the F-Interop site and allows him/her to connect the IUT to the F-Interop server. More details in : agent’s gitlab repo

The agent uses the serial mode to communicate with “probe-mote”.

Probe-mote

The probe mote is a mote which is flashed with the firmwares available in this directory “sixlowpan_motes_firmware” so that it can sniff the 6LoWPAN packets and send them to the agent.

We have two types of probe mote:

Passive-probe

                        +----------------+
                        |                |
                        |   AMQP broker  |
                        |                |
                        |                |
                        +----------------+

                                 ^
                                 |
data.serial.fromAgent.agent_name |
                                 |
                                 +

                         +----------------+
                         |                |
                         |                |
                         |     Agent      |
                         | (serial mode)  |
                         |                |
                         |                |
                         +-------+--------+
                                 | USB interface
                                 | (SLIP protocol)
                         +-------+--------+
                         |                |
                         |                |
                         |    probe mote  |
                         |  (passive mode)|
                         |                |
                         |                |
                         +-------+--------+
                                 |
                                 |
                                 |
                                 |
+---------------+                |              +---------------+
|               |                |              |               |
|   802.15.4    |       <--------+--------+     |   802.15.4    |
|     user      |         802.15.4 frames       |     user      |
|    device     |       +----------------->     |    device     |
|               |                               |               |
|               |                               |               |
+---------------+                               +---------------+
Active-probe

                           +----------------+
                           |                |
                           |   AMQP broker  |
                           |                |
                           |                |
                           +----------------+


                                 ^     +
                                 |     |
data.serial.fromAgent.agent_name |     | data.serial.toAgent.agent_name
                                 |     |
                                 +     v

                           +----------------+
                           |                |
                           |                |
                           |     Agent      |
                           | (serial mode)  |
                           |                |
                           |                |
                           +-------+--------+
                                   | USB interface
                                   | (SLIP protocol)
                           +-------+--------+                        +---------------+
                           |                |     802.15.4 frame     |               |
                           |                |  <-----------------+   |   802.15.4    |
                           |    probe mote  |                        |     user      |
                           |  (active mode) |  +----------------->   |    device     |
                           |                |                        |               |
                           |                |                        |               |
                           +----------------+                        +---------------+

2. Preparing the testing environment

Configuration of probe-mote

The source code of probe-mote was implemented in contiki 3.0, it is available in :
https://gitlab.f-interop.eu/fsismondi/probes_802_15_4/tree/twoway_sniffer/examples/twowaysniffer. It supports any contiki hardware platform (only openmote and skymote were tested).

Steps to configure the motes in order to be used in our project

The configuration of the motes has been made on linux_OS.
The hardware motes we tested, are:

The Gitlab source code will be found here : gitlab_probe_motes_fw
To be able to execute the code, you must have Python installed on your machine.
- Type the following command to download the “coap_testing_tool” repository:

sudo git clone --recursive -b probe_motes_fw https://gitlab.f-interop.eu/f-interop-contributors/coap_testing_tool.git cd coap_testing_tool

ls /dev/tty*

export FINTEROP_CONNECTOR_SERIAL_PORT='/dev/ttyUSB0'

python -m automated_IUTs.utils.mote_flasher flash [type_of_probe_mote] --channel [channel_number] --port [port_number] --platform [platform]

Example: Passive-probe for “openMote cc2538”:

python -m automated_IUTs.utils.mote_flasher flash passive_probe --channel 26 --port /dev/ttyUSB0 --platform openmote-RevA1

Steps for configuring other motes

TBD

3. Account and interop session creation

Create an acount on: [F-Interop portal][http://go.f-interop.eu/registration]

Account creation

Screenshot

6LoWPAN Interop session creation

after creating account and logging in, you should see:

Screenshot

Enter New Test Session, and select 6LoWPAN test suite (single user) for Local model or 6LoWPAN test suite (user to user) for Remote model

Screenshot

6LoWPAN Interop session configuration

Add the name of your device in the appropriate place, then select the test cases you want to test.

Screenshot

Click on save configuration button, and then click on start.

You should click on Yes button 2 times in a row in the case of 6LoWPAN test suite (single user).

Screenshot

In the case of 6LoWPAN test suite (user to user), each user need to click on the Yes button. For that the user who was created the session need to share the url_link with the second user to let him join the session. Then click on “continue” button.

Now you should run the agent. But before you need to copy the AMQP_URL of your session. You will need it for launching the agent. In the case of 6LoWPAN test suite (user to user), both users need to copy the amqp_url and past it in their terminal (see description below).

Screenshot

Then click on the “confirm” button.

4. Local model (using passive-probe)

We have used the ATREB233_XPRO module (EUT) for creating a 6LoWPAN network in the raspberry pi3. The ATREB233_XPRO module has been pluged into the pins of the raspberry, and the probe-mote has been connected to the serial port of the raspberry. the code of the agent has been cloned on the raspberry pi, see picture below:

Screenshot

Launching the agent
For launching the code of the agent, you must have your “probe-mote” configured as passive-probe (see previous configurations).

pip install ioppytest-agent

export FINTEROP_CONNECTOR_SERIAL_PORT=/dev/ttyXXXX

export FINTEROP_CONNECTOR_BAUDRATE=XXXXX

export AMQP_EXCHANGE='amq.topic'

export AMQP_URL="amqp://xxxxx@mq.dev.f-interop.eu:443/xxxxx?heartbeat_interval=0&blocked_connection_timeout=2&retry_delay=1&socket_timeout=1&connection_attempts=3"

python -m agent connect --url $AMQP_URL --name eut1 --serial Now you can follow the instructions presented in the F-interop platform.

For making sure that the agent works correctly, you have to see the uplink messages when you will sent an “echo request”:

         _   
        / \   
       /   \   
      /     \   
     /       \   
    /__     __\      
       |   |              _ _       _         
       |   |             | (_)     | |            
       |   |  _   _ _ __ | |_ _ __ | | __           
       |   | | | | | '_ \| | | '_ \| |/ /         
       |   | | |_| | |_) | | | | | |   <   
       |   |  \__,_| .__/|_|_|_| |_|_|\_\                 
       |   |       | |              
       |   |       |_|                     
       !___!      
      \  O  /    
       \/|\/    
         |    
        / \   
      _/   \ _        

5. Remote model (using active-probe) (e.g. eut1 @ Paris, eut2 @ china)

In the case of remote model, you are testing your implementation with another user who is located in another place in the world.

Both users must configure their probe-motes as an active probe. (see how to configure your probe mote as active probe in the section ‘Configuration of probe-mote’ below)

Here is an example of implementation that each user must set up in their site :

Screenshot

Launching the agent
For launching the code of the agent, each user must have his “probe-mote” configured as active-probe.

pip install ioppytest-agent

export FINTEROP_CONNECTOR_SERIAL_PORT=/dev/ttyXXXX

export FINTEROP_CONNECTOR_BAUDRATE=XXXXX

export AMQP_EXCHANGE='amq.topic'

export AMQP_URL="amqp://xxxxx@mq.dev.f-interop.eu:443/xxxxx?heartbeat_interval=0&blocked_connection_timeout=2&retry_delay=1&socket_timeout=1&connection_attempts=3"

For making sure that the agent works correctly, send an “echo request” command (ping6). You have to see the uplink and the downlink messages:

         _   
        / \   
       /   \   
      /     \   
     /       \   
    /__     __\      
       |   |              _ _       _         
       |   |             | (_)     | |            
       |   |  _   _ _ __ | |_ _ __ | | __           
       |   | | | | | '_ \| | | '_ \| |/ /         
       |   | | |_| | |_) | | | | | |   <   
       |   |  \__,_| .__/|_|_|_| |_|_|\_\                 
       |   |       | |              
       |   |       |_|                     
       !___!      
      \  O  /    
       \/|\/    
         |    
        / \   
      _/   \ _        
   ___      
  |   |      
  |   |       _                     _ _       _    
  |   |      | |                   | (_)     | |  
  |   |    __| | _____      ___ __ | |_ _ __ | | __
  |   |   / _` |/_ \ \ /\ / / '_ \| | | '_ '\| |/ /
  |   |  | (_| | (_) \ V  V /| | | | | | | | |   <
  |   |   \__,_|\___/ \_/\_/ |_| |_|_|_|_| |_|_|\_\
  |   |
__!   !__,
\       / \O
 \     / \/|
  \   /    |
   \ /    / \
    Y   _/  _\

Contact me

ioppytest-cli: kick-off document

This tutorial shows how to install and use the ioppytest-cli for interacting with GUI

Installation

Tested with python3:

python3 -m pip install ioppytest-utils --user

$ python3 -m pip install ioppytest-utils --user
Collecting ioppytest-utils
  Using cached ioppytest-utils-0.1.2.tar.gz
Collecting click==6.7 (from ioppytest-utils)
  Using cached click-6.7-py2.py3-none-any.whl
Collecting click_repl==0.1.2 (from ioppytest-utils)
  Downloading click_repl-0.1.2-py3-none-any.whl
Collecting pika==0.11.0 (from ioppytest-utils)
  Using cached pika-0.11.0-py2.py3-none-any.whl
Collecting prompt_toolkit==1.0.15 (from ioppytest-utils)
  Downloading prompt_toolkit-1.0.15-py3-none-any.whl (247kB)
    100% |████████████████████████████████| 256kB 3.1MB/s
Collecting wcwidth==0.1.7 (from ioppytest-utils)
  Using cached wcwidth-0.1.7-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from click_repl==0.1.2->ioppytest-utils)
Building wheels for collected packages: ioppytest-utils
  Running setup.py bdist_wheel for ioppytest-utils ... done
  Stored in directory: /home/fedesismo/.cache/pip/wheels/e1/33/a6/a1b6aa36ec170c5057606ebe290e532812cd4adffb6a81f948
Successfully built ioppytest-utils
Installing collected packages: click, wcwidth, prompt-toolkit, click-repl, pika, ioppytest-utils
Successfully installed click-6.7 click-repl-0.1.2 ioppytest-utils-0.1.2 pika-0.11.0 prompt-toolkit-1.0.15 wcwidth-0.1.7
You are using pip version 9.0.1, however version 9.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Test installation

$ ioppytest-cli
Usage: ioppytest-cli [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  action                   Execute interop test action
  chat                     Send chat message, useful for user-to-user...
  check_connection         (REPL only) Check if AMQP connection is...
  clear                    Clear screen
  connect                  Connect to an AMQP session and start...
  download_network_traces  Downloads all networks traces generated...
  enter_debug_context      (REPL only) Provides user with some extra...
  exit                     Exits REPL
  get_session_parameters   Print session state and parameters
  get_session_status       Retrieves status information from testing...
  gui_display_message      Sends message to GUI
  gui_request_file_upload  Request user to upload a file, saves it in...
  ignore                   (REPL only) Do not notify any more on message...
  repl                     Interactive shell, allows user to interact...

Interacting with the GUI demo:

1. create a dummy session GUI instance and get AMQP URL for communicating with GUI:

Create a dummy session using F-Interop DEV platform

Screenshot

Deploy the session, go to INFO and copy AMQP_URL (right-top), this is a standard uri schema and should look like:

amqp://3UO5U518:GQZUNV2L@mq.dev.f-interop.eu:443/2f5f0dd9-e933-4d33-8d90-f3ca6a373121

Screenshot

2. Open a terminal and export this as an env var:

$ export AMQP_URL=amqp://3UO5U518:GQZUNV2L@mq.dev.f-interop.eu:443/2f5f0dd9-e933-4d33-8d90-f3ca6a373121

3. GUI Display: “Hello world”

$ ioppytest-cli gui_display_message
Usage: ioppytest-cli gui_display_message [OPTIONS] TEXT_MESSAGE

Error: Missing argument "text-message".
$ ioppytest-cli gui_display_message --help
Usage: ioppytest-cli gui_display_message [OPTIONS] TEXT_MESSAGE

  Sends message to GUI

Options:
  --user-id TEXT  User ID in case there are several users in session
  --help          Show this message and exit.
$ ioppytest-cli gui_display_message "hello world"
No connection established yet, setting up one..
Sending message..
[User input] message display sent to ui.user.all.display

Screenshot

4. GUI Requests: Uploading a file

$ ioppytest-cli gui_request_file_upload
Usage: ioppytest-cli gui_request_file_upload [OPTIONS] PATH_TO_FILE

Error: Missing argument "path-to-file".
$ ioppytest-cli gui_request_file_upload --help
Usage: ioppytest-cli gui_request_file_upload [OPTIONS] PATH_TO_FILE

  Request user to upload a file, saves it in directory (if provided) or else
  in .tmp

Options:
  --text-message TEXT  Message to be displayed in GUI
  --user-id TEXT       User ID in case there are several users in session
  --help               Show this message and exit.
$ ioppytest-cli gui_display_message^C
$ ioppytest-cli gui_display_message --help
Usage: ioppytest-cli gui_display_message [OPTIONS] TEXT_MESSAGE

  Sends message to GUI

Options:
  --user-id TEXT  User ID in case there are several users in session
  --help          Show this message and exit.
$ mkdir ./tmp
$ ls
tmp
$ ioppytest-cli gui_request_file_upload tmp --text-message "please upload file now :D"
[User input] sending request to ui.user.all.request
No connection established yet, setting up one..
Traceback (most recent call last):
  File "/home/fedesismo/.local/bin/ioppytest-cli", line 11, in <module>
    sys.exit(main())
  File "/home/fedesismo/.local/lib/python3.5/site-packages/ioppytest_cli/ioppytest_cli.py", line 1351, in main
    cli()
  File "/home/fedesismo/.local/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/fedesismo/.local/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/fedesismo/.local/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/fedesismo/.local/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/fedesismo/.local/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/fedesismo/.local/lib/python3.5/site-packages/ioppytest_cli/ioppytest_cli.py", line 204, in gui_request_file_upload
    msg_response = _amqp_request(msg_request, COMPONENT_ID, timeout=WAIT_TIME_FOR_USER_INPUT)
  File "/home/fedesismo/.local/lib/python3.5/site-packages/ioppytest_cli/ioppytest_cli.py", line 1285, in _amqp_request
    type(request_message)
Exception: Response timeout! rkey: ui.user.all.request , request type: <class 'messages.messages.MsgUiRequestUploadFile'>

In GUI we will see:

Screenshot

about the Exception: Response timeout! this is what happens when user doesnt upload anything in GUI :/

If we try again and we upload some file (any extension):

$ ioppytest-cli gui_request_file_upload tmp --text-message "please upload file now :D"
[User input] sending request to ui.user.all.request
No connection established yet, setting up one..
[User input] saved file OMA-ETS-LightweightM2M-V1_0-20140226-C.pdf in path tmp

Screenshot

$ ls tmp
OMA-ETS-LightweightM2M-V1_0-20140226-C.pdf

the file provided by the user now is in our tmp directory :D

5. Under the hood

Repeat the whole sequence from 1 to 3, but now using a second terminal which runs ioppytest-cli connect

$ ioppytest-cli connect

[Event bus message] [<class 'messages.messages.MsgTestingToolComponentReady'>]

------------  -------------------------------
_api_version  1.0.15
component     amqp_listener_5b04caf6
description   amqp_listener_5b04caf6 is READY
------------  -------------------------------
[UI message]
    Message: MsgUiDisplay(_api_version = 1.0.15, fields = [{'name': 'hola', 'type':
    tags: {}
    Fields: [{'name': 'hola', 'type': 'p'}]
    R_key: ui.user.all.display
    corr_id: None

[Event bus message] [<class 'messages.messages.MsgUiRequest'>]

------------  -----------------------------------------------
 _api_version    1.0.15
 fields          {'name': 'yo give me the file', 'type': 'file'}
 tags            {}
------------  -----------------------------------------------

(...)

for more info:

http://doc.f-interop.eu/#graphical-user-interface

Events (core API)

This section describes the format of the messages used in F-Interop.

This section of the documentation is autogenerated by tool

Check out the messages library tool

Version 1.0.15

Orchestrator events

User interface events

ui.user.all.display

Routing key ui.user.all.display
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for displaying Markdown text to user interface

Source code: MsgUiDisplayMarkdownText

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: ui.user.all.display
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "5777421b-890b-4fe9-9d5c-967c90482a58",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "fields": [
        {
            "type": "p",
            "value": "Hello World!"
        }
    ],
    "level": null,
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "fields": [
        {
            "type": "p",
            "value": "Hello World!"
        }
    ],
    "level": null,
    "tags": {}
}

ui.user.all.request

Routing key ui.user.all.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for requesting a text input on UI

Source code: MsgUiRequestTextInput

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: ui.user.all.request
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "bf897af9-e4ac-40c2-a017-68cac99ad538",
    "timestamp": 1523606906,
    "reply_to": "ui.user.all.reply",
    "correlation_id": "bf897af9-e4ac-40c2-a017-68cac99ad538"
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "fields": [
        {
            "name": "input_name",
            "type": "text"
        }
    ],
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "fields": [
        {
            "name": "input_name",
            "type": "text"
        }
    ],
    "tags": {}
}

ui.user.all.request

Routing key ui.user.all.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for requesting confirmation button

Source code: MsgUiRequestConfirmationButton

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: ui.user.all.request
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "3349a0e2-4bfd-4e5a-a9e7-dbed422fd740",
    "timestamp": 1523606906,
    "reply_to": "ui.user.all.reply",
    "correlation_id": "3349a0e2-4bfd-4e5a-a9e7-dbed422fd740"
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "fields": [
        {
            "name": "test_button",
            "type": "button",
            "value": true
        }
    ],
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "fields": [
        {
            "name": "test_button",
            "type": "button",
            "value": true
        }
    ],
    "tags": {}
}

ui.core.session.get.request

Routing key ui.core.session.get.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for requesting session information to UI

Source code: MsgUiRequestSessionConfiguration

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: ui.core.session.get.request
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "9b8aba2e-a728-4937-b192-98ad76c79ee0",
    "timestamp": 1523606906,
    "reply_to": "ui.core.session.get.reply",
    "correlation_id": "9b8aba2e-a728-4937-b192-98ad76c79ee0"
}
 -  -  -
Message body: {
    "_api_version": "1.0.15"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15"
}

ui.core.session.get.reply

Routing key ui.core.session.get.reply
Requirements
Type Event
Pub/Sub UI -> TT
Description: Message for requesting session information to UI

Source code: MsgUiSessionConfigurationReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: ui.core.session.get.reply
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "787a007b-4ea1-4465-a03d-ca299b3b3587",
    "timestamp": 1523606906,
    "correlation_id": "b93cb41c-874b-4a47-bc6b-83032e1852ab"
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "ok": true
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "ok": true
}

ui.user.all.request

Routing key ui.user.all.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for file upload request on UI

Source code: MsgUiRequestUploadFile

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: ui.user.all.request
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "b1714998-1bcc-464a-a2e3-e425aeafe5fa",
    "timestamp": 1523606906,
    "reply_to": "ui.user.all.reply",
    "correlation_id": "b1714998-1bcc-464a-a2e3-e425aeafe5fa"
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "fields": [
        {
            "name": "upload a file",
            "type": "file"
        }
    ],
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "fields": [
        {
            "name": "upload a file",
            "type": "file"
        }
    ],
    "tags": {}
}

ui.user.all.request

Routing key ui.user.all.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for checkbox request on UI

Source code: MsgUiRequestQuestionCheckbox

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: ui.user.all.request
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "620a41f7-6e02-44b2-98a9-a8469fa6c0b8",
    "timestamp": 1523606906,
    "reply_to": "ui.user.all.reply",
    "correlation_id": "620a41f7-6e02-44b2-98a9-a8469fa6c0b8"
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "fields": [
        {
            "name": "Choice1",
            "label": "Choice1",
            "type": "checkbox",
            "value": 0
        },
        {
            "name": "Choice2",
            "label": "Choice2",
            "type": "checkbox",
            "value": 1
        }
    ],
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "fields": [
        {
            "name": "Choice1",
            "label": "Choice1",
            "type": "checkbox",
            "value": 0
        },
        {
            "name": "Choice2",
            "label": "Choice2",
            "type": "checkbox",
            "value": 1
        }
    ],
    "tags": {}
}

ui.user.all.request

Routing key ui.user.all.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for radio request on UI

Source code: MsgUiRequestQuestionRadio

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: ui.user.all.request
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "7aad495d-6c72-49db-a8eb-10d2e2ad1ead",
    "timestamp": 1523606906,
    "reply_to": "ui.user.all.reply",
    "correlation_id": "7aad495d-6c72-49db-a8eb-10d2e2ad1ead"
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "fields": [
        {
            "name": "True",
            "type": "radio",
            "value": true
        },
        {
            "name": "False",
            "type": "radio",
            "value": false
        }
    ],
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "fields": [
        {
            "name": "True",
            "type": "radio",
            "value": true
        },
        {
            "name": "False",
            "type": "radio",
            "value": false
        }
    ],
    "tags": {}
}

Testing Tool events

testingtool.ready

Routing key testingtool.ready
Requirements TT MUST publish event as soon as TT is up and listening on the event bus
Type Event
Pub/Sub Testing Tool -> GUI
Description: Used to indicate to the GUI that testing is ready to start the test suite

Source code: MsgTestingToolReady

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: testingtool.ready
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "ad51725b-366e-4266-a369-4717cd4f841f",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "description": "Testing tool READY to start test suite."
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "description": "Testing tool READY to start test suite."
}

session.configuration

Routing key session.configuration
Requirements TT MUST listen to event, and configure accordingly
Type Event
Pub/Sub Orchestrator -> Testing Tool
Description: TT MUST listen to this message and configure the testsuite correspondingly

Source code: MsgSessionConfiguration

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: session.configuration
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "809baed1-4afa-4b9b-8f56-d4b657acdc7d",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "configuration": {
        "testsuite.testcases": [
            "someTestCaseId1",
            "someTestCaseId2"
        ]
    },
    "session_id": "666",
    "testing_tools": "f-interop/someTestToolId",
    "users": [
        "u1",
        "f-interop"
    ]
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "configuration": {
        "testsuite.testcases": [
            "someTestCaseId1",
            "someTestCaseId2"
        ]
    },
    "session_id": "666",
    "testing_tools": "f-interop/someTestToolId",
    "users": [
        "u1",
        "f-interop"
    ]
}

testingtool.configured

Routing key testingtool.configured
Requirements TT MUST publish event once session.configuration message has been processed.
Type Event
Pub/Sub Testing Tool -> Orchestrator, GUI
Description: The goal is to notify orchestrator and other components that the testing tool has been configured

Source code: MsgTestingToolConfigured

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: testingtool.configured
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "c12fef21-3284-457b-80e4-76de843eb609",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "description": "Testing tool CONFIGURED",
    "session_id": "TBD",
    "testing_tools": "f-interop/interoperability-coap"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "description": "Testing tool CONFIGURED",
    "session_id": "TBD",
    "testing_tools": "f-interop/interoperability-coap"
}

testsuite.start

Routing key testsuite.start
Requirements TT MUST listen to event and start the test suite right after reception. MsgTestSuiteStarted
Type Event
Pub/Sub GUI -> Testing Tool
Description: tbd

Source code: MsgTestSuiteStart

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: testsuite.start
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "d8699913-3d43-4982-9806-e29e6f9daf90",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "description": "Test suite START command"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "description": "Test suite START command"
}

testsuite.testcase.step.stimuli.execute

Routing key testsuite.testcase.step.stimuli.execute
Requirements TT MUST publish event
Type Event
Pub/Sub Testing Tool -> GUI
Description:
- Used to indicate to the GUI (or automated-iut) which is the stimuli step to be executed by the user (or
automated-IUT).

Source code: MsgStepStimuliExecute

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: testsuite.testcase.step.stimuli.execute
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "5ddb52b8-08ca-462f-a0e7-8b8b529a5ac5",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "description": "Please execute TD_COAP_CORE_01_step_01",
    "node": "coap_client",
    "node_execution_mode": "user_assisted",
    "step_id": "TD_COAP_CORE_01_step_01",
    "step_info": [
        "Client is requested to send a GET request with",
        "Type = 0(CON)",
        "Code = 1(GET)"
    ],
    "step_state": "executing",
    "step_type": "stimuli",
    "target_address": "TBD",
    "testcase_id": "TBD",
    "testcase_ref": "TBD"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "description": "Please execute TD_COAP_CORE_01_step_01",
    "node": "coap_client",
    "node_execution_mode": "user_assisted",
    "step_id": "TD_COAP_CORE_01_step_01",
    "step_info": [
        "Client is requested to send a GET request with",
        "Type = 0(CON)",
        "Code = 1(GET)"
    ],
    "step_state": "executing",
    "step_type": "stimuli",
    "target_address": "TBD",
    "testcase_id": "TBD",
    "testcase_ref": "TBD"
}

testsuite.testcase.step.stimuli.executed

Routing key testsuite.testcase.step.stimuli.executed
Requirements TT MUST listen to event
Type Event
Pub/Sub GUI (or automated-IUT)-> Testing Tool
Description:
- Used to indicate stimuli has been executed by user (and it’s user-assisted iut) or by automated-iut

Source code: MsgStepStimuliExecuted

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: testsuite.testcase.step.stimuli.executed
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "fd2b3c44-bc36-4f59-9d44-096ffbebb162",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "description": "Step (stimuli) EXECUTED",
    "node": "coap_client",
    "node_execution_mode": "user_assisted"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "description": "Step (stimuli) EXECUTED",
    "node": "coap_client",
    "node_execution_mode": "user_assisted"
}

testsuite.testcase.step.verify.execute

Routing key testsuite.testcase.step.verify.execute
Requirements TT MUST publish event
Type Event
Pub/Sub Testing Tool -> GUI (or automated-IUT)
Description:
- Used to indicate to the GUI (or automated-iut) which is the verify step to be executed by the user (or
automated-IUT).

Source code: MsgStepVerifyExecute

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: testsuite.testcase.step.verify.execute
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "1712d1a4-ed7c-4938-95ca-5f60c47c5069",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "description": "Please execute TD_COAP_CORE_01_step_04",
    "node": "coap_client",
    "node_execution_mode": "user_assisted",
    "response_type": "bool",
    "step_id": "TD_COAP_CORE_01_step_04",
    "step_info": [
        "Client displays the received information"
    ],
    "step_state": "executing",
    "step_type": "verify",
    "testcase_id": "TBD",
    "testcase_ref": "TBD"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "description": "Please execute TD_COAP_CORE_01_step_04",
    "node": "coap_client",
    "node_execution_mode": "user_assisted",
    "response_type": "bool",
    "step_id": "TD_COAP_CORE_01_step_04",
    "step_info": [
        "Client displays the received information"
    ],
    "step_state": "executing",
    "step_type": "verify",
    "testcase_id": "TBD",
    "testcase_ref": "TBD"
}

testsuite.testcase.step.verify.executed

Routing key testsuite.testcase.step.verify.executed
Requirements TT MUST listen to event
Type Event
Pub/Sub GUI (or automated-IUT)-> Testing Tool
Description:
- Message generated by user (GUI or automated-IUT) declaring if the IUT VERIFY verifies the expected behaviour.

Source code: MsgStepVerifyExecuted

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: testsuite.testcase.step.verify.executed
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "06842fe2-b042-43b4-964e-eede27ff49dd",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "description": "Step (verify) EXECUTED",
    "node": "coap_client",
    "node_execution_mode": "user_assisted",
    "response_type": "bool",
    "verify_response": true
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "description": "Step (verify) EXECUTED",
    "node": "coap_client",
    "node_execution_mode": "user_assisted",
    "response_type": "bool",
    "verify_response": true
}

testsuite.testcase.verdict

Routing key testsuite.testcase.verdict
Requirements TT MUST publish event
Type Event
Pub/Sub Testing Tool -> GUI
Description: Used to indicate to the GUI (or automated-iut) which is the final verdict of the testcase.

Source code: MsgTestCaseVerdict

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: testsuite.testcase.verdict
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "64221bfb-8c34-4953-a955-b4108f821891",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "description": "No interoperability error was detected,",
    "objective": "Perform GET transaction(CON mode)",
    "partial_verdicts": [
        [
            "TD_COAP_CORE_01_step_02",
            null,
            "CHECK postponed",
            ""
        ],
        [
            "TD_COAP_CORE_01_step_03",
            null,
            "CHECK postponed",
            ""
        ],
        [
            "TD_COAP_CORE_01_step_04",
            "pass",
            "VERIFY step: User informed that the information was displayed correclty on his/her IUT",
            ""
        ],
        [
            "CHECK_1_post_mortem_analysis",
            "pass",
            "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
        ],
        [
            "CHECK_2_post_mortem_analysis",
            "pass",
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
        ],
        [
            "CHECK_3_post_mortem_analysis",
            "pass",
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
        ]
    ],
    "state": "finished",
    "testcase_id": "TD_COAP_CORE_01",
    "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01",
    "verdict": "pass"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "description": "No interoperability error was detected,",
    "objective": "Perform GET transaction(CON mode)",
    "partial_verdicts": [
        [
            "TD_COAP_CORE_01_step_02",
            null,
            "CHECK postponed",
            ""
        ],
        [
            "TD_COAP_CORE_01_step_03",
            null,
            "CHECK postponed",
            ""
        ],
        [
            "TD_COAP_CORE_01_step_04",
            "pass",
            "VERIFY step: User informed that the information was displayed correclty on his/her IUT",
            ""
        ],
        [
            "CHECK_1_post_mortem_analysis",
            "pass",
            "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
        ],
        [
            "CHECK_2_post_mortem_analysis",
            "pass",
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
        ],
        [
            "CHECK_3_post_mortem_analysis",
            "pass",
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
        ]
    ],
    "state": "finished",
    "testcase_id": "TD_COAP_CORE_01",
    "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01",
    "verdict": "pass"
}

testsuite.report

Routing key testsuite.report
Requirements TT MUST publish event
Type Event
Pub/Sub Testing Tool -> GUI
Description: Used to indicate to the GUI (or automated-iut) the final results of the test session.

Source code: MsgTestSuiteReport

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: testsuite.report
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "7d9c6e09-1ad0-4990-bb41-8caa22288da3",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "tc_results": [
        {
            "testcase_id": "TD_COAP_CORE_01",
            "verdict": "pass",
            "description": "No interoperability error was detected,",
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_01_step_02",
                    null,
                    "CHECK postponed",
                    ""
                ],
                [
                    "TD_COAP_CORE_01_step_03",
                    null,
                    "CHECK postponed",
                    ""
                ],
                [
                    "TD_COAP_CORE_01_step_04",
                    "pass",
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT",
                    ""
                ],
                [
                    "CHECK_1_post_mortem_analysis",
                    "pass",
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
                ],
                [
                    "CHECK_2_post_mortem_analysis",
                    "pass",
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
                ],
                [
                    "CHECK_3_post_mortem_analysis",
                    "pass",
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
                ]
            ]
        },
        {
            "testcase_id": "TD_COAP_CORE_02",
            "verdict": "pass",
            "description": "No interoperability error was detected,",
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_02_step_02",
                    null,
                    "CHECK postponed",
                    ""
                ],
                [
                    "TD_COAP_CORE_02_step_03",
                    null,
                    "CHECK postponed",
                    ""
                ],
                [
                    "TD_COAP_CORE_02_step_04",
                    "pass",
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT",
                    ""
                ],
                [
                    "CHECK_1_post_mortem_analysis",
                    "pass",
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43213] DELETE /test> Match: CoAP(type=0, code=4)"
                ],
                [
                    "CHECK_2_post_mortem_analysis",
                    "pass",
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43213] 2.02 Deleted > Match: CoAP(code=66, mid=0xa8cd, tok=b'')"
                ]
            ]
        }
    ]
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "tc_results": [
        {
            "testcase_id": "TD_COAP_CORE_01",
            "verdict": "pass",
            "description": "No interoperability error was detected,",
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_01_step_02",
                    null,
                    "CHECK postponed",
                    ""
                ],
                [
                    "TD_COAP_CORE_01_step_03",
                    null,
                    "CHECK postponed",
                    ""
                ],
                [
                    "TD_COAP_CORE_01_step_04",
                    "pass",
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT",
                    ""
                ],
                [
                    "CHECK_1_post_mortem_analysis",
                    "pass",
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
                ],
                [
                    "CHECK_2_post_mortem_analysis",
                    "pass",
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
                ],
                [
                    "CHECK_3_post_mortem_analysis",
                    "pass",
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
                ]
            ]
        },
        {
            "testcase_id": "TD_COAP_CORE_02",
            "verdict": "pass",
            "description": "No interoperability error was detected,",
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_02_step_02",
                    null,
                    "CHECK postponed",
                    ""
                ],
                [
                    "TD_COAP_CORE_02_step_03",
                    null,
                    "CHECK postponed",
                    ""
                ],
                [
                    "TD_COAP_CORE_02_step_04",
                    "pass",
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT",
                    ""
                ],
                [
                    "CHECK_1_post_mortem_analysis",
                    "pass",
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43213] DELETE /test> Match: CoAP(type=0, code=4)"
                ],
                [
                    "CHECK_2_post_mortem_analysis",
                    "pass",
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43213] 2.02 Deleted > Match: CoAP(code=66, mid=0xa8cd, tok=b'')"
                ]
            ]
        }
    ]
}

testingtool.terminate

Routing key testingtool.terminate
Requirements TT MUST listen to event, and handle a gracefully termination of all it’s processes
Type Event
Pub/Sub GUI, (or Orchestrator) -> Testing Tool
Description: Testing tool should stop all it’s processes gracefully.

Source code: MsgTestingToolTerminate

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: testingtool.terminate
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "fdc96247-26b3-4b8d-8da4-d04cc28b6441",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "description": "Command TERMINATE testing tool execution"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "description": "Command TERMINATE testing tool execution"
}

Results Repository events

Resources Repository events

Visualization tools events

viztool-grafana.init.request

Routing key viztool-grafana.init.request
Requirements Implementing Test Tools should send this at start
Type Event
Pub/Sub Testing Tool -> Visualization Tool
Description: Visualization Tool is waiting for this message
Visualization Tool MUST listen to this
to start init routines

Source code: MsgVizInitRequest

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: viztool-grafana.init.request
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "02677749-3dac-4910-8009-a9720cad9679",
    "timestamp": 1523606906,
    "reply_to": "viztool-grafana.init.reply",
    "correlation_id": "02677749-3dac-4910-8009-a9720cad9679"
}
 -  -  -
Message body: {
    "_api_version": "1.0.15"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15"
}

viztool-grafana.init.reply

Routing key viztool-grafana.init.reply
Requirements Visualization MUST send this after recieving MsgVizInitRequest
Type Event
Pub/Sub Visualization Tool -> Testing Tool
Description: This message contains the URL to access the internal Webserver
Test Tool MUST listen to this
that serves the Grafana Instance

Source code: MsgVizInitReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: viztool-grafana.init.reply
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "26e3eb18-5fc9-4a77-9f4a-0aeceedadba0",
    "timestamp": 1523606906,
    "correlation_id": "acc0275f-f60c-41b6-aa61-78edea393206"
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "ok": true,
    "url": "http://url-to-access-grafana:1234"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "ok": true,
    "url": "http://url-to-access-grafana:1234"
}

viztool-grafana.set_dashboard.request

Routing key viztool-grafana.set_dashboard.request
Requirements
Type Event
Pub/Sub Testing Tool -> Visualization Tool
Description: Visualization Tool uses this message to configure the Dashboard based on the JSON config
- Visualization Tool MUST listen to this.
- Test Tools should send this message after the Visualization Tool has confirmed the initialization

Source code: MsgVizDashboardRequest

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: viztool-grafana.set_dashboard.request
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "329c9e96-15a9-4984-b22d-8ee4629a7c06",
    "timestamp": 1523606906,
    "reply_to": "viztool-grafana.set_dashboard.reply",
    "correlation_id": "329c9e96-15a9-4984-b22d-8ee4629a7c06"
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "config": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "config": {}
}

viztool-grafana.set_dashboard.reply

Routing key viztool-grafana.set_dashboard.reply
Requirements
Type Event
Pub/Sub Visualization Tool -> Testing Tool
Description: This message contains the URL to access the internal Webserver that serves the Grafana Instance
- Visualization MUST send this after recieving MsgVizDashboardRequest
- Test Tool MUST listen to this

Source code: MsgVizDashboardReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: viztool-grafana.set_dashboard.reply
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "353e2a96-5824-46f4-8af2-d797b4af3a46",
    "timestamp": 1523606906,
    "correlation_id": "ecf3ad7e-d5e0-4e60-9b77-fea0522fc731"
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "ok": true
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "ok": true
}

viztool-grafana.write_data

Routing key viztool-grafana.write_data
Requirements
Type Event
Pub/Sub Performance Testing Tool -> Visualization
Description:
- Performance Testing Tool SHOULD emit this event periodically
- Visualization Tool MUST listen to this event
- During the test execution, the Performance Testing Tool MUST periodically emit this event carrying current performance statistics/measurements

Source code: MsgVizWrite

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Message routing key: viztool-grafana.write_data
 -  -  -
Message properties: {
    "content_type": "application/json",
    "message_id": "c3bd0718-1d4c-4c9b-a4e4-8e6beeb8c479",
    "timestamp": 1523606906
}
 -  -  -
Message body: {
    "_api_version": "1.0.15",
    "fields": {
        "value": 0
    },
    "measurement": "name",
    "tags": {},
    "time": 0
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
 {
    "_api_version": "1.0.15",
    "fields": {
        "value": 0
    },
    "measurement": "name",
    "tags": {},
    "time": 0
}