Introduction

In this doc we present the important information regarding the architecture, the core components and the description of the messages exchanged between those core components.

Must-Reads

Developers should read architecture public deliverable before getting started here:

D1.3 - Initial architecture design

I would be extremely useful you also take a look at the discussions and announcements at:

F-Interop google-group forum

“At the early stage [of an API defintion], agility trumps completeness”.

Joshua J. Bloch (Sun Microsystems and Google)

Architecture

The F-Interop architecture is composed of different components exchanging messages through an “Event Bus”.

F-Interop’s architecture follows the event bus design pattern:

generic messages are exchanged between the different software components
these components can be running in different locations
any component connected to the bus can publish any type of message
any component connected to the bus can subscribe to any type of message

All communication is done through this mechanism, including control messages, network data packets, logs, etc.

We use the RabbitMQ as the underlying message-passing mechanism. It acts as a secure message broker between all the components through encrypted channels.

Screenshot

Scope of the documentation

This documentation describes the core API for building testing tools and other components in the F-Interop architecture. The interfaces of the core API are outlined in red in the following diagram:

Screenshot

If you are a testing tool implementor which is re-using some of the reference implementations and you want to read their documentation please refer to:

coap_testing_tool documentation

6tisch_testing_tool documentation

performance_testing_tool documentation

privacy_testing_tool documentation

Event Bus

(updated for API v2.0)

Introduction

Components of the system communicates with each other using messages. We need message delivery to be:

reliable (A message MUST be managed with reliability guarantee)
scalable (We MUST be able to manage many messages during different concurrent and isolated test session)
scriptable (Adding new messages queues to add new components should be easy)

This architecture enables many independent components to talk to each other and avoid a monolith that would be difficult to manage as the number of components increase.

Screenshot

open image

We use the RabbitMQ as the underlying message-passing mechanism.

It acts as a secure message broker between all the components through encrypted channels.

Important facts about the event bus:

Every message in the event bus contains a routing key (called also topic) which categorizes the message. These are standarized by the f-interop platform.
Every message in the event bus contains a payload.
Payloads of messages which are part of the CORE API are standardized.
Payloads of some components of the architecture which are not part of the CORE API are not standardized. The testing tool implementors are free to defined their own messages.
JSON is the data format used for the application messages exchanged in the bus.
Isolation of test session is performed using virtual hosts (vhosts).
Each session has a interface for monitoring the load and enable easy debugging. If you are implementor request access to it to any of the developers of the project.

Messaging pattern

For producing/consuming events we use the publish/subscribe pattern which implementation is based on topic exchanges of AMQP (more about topic exchanges)

Topic exchanges route messages to one or many queues based on matching between a message routing key and the pattern that was used to bind a queue to an exchange.

Topics/routing keys

How the topics/routing keys are composed is standardized by F-Interop, by using certain conventions.

As you may know, messages are sent using topic exchanges, and when using this feature in amqp routing keys / topics are formed as a list of words delimited by dots:

quick.orange.rabbit

Conventions

Conventions on routing keys/topics

By convention in F-Interop the first keyword of every routing key will be the namespace described before, the rest of the routing key terms will be identify the type of the message.

Some examples:

results.put.request, orchestrator.sesssion.get.request, ui.user.john.display

Conventions on request/reply interactions

In some situations components will need to use a request/reply type of interaction between instead of just pushing push & forget a message into the bus. But note that the event bus follows the pub/subs which doesnt make it ideal for this purpose, nevertheless F-Interop proposes the following convention for sending a request and receiving a reply over the pub/sub pattern.

a component making a request will publish a message into:

(->) routing_key = <namespace>.<typeOfMessageOrAction>.request

with ‘reply_to’ amqp property set on:

reply_to = <namespace>.<typeOfMessageOrAction>.reply

the second component replying to the message will publish into (reply_to routing key):

(<-) routing_key = <namespace>.<typeOfMessageOrAction>.reply

Also, for solving the problem where two simultaneous requests are sent to the same routing key each request and reply must contain correlation_id amqp property. This will allow components to correlate uniquely a reply to a request.

AMQP messages library

A comprehensive list of messages can be found in here: messages library

AMQP message attributes/properties

These are the attributes / properties typically used in the F-Interop event bus:

Field	Type	Description
routing_key	string	Topic of the message being sent.
reply_to	string	Topic used for the reply. Every request message needs to include this.
msg_id	number	A globally unique ID for this message.
correlation_id	string	Whenever the message is a request or a reply this field MUST be included.
timestamp	number	the Posix/Unix time in second when this message was generated by its publisher.
body	JSON	payload of the message in JSON format and including a _type.

    -> add an example here

Session Orchestrator (SO)

The orchestrator main roles are:

it monitors the users that are connected,
activates the rooms currently in use and starts/stops the test sessions,

Session deployment sequence

The session deployment phase includes interactions between GUI, SO, TT and eventually other requested resources if the testing suite requires them.

The steps during this phase are:

GUI -> SO : http PUT sessions/session_id {configuration= [..]}

( SO creates vhost )

( SO adds users to vhost )

( SO creates session)
GUI -> SO : http PATCH sessions/session_id {status:started}

( SO spawns TT )

( SO spawns other requested resources )
TT -> GUI: amqp event ui.core.session.confirguration.get.resquest

(requests session anad testing tool configuration)
GUI -> TT: amqp event ui.core.session.confirguration.get.reply {configuration= [..]}

(this returns information about the session, and about user input configuration for testing tool)

After this point the session is ready to start. It’s up to the testing tool developer to implement the test logic.

FAQ:

I’m developing a new testing tool, is there any specific testing tool requirements I need to comply to?
see: Testing Tool section

SO: Session Orchestrator

TT: Testing Tool (any type)

AMQP interface

(WIP)

HTTP interface

Create a new session

PUT http://orchestrator.f-interop.eu/sessions/<session_id>

users: Allowed user to connect to the session
testing_tool: The testing tool that you want F-Interop to spawn in the session available testing tool: http://orchestrator.dev.f-interop.eu/tests
configuration: this is the information required by the testing tool being deployed, see as an example of the information that a CoAP tests requires when deploying a CoAP session here: http://orchestrator.dev.f-interop.eu/tests/f-interop/interoperability-coap

{
   "testing_tools": "f-interop/interoperability-coap",
    "users": [
        "u1",
        "f-interop"
    ],
    "configuration": {
        "testsuite.additional_session_resource": "automated_iut-coap_client-californium",
        "testsuite.testcases": [
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_01",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_03",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_02",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_04",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_05",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_06"
        ]
    }
}

import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>"

data = {
    "users": [
        "a",
        "b",
        "c"
    ],
    "testing_tools": "http://orchestrator.f-interop.eu/tests/f-interop/interoperability-coap",
    "configuration": {
        "testsuite.additional_session_resource": "automated_iut-coap_client-californium",
        "testsuite.testcases": [
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_01",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_03",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_02",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_04",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_05",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_06"
        ]
    }
}

register_response = requests.put(url,
                                 json=data,
                                 auth=("username", "password"))

Get a session

GET http://orchestrator.f-interop.eu/sessions/<session_id>

import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>"

register_response = requests.get(url,
                                 auth=("username", "password"))

Start a session

PATCH http://orchestrator.f-interop.eu/sessions/<session_id>

{
  "status": "started"
}

import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>"

register_response = requests.patch(url,
                                   json={"status": "started"},
                                   auth=("username", "password"))

Stop a session

PATCH http://orchestrator.f-interop.eu/sessions/<session_id>

{
  "status": "stopped"
}

import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>"

register_response = requests.patch(url,
                                   json={"status": "stopped"},
                                   auth=("username", "password"))

Delete a session

DELETE http://orchestrator.f-interop.eu/sessions/<session_id>

import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>"

register_response = requests.delete(url,
                                    auth=("username", "password"))

Create a new user

PUT http://orchestrator.f-interop.eu/users/<user_id>

import requests

url = "http://orchestrator.f-interop.eu/users/<user_id>"
register_response = requests.put(url,
                                 json={"password": "foo"})

{
  "password": "foo"
}

Get information about a user

GET http://orchestrator.f-interop.eu/users/<user_id>

import requests

url = "http://orchestrator.f-interop.eu/users/<user_id>"
register_response = requests.get(url)

Delete a user

DELETE http://orchestrator.f-interop.eu/users/<user_id>

import requests

url = "http://orchestrator.f-interop.eu/users/<user_id>"
register_response = requests.delete(url,
                                    auth=("username", "password"))

Post an event on the event bus of a session

HTTP is not designed to handle event flow such as AMQP is. Sending event is supported but the reply will only tell the sender whether or not the event is arrived inside the broker but it won’t contains anything about the effect.

Publish a message to a given exchange. You will need a body looking something like:

{ "properties": {}, "routing_key": "my key", "payload": "my body", "payload_encoding": "string" }

All keys are mandatory. The payload_encoding key should be either “string” (in which case the payload will be taken to be the UTF-8 encoding of the payload field) or “base64” (in which case the payload field is taken to be base64 encoded). If the message is published successfully, the response will look like:

{ "routed": true }

Routed will be true if the message was sent to at least one queue. Please note that the HTTP API is not ideal for high performance publishing; the need to create a new TCP connection for each message published can limit message throughput compared to AMQP or other protocols using long-lived connections.

POST http://orchestrator.f-interop.eu/sessions/<session_id>/publish

{
  "properties": {},
  "routing_key": "my key",
  "payload": "my body",
  "payload_encoding": "string"
}

import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>/publish"

register_response = requests.post(url,
                                  json={"password": "foo"},
                                  auth=("username", "password"))

Need to integrate a new testing tool to orchestrator?

read the F-Interop SDK documentation

for any questions, please use :

https://groups.google.com/forum/#!forum/technical-support-f-interop-industrial-experiments

Testing tools (TT)

This section of the documentation describes the testing tool’s minimal requirements.

General requirements on Testing Tools (TT):

(The testing tool…)

MUST issue a final report with the results of the test session, this must be displayed in the GUI.
MUST use user interface to coordinate the test session.
MUST publish/submit a final report into the results store
MUST provide the user with a mechanism to download all data related to the session before termination (pcap files, result files, logs, influx db dumps, etc..)
SHOULD use an AMQP log handler for easing debugging.

Extra requirements on interop and conformance TTs:

(The testing tool…)

MUST follow a test description or test specification reference document.
SHOULD implement a test suite model where: test suite = list of [test case], testcase = list of [test step]
SHOULD issue IUT configuration messages at the beginning of the test to take the IUT to original state
SHOULD issue progress information as the testing transition from step to step
MAY issue addition configuration messages to take the IUT to original state and other configurations specified in the test descriptions) at the beginning or during the test cases

Interfaces

The abstraction used for describing the interfaces in this document (the CORE API) are indicated in this diagram:

Screenshot

More details on thos API calls can be found on the previously presented Event Bus CORE API

When contributing with a testing tool (TT) to F-Interop you should have in mind that for us your testing tool is a black box. F-Interop won’t send any requests to the TT, it’s the TT who must use F-Interop services (GUI services, RR, RS, etc..) for executing the complete test session.

Reference testing tools are available for re-usablity, some links to the source code is provided below. TT developers are free to start building their tools from scratch using the CORE API or extend these reference implementations which already make use of the CORE API

Snippets

Decode from base64 any uploaded file to GUI

One of the typical steps of a test is asking the user to upload a pcap file so the testing tool can check the correctness of the packet format in the file. This step is done through GUI API for file (upload). The file content will be uploaded through the GUI to Testing tool. For a testing tool developer, she needs to decode the file content using base64 to a pcap file.

An example is showing on the right side:

import base64

# ask for uploading pcap files (the function could be difference for different testing tool)
uploadFile = FileUploadQuery(question=question, tag=step_id).ask()

for item in uploadFile:
    # looking for the file content
    if 'filename' in item:
        capture_path = get_from_environment("PCAP_PATH", "/capture.pcapng")
        with open(capture_path,'wb') as f:
            # item['value'] is the file content
            f.write(base64.b64decode(item['value']))

Testing Tool source code, READMEs and tutorials:

The complete list of F-Interop public git repositories at:

Explore F-Interop’s gitlab repos

CoAP Testing Tools tutorial

6LoWPAN Testing Tools tutorial

CoAP/6LoWPAN/oneM2M Testing Tools API doc

CoAP/6LoWPAN/oneM2M Testing Tools README doc

CoAP/6LoWPAN/oneM2M Testing Tools git repo

(WIP)

6TiSCH README Testing Tool doc

Privacy Testing Tool doc

Performance Testing Tool doc

Agent

(updated for API v2.0)

Introduction

The “agent” is a program a user downloads from the F-Interop platform, and which allows him/her to connect the IUT to the F-Interop’s testing tools.

Communication between the agent and the server is authenticated and secure. Through the agent, the F-Interop’s testing tools can (remotely) interact with the IUTs, for example by changing its configuration or injecting packets in the link/interface the IUT is listening to. Similarly, the agent reports to F-Interop the data packets sent from the IUT.

Routing key / topics basics

There can be several agents during a session. Therefore, there is a need to address each agent individually. The exact process of downloading and launching an agent is defined by the testing tools’ developed, this information must be provided to the user using the GUI.

the “name spaces” for the routing keys reserved for agent use are:

toAgent.# fromAgent.#

An agent listens to several routing keys/topics. A typical agent’s routing key / topic used for forwarding the IP packets through the a previously created IP tunnels looks like this:

toAgent.coap_client_agent.packet.raw.tun.

fromAgent.coap_server_agent.packet.raw.tun.

etc..

NOTE: The routing of the data messages between agents in not handled by the agents themselves - they just publish/listen to events under those topics which will end up being stored in some queue in RMQ broker. It’s up to the testing tool later to route those messages to other agents and for establishing a communication between IUTs. Testing Tool can also implement message dropping mechanisms for testing certain network features like re-transmitting, etc.

This way, components of the testing tools can publish messages to this topic, then the agent will receive the message and e.g. inject it in the tun interface.

Besides the packet injection, there are other types of actions we can trigger by publishing to these routing keys / topics. Like network interfaces remote control (from testing tool to the PC where the agent is running)

etc..

Events and API

F-Interop doesnt standardised messages or routing keys for these components given that these interfaces are test suite imeplementation dependent. For examples you can check out section Source-code-implementations

Source code implementations

Some examples on how to build your own agent component:

agent for builing an IPv6 vpn-like setup for running inteorp tests (used in CoAP test suites, and oneM2M test suites):

network-tunneling agent’s gitlab repo

agent for flashing and controlling motes: 6TiSCH agent’s gitlab repo

Graphical User Interface

( version of the API v.2.0 )

BASICS

The F-Interop GUI is based on MySlice v2 software.

It provides the necessary registration, validation and authentication of users.

Authentication to the GUI relies on login/password, but authentication to the testbeds requires ssh-keys and X509 certificates.

For the specific F-Interop needs, new modules have been developed in order to communicate with the Message Bus.

The following architecture has been implemented:

Screenshot

SERVICES PROVIDED

Registration of users
Creation of a test session
- conformance
- interoperability
- performance
- privacy
Execution of the session
- visualization of messages consumed
- action buttons pushing messages
History of past test sessions

MESSAGES FLOW

ui.user.* messages

Screenshot

ui.user.* routing_key is dedicated to communication with the user interface

ui.user.all.*

Any component on the bus can direct messages to all users involved in the session.

ui.user.<user_id>.*

Any component on the bus can direct messages to a specific user in the session.

ui.user.<user_id>.request

PROPS: correlation_id is sent by the testing tool on the ui.user.<user_id>.request, the answer uses the same correlation_id in the ui.user.<user_id>.reply

The GUI will use correlation_id to identify the actions that have already been triggered and disable the submit button accordingly

BODY of the ui.user.<user_id>.request message is used as a form, which contains a list of input fields.

fields: list of input fields, several fields of different types can be combined.

USER INPUT (requests)

text

Text is used to request to the user some text input. Input fields must must have the same name. The value of each input field must be defined.

ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "input_name",
            type: text,
        }
    ]
 }

radio

Radio is used to offer the possibility to select one choice among several. Input fields must must have the same name. The value of each input field must be defined.

ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            label: "choice number 1”,
            type: radio,
            value: 1
        },
        {
            name: "example",
            label: "choice number 2”,
            type: radio,
            value: 2
        }
    ]
 }

checkbox

Checkbox is used to offer the possibility to select several choices. The value of each input must be defined

ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example1",
            label: "choice number 1”,
            type: checkbox,
            value: 1
        },
        {
            name: "example2",
            label: "choice number 2”,
            type: checkbox,
            value: 2
        }
    ]
}

select

Select is used to offer the user to select one option, a default choice can be defined using value = x

ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    title: "Select example",
    fields: [
        {
            name: "example",
            type: "select",
            options: [
                {label:"choice 1”, value:1},
                {label:"choice 2”, value:2},
                {label:"choice 3”, value:3},
            ],
            value: 1
        }
    ]
 }

file (upload)

file allow users to upload a file to the testing tool

ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            type: "file"
        }
    ]
 }

button

Value must be specified. Clicking a button triggers the action to send the form as a message to the bus. Several buttons with different values can be proposed in the same form.

ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "button 1",
            type: "button",
            value: 1
        },
        {
            name: "button 2",
            type: "button",
            value: 2
        }
    ]
}

OPTIONAL ATTRIBUTES (recommended)

title: will be placed in the header of the form

label: each field can have a label to describe it

ROUTING_KEY: ui.user.<user_id>.request
PROPS:
        correlation_id: correlation_id
BODY:    
{
    title: "My Action",
    fields: [
        {
            name: "input_name",
            type: "text",
            label:"demo input”,
            value: "default value",
        }
    ]
}

USER INPUT (replies)

ui.user.<user_id>.reply

ui.user.user_id.reply is sent with the same correlation_id as the one received in the corresponding ui.user.user_id.request, enabling the testingtool to identify what question has been answered. fields: list of the input fields using the same names as ui.user.user_id.request to map the values.

ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "input_name",
            value: "user’s answer",
        }
    ]
}

text reply

see json example

ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "input_name",
            value: "user’s answer",
        }
    ]
}

radio reply

see json example

ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            value: 1
        }
    ]
}

checkbox reply

see json example

ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            value: 1
        }
    ]
}

select reply

see json example

ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            value: 2
        }
    ]
}

file upload reply

see json example

ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            value: base64_encoded <file>
        }
    ]
}

button reply

see json example

ROUTING_KEY: ui.user.<user_id>.reply
BODY:    
{
    fields: [
        {
            name: "button 1",
            value: 1
        }
    ]
}

USER DISPLAY

ui.user.user_id.display is sent by the testingtool to the ui.user.user_id. fields: list of fields, allowing to display several elements within the same message (text, link, file, img, iframe)

paragraph (<p>)

markdown format is accepted https://learn.getgrav.org/content/markdown

ROUTING_KEY: ui.user.user_id.display
PROPS:
        correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "my_message",
            type: "p",
            value: "Hello World!"
        }
    ]
}

image

see json example

ROUTING_KEY: ui.user.user_id.display
PROPS:
   correlation_id: correlation_id
BODY:    
{
    title: "Example image"
   fields: [
        {
            name: "my_image.png",
            type: "img",
            value: <url> | base_64_encoded <file>
       },
    ]
}

html

see json example

ROUTING_KEY: ui.user.user_id.display
PROPS:
   correlation_id: correlation_id
BODY:    
{
    title: "Example html"
   fields: [
        {
            name: "my_html",
            type: "html",
            value: "<div>Hello World</div>"
       },
    ]
}

link (<a>)

see json example

ROUTING_KEY: ui.user.user_id.display
PROPS:
   correlation_id: correlation_id
BODY:    
{
    title: "Example link"
   fields: [
        {
            name: "my_link",
            type: "a",
            value: <url>
        },
    ]
}

data (download)

Value must be base64 encoded url the format of the file is based on the extension of its name

ROUTING_KEY: ui.user.user_id.display
PROPS:
   correlation_id: correlation_id
BODY:    
{
    title: "Download your report for testcase 1"
   fields: [
        {
            name: "report_testcase_1.pcap",
            type: "data",
            value: base64_encoded <file>
       },
    ]
}

iframe

see json example

ROUTING_KEY: ui.user.user_id.display
PROPS:
   correlation_id: correlation_id
BODY:    
{
    title: "Example iframe"
   fields: [
        {
            name: "my_iframe",
            type: "iframe",
            value: <url>
       },
    ]
}

LOG LEVELS

Level is optional, by default it is considered as info. Level can be applied to any message type (ui.user.user_id.request, ui.user.user_id.display).

ROUTING_KEY: ui.user.user_id.display
PROPS:
    correlation_id: correlation_id
BODY:    
{
    level: debug | info | highlighted | warning | error | critical,
    fields: [
        {
            name: "my_message",
            type: "p",
            value: "this is a log message"
        }
    ]
}

The log level is meant to inform about the execution of the session, not about the outcome (verdict) of the tests themselves.

debug	Only displayed in the development environment.
info	Default log level, if not specified is considered as info.
highlighted	Highlighted to the user, ex: download pcap file, test verdict
warning	Highlighted to the user, ex: your session will expire in 5 minutes!
error	Propose user to reset
critical	Terminate the session

TAGS (optional filters)

Tags allow to categorize messages and to apply filters on the messages displayed to the user

ROUTING_KEY: ui.user.user_id.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    tags: {testcase_id: "TD_COAP_CORE_03_v01"}
    fields: [
        {
            name: "input_name",
            type: "text",
        }
    ]
 }

ROUTING_KEY: ui.user.user_id.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    tags: {testcase_id: "TD_COAP_CORE_03_v01"}
    fields: [
        {
            name: "input_name",
            type: "text",
        }
    ]
 }

ROUTING_KEY: ui.user.user_id.display
PROPS:     
    correlation_id: correlation_id
BODY:    
{
    tags: {testcase_id: "TD_COAP_CORE_03_v01"}
    fields: [
        {
            name: "input_name",
            value: "user’s answer"
        }
    ]
}

Results Store (RS)

(updated for API v2.0)

The Results Store (RS) is a core F-Interop service that allow to store and retrieve results or intermediate results generated from Testing Tools (TT).

The Results Store repository can be found here.

API Description

The RS uses MongoDB as database. The results will be stored as Binary JSON (BSON) by following the format below:

Key	Type	Value
`resources`	Array of Objects	The resources involved in the session. Each object has the keys `resource_id` and `resource_version`
`owners`	Array of Strings	The ids of the owners of the resources
`session_id`	String	The id of the session
`testing_tool_id`	String	The id of the testing tool that produced this result
`timestamp`	Date	The time this result was inserted in the DB (UTC datetime)
`type`	String	The type of the result. It can be `intermediate` or `final`
`data`	Object	The data produced by the testing tool as a result. There is no schema for this object

Here an example of a result:

{
    "resources": [
        {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
        {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
    ],
    "owners": [
        "user-01",
        "user-02"
    ],
    "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
    "testing_tool_id": "testing-tool-01-id",
    "timestamp": 1520437111,
    "type": "final",
    "data": {
        "some_data_produced_by_tt": 42
    }
}

The RS provides on virtual host / the following services:

Service	Description
`results_store.insert_result`	Insert a result in the Results Store
`results_store.get_result`	Query the Results Store for results
`results_store.delete_result`	Delete one or more results

Insert Result

Request parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`results_store.insert_result.request`	The routing key used for the request
reply_to	`results_store.insert_result.reply`	The routing key used for the reply
correlation_id	`str(uuid.uuid4())`	The correlation id of the request
content_type	`application/json`	The content type

Request body:

See database format described on top.

Reply parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`results_store.insert_result.reply`	The routing key used for the reply
correlation_id	same as request	The correlation id of the request
content_type	`application/json`	The content type

Reply body:

Key	Type	Value
`ok`	Boolean	`true` if the result was inserted correctly, `false` otherwise
`error`	String	A message describing the error (present only if `ok` is `false`)

Insert Result Request example:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "resources": [
        {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
        {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
    ],
    "owners": [
        "user-01",
        "user-02"
    ],
    "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
    "testing_tool_id": "testing-tool-01-id",
    "timestamp": 1520437111,
    "type": "final",
    "data": {
        "some_data_produced_by_tt": 42
    }
}

Insert Result Reply example:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Get Result

Request parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`results_store.get_result.request`	The routing key used for the request
reply_to	`results_store.get_result.reply`	The routing key used for the reply
correlation_id	`str(uuid.uuid4())`	The correlation id of the request
content_type	`application/json`	The content type

Request body:

The body must contain a query matching the MongoDB query language. See examples below.

Reply parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`results_store.get_result.reply`	The routing key used for the reply
correlation_id	same as request	The correlation id of the request
content_type	`application/json`	The content type

Reply body:

Key	Type	Value
`ok`	Boolean	`true` if the query was executed correctly, `false` otherwise
`results`	Array of Objects	The result(s) retrieved by the query. The schema of one result is the one described at the top but with one additional key `_id`, which is added by MongoDB
`error`	String	A message describing the error (present only if `ok` is `false`)

Examples

The following examples assume that the RS contain the three results above.
The two first results are produced in the same session and there are two
involved resources owned by two different users. The third one is produced
in another session and there is one resource owned by one user. The
resource is a new version of the one present in the previous results.
Note that the `_id` key is added by MongoDB as unique reference to that
object.

{
    "_id": {"$oid": "5aa7d1b8dbbb46608189119a"},
    "resources": [
        {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
        {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
    ],
    "owners": [
        "user-01",
        "user-02"
    ],
    "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
    "timestamp": 1520437111,
    "testing_tool_id": "testing-tool-01-id",
    "type": "intermediate",
    "data": {
        "some_data_produced_by_tt": 42
    }
}

{
    "_id": {"$oid": "5aa7d1b8dbbb46608189119b"},
    "resources": [
        {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
        {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
    ],
    "owners": [
        "user-01",
        "user-02"
    ],
    "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
    "timestamp": 1520437120,
    "testing_tool_id": "testing-tool-01-id",
    "type": "final",
    "data": {
        "some_data_produced_by_tt": {
            "some_data": 3.14,
            "other_data": "some_string"
        }
    }
}

{
    "_id": {"$oid": "5aa7d1b8dbbb46608189119c"},
    "resources": [
        {"resource_id": "user-01-resource-id", "resource_version": "1.1"}
    ],
    "owners": [
        "user-01"
    ],
    "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999",
    "timestamp": 1520437199,
    "testing_tool_id": "testing-tool-id-654321",
    "type": "final",
    "data": {
        "some_data_produced_by_tt": 1234567890
    }
}

Example 1: Get results of all resources for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01"
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true,
    "results": [
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119a"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
                {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
            ],
            "owners": [
                "user-01",
                "user-02"
            ],
            "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
            "timestamp": 1520437111,
            "testing_tool_id": "testing-tool-01-id",
            "type": "intermediate",
            "data": {
                "some_data_produced_by_tt": 42
            }
        },
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119b"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
                {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
            ],
            "owners": [
                "user-01",
                "user-02"
            ],
            "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
            "timestamp": 1520437120,
            "testing_tool_id": "testing-tool-01-id",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": {
                    "some_data": 3.14,
                    "other_data": "some_string"
                }
            }
        },
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119c"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.1"}
            ],
            "owners": [
                "user-01"
            ],
            "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999",
            "timestamp": 1520437199,
            "testing_tool_id": "testing-tool-id-654321",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": 1234567890
            }
        }
    ]
}

Example 2: Get results of specific resource for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "resources": {"$elemMatch": {"resource_id": "user-01-resource-id"}}
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true,
    "results": [
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119a"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
                {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
            ],
            "owners": [
                "user-01",
                "user-02"
            ],
            "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
            "timestamp": 1520437111,
            "testing_tool_id": "testing-tool-01-id",
            "type": "intermediate",
            "data": {
                "some_data_produced_by_tt": 42
            }
        },
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119b"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
                {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
            ],
            "owners": [
                "user-01",
                "user-02"
            ],
            "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
            "timestamp": 1520437120,
            "testing_tool_id": "testing-tool-01-id",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": {
                    "some_data": 3.14,
                    "other_data": "some_string"
                }
            }
        },
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119c"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.1"}
            ],
            "owners": [
                "user-01"
            ],
            "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999",
            "timestamp": 1520437199,
            "testing_tool_id": "testing-tool-id-654321",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": 1234567890
            }
        }
    ]
}

Example 3: Get results of specific resource + version for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "resources": {"$elemMatch": {"resource_id": "user-01-resource-id", "resource_version": "1.1"}}
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true,
    "results": [
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119c"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.1"}
            ],
            "owners": [
                "user-01"
            ],
            "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999",
            "timestamp": 1520437199,
            "testing_tool_id": "testing-tool-id-654321",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": 1234567890
            }
        }
    ]
}

Example 4: Get results of specific session for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999"
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true,
    "results": [
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119c"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.1"}
            ],
            "owners": [
                "user-01"
            ],
            "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999",
            "timestamp": 1520437199,
            "testing_tool_id": "testing-tool-id-654321",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": 1234567890
            }
        }
    ]
}

Delete Result

Request parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`results_store.delete_result.request`	The routing key used for the request
reply_to	`results_store.delete_result.reply`	The routing key used for the reply
correlation_id	`str(uuid.uuid4())`	The correlation id of the request
content_type	`application/json`	The content type

Request body:

The body is a query as in the case of getting the results. All results matching that query will be deleted.

WARNING: Passing an empty document {} will delete all documents (TBD: disable this case from API).

Reply parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`results_store.delete_result.reply`	The routing key used for the reply
correlation_id	same as request	The correlation id of the request
content_type	`application/json`	The content type

Reply body:

Key	Type	Value
`ok`	Boolean	`true` if the result(s) was deleted correctly, `false` otherwise
`error`	String	A message describing the error (present only if `ok` is `false`)

Examples

The following examples assume that the RS contain the three results
described in the Get Result section.

Example 1: Delete all results for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01"
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Example 2: Delete all results of specific resource for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "resources": {"$elemMatch": {"resource_id": "user-01-resource-id"}}
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Example 3: Delete all results of specific resource + version for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "resources": {"$elemMatch": {"resource_id": "user-01-resource-id", "resource_version": "1.1"}}
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Example 4: Delete all results of specific session for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999"
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Results Store Service (RSS)

( version of the API v.1.0 )

The Results Store Service (RSS) is a F-Interop service that acts as a bridge between a session virtual host and the Results Store. During a session it will receive raw data (results or intermediate results) directly from the Testing Tool. This data is then packaged with session meta-data and sent to the Results Store (RS).

To enable the RSS in a session, its Supervisor configuration template must be added inside the Testing Tool configuration file. See the example.

Supervisor configuration template of RSS

[program:{{ session }}|service_results_store]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service-results-store-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service-results-store-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env RS_AMQP_URL={{ rs_amqp_url }}
    --env RS_AMQP_EXCHANGE={{ rs_amqp_exchange }}
    --rm
    --name="session_{{ session }}-service-results-store"
    service-results-store

Example of Testing Tool configuration using the RSS

[program:{{ session }}|my_testing_tool]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-my_testing_tool-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-my_testing_tool-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --rm
    --name="session_{{ session }}-my_testing_tool"
    my_testing_tool-image-name

[program:{{ session }}|service_results_store]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service-results-store-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service-results-store-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env RS_AMQP_URL={{ rs_amqp_url }}
    --env RS_AMQP_EXCHANGE={{ rs_amqp_exchange }}
    --rm
    --name="session_{{ session }}-service-results-store"
    service-results-store

API Description

This API is used, in principle only by Testing Tools, to insert results in the RS. The schema of the data pushed by TTs is completely free as long it is valid JSON. The RS provides on any session virtual host the following services:

Service	Description
`results_store.session.report.save`	Insert a result in the Results Store

API Description: Insert Result (through RSS)

Request parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`results_store.session.report.save.request`	The routing key used for the request
reply_to	`results_store.session.report.save.reply`	The routing key used for the reply
correlation_id	`str(uuid.uuid4())`	The correlation id of the request
content_type	`application/json`	The content type

Request body:

Key	Type	Value
`type`	String	The type of the result. It can be `intermediate` or `final`
`data`	Object	The data produced by the testing tool as a result. There is no schema for this object

Reply parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`results_store.session.report.save.reply`	The routing key used for the reply
correlation_id	same as request	The correlation id of the request
content_type	`application/json`	The content type

Reply body:

Key	Type	Value
`ok`	Boolean	`true` if the result was inserted correctly, `false` otherwise
`error`	String	A message describing the error (present only if `ok` is `false`)

Insert Result Request example:

 - - -
ROUTING_KEY: results_store.session.report.save.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.session.report.save.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "type": "final",
    "data": {
        "some_key": {
            "another_key": "some_value"
        },
        "yet_another_key": 42
    }
}

Insert Result Reply example:

 - - -
ROUTING_KEY: results_store.session.report.save.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Resource Repository (RR)

( version of the API v.2.0 )

The Resource Repository (RR) is a core F-Interop service that allow to store and retrieve the resources used in the tests.

The Results Store repository can be found here.

API Description

The RR uses MySQL as database. A resource is described by the JSON schema showed in the example. All fields are optional except resource_id and owner_id.

Resource Example

{
    "resource_id": "unique-resource-identifier",
    "privacy_flag": "public",
    "available": true,
    "owner_id": "unique-owner-identifier",
    "ts_creation": 1256953732,
    "ts_update": 1256953732,
    "hardware": {
        "manufacturer": "HW manufacturer",
        "machine_type": "phy",
        "hw_platform": "HW platform (board name)",
        "power_supply": "Power supply",
        "processor": "Processor name",
        "cpu_cache": "CPU cache",
        "ram": "RAM",
        "disk": "Secondary storage info",
        "supported_medias": [
            {
                "name": "Media name",
                "version": "Media version",
                "network_card": "Network card info",
                "mac_address": "aa:bb:cc:dd:ee:ff",
                "if_bandwidth": "Interface bandwidth"
            }
        ],
        "other_hardware_info": [
            {"parameter": "info_key", "value": "info_value"}
        ]
    },
    "fw_os": {
        "name": "OS/Firmware name",
        "version": "OS/Firmware version",
        "ip_address": "IP address",
        "supported_protocols_services": [
            {
                "name": "Protocol name 1",
                "version": "Protocol version 1",
                "port": 123,
                "other_protocol_info": [
                    {"parameter": "info_key", "value": "info_value"}
                ]
            }
        ],
        "other_fw_os_info": [
            {"parameter": "fw_os_info", "value": "fw_os_value"}
        ]
    },
    "software": {
        "name": "SW name",
        "type": "SW type",
        "version": [
            {"version": "1.0"},
            {"version": "1.1"},
            {"version": "2.0"}
        ],
        "role": "client",
        "automatic_flag": "auto",
        "test_environment": [
            {"test_environment": "central_server"},
            {"test_environment": "docker"}
        ],
        "testing_protocol": {
            "name": "Protocol name 2",
            "version": "Protocol version 2",
            "port": 1234,
            "other_protocol_info": [
                {"parameter": "info_key", "value": "info_value"}
            ]
        },
        "testing_tool": [
            {
                "name": "Testing Tool name",
                "version": "Testing Tool version"
            }
        ],
        "other_software_info": [
            {"parameter": "info_key", "value": "info_value"}
        ]
    },
    "location": {
        "latitude": 11.1111,
        "longitude": 22.2222,
        "country": "CH",
        "city": "City",
        "address": "Address",
        "x": 0,
        "y": 0,
        "z": 0
    }
}

The RR provides on virtual host / the following services:

Service	Description
`resource_repository.insert_resource`	Insert a resource in the RR
`resource_repository.get_resource`	Query the RR for resources
`resource_repository.delete_resource`	Delete one resource

Insert Resource

Request parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`resource_repository.insert_resource.request`	The routing key used for the request
reply_to	`resource_repository.insert_resource.reply`	The routing key used for the reply
correlation_id	`str(uuid.uuid4())`	The correlation id of the request
content_type	`application/json`	The content type

Request body:

A JSON object matching the resource schema defined above. All fields are optional except resource_id and owner_id.

Reply parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`resource_repository.insert_resource.reply`	The routing key used for the reply
correlation_id	same as request	The correlation id of the request
content_type	`application/json`	The content type

Reply body:

Key	Type	Value
`ok`	Boolean	`true` if the resource was inserted correctly, `false` otherwise
`error`	String	A message describing the error (present only if `ok` is `false`)

Insert Resource Request example:

 - - -
ROUTING_KEY: resource_repository.insert_resource.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "resource_repository.insert_resource.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "resource_id": "unique-resource-identifier",
    "privacy_flag": "public",
    "available": true,
    "owner_id": "unique-owner-identifier",
    "location": {
        "latitude": 46.204391,
        "longitude": 6.143158,
        "country": "CH",
        "city": "Geneva"
    }
}

Insert Resource Reply example:

 - - -
ROUTING_KEY: resource_repository.insert_resource.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Get Resource

Request parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`resource_repository.get_resource.request`	The routing key used for the request
reply_to	`resource_repository.get_resource.reply`	The routing key used for the reply
correlation_id	`str(uuid.uuid4())`	The correlation id of the request
content_type	`application/json`	The content type

Request body:

An object matching the structure of a resource object. All fields that are present in the query will be used to match resources. All resources that match all the fields present in the query will be returned (see example).

Reply parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`resource_repository.get_resource.reply`	The routing key used for the reply
correlation_id	same as request	The correlation id of the request
content_type	`application/json`	The content type

Reply body:

Key	Type	Value
`ok`	Boolean	`true` if the query was executed correctly, `false` otherwise
`results`	Array of Objects	The resource(s) retrieved by the query
`error`	String	A message describing the error (present only if `ok` is `false`)

Get Resource Request example:

The following example shows how to retrieve all resources owned by
the owner id `unique-owner-identifier`.

 - - -
ROUTING_KEY: resource_repository.get_resource.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "resource_repository.get_resource.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owner_id": "unique-owner-identifier"
}

Get Resouce Reply example:

 - - -
ROUTING_KEY: resource_repository.get_resource.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true,
    "results": [
        {
            "resource_id": "unique-resource-identifier",
            "privacy_flag": "public",
            "available": true,
            "owner_id": "unique-owner-identifier",
            "location": {
                "latitude": 46.204391,
                "longitude": 6.143158,
                "country": "CH",
                "city": "Geneva"
            }
        }
    ]
}

Delete Resource

Request parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`resource_repository.delete_resource.request`	The routing key used for the request
reply_to	`resource_repository.delete_resource.reply`	The routing key used for the reply
correlation_id	`str(uuid.uuid4())`	The correlation id of the request
content_type	`application/json`	The content type

Request body:

Key	Type	Value
`resource_id`	String	The id of the resource
`owner_id`	String	The id of the owner user

Reply parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`resource_repository.delete_resource.reply`	The routing key used for the reply
correlation_id	same as request	The correlation id of the request
content_type	`application/json`	The content type

Reply body:

Key	Type	Value
`ok`	Boolean	`true` if the result(s) was deleted correctly, `false` otherwise
`error`	String	A message describing the error (present only if `ok` is `false`)

Delete Resouce Request example:

 - - -
ROUTING_KEY: rresource_repository.delete_resource.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "resource_repository.delete_resource.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "resource_id": "unique-resource-identifier",
    "owner_id": "unique-owner-identifier"
}

Delete Resource Reply example:

 - - -
ROUTING_KEY: resource_repository.delete_resource.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Events (core API)

This section describes the format of the messages used in F-Interop.

This section of the documentation is autogenerated by tool

Check out the messages library tool

Version 1.1.0

Orchestrator events

User interface events

ui.user.all.display


Routing key	ui.user.all.display
Requirements	…
Type	Event
Pub/Sub	TT -> UI
Description:	Message for displaying Markdown text to user interface

Source code: MsgUiDisplayMarkdownText

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.display
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "8580421b-f568-464e-a01c-989dd25834c6", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "p", 
            "value": "Hello World!"
        }
    ], 
    "level": null, 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "p", 
            "value": "Hello World!"
        }
    ], 
    "tags": {}, 
    "level": null
}

ui.user.all.request


Routing key	ui.user.all.request
Requirements	…
Type	Event
Pub/Sub	TT -> UI
Description:	Message for requesting a text input on UI

Source code: MsgUiRequestTextInput

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.request
 -  -  - 
Message properties: {
    "reply_to": "ui.user.all.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "039ea17d-3b98-4e41-9539-ebc6ccd424c5", 
    "message_id": "039ea17d-3b98-4e41-9539-ebc6ccd424c5", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "text", 
            "name": "input_name"
        }
    ], 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "text", 
            "name": "input_name"
        }
    ], 
    "tags": {}
}

ui.user.all.request


Routing key	ui.user.all.request
Requirements	…
Type	Event
Pub/Sub	TT -> UI
Description:	Message for requesting confirmation button

Source code: MsgUiRequestConfirmationButton

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.request
 -  -  - 
Message properties: {
    "reply_to": "ui.user.all.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "26c87027-6a69-4fa2-8b12-d4bce9f9fddb", 
    "message_id": "26c87027-6a69-4fa2-8b12-d4bce9f9fddb", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "button", 
            "name": "test_button", 
            "value": true
        }
    ], 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "button", 
            "name": "test_button", 
            "value": true
        }
    ], 
    "tags": {}
}

ui.core.session.get.request


Routing key	ui.core.session.get.request
Requirements	…
Type	Event
Pub/Sub	TT -> UI
Description:	Message for requesting session information to UI

Source code: MsgUiRequestSessionConfiguration

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.core.session.get.request
 -  -  - 
Message properties: {
    "reply_to": "ui.core.session.get.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "5549a795-3143-4bb6-83af-b337691466fb", 
    "message_id": "5549a795-3143-4bb6-83af-b337691466fb", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0"
}

ui.core.session.get.reply


Routing key	ui.core.session.get.reply
Requirements	…
Type	Event
Pub/Sub	UI -> TT
Description:	Message for requesting session information to UI

Source code: MsgUiSessionConfigurationReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.core.session.get.reply
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "correlation_id": "127104de-6eaf-450c-9894-ab57423dac0d", 
    "message_id": "ce6d27b7-a579-49e4-b0eb-de37f7ef3c25", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "amqp_url": "amqp://WX9D3L5A:5S68CRDC@mq.dev.f-interop.eu:443/277704a1-03c0-467c-b00d-c984976692d7", 
    "logs": [
        {
            "date": "2018-05-07T12:50:47.224000+00:00", 
            "message": "Session created locally", 
            "type": "info"
        }
    ], 
    "ok": true, 
    "resources": [
        {}
    ], 
    "shared": true, 
    "slice_id": "urn:publicid:IDN+finterop:project1+slice+testing", 
    "start_date": "2018-05-07T12:50:48.128000+00:00", 
    "status": "open", 
    "testSuite": "http://orchestrator.dev.f-interop.eu:8181/tests/f-interop/dummy-tool-shared", 
    "testSuiteType": "interoperability", 
    "users": [
        "federico_sismondiojxu", 
        "myslice", 
        "federicosismondiparu"
    ]
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "status": "open", 
    "testSuite": "http://orchestrator.dev.f-interop.eu:8181/tests/f-interop/dummy-tool-shared", 
    "ok": true, 
    "logs": [
        {
            "date": "2018-05-07T12:50:47.224000+00:00", 
            "message": "Session created locally", 
            "type": "info"
        }
    ], 
    "slice_id": "urn:publicid:IDN+finterop:project1+slice+testing", 
    "_api_version": "1.1.0", 
    "testSuiteType": "interoperability", 
    "amqp_url": "amqp://WX9D3L5A:5S68CRDC@mq.dev.f-interop.eu:443/277704a1-03c0-467c-b00d-c984976692d7", 
    "shared": true, 
    "start_date": "2018-05-07T12:50:48.128000+00:00", 
    "resources": [
        {}
    ], 
    "users": [
        "federico_sismondiojxu", 
        "myslice", 
        "federicosismondiparu"
    ]
}

ui.user.all.request


Routing key	ui.user.all.request
Requirements	…
Type	Event
Pub/Sub	TT -> UI
Description:	Message for file upload request on UI

Source code: MsgUiRequestUploadFile

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.request
 -  -  - 
Message properties: {
    "reply_to": "ui.user.all.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "b7da591f-b651-404c-99c6-2d587d558e03", 
    "message_id": "b7da591f-b651-404c-99c6-2d587d558e03", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "file", 
            "name": "upload a file"
        }
    ], 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "file", 
            "name": "upload a file"
        }
    ], 
    "tags": {}
}

ui.user.all.request


Routing key	ui.user.all.request
Requirements	…
Type	Event
Pub/Sub	TT -> UI
Description:	Message for checkbox request on UI

Source code: MsgUiRequestQuestionCheckbox

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.request
 -  -  - 
Message properties: {
    "reply_to": "ui.user.all.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "8348c8e8-24f9-44c8-bd98-140e5c81bae9", 
    "message_id": "8348c8e8-24f9-44c8-bd98-140e5c81bae9", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "checkbox", 
            "name": "Choice1", 
            "value": 0, 
            "label": "Choice1"
        }, 
        {
            "type": "checkbox", 
            "name": "Choice2", 
            "value": 1, 
            "label": "Choice2"
        }
    ], 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "checkbox", 
            "name": "Choice1", 
            "value": 0, 
            "label": "Choice1"
        }, 
        {
            "type": "checkbox", 
            "name": "Choice2", 
            "value": 1, 
            "label": "Choice2"
        }
    ], 
    "tags": {}
}

ui.user.all.request


Routing key	ui.user.all.request
Requirements	…
Type	Event
Pub/Sub	TT -> UI
Description:	Message for radio request on UI

Source code: MsgUiRequestQuestionRadio

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.request
 -  -  - 
Message properties: {
    "reply_to": "ui.user.all.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "2526ab40-2906-489b-b188-1005d2fad282", 
    "message_id": "2526ab40-2906-489b-b188-1005d2fad282", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "radio", 
            "name": "True", 
            "value": true
        }, 
        {
            "type": "radio", 
            "name": "False", 
            "value": false
        }
    ], 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "radio", 
            "name": "True", 
            "value": true
        }, 
        {
            "type": "radio", 
            "name": "False", 
            "value": false
        }
    ], 
    "tags": {}
}

Testing Tool events

testingtool.ready


Routing key	testingtool.ready
Requirements	TT SHOULD publish event as soon as TT is up and listening on the event bus
Type	Event
Pub/Sub	Testing Tool -> GUI
Description:	Used to indicate to the GUI that testing is ready to start the test suite

Source code: MsgTestingToolReady

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testingtool.ready
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "5a5d99ed-954c-4b99-8b5f-8f3bf8396f8c", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Testing tool READY to start test suite."
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "description": "Testing tool READY to start test suite."
}

session.configuration


Routing key	session.configuration
Requirements	TT SHOULD listen to event, and configure accordingly
Type	Event
Pub/Sub	Orchestrator -> Testing Tool
Description:	TT SHOULD listen to this message and configure the testsuite correspondingly

Source code: MsgSessionConfiguration

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: session.configuration
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "b46a5a1d-765f-4232-a494-ed5a5eb7d936", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "configuration": {
        "testsuite.testcases": [
            "someTestCaseId1", 
            "someTestCaseId2"
        ]
    }, 
    "session_id": "666", 
    "testing_tools": "f-interop/someTestToolId", 
    "users": [
        "u1", 
        "f-interop"
    ]
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "configuration": {
        "testsuite.testcases": [
            "someTestCaseId1", 
            "someTestCaseId2"
        ]
    }, 
    "testing_tools": "f-interop/someTestToolId", 
    "session_id": "666", 
    "users": [
        "u1", 
        "f-interop"
    ]
}

testingtool.configured


Routing key	testingtool.configured
Requirements	TT SHOULD publish event once session.configuration message has been processed.
Type	Event
Pub/Sub	Testing Tool -> Orchestrator, GUI
Description:	The goal is to notify orchestrator and other components that the testing tool has been configured

Source code: MsgTestingToolConfigured

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testingtool.configured
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "66275e5b-3ec1-4c2e-a3d0-6cec3c17357a", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Testing tool CONFIGURED", 
    "session_id": null, 
    "testing_tools": "f-interop/interoperability-coap"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "description": "Testing tool CONFIGURED", 
    "session_id": null, 
    "testing_tools": "f-interop/interoperability-coap"
}

testsuite.start


Routing key	testsuite.start
Requirements	TT SHOULD listen to event and start the test suite right after reception. MsgTestSuiteStarted
Type	Event
Pub/Sub	GUI -> Testing Tool
Description:	tbd

Source code: MsgTestSuiteStart

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.start
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "cad6dc20-1a51-447c-873f-ecd69ec915ee", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Test suite START command"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "description": "Test suite START command"
}

testsuite.testcase.step.stimuli.execute


Routing key	testsuite.testcase.step.stimuli.execute
Requirements	TT SHOULD publish event
Type	Event
Pub/Sub	Testing Tool -> GUI
Description:
	- Used to indicate to the GUI (or automated-iut) which is the stimuli step to be executed by the user (or
	automated-IUT).

Source code: MsgStepStimuliExecute

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.testcase.step.stimuli.execute
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "c13e6e99-ac85-4ae1-bd74-20aaa97b0555", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Please execute TD_COAP_CORE_01_step_01", 
    "node": "coap_client", 
    "node_execution_mode": "user_assisted", 
    "step_id": "TD_COAP_CORE_01_step_01", 
    "step_info": [
        "Client is requested to send a GET request with", 
        "Type = 0(CON)", 
        "Code = 1(GET)"
    ], 
    "step_state": "executing", 
    "step_type": "stimuli", 
    "target_address": null, 
    "testcase_id": null, 
    "testcase_ref": null
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "node": "coap_client", 
    "step_state": "executing", 
    "description": "Please execute TD_COAP_CORE_01_step_01", 
    "_api_version": "1.1.0", 
    "testcase_ref": null, 
    "testcase_id": null, 
    "step_type": "stimuli", 
    "target_address": null, 
    "step_info": [
        "Client is requested to send a GET request with", 
        "Type = 0(CON)", 
        "Code = 1(GET)"
    ], 
    "node_execution_mode": "user_assisted", 
    "step_id": "TD_COAP_CORE_01_step_01"
}

testsuite.testcase.step.stimuli.executed


Routing key	testsuite.testcase.step.stimuli.executed
Requirements	TT SHOULD listen to event
Type	Event
Pub/Sub	GUI (or automated-IUT)-> Testing Tool
Description:
	- Used to indicate stimuli has been executed by user (and it’s user-assisted iut) or by automated-iut

Source code: MsgStepStimuliExecuted

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.testcase.step.stimuli.executed
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "2a31d848-1689-4c01-9e1c-a87fbbcbcc7e", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Step (stimuli) EXECUTED", 
    "node": "coap_client", 
    "node_execution_mode": "user_assisted"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "node": "coap_client", 
    "node_execution_mode": "user_assisted", 
    "description": "Step (stimuli) EXECUTED"
}

testsuite.testcase.step.verify.execute


Routing key	testsuite.testcase.step.verify.execute
Requirements	TT SHOULD publish event
Type	Event
Pub/Sub	Testing Tool -> GUI (or automated-IUT)
Description:
	- Used to indicate to the GUI (or automated-iut) which is the verify step to be executed by the user (or
	automated-IUT).

Source code: MsgStepVerifyExecute

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.testcase.step.verify.execute
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "474476b9-d879-4332-97d1-a118528bcbe9", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Please execute TD_COAP_CORE_01_step_04", 
    "node": "coap_client", 
    "node_execution_mode": "user_assisted", 
    "response_type": "bool", 
    "step_id": "TD_COAP_CORE_01_step_04", 
    "step_info": [
        "Client displays the received information"
    ], 
    "step_state": "executing", 
    "step_type": "verify", 
    "testcase_id": null, 
    "testcase_ref": null
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "node": "coap_client", 
    "step_state": "executing", 
    "description": "Please execute TD_COAP_CORE_01_step_04", 
    "_api_version": "1.1.0", 
    "testcase_ref": null, 
    "testcase_id": null, 
    "step_type": "verify", 
    "step_info": [
        "Client displays the received information"
    ], 
    "response_type": "bool", 
    "node_execution_mode": "user_assisted", 
    "step_id": "TD_COAP_CORE_01_step_04"
}

testsuite.testcase.step.verify.executed


Routing key	testsuite.testcase.step.verify.executed
Requirements	TT SHOULD listen to event
Type	Event
Pub/Sub	GUI (or automated-IUT)-> Testing Tool
Description:
	- Message generated by user (GUI or automated-IUT) declaring if the IUT VERIFY verifies the expected behaviour.

Source code: MsgStepVerifyExecuted

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.testcase.step.verify.executed
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "b2aff90a-092e-4eee-a15c-ff2ceb28c3b0", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Step (verify) EXECUTED", 
    "node": "coap_client", 
    "node_execution_mode": "user_assisted", 
    "response_type": "bool", 
    "verify_response": true
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "node": "coap_client", 
    "description": "Step (verify) EXECUTED", 
    "_api_version": "1.1.0", 
    "response_type": "bool", 
    "node_execution_mode": "user_assisted", 
    "verify_response": true
}

testsuite.testcase.verdict


Routing key	testsuite.testcase.verdict
Requirements	TT SHOULD publish event
Type	Event
Pub/Sub	Testing Tool -> GUI
Description:	Used to indicate to the GUI (or automated-iut) which is the final verdict of the testcase.

Source code: MsgTestCaseVerdict

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.testcase.verdict
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "d839ebb1-f115-4a0d-b799-4c07d7a9755a", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "No interoperability error was detected,", 
    "objective": "Perform GET transaction(CON mode)", 
    "partial_verdicts": [
        [
            "TD_COAP_CORE_01_step_02", 
            null, 
            "CHECK postponed", 
            ""
        ], 
        [
            "TD_COAP_CORE_01_step_03", 
            null, 
            "CHECK postponed", 
            ""
        ], 
        [
            "TD_COAP_CORE_01_step_04", 
            "pass", 
            "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
            ""
        ], 
        [
            "CHECK_1_post_mortem_analysis", 
            "pass", 
            "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
        ], 
        [
            "CHECK_2_post_mortem_analysis", 
            "pass", 
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
        ], 
        [
            "CHECK_3_post_mortem_analysis", 
            "pass", 
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
        ]
    ], 
    "state": "finished", 
    "testcase_id": "TD_COAP_CORE_01", 
    "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01", 
    "verdict": "pass"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "description": "No interoperability error was detected,", 
    "_api_version": "1.1.0", 
    "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01", 
    "testcase_id": "TD_COAP_CORE_01", 
    "partial_verdicts": [
        [
            "TD_COAP_CORE_01_step_02", 
            null, 
            "CHECK postponed", 
            ""
        ], 
        [
            "TD_COAP_CORE_01_step_03", 
            null, 
            "CHECK postponed", 
            ""
        ], 
        [
            "TD_COAP_CORE_01_step_04", 
            "pass", 
            "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
            ""
        ], 
        [
            "CHECK_1_post_mortem_analysis", 
            "pass", 
            "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
        ], 
        [
            "CHECK_2_post_mortem_analysis", 
            "pass", 
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
        ], 
        [
            "CHECK_3_post_mortem_analysis", 
            "pass", 
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
        ]
    ], 
    "state": "finished", 
    "verdict": "pass", 
    "objective": "Perform GET transaction(CON mode)"
}

testsuite.report


Routing key	testsuite.report
Requirements	TT SHOULD publish event
Type	Event
Pub/Sub	Testing Tool -> GUI
Description:	Used to indicate to the GUI (or automated-iut) the final results of the test session.

Source code: MsgTestSuiteReport

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.report
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "39d709a8-ade1-4f0f-8117-daad33dbed0d", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "tc_results": [
        {
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_01_step_02", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_01_step_03", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_01_step_04", 
                    "pass", 
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
                    ""
                ], 
                [
                    "CHECK_1_post_mortem_analysis", 
                    "pass", 
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
                ], 
                [
                    "CHECK_2_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
                ], 
                [
                    "CHECK_3_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
                ]
            ], 
            "testcase_id": "TD_COAP_CORE_01", 
            "verdict": "pass", 
            "description": "No interoperability error was detected,"
        }, 
        {
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_02_step_02", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_02_step_03", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_02_step_04", 
                    "pass", 
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
                    ""
                ], 
                [
                    "CHECK_1_post_mortem_analysis", 
                    "pass", 
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43213] DELETE /test> Match: CoAP(type=0, code=4)"
                ], 
                [
                    "CHECK_2_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43213] 2.02 Deleted > Match: CoAP(code=66, mid=0xa8cd, tok=b'')"
                ]
            ], 
            "testcase_id": "TD_COAP_CORE_02", 
            "verdict": "pass", 
            "description": "No interoperability error was detected,"
        }
    ]
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "tc_results": [
        {
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_01_step_02", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_01_step_03", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_01_step_04", 
                    "pass", 
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
                    ""
                ], 
                [
                    "CHECK_1_post_mortem_analysis", 
                    "pass", 
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
                ], 
                [
                    "CHECK_2_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
                ], 
                [
                    "CHECK_3_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
                ]
            ], 
            "testcase_id": "TD_COAP_CORE_01", 
            "verdict": "pass", 
            "description": "No interoperability error was detected,"
        }, 
        {
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_02_step_02", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_02_step_03", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_02_step_04", 
                    "pass", 
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
                    ""
                ], 
                [
                    "CHECK_1_post_mortem_analysis", 
                    "pass", 
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43213] DELETE /test> Match: CoAP(type=0, code=4)"
                ], 
                [
                    "CHECK_2_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43213] 2.02 Deleted > Match: CoAP(code=66, mid=0xa8cd, tok=b'')"
                ]
            ], 
            "testcase_id": "TD_COAP_CORE_02", 
            "verdict": "pass", 
            "description": "No interoperability error was detected,"
        }
    ]
}

testingtool.terminate


Routing key	testingtool.terminate
Requirements	TT SHOULD listen to event, and handle a gracefully termination of all it’s processes
Type	Event
Pub/Sub	GUI, (or Orchestrator) -> Testing Tool
Description:	Testing tool should stop all it’s processes gracefully.

Source code: MsgTestingToolTerminate

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testingtool.terminate
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "12f42067-768f-4a05-9b16-3a73b4860084", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Command TERMINATE testing tool execution"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "description": "Command TERMINATE testing tool execution"
}

sniffing.start.request


Routing key	sniffing.start.request
Requirements	Testing Tool SHOULD implement (other components should not subscribe to event)
Type	Request (service)
Pub/Sub	coordination -> sniffing
Description:	tbd

Source code: MsgSniffingStart

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: sniffing.start.request
 -  -  - 
Message properties: {
    "reply_to": "sniffing.start.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "06cccbb3-6a24-4c10-9110-2a2c59a502dd", 
    "message_id": "06cccbb3-6a24-4c10-9110-2a2c59a502dd", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "capture_id": "TD_COAP_CORE_01", 
    "filter_if": "tun0", 
    "filter_proto": "udp"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "capture_id": "TD_COAP_CORE_01", 
    "filter_if": "tun0", 
    "filter_proto": "udp"
}

sniffing.stop.request


Routing key	sniffing.stop.request
Requirements	Testing Tool SHOULD implement (other components should not subscribe to event)
Type	Request (service)
Pub/Sub	coordination -> sniffing
Description:	tbd

Source code: MsgSniffingStop

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: sniffing.stop.request
 -  -  - 
Message properties: {
    "reply_to": "sniffing.stop.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "cdd45f6d-7c03-4625-a466-16d1892a3199", 
    "message_id": "cdd45f6d-7c03-4625-a466-16d1892a3199", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0"
}

sniffing.getcapture.request


Routing key	sniffing.getcapture.request
Requirements	Testing Tool SHOULD implement (other components should not subscribe to event)
Type	Request (service)
Pub/Sub	coordination -> sniffing
Description:	tbd

Source code: MsgSniffingGetCapture

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: sniffing.getcapture.request
 -  -  - 
Message properties: {
    "reply_to": "sniffing.getcapture.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "62a982a8-bd52-447a-87cf-bc039b799616", 
    "message_id": "62a982a8-bd52-447a-87cf-bc039b799616", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "capture_id": "TD_COAP_CORE_01"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "capture_id": "TD_COAP_CORE_01"
}

sniffing.getlastcapture.request


Routing key	sniffing.getlastcapture.request
Requirements	Testing Tool SHOULD implement (other components should not subscribe to event)
Type	Request (service)
Pub/Sub	coordination -> sniffing
Description:	tbd

Source code: MsgSniffingGetCaptureLast

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: sniffing.getlastcapture.request
 -  -  - 
Message properties: {
    "reply_to": "sniffing.getlastcapture.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "276773c2-40cc-461d-bede-601c9bff96e5", 
    "message_id": "276773c2-40cc-461d-bede-601c9bff96e5", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0"
}

dissection.dissectcapture.request


Routing key	dissection.dissectcapture.request
Requirements	Testing Tool SHOULD implement (other components should not subscribe to event)
Type	Request (service)
Pub/Sub	coordination -> dissection, analysis -> dissection
Description:	TBD

Source code: MsgDissectionDissectCapture

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: dissection.dissectcapture.request
 -  -  - 
Message properties: {
    "reply_to": "dissection.dissectcapture.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "86ec1ea6-93c0-41f5-be42-4a4a7e5bce8b", 
    "message_id": "86ec1ea6-93c0-41f5-be42-4a4a7e5bce8b", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "file_enc": "pcap_base64", 
    "filename": "TD_COAP_CORE_01.pcap", 
    "protocol_selection": "coap", 
    "value": "1MOyoQIABAAAAAAAAAAAAMgAAABlAAAAqgl9WK8aBgA7AAAAOwAAAGADPxUAExFAu7sAAAAAAAAAAAAAAAAAAbu7AAAAAAAAAAAAAAAAAALXvBYzABNZUEABcGO0dGVzdMECqgl9WMcaBgCQAAAAkAAAAGAAAAAAaDr//oAAAAAAAAAAAAAAAAAAA7u7AAAAAAAAAAAAAAAAAAGJAAcTAAAAALu7AAAAAAAAAAAAAAAAAAK7uwAAAAAAAAAAAAAAAAACBAgAAAAAAABgAz8VABMRQLu7AAAAAAAAAAAAAAAAAAG7uwAAAAAAAAAAAAAAAAAC17wWMwATWVBAAXBjtHRlc6oJfVjSGgYAOwAAADsAAABgAz8VABMRP7u7AAAAAAAAAAAAAAAAAAG7uwAAAAAAAAAAAAAAAAAC17wWMwATWVBAAXBjtHRlc3TBAg=="
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "protocol_selection": "coap", 
    "filename": "TD_COAP_CORE_01.pcap", 
    "value": "1MOyoQIABAAAAAAAAAAAAMgAAABlAAAAqgl9WK8aBgA7AAAAOwAAAGADPxUAExFAu7sAAAAAAAAAAAAAAAAAAbu7AAAAAAAAAAAAAAAAAALXvBYzABNZUEABcGO0dGVzdMECqgl9WMcaBgCQAAAAkAAAAGAAAAAAaDr//oAAAAAAAAAAAAAAAAAAA7u7AAAAAAAAAAAAAAAAAAGJAAcTAAAAALu7AAAAAAAAAAAAAAAAAAK7uwAAAAAAAAAAAAAAAAACBAgAAAAAAABgAz8VABMRQLu7AAAAAAAAAAAAAAAAAAG7uwAAAAAAAAAAAAAAAAAC17wWMwATWVBAAXBjtHRlc6oJfVjSGgYAOwAAADsAAABgAz8VABMRP7u7AAAAAAAAAAAAAAAAAAG7uwAAAAAAAAAAAAAAAAAC17wWMwATWVBAAXBjtHRlc3TBAg==", 
    "file_enc": "pcap_base64"
}

dissection.dissectcapture.reply


Routing key	dissection.dissectcapture.reply
Requirements	Testing Tool SHOULD implement (other components should not subscribe to event)
Type	Reply (service)
Pub/Sub	Dissector -> Coordinator, Dissector -> Analyzer
Description:	TBD

Source code: MsgDissectionDissectCaptureReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: dissection.dissectcapture.reply
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "correlation_id": "1e58eab1-f7e2-4a5d-b8f5-bc9edbc1fabb", 
    "message_id": "94691506-af7e-4a6d-992a-4d4752a02534", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "frames": [
        {
            "timestamp": 1464858393.547275, 
            "_type": "frame", 
            "protocol_stack": [
                {
                    "_type": "protocol", 
                    "ProtocolFamily": "0", 
                    "AddressFamily": "2", 
                    "_protocol": "NullLoopback"
                }, 
                {
                    "TotalLength": "41", 
                    "_type": "protocol", 
                    "DestinationAddress": "127.0.0.1", 
                    "Reserved": "0", 
                    "MoreFragments": "0", 
                    "HeaderLength": "5", 
                    "DontFragment": "0", 
                    "HeaderChecksum": "0x0000", 
                    "Version": "4", 
                    "Identification": "0x71ac", 
                    "TypeOfService": "0x00", 
                    "_protocol": "IPv4", 
                    "Protocol": "17", 
                    "TimeToLive": "64", 
                    "SourceAddress": "127.0.0.1", 
                    "Options": "b''", 
                    "FragmentOffset": "0"
                }
            ], 
            "id": 1, 
            "error": null
        }
    ], 
    "frames_simple_text": null, 
    "ok": true, 
    "token": "0lzzb_Bx30u8Gu-xkt1DFE1GmB4"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "frames": [
        {
            "timestamp": 1464858393.547275, 
            "_type": "frame", 
            "protocol_stack": [
                {
                    "_type": "protocol", 
                    "ProtocolFamily": "0", 
                    "AddressFamily": "2", 
                    "_protocol": "NullLoopback"
                }, 
                {
                    "TotalLength": "41", 
                    "_type": "protocol", 
                    "DestinationAddress": "127.0.0.1", 
                    "Reserved": "0", 
                    "MoreFragments": "0", 
                    "HeaderLength": "5", 
                    "DontFragment": "0", 
                    "HeaderChecksum": "0x0000", 
                    "Version": "4", 
                    "Identification": "0x71ac", 
                    "TypeOfService": "0x00", 
                    "_protocol": "IPv4", 
                    "Protocol": "17", 
                    "TimeToLive": "64", 
                    "SourceAddress": "127.0.0.1", 
                    "Options": "b''", 
                    "FragmentOffset": "0"
                }
            ], 
            "id": 1, 
            "error": null
        }
    ], 
    "token": "0lzzb_Bx30u8Gu-xkt1DFE1GmB4", 
    "ok": true, 
    "frames_simple_text": null
}

analysis.interop.testcase.analyze.request


Routing key	analysis.interop.testcase.analyze.request
Requirements	Testing Tool SHOULD implement (other components should not subscribe to event)
Type	Request (service)
Pub/Sub	coordination -> analysis
Description:
	- Method to launch an analysis from a pcap file or a token if the pcap file has already been provided.
	- The method need a token or a pcap_file but doesn’t allow someone to provide both.

Source code: MsgInteropTestCaseAnalyze

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: analysis.interop.testcase.analyze.request
 -  -  - 
Message properties: {
    "reply_to": "analysis.interop.testcase.analyze.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "e1133f12-1dab-4c14-b2ed-cd3698138ef1", 
    "message_id": "e1133f12-1dab-4c14-b2ed-cd3698138ef1", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "file_enc": "pcap_base64", 
    "filename": "TD_COAP_CORE_01.pcap", 
    "protocol": "coap", 
    "testcase_id": "TD_COAP_CORE_01", 
    "testcase_ref": "http://doc.f-interop.eu/tests/TD_COAP_CORE_01", 
    "value": "1MOyoQIABAAAAAAAAAAAAMgAAAAAAAAA"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "protocol": "coap", 
    "_api_version": "1.1.0", 
    "testcase_ref": "http://doc.f-interop.eu/tests/TD_COAP_CORE_01", 
    "testcase_id": "TD_COAP_CORE_01", 
    "value": "1MOyoQIABAAAAAAAAAAAAMgAAAAAAAAA", 
    "filename": "TD_COAP_CORE_01.pcap", 
    "file_enc": "pcap_base64"
}

analysis.interop.testcase.analyze.reply


Routing key	analysis.interop.testcase.analyze.reply
Requirements	Testing Tool SHOULD implement (other components should not subscribe to event)
Type	Reply (service)
Pub/Sub	analysis -> coordination
Description:
	- The recommended structure for the partial_verdicts field is a list of partial verdicts which complies to:
	- each one of those elements of the list correspond to one CHECK or VERIFY steps of the test description
	- first value of the list MUST be a “pass”, “fail”, “inconclusive” or eventually “error” partial verdict (
	string)
	- the second value MUST be a string with a description of partial verdict (intended for the user)
	- more values elements MAY be added to the list

Source code: MsgInteropTestCaseAnalyzeReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: analysis.interop.testcase.analyze.reply
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "correlation_id": "953df231-48a8-4215-ace2-140b7e221537", 
    "message_id": "a935b017-40fe-4851-8402-53af99691863", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "analysis_type": "postmortem", 
    "description": "The test purpose has been verified without any fault detected", 
    "ok": true, 
    "partial_verdicts": [
        [
            "pass", 
            "<Frame   1: [127.0.0.1 -> 127.0.0.1] CoAP [CON 43521] GET /test> Match: CoAP(type=0, code=1)"
        ], 
        [
            "pass", 
            "<Frame   2: [127.0.0.1 -> 127.0.0.1] CoAP [ACK 43521] 2.05 Content > Match: CoAP(code=69, mid=0xaa01,                 tok=b'b\\xda', pl=Not(b''))"
        ], 
        [
            "pass", 
            "<Frame   2: [127.0.0.1 -> 127.0.0.1] CoAP [ACK 43521] 2.05 Content >                 Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
        ]
    ], 
    "review_frames": [], 
    "testcase_id": "TD_COAP_CORE_01", 
    "testcase_ref": "http://doc.f-interop.eu/tests/TD_COAP_CORE_01", 
    "token": "0lzzb_Bx30u8Gu-xkt1DFE1GmB4", 
    "verdict": "pass"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "ok": true, 
    "description": "The test purpose has been verified without any fault detected", 
    "_api_version": "1.1.0", 
    "testcase_ref": "http://doc.f-interop.eu/tests/TD_COAP_CORE_01", 
    "testcase_id": "TD_COAP_CORE_01", 
    "review_frames": [], 
    "partial_verdicts": [
        [
            "pass", 
            "<Frame   1: [127.0.0.1 -> 127.0.0.1] CoAP [CON 43521] GET /test> Match: CoAP(type=0, code=1)"
        ], 
        [
            "pass", 
            "<Frame   2: [127.0.0.1 -> 127.0.0.1] CoAP [ACK 43521] 2.05 Content > Match: CoAP(code=69, mid=0xaa01,                 tok=b'b\\xda', pl=Not(b''))"
        ], 
        [
            "pass", 
            "<Frame   2: [127.0.0.1 -> 127.0.0.1] CoAP [ACK 43521] 2.05 Content >                 Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
        ]
    ], 
    "token": "0lzzb_Bx30u8Gu-xkt1DFE1GmB4", 
    "verdict": "pass", 
    "analysis_type": "postmortem"
}

Results Repository events

Resources Repository events

Visualization tools events

viztool-grafana.init.request


Routing key	viztool-grafana.init.request
Requirements	Implementing Test Tools should send this at start
Type	Event
Pub/Sub	Testing Tool -> Visualization Tool
Description:	Visualization Tool is waiting for this message
	Visualization Tool MUST listen to this
	to start init routines

Source code: MsgVizInitRequest

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: viztool-grafana.init.request
 -  -  - 
Message properties: {
    "reply_to": "viztool-grafana.init.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "956eb46c-e161-469c-b427-a1711e4287f9", 
    "message_id": "956eb46c-e161-469c-b427-a1711e4287f9", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0"
}

viztool-grafana.init.reply


Routing key	viztool-grafana.init.reply
Requirements	Visualization MUST send this after recieving MsgVizInitRequest
Type	Event
Pub/Sub	Visualization Tool -> Testing Tool
Description:	This message contains the URL to access the internal Webserver
	Test Tool MUST listen to this
	that serves the Grafana Instance

Source code: MsgVizInitReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: viztool-grafana.init.reply
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "correlation_id": "ac0c6803-1606-4f88-9f87-37e2e7107fa5", 
    "message_id": "1e62d419-166b-4aaa-a224-41ddfe507dc6", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "ok": true, 
    "url": "http://url-to-access-grafana:1234"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "url": "http://url-to-access-grafana:1234", 
    "ok": true
}

viztool-grafana.set_dashboard.request


Routing key	viztool-grafana.set_dashboard.request
Requirements
Type	Event
Pub/Sub	Testing Tool -> Visualization Tool
Description:	Visualization Tool uses this message to configure the Dashboard based on the JSON config
	- Visualization Tool MUST listen to this.
	- Test Tools should send this message after the Visualization Tool has confirmed the initialization

Source code: MsgVizDashboardRequest

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: viztool-grafana.set_dashboard.request
 -  -  - 
Message properties: {
    "reply_to": "viztool-grafana.set_dashboard.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "cb5c4eeb-0797-4922-98d3-44b9cddcf11c", 
    "message_id": "cb5c4eeb-0797-4922-98d3-44b9cddcf11c", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "config": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "config": {}
}

viztool-grafana.set_dashboard.reply


Routing key	viztool-grafana.set_dashboard.reply
Requirements
Type	Event
Pub/Sub	Visualization Tool -> Testing Tool
Description:	This message contains the URL to access the internal Webserver that serves the Grafana Instance
	- Visualization MUST send this after recieving MsgVizDashboardRequest
	- Test Tool MUST listen to this

Source code: MsgVizDashboardReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: viztool-grafana.set_dashboard.reply
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "correlation_id": "21e86b23-4d0a-4551-9392-bb653b1e6328", 
    "message_id": "64d4784e-76cd-4699-b3a5-61a217d65626", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "ok": true
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "ok": true
}

viztool-grafana.write_data


Routing key	viztool-grafana.write_data
Requirements
Type	Event
Pub/Sub	Performance Testing Tool -> Visualization
Description:
	- Performance Testing Tool SHOULD emit this event periodically
	- Visualization Tool MUST listen to this event
	- During the test execution, the Performance Testing Tool MUST periodically emit this event carrying current performance statistics/measurements

Source code: MsgVizWrite

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: viztool-grafana.write_data
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "a7968257-9348-49a4-9986-1d291b1f3899", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": {
        "value": 0
    }, 
    "measurement": "name", 
    "tags": {}, 
    "time": 0
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -

 {
    "_api_version": "1.1.0", 
    "fields": {
        "value": 0
    }, 
    "tags": {}, 
    "time": 0, 
    "measurement": "name"
}

Container services (CS)

This section introduces the Container Services (CS) and explain how to use them. This feature allows containers like Testing Tools (TT) or Visualization Tools (VT) to expose their own web services through the Session Orchestrator (SO).

URL-port Mappings

When a container, having inside a web server, is spawned on the SO then a particular URL and a port are created. The URL specifies where to find that service on the SO host while the container must use the given port to expose its service inside the SO host. Then an NGINX proxy server will redirect HTTP traffic from that URL to the corresponding port.

For example, if we have two containers service_1 and service_2 both willing to expose their web services, then the SO will create two mappings like:

http(s)://so.f-interop.eu:8282/container-services/session_id/service_1 -> localhost:6123 http(s)://so.f-interop.eu:8282/container-services/session_id/service_2 -> localhost:6124

As a consequence service_1 must use port 6123 while service_2 must use port 6124. Containers are able to know what URL has been assigned to them, this way they can inform other parties about how to reach them. For example they can use GUI API to have their web service embedded as an iframe into the GUI.

URL-port mappings are stored internally by the SO.

Container Template

The supervisor template of a container (stored in the SO) willing to expose a service must follow a few rules. On the right a minimal example.

Minimal example of a container template

[program:{{ session }}|service_1]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service_1-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service_1-stderr.log
command = docker run
    --env SERVICE_URL={{ service_1_url }}
    --rm
    --name="session_{{ session }}-service_1"
    -p {{ service_1_port }}:8080
    service_1-image-name

The variables used to access the given url and port are structured as follows:

Name	Value
`<program_name>_url`	The URL to access the service
`<program_name>_port`	The port number to be used internally

Where <program_name> is the name of the supervisor program described in the template but without the prefix {{ session }}|. For example, if the template defines the program as [program:{{ session }}|my_complex_program_name] then <program_name> will correspond to my_complex_program_name. As a consequence <program_name>_url will correspond to my_complex_program_name_url, while <program_name>_port will correspond to my_complex_program_name_port.

Note that in the example we use Docker and so we can explicitely define what port the Docker container should expose. The port used inside the container, in this case 8080, can be any port as long as the service inside the container will use the same port.

The program name should not contain invalid or reserved URL characters and also invalid Jinja variable characters like -. In general, avoid all these characters:

-{}|\^[]`;/?:@&=+$,

We strongly suggest to only use (lowercase) alphanumeric characters plus the underscore _.

The template can, of course, still use the usual variables like amqp_url and amqp_exchange if the container needs to connect to the session AMQP bus.

There can also be more than one service per session in the same config file, just use different valid names.

URLs can also be directly passed to other programs, for example in the case there is a container that needs to communicate with the other container through HTTP. See the examples on the right.

Example 1: Container using amqp_url and amqp_exchange variables

[program:{{ session }}|service_2]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service_2-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service_2-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env SERVICE_URL={{ service_2_url }}
    --rm
    --name="session_{{ session }}-service_2"
    -p {{ service_2_port }}:8080
    service_2-image-name

Example 2: Two containers exposing two different services

[program:{{ session }}|service_1]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service_1-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service_1-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env SERVICE_URL={{ service_1_url }}
    --rm
    --name="session_{{ session }}-service_1"
    -p {{ service_1_port }}:8080
    service_1-image-name

[program:{{ session }}|service_2]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service_2-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service_2-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env SERVICE_URL={{ service_2_url }}
    --rm
    --name="session_{{ session }}-service_2"
    -p {{ service_2_port }}:8080
    service_2-image-name

Example 3: A container tool using the service of the second container

[program:{{ session }}|my_tool]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-my_tool-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-my_tool-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env SERVICE_2_URL={{ service_2_url }}
    --rm
    --name="session_{{ session }}-my_tool"
    my_tool-image-name

[program:{{ session }}|my_service]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-my_service-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-my_service-stderr.log
command = docker run
    --rm
    --name="session_{{ session }}-my_service"
    -p {{ my_service_port }}:8080
    my_service-image-name

Visualization Tools (VT)

This section of the documentation describes the visualization tool’s minimal requirements.

General requirements on Visualization Tools (VT):

Visualization Tools are an implementation of Container Services (CS) and therefore they must respect CS requirements and conventions.

Minimal set of events to handle

There is no minimal set of events to handle.

Reference implementations of VTs:

See viztool-grafana for reference implementation.

Viztool-Grafana

( version of the API v.2.0 )

This tool offers real-time visualization support for Time Series data to F-Interop testing tools. The data is stored internally by using InfluxDB and the visualization is handled by Grafana.

The viztool-grafana repository can be found here.

Using the tool

To use the tool, its Supervisor configuration template must be copied inside the configuration template of the tool that wants to use it. By doing this way, an instance of this visualization tool will be spawned alongside the other tools defined for that session. See the example.

Usage example

[program:{{ session }}|my_testing_tool]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-my_testing_tool-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-my_testing_tool-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --rm
    --privileged=true
    --name="session_{{ session }}-my_testing_tool"
    my_testing_tool

[program:{{ session }}|service_viztool_grafana]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service_viztool_grafana-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service_viztool_grafana-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env VIZTOOL_URL={{ service_viztool_grafana_url }}
    --env GF_SERVER_ROOT_URL={{ service_viztool_grafana_url }}
    --rm
    --name="session_{{ session }}-service_viztool_grafana"
    -p {{ service_viztool_grafana_port }}:3000
    service-viztool-grafana

API Description

The viztool-grafana provides on the current virtual host the following services:

Service	Description
`viztool-grafana.init`	Initialize the tool
`viztool-grafana.set_dashboard`	Set the Grafana dashboard
`viztool-grafana.write_data`	Write data to be visualized by Grafana

API Description: Init

This RPC should be called first and it initializes the tool. It is also used to retrieve the URL used to access the Grafana web server.

Request parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`viztool-grafana.init.request`	The routing key used for the request
reply_to	`viztool-grafana.init.reply`	The routing key used for the reply
correlation_id	`str(uuid.uuid4())`	The correlation id of the request
content_type	`application/json`	The content type

Request body:

An empty JSON object (not used).

Reply parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`viztool-grafana.init.reply`	The routing key used for the reply
correlation_id	same as request	The correlation id of the request
content_type	`application/json`	The content type

Reply body:

Key	Type	Value
`ok`	Boolean	`true` if the operation succeded, `false` otherwise
`url`	String	The URL to access Grafana (present only if `ok` is `true`)
`error`	String	A message describing the error (present only if `ok` is `false`)

INIT REQUEST EXAMPLE:

 - - -
ROUTING_KEY: viztool-grafana.init.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": "viztool-grafana.init.reply",
    ...
}
 - - -
BODY {}
 - - -

INIT RESPONSE EXAMPLE

 - - -
ROUTING_KEY: viztool-grafana.init.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": null,
    ...
}
 - - -
BODY {
    "ok": true,
    "url": "https://someHost:somePort/path/to/service_viztool_grafana"
}
 - - -

API Description: Set Dashboard

This RPC is called after initialization and it is used to set the dashboard to be used by Grafana to display the data.

Request parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`viztool-grafana.set_dashboard.request`	The routing key used for the request
reply_to	`viztool-grafana.set_dashboard.reply`	The routing key used for the reply
correlation_id	`str(uuid.uuid4())`	The correlation id of the request
content_type	`application/json`	The content type

Request body:

Key	Type	Value
`config`	Object	The JSON object representing the Grafana dashboard to be used.

See Grafana doc. An easy way to get a JSON dashboard is to manually create and edit one in a local Grafana instance and then export it as JSON.

Reply parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`viztool-grafana.set_dashboard.reply`	The routing key used for the reply
correlation_id	same as request	The correlation id of the request
content_type	`application/json`	The content type

Reply body:

Key	Type	Value
`ok`	Boolean	`true` if the operation succeded, `false` otherwise
`error`	String	A message describing the error (present only if `ok` is `false`)

SET DASHBOARD REQUEST EXAMPLE:

 - - -
ROUTING_KEY: viztool-grafana.set_dashboard.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": "viztool-grafana.set_dashboard.reply",
    ...
}
 - - -
BODY {
  "config": {
      "annotations": {
        "list": [
          {
            "builtIn": 1,
            "datasource": "-- Grafana --",
            "enable": true,
            "hide": true,
            "iconColor": "rgba(0, 211, 255, 1)",
            "name": "Annotations & Alerts",
            "type": "dashboard"
          }
        ]
      },
      "editable": true,
      "gnetId": null,
      "graphTooltip": 0,
      "hideControls": false,
      "id": 1,
      "links": [],
      "refresh": "5s",
      "rows": [
        {
          "collapse": false,
          "height": 318,
          "panels": [
            {
              "aliasColors": {
                "cpu.mean": "#65c5db"
              },
              "bars": true,
              "dashLength": 10,
              "dashes": false,
              "datasource": null,
              "fill": 1,
              "id": 1,
              "legend": {
                "avg": false,
                "current": false,
                "max": false,
                "min": false,
                "show": true,
                "total": false,
                "values": false
              },
              "lines": false,
              "linewidth": 1,
              "links": [],
              "nullPointMode": "null",
              "percentage": false,
              "pointradius": 2,
              "points": true,
              "renderer": "flot",
              "seriesOverrides": [],
              "spaceLength": 10,
              "span": 12,
              "stack": false,
              "steppedLine": false,
              "targets": [
                {
                  "dsType": "influxdb",
                  "groupBy": [
                    {
                      "params": [
                        "$__interval"
                      ],
                      "type": "time"
                    },
                    {
                      "params": [
                        "null"
                      ],
                      "type": "fill"
                    }
                  ],
                  "measurement": "cpu",
                  "orderByTime": "ASC",
                  "policy": "default",
                  "refId": "A",
                  "resultFormat": "time_series",
                  "select": [
                    [
                      {
                        "params": [
                          "value"
                        ],
                        "type": "field"
                      },
                      {
                        "params": [],
                        "type": "mean"
                      }
                    ]
                  ],
                  "tags": []
                }
              ],
              "thresholds": [],
              "timeFrom": null,
              "timeShift": null,
              "title": "Demo Panel",
              "tooltip": {
                "shared": true,
                "sort": 0,
                "value_type": "individual"
              },
              "transparent": false,
              "type": "graph",
              "xaxis": {
                "buckets": null,
                "mode": "time",
                "name": null,
                "show": true,
                "values": []
              },
              "yaxes": [
                {
                  "format": "short",
                  "label": null,
                  "logBase": 1,
                  "max": null,
                  "min": null,
                  "show": true
                },
                {
                  "format": "short",
                  "label": null,
                  "logBase": 1,
                  "max": null,
                  "min": null,
                  "show": true
                }
              ]
            }
          ],
          "repeat": null,
          "repeatIteration": null,
          "repeatRowId": null,
          "showTitle": false,
          "title": "Dashboard Row",
          "titleSize": "h6"
        }
      ],
      "schemaVersion": 14,
      "style": "dark",
      "tags": [],
      "templating": {
        "list": []
      },
      "time": {
        "from": "now-5m",
        "to": "now"
      },
      "timepicker": {
        "nowDelay": "",
        "refresh_intervals": [
          "1s",
          "5s",
          "10s",
          "30s",
          "1m",
          "5m",
          "15m",
          "30m",
          "1h",
          "2h",
          "1d"
        ],
        "time_options": [
          "5m",
          "15m",
          "1h",
          "6h",
          "12h",
          "24h",
          "2d",
          "7d",
          "30d"
        ]
      },
      "timezone": "browser",
      "title": "Demo Dashboard",
      "version": 1
    }
}
 - - -

SET DASHBOARD RESPONSE EXAMPLE

 - - -
ROUTING_KEY: viztool-grafana.set_dashboard.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": null,
    ...
}
 - - -
BODY {
    "ok": true
}
 - - -

API Description: Write Data

This method is used to push data into Grafana to be visualized in the previously set dashboard.

Method parameters:

Parameter	Value	Description
exchange	`amq.topic`	The AMQP exchange
routing_key	`viztool-grafana.write_data`	The routing key used for the request
content_type	`application/json`	The content type

Request body:

The format of the data reflects the Line Protocol specified by InfluxDB (reference). Note that the data you are sending must be compatible with the queries you specify in the dashboard. See Grafana doc.

WRITE DATA EXAMPLE

 - - -
ROUTING_KEY: viztool-grafana.write_data
 - - -
HEADERS: None
 - - -
PROPS: {
    "content_type": "application/json", 
    ...
}
 - - -
BODY {
  "measurement": "cpu",
  "tags": {
    "host": "server01",
    "region": "us-west"
  },
  "fields": {
    "value_1": 0.42,
    "value_2": 42
  }
}

 - - -

Online Event Tool (OET)

The Online Event tool provides a virtual space for collaborative testing. It includes three components that are fully integrated in the GUI frontend, as separate web pages. Services are provided by the GUI backend, through a REST API and a WebSocket API. The Event Administration component enables the creation and modification of virtual events. The Event Selection component displays the events to which the user is registered. Clicking an event takes the user to that event’s Dashboard. The Event Dashboard is the virtual space for an event. It is made of a chat component, enabling participants to discuss results and issues, and multiple widgets displaying statistics of the event.

Composition

The Online Event Tool is a React component library. It is imported as a dependency in the GUI front-end (itself a React web-application). The components of the Online Event Tool are then embedded in their own page. All three of them send XML Http Requests (XHR) to the GUI backend to send and receive data.

Additionally, using SockJS, the Event Dashboard component opens a WebSocket connection to the GUI backend to receive live status updates. These updates come from monitoring users’ WebSocket connections, as well as session modifications in the GUI database. RethinkDB, the database engine, provides update feeds out of the box. The GUI back-end simply forwards the updates to the relevant WebSocket clients.

Front-end Integration

Two views have been added to the GUI Front-end. The EventAdministrationView imports and renders the EventAdministration react component from the Online Event Tool library. The EventDashboardView imports both the Dashboard and EventSelector components from the library, and renders one or the other if an event id is provided or not, respectively.

The library itself is hosted on npm. To update it, it has to be published (and tagged, please!) after being built (yarn build and yarn publish) and the GUI Front-end has to upgrade the dependency.

Architecture

As previously mentioned, the library exports three React Components. Each of these is dependent on multiple sub-components. Among these, three are third-party: react-copy, react-date-range and react-select.

Peer dependencies

  {
    "axios": ">=0.18.0",
    "react": ">=15.0.1",
    "react-addons-css-transition-group": ">=15.5.2",
    "react-copy": ">=0.2.1",
    "react-dom": ">=15.0.1",
    "sockjs-client": ">=1.1.4"
  }

The project is also dependent on non-component third-party libraries. Some of these dependencies of the Online Event Tool Library are already present in the GUI Front-end, and as such have been declared as peer-dependencies, to avoid including them multiple times. The others are specific to the Online Event Tool Library.

The charts that are part of the dashboard make use of d3, a powerful library which provides geometric math and visualization helpers. We do not use d3’s svg construction helpers (e.g. axes), because they would interfere with React’s handling of the dom.

The styles are not defined in css files. Instead, styled-components is used. This library implements “css-in-js”. The styles are generated from javascript code that creates React components. These components can then be used by other components to render the dom. They are defined in modules suffixed with “Styles”, for easy navigation. Each component has its own “Styles” module from which it imports its styled components to render.

Events API

Event services are accessed through a REST API. The following endpoints are available:

GET finterop/events

Response example for GET finterop/events

  {
    "events": [
      {
        "id": "ed6ff550-c208-4c2a-b822-3ed502de218b",
        "name": "Example event",
        "description": "description of the example event",
        "registered": true
      }
    ]
  }

Returns the list of events the current user is registered to. Includes the id, name and description of each event.

GET finterop/events?all

Response example for GET finterop/events?all

  {
    "events": [
      {
        "id": "ed6ff550-c208-4c2a-b822-3ed502de218b",
        "participants": [
            "urn:publicid:IDN+localhost+user+user1zkdc",
            "urn:publicid:IDN+localhost+user+user2ozvc"
        ],
        "name": "Example archived event",
        "description": "Description of the event",
        "period": {
            "end": "2018-07-19",
            "start": "2018-07-16"
        },
        "archived": true
      }
    ]
  }

Returns the full list of events, including archived events, along with all their attributes. Restricted to administrators.

GET finterop/events/[uuid]

Response example for GET finterop/events/[uuid] unregistered

  {
    "events": [
      {
        "id": "ed6ff550-c208-4c2a-b822-3ed502de218b",
        "name": "Example event",
        "description": "description of the example event"
      }
    ]
  }

Response example for GET finterop/events/[uuid] registered

  {
    "event": {
      "id": "ed6ff550-c208-4c2a-b822-3ed502de218b",
      "name": "Event participant test",
      "description": "Description of event",
      "participants": [
        {
          "id": "urn:publicid:IDN+localhost+user+user3aodh",
          "first_name": "Steve",
          "last_name": "Jackson",
          "shortname": "user3aodh",
          "sessions": [
            {
              "id": "5113d721-c4b7-4eba-86f9-bce339a7e287",
              "users": [
                  "user4qccc",
                  "myslice",
                  "user3aodh"
              ],
              "status": "open",
              "resources": [
                {
                  "id": "5b3b8b5b-530e-4078-ae3c-def94918cebf",
                  "owner_id": "urn:publicid:IDN+localhost+user+user4qccc",
                  "software": {
                    "testing_tool": [
                      {
                        "name": "6top Protocol (6P)",
                        "version": "Unknown"
                      }
                    ],
                    "name": "6lowpan",
                    "version": [
                      {
                        "version": "4.3"
                      }
                    ],
                    "testing_protocol": {
                      "name": "6top Protocol (6P)",
                      "version": "Unknown"
                    }
                  },
                  "resource_id": "urn:publicid:IDN+localhost:user4qccc+node+6lowpan"
                }
              ]
            }
          ]
        },
        {  
          "id": "urn:publicid:IDN+localhost+user+user4qccc",
          "first_name": "Ada",
          "last_name": "Lovelace",
          "shortname": "user4qccc",
          "sessions": []
        },
      ],
      "sessionsByTestSuite": [
        {
          "reduction": 1,
          "group": [
            "conformance",
            "http://orchestrator.dev.f-interop.eu:8181/tests/f-interop/conformance-coap"
          ]
        },
        {
          "reduction": 3,
          "group": [
            "interoperability",
            "http://orchestrator.dev.f-interop.eu:8181/tests/f-interop/interoperability-6lowpan-user-to-user"
          ]
        }
      ],    
      "sessionStats": [
        {
          "reduction": 1,
          "group": "closed"
        },
        {
          "reduction": 1,
          "group": "open"
        },
        {
          "reduction": 6,
          "group": "terminated"
        }
      ],
      "chats": [
        {
          "id": "04ef4401-1333-4b79-a3c1-bb24185e3be9",
          "event": "ed6ff550-c208-4c2a-b822-3ed502de218b",
          "users": [
            "urn:publicid:IDN+localhost+user+admincsfs",
            "urn:publicid:IDN+localhost+user+user3aodh"
          ],
          "messages": [
            {
              "text": "hi?",
              "timestamp": "2018-06-01T16:43:22.778+09:00",
              "author": "urn:publicid:IDN+localhost+user+admincsfs",
              "to": "urn:publicid:IDN+localhost+user+user3aodh"
            }
          ]
        },  
        {
          "id": "ed6ff550-c208-4c2a-b822-3ed502de218b",
          "event": "ed6ff550-c208-4c2a-b822-3ed502de218b",
          "messages": [
            {
              "text": "test event chat",
              "timestamp": "2018-07-10T06:08:50.978+09:00",
              "author": "urn:publicid:IDN+localhost+user+user2ozvc"
            }
          ]
        }
      ]
    }
  }

Returns the requested event. If the user is not registered to it, only returns the id, name and description. Otherwise, returns the full event, to which are added the list of participants, existing chats, session count by status, and session count by test suite.

POST finterop/events

Example of data to send in POST finterop/events

  {
    "name": "new event name",
    "description": "description of new event",
    "period": {
      "start": "2018-06-23",
      "end": "2018-07-18"
    },
    "protocols": ["CoAP", "CoAP_CORE"],
    "testSuites": [7]
  }

Response example for POST finterop/events

  {
    "id": "ed6ff550-c208-4c2a-b822-3ed502de218b"
  }

Creates an event. The following fields are nullable: period, protocols, testSuites. Returns the event id. Restricted to administrators. This also creates the event chat (the common channel for that event).

PUT finterop/events/[uuid]

Example of data to send in PUT finterop/events/uuid

  {
    "name": "event name",
    "description": "description of event",
    "participants": ["urn:publicid:IDN+localhost+user+user2ozvc"],
    "period": {
      "start": "2018-06-23",
      "end": "2018-07-18"
    },
    "protocols": ["CoAP", "CoAP_CORE"],
    "testSuites": [7]
  }

Response example for PUT finterop/events/[uuid]

  {
    "success": true
  }

If the user is an administrator, the event is updated according to the new data passed. Otherwise, the user is registered to the event by being added to the participants. If no body is sent, an administrator will also simply be registered to the event.

Chats API

GET finterop/events/[uuid]/chats?partner=[urn]

Response example for GET finterop/events/[uuid]/chats?partner=[urn]

  {
    "chat": {
      "id": "04ef4401-1333-4b79-a3c1-bb24185e3be9",
      "event": "ed6ff550-c208-4c2a-b822-3ed502de218b",
      "users": [
        "urn:publicid:IDN+localhost+user+user3aodh",
        "urn:publicid:IDN+localhost+user+user5fzfi"
      ],
      "messages": [
        {
          "text": "hi",
          "to": "urn:publicid:IDN+localhost+user+user3aodh",
          "author": "urn:publicid:IDN+localhost+user+user5fzfi",
          "timestamp": "2018-06-01T16:43:22.778+09:00"
        }
      ]
    }
  }

Find and return the chat for the specified event between the user and the specified partner. If it does not exist, it is first created in the DB.

GET finterop/events/[uuid]/chats/[uuid]

Returns the chat by its id. The user must be a part of the chat. Same response as GET finterop/events/[uuid]/chats?partner=[urn].

WebSocket API

Because web sockets only accept plain text to be sent, we send serialized JSON.

General WebSocket message format

  {
    "command": "commandName",
    "data": "the format of the data member is command-dependent"
  }

Accepted messages

Authenticate

Command: “authenticate”

  {
    "token": "eyJxcTIyrGFIxcTIyrufp1ntuDcVcwgqjG3_XOY [...]",
    "event_id": "ed6ff550-c208-4c2a-b822-3ed502de218b"
  }

Because the cookie cannot be sent through the websocket, we have to authenticate the user again. This command expects the token obtained from the usertoken REST endpoint, as well as the event uuid. If the token is legitimate, the user is added to the list of the event’s connected clients, and all users connected to that event are notified.

Send Text

Command: “sendText”

  {
    "text": "hello",
    "to": "urn:publicid:IDN+localhost+user+user3aodh",
    "event": "ed6ff550-c208-4c2a-b822-3ed502de218b",
    "timestamp": "2018-06-01T16:43:22.778+09:00"
  }

This sends a text message to a user. It is bound to an event. If no recipient is specified (to parameter is absent or empty), the text message is added to the event chat. The timestamp is expected to be an ISO date with time: YYYY-MM-DDTHH:mm:ss.SSSZ

Emitted messages

Connect

Command: “connect”

  [
    "urn:publicid:IDN+localhost+user+user3aodh",
    "urn:publicid:IDN+localhost+user+user4zghf"
  ]

Emitted when a user connects to the event dashboard. Also emitted to a user after authentication, with the list of all connected users.

Disconnect

Command: “disconnect”

  "urn:publicid:IDN+localhost+user+user3aodh"

Emitted when a user disconnects from the event dashboard.

Status Update

Command: “statusUpdate”

  {
    "old_val": {
      "event": "87f4f22f-67ae-48fa-9f96-77a8841f5fc1",
      "id": "679a6b9e-9a19-48bf-875d-857537967fd9",
      "resources": [
        {
          "id": "941f7607-9365-489a-b424-505761cea517",
          "software": {
            "name": "device2",
            "testing_protocol": {
              "name": "CoAP_CORE"
            },
            "testing_tool": [
              {
                "name": "CoAP_CORE"
              }
            ]
          }
        }
      ],
      "status": "open",
      "testSuite": "http://orchestrator.dev.f-interop.eu:8181/tests/f-interop/interoperability-coap-single-user",
      "testSuiteType": "interoperability",
      "users": [
        "admincsfs",
        "myslice"
      ]
    },
    "new_val": {
      "event": "87f4f22f-67ae-48fa-9f96-77a8841f5fc1",
      "id": "679a6b9e-9a19-48bf-875d-857537967fd9",
      "resources": [
        {
          "id": "941f7607-9365-489a-b424-505761cea517",
          "software": {
            "name": "device2",
            "testing_protocol": {
              "name": "CoAP_CORE"
            },
            "testing_tool": [
              {
                "name": "CoAP_CORE"
              }
            ]
          }
        }
      ],
      "status": "closed",
      "testSuite": "http://orchestrator.dev.f-interop.eu:8181/tests/f-interop/interoperability-coap-single-user",
      "testSuiteType": "interoperability",
      "users": [
        "admincsfs",
        "myslice"
      ]
    }
  }

Emitted when a session linked to the event changes status, or a new one is created. If old_val is null, it is a new session.

Users Update

Command: “usersUpdate”

  [
    {
      "id": "urn:publicid:IDN+localhost+user+user1zkdc",
      "shortname": "user1zkdc",
      "first_name": "Michael",
      "last_name": "Terri"
    }
  ]

Emitted when a user registers to the event.

Text

Command: “text”

  {
    "text": "hello",
    "to": "urn:publicid:IDN+localhost+user+user3aodh",
    "author": "urn:publicid:IDN+localhost+user+user1zkdc",
    "timestamp": "2018-06-01T16:43:22.778+09:00"
  }

Emitted to a user when another user sends a text message to him or to the event chat. If no to is defined, the message is destined to the general event chat.