NAV
shell json python

Introduction

In this doc we present the important information regarding the architecture, the core components and the description of the messages exchanged between those core components.

Must-Reads

Developers should read architecture public deliverable before getting started here:

D1.3 - Initial architecture design

I would be extremely useful you also take a look at the discussions and announcements at:

F-Interop google-group forum

“At the early stage [of an API defintion], agility trumps completeness”.

Joshua J. Bloch (Sun Microsystems and Google)

Architecture

The F-Interop architecture is composed of different components exchanging messages through an “Event Bus”.

F-Interop’s architecture follows the event bus design pattern:

All communication is done through this mechanism, including control messages, network data packets, logs, etc.

We use the RabbitMQ as the underlying message-passing mechanism. It acts as a secure message broker between all the components through encrypted channels.

Screenshot

Scope of the documentation

This documentation describes the core API for building testing tools and other components in the F-Interop architecture. The interfaces of the core API are outlined in red in the following diagram:

Screenshot

If you are a testing tool implementor which is re-using some of the reference implementations and you want to read their documentation please refer to:

coap_testing_tool documentation

6tisch_testing_tool documentation

performance_testing_tool documentation

privacy_testing_tool documentation

Event Bus

(updated for API v2.0)

Introduction

Components of the system communicates with each other using messages. We need message delivery to be:

This architecture enables many independent components to talk to each other and avoid a monolith that would be difficult to manage as the number of components increase.

Screenshot

open image

We use the RabbitMQ as the underlying message-passing mechanism.

It acts as a secure message broker between all the components through encrypted channels.

Important facts about the event bus:

Messaging pattern

For producing/consuming events we use the publish/subscribe pattern which implementation is based on topic exchanges of AMQP (more about topic exchanges)

Topic exchanges route messages to one or many queues based on matching between a message routing key and the pattern that was used to bind a queue to an exchange.

Topics/routing keys

How the topics/routing keys are composed is standardized by F-Interop, by using certain conventions.

As you may know, messages are sent using topic exchanges, and when using this feature in amqp routing keys / topics are formed as a list of words delimited by dots:

quick.orange.rabbit

more about topics

Namespaces: For categorizing certain type of messages which share similar functionalities we defined some namespaces. These describe a category of events and services which are similar in some way.

A non exhaustive list of namespaces used in the F-Interop:

(used for resources discovery, management & orchestration) - orchestration.# - resource_repository.# - session.# - results.# - ui.#

(used by testing tools)

Data plane (fromAgent.# , toAgent.# topics) messages contains the network data (raw) exchanged between the implementations under test (IUT). This is used for sending and forwarding packets between IUTs.

Control plane (orchestration.#, results.# , ui.#) messages relate to the management of an ongoing test session: User interface messages one of the most important ones handled by the control plane of the API.

Conventions

Conventions on routing keys/topics

By convention in F-Interop the first keyword of every routing key will be the namespace described before, the rest of the routing key terms will be identify the type of the message.

Some examples:

results.put.request, orchestrator.sesssion.get.request, ui.user.john.display

Conventions on request/reply interactions

In some situations components will need to use a request/reply type of interaction between instead of just pushing push & forget a message into the bus. But note that the event bus follows the pub/subs which doesnt make it ideal for this purpose, nevertheless F-Interop proposes the following convention for sending a request and receiving a reply over the pub/sub pattern.

a component making a request will publish a message into:

(->) routing_key = <namespace>.<typeOfMessageOrAction>.request

with ‘reply_to’ amqp property set on:

reply_to = <namespace>.<typeOfMessageOrAction>.reply

the second component replying to the message will publish into (reply_to routing key):

(<-) routing_key = <namespace>.<typeOfMessageOrAction>.reply

Also, for solving the problem where two simultaneous requests are sent to the same routing key each request and reply must contain correlation_id amqp property. This will allow components to correlate uniquely a reply to a request.

AMQP messages library

A comprehensive list of messages can be found in here: messages library

AMQP message attributes/properties

These are the attributes / properties typically used in the F-Interop event bus:

Field Type Description
routing_key string Topic of the message being sent.
reply_to string Topic used for the reply. Every request message needs to include this.
msg_id number A globally unique ID for this message.
correlation_id string Whenever the message is a request or a reply this field MUST be included.
timestamp number the Posix/Unix time in second when this message was generated by its publisher.
body JSON payload of the message in JSON format and including a _type.
    -> add an example here

Session Orchestrator (SO)

The orchestrator main roles are:

Session deployment sequence

The session deployment phase includes interactions between GUI, SO, TT and eventually other requested resources if the testing suite requires them.

The steps during this phase are:

  1. GUI -> SO : http PUT sessions/session_id {configuration= [..]}

    ( SO creates vhost )

    ( SO adds users to vhost )

    ( SO creates session)

  2. GUI -> SO : http PATCH sessions/session_id {status:started}

    ( SO spawns TT )

    ( SO spawns other requested resources )

  3. TT -> GUI: amqp event ui.core.session.confirguration.get.resquest

    (requests session anad testing tool configuration)

  4. GUI -> TT: amqp event ui.core.session.confirguration.get.reply {configuration= [..]}

    (this returns information about the session, and about user input configuration for testing tool)

After this point the session is ready to start. It’s up to the testing tool developer to implement the test logic.

FAQ:

SO: Session Orchestrator

TT: Testing Tool (any type)

AMQP interface

(WIP)

HTTP interface

Create a new session

PUT http://orchestrator.f-interop.eu/sessions/<session_id>

{
   "testing_tools": "f-interop/interoperability-coap",
    "users": [
        "u1",
        "f-interop"
    ],
    "configuration": {
        "testsuite.additional_session_resource": "automated_iut-coap_client-californium",
        "testsuite.testcases": [
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_01",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_03",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_02",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_04",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_05",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_06"
        ]
    }
}

import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>"

data = {
    "users": [
        "a",
        "b",
        "c"
    ],
    "testing_tools": "http://orchestrator.f-interop.eu/tests/f-interop/interoperability-coap",
    "configuration": {
        "testsuite.additional_session_resource": "automated_iut-coap_client-californium",
        "testsuite.testcases": [
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_01",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_03",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_02",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_04",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_05",
            "http://doc.f-interop.eu/tests/TD_COAP_CORE_06"
        ]
    }
}

register_response = requests.put(url,
                                 json=data,
                                 auth=("username", "password"))

Get a session

GET http://orchestrator.f-interop.eu/sessions/<session_id>

import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>"

register_response = requests.get(url,
                                 auth=("username", "password"))

Start a session

PATCH http://orchestrator.f-interop.eu/sessions/<session_id>

{
  "status": "started"
}
import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>"

register_response = requests.patch(url,
                                   json={"status": "started"},
                                   auth=("username", "password"))

Stop a session

PATCH http://orchestrator.f-interop.eu/sessions/<session_id>

{
  "status": "stopped"
}
import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>"

register_response = requests.patch(url,
                                   json={"status": "stopped"},
                                   auth=("username", "password"))

Delete a session

DELETE http://orchestrator.f-interop.eu/sessions/<session_id>

import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>"

register_response = requests.delete(url,
                                    auth=("username", "password"))

Create a new user

PUT http://orchestrator.f-interop.eu/users/<user_id>

import requests

url = "http://orchestrator.f-interop.eu/users/<user_id>"
register_response = requests.put(url,
                                 json={"password": "foo"})
{
  "password": "foo"
}

Get information about a user

GET http://orchestrator.f-interop.eu/users/<user_id>

import requests

url = "http://orchestrator.f-interop.eu/users/<user_id>"
register_response = requests.get(url)

Delete a user

DELETE http://orchestrator.f-interop.eu/users/<user_id>

import requests

url = "http://orchestrator.f-interop.eu/users/<user_id>"
register_response = requests.delete(url,
                                    auth=("username", "password"))

Post an event on the event bus of a session

HTTP is not designed to handle event flow such as AMQP is. Sending event is supported but the reply will only tell the sender whether or not the event is arrived inside the broker but it won’t contains anything about the effect.

Publish a message to a given exchange. You will need a body looking something like:

{ "properties": {}, "routing_key": "my key", "payload": "my body", "payload_encoding": "string" }

All keys are mandatory. The payload_encoding key should be either “string” (in which case the payload will be taken to be the UTF-8 encoding of the payload field) or “base64” (in which case the payload field is taken to be base64 encoded). If the message is published successfully, the response will look like:

{ "routed": true }

Routed will be true if the message was sent to at least one queue. Please note that the HTTP API is not ideal for high performance publishing; the need to create a new TCP connection for each message published can limit message throughput compared to AMQP or other protocols using long-lived connections.

POST http://orchestrator.f-interop.eu/sessions/<session_id>/publish

{
  "properties": {},
  "routing_key": "my key",
  "payload": "my body",
  "payload_encoding": "string"
}
import requests

url = "http://orchestrator.f-interop.eu/sessions/<session_id>/publish"

register_response = requests.post(url,
                                  json={"password": "foo"},
                                  auth=("username", "password"))

Need to integrate a new testing tool to orchestrator?

read the F-Interop SDK documentation

for any questions, please use :

https://groups.google.com/forum/#!forum/technical-support-f-interop-industrial-experiments

Testing tools (TT)

This section of the documentation describes the testing tool’s minimal requirements.

General requirements on Testing Tools (TT):

(The testing tool…)

Extra requirements on interop and conformance TTs:

(The testing tool…)

Interfaces

The abstraction used for describing the interfaces in this document (the CORE API) are indicated in this diagram:

Screenshot

More details on thos API calls can be found on the previously presented Event Bus CORE API

When contributing with a testing tool (TT) to F-Interop you should have in mind that for us your testing tool is a black box. F-Interop won’t send any requests to the TT, it’s the TT who must use F-Interop services (GUI services, RR, RS, etc..) for executing the complete test session.

Reference testing tools are available for re-usablity, some links to the source code is provided below. TT developers are free to start building their tools from scratch using the CORE API or extend these reference implementations which already make use of the CORE API

Snippets

Decode from base64 any uploaded file to GUI

One of the typical steps of a test is asking the user to upload a pcap file so the testing tool can check the correctness of the packet format in the file. This step is done through GUI API for file (upload). The file content will be uploaded through the GUI to Testing tool. For a testing tool developer, she needs to decode the file content using base64 to a pcap file.

An example is showing on the right side:

import base64

# ask for uploading pcap files (the function could be difference for different testing tool)
uploadFile = FileUploadQuery(question=question, tag=step_id).ask()

for item in uploadFile:
    # looking for the file content
    if 'filename' in item:
        capture_path = get_from_environment("PCAP_PATH", "/capture.pcapng")
        with open(capture_path,'wb') as f:
            # item['value'] is the file content
            f.write(base64.b64decode(item['value']))

Testing Tool source code, READMEs and tutorials:

The complete list of F-Interop public git repositories at:

Explore F-Interop’s gitlab repos

CoAP Testing Tools tutorial

6LoWPAN Testing Tools tutorial

CoAP/6LoWPAN/oneM2M Testing Tools API doc

CoAP/6LoWPAN/oneM2M Testing Tools README doc

CoAP/6LoWPAN/oneM2M Testing Tools git repo

(WIP)

6TiSCH README Testing Tool doc

Privacy Testing Tool doc

Performance Testing Tool doc

Agent

(updated for API v2.0)

Introduction

The “agent” is a program a user downloads from the F-Interop platform, and which allows him/her to connect the IUT to the F-Interop’s testing tools.

Communication between the agent and the server is authenticated and secure. Through the agent, the F-Interop’s testing tools can (remotely) interact with the IUTs, for example by changing its configuration or injecting packets in the link/interface the IUT is listening to. Similarly, the agent reports to F-Interop the data packets sent from the IUT.

Routing key / topics basics

There can be several agents during a session. Therefore, there is a need to address each agent individually. The exact process of downloading and launching an agent is defined by the testing tools’ developed, this information must be provided to the user using the GUI.

the “name spaces” for the routing keys reserved for agent use are:

toAgent.# fromAgent.#

An agent listens to several routing keys/topics. A typical agent’s routing key / topic used for forwarding the IP packets through the a previously created IP tunnels looks like this:

toAgent.coap_client_agent.packet.raw.tun.

fromAgent.coap_server_agent.packet.raw.tun.

etc..

NOTE: The routing of the data messages between agents in not handled by the agents themselves - they just publish/listen to events under those topics which will end up being stored in some queue in RMQ broker. It’s up to the testing tool later to route those messages to other agents and for establishing a communication between IUTs. Testing Tool can also implement message dropping mechanisms for testing certain network features like re-transmitting, etc.

This way, components of the testing tools can publish messages to this topic, then the agent will receive the message and e.g. inject it in the tun interface.

Besides the packet injection, there are other types of actions we can trigger by publishing to these routing keys / topics. Like network interfaces remote control (from testing tool to the PC where the agent is running)

etc..

Events and API

F-Interop doesnt standardised messages or routing keys for these components given that these interfaces are test suite imeplementation dependent. For examples you can check out section Source-code-implementations

Source code implementations

Some examples on how to build your own agent component:

network-tunneling agent’s gitlab repo

Graphical User Interface

( version of the API v.2.0 )

BASICS

The F-Interop GUI is based on MySlice v2 software.

It provides the necessary registration, validation and authentication of users.

Authentication to the GUI relies on login/password, but authentication to the testbeds requires ssh-keys and X509 certificates.

For the specific F-Interop needs, new modules have been developed in order to communicate with the Message Bus.

The following architecture has been implemented:

Screenshot

SERVICES PROVIDED

MESSAGES FLOW

ui.user.* messages

Screenshot

ui.user.* routing_key is dedicated to communication with the user interface

ui.user.all.*

Any component on the bus can direct messages to all users involved in the session.

ui.user.<user_id>.*

Any component on the bus can direct messages to a specific user in the session.

ui.user.<user_id>.request

PROPS: correlation_id is sent by the testing tool on the ui.user.<user_id>.request, the answer uses the same correlation_id in the ui.user.<user_id>.reply

The GUI will use correlation_id to identify the actions that have already been triggered and disable the submit button accordingly

BODY of the ui.user.<user_id>.request message is used as a form, which contains a list of input fields.

fields: list of input fields, several fields of different types can be combined.

USER INPUT (requests)

text

Text is used to request to the user some text input. Input fields must must have the same name. The value of each input field must be defined.
ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "input_name",
            type: text,
        }
    ]
 }

radio

Radio is used to offer the possibility to select one choice among several. Input fields must must have the same name. The value of each input field must be defined.
ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            label: "choice number 1”,
            type: radio,
            value: 1
        },
        {
            name: "example",
            label: "choice number 2”,
            type: radio,
            value: 2
        }
    ]
 }

checkbox

Checkbox is used to offer the possibility to select several choices. The value of each input must be defined
ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example1",
            label: "choice number 1”,
            type: checkbox,
            value: 1
        },
        {
            name: "example2",
            label: "choice number 2”,
            type: checkbox,
            value: 2
        }
    ]
}

select

Select is used to offer the user to select one option, a default choice can be defined using value = x
ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    title: "Select example",
    fields: [
        {
            name: "example",
            type: "select",
            options: [
                {label:"choice 1”, value:1},
                {label:"choice 2”, value:2},
                {label:"choice 3”, value:3},
            ],
            value: 1
        }
    ]
 }

file (upload)

file allow users to upload a file to the testing tool
ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            type: "file"
        }
    ]
 }

button

Value must be specified. Clicking a button triggers the action to send the form as a message to the bus. Several buttons with different values can be proposed in the same form.
ROUTING_KEY: ui.user.<user_id>.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "button 1",
            type: "button",
            value: 1
        },
        {
            name: "button 2",
            type: "button",
            value: 2
        }
    ]
}

title: will be placed in the header of the form

label: each field can have a label to describe it

ROUTING_KEY: ui.user.<user_id>.request
PROPS:
        correlation_id: correlation_id
BODY:    
{
    title: "My Action",
    fields: [
        {
            name: "input_name",
            type: "text",
            label:"demo input”,
            value: "default value",
        }
    ]
}

USER INPUT (replies)

ui.user.<user_id>.reply

ui.user.user_id.reply is sent with the same correlation_id as the one received in the corresponding ui.user.user_id.request, enabling the testingtool to identify what question has been answered. fields: list of the input fields using the same names as ui.user.user_id.request to map the values.
ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "input_name",
            value: "user’s answer",
        }
    ]
}

text reply

see json example
ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "input_name",
            value: "user’s answer",
        }
    ]
}

radio reply

see json example
ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            value: 1
        }
    ]
}

checkbox reply

see json example
ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            value: 1
        }
    ]
}

select reply

see json example
ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            value: 2
        }
    ]
}

file upload reply

see json example
ROUTING_KEY: ui.user.<user_id>.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "example",
            value: base64_encoded <file>
        }
    ]
}

button reply

see json example
ROUTING_KEY: ui.user.<user_id>.reply
BODY:    
{
    fields: [
        {
            name: "button 1",
            value: 1
        }
    ]
}

USER DISPLAY

ui.user.user_id.display is sent by the testingtool to the ui.user.user_id. fields: list of fields, allowing to display several elements within the same message (text, link, file, img, iframe)

paragraph (<p>)

markdown format is accepted https://learn.getgrav.org/content/markdown
ROUTING_KEY: ui.user.user_id.display
PROPS:
        correlation_id: correlation_id
BODY:    
{
    fields: [
        {
            name: "my_message",
            type: "p",
            value: "Hello World!"
        }
    ]
}

image

see json example
ROUTING_KEY: ui.user.user_id.display
PROPS:
   correlation_id: correlation_id
BODY:    
{
    title: "Example image"
   fields: [
        {
            name: "my_image.png",
            type: "img",
            value: <url> | base_64_encoded <file>
       },
    ]
}

html

see json example
ROUTING_KEY: ui.user.user_id.display
PROPS:
   correlation_id: correlation_id
BODY:    
{
    title: "Example html"
   fields: [
        {
            name: "my_html",
            type: "html",
            value: "<div>Hello World</div>"
       },
    ]
}
see json example
ROUTING_KEY: ui.user.user_id.display
PROPS:
   correlation_id: correlation_id
BODY:    
{
    title: "Example link"
   fields: [
        {
            name: "my_link",
            type: "a",
            value: <url>
        },
    ]
}

data (download)

Value must be base64 encoded url the format of the file is based on the extension of its name
ROUTING_KEY: ui.user.user_id.display
PROPS:
   correlation_id: correlation_id
BODY:    
{
    title: "Download your report for testcase 1"
   fields: [
        {
            name: "report_testcase_1.pcap",
            type: "data",
            value: base64_encoded <file>
       },
    ]
}

iframe

see json example
ROUTING_KEY: ui.user.user_id.display
PROPS:
   correlation_id: correlation_id
BODY:    
{
    title: "Example iframe"
   fields: [
        {
            name: "my_iframe",
            type: "iframe",
            value: <url>
       },
    ]
}

LOG LEVELS

Level is optional, by default it is considered as info. Level can be applied to any message type (ui.user.user_id.request, ui.user.user_id.display).
ROUTING_KEY: ui.user.user_id.display
PROPS:
    correlation_id: correlation_id
BODY:    
{
    level: debug | info | highlighted | warning | error | critical,
    fields: [
        {
            name: "my_message",
            type: "p",
            value: "this is a log message"
        }
    ]
}

The log level is meant to inform about the execution of the session, not about the outcome (verdict) of the tests themselves.

debug Only displayed in the development environment.
info Default log level, if not specified is considered as info.
highlighted Highlighted to the user, ex: download pcap file, test verdict
warning Highlighted to the user, ex: your session will expire in 5 minutes!
error Propose user to reset
critical Terminate the session

TAGS (optional filters)

Tags allow to categorize messages and to apply filters on the messages displayed to the user
ROUTING_KEY: ui.user.user_id.request
PROPS:
    correlation_id: correlation_id
BODY:    
{
    tags: {testcase_id: "TD_COAP_CORE_03_v01"}
    fields: [
        {
            name: "input_name",
            type: "text",
        }
    ]
 }
ROUTING_KEY: ui.user.user_id.reply
PROPS:
    correlation_id: correlation_id
BODY:    
{
    tags: {testcase_id: "TD_COAP_CORE_03_v01"}
    fields: [
        {
            name: "input_name",
            type: "text",
        }
    ]
 }
ROUTING_KEY: ui.user.user_id.display
PROPS:     
    correlation_id: correlation_id
BODY:    
{
    tags: {testcase_id: "TD_COAP_CORE_03_v01"}
    fields: [
        {
            name: "input_name",
            value: "user’s answer"
        }
    ]
}

Results Store (RS)

(updated for API v2.0)

The Results Store (RS) is a core F-Interop service that allow to store and retrieve results or intermediate results generated from Testing Tools (TT).

The Results Store repository can be found here.

API Description

The RS uses MongoDB as database. The results will be stored as Binary JSON (BSON) by following the format below:

Key Type Value
resources Array of Objects The resources involved in the session.
Each object has the keys resource_id and resource_version
owners Array of Strings The ids of the owners of the resources
session_id String The id of the session
testing_tool_id String The id of the testing tool that produced this result
timestamp Date The time this result was inserted in the DB (UTC datetime)
type String The type of the result.
It can be intermediate or final
data Object The data produced by the testing tool as a result.
There is no schema for this object

Here an example of a result:

{
    "resources": [
        {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
        {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
    ],
    "owners": [
        "user-01",
        "user-02"
    ],
    "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
    "testing_tool_id": "testing-tool-01-id",
    "timestamp": 1520437111,
    "type": "final",
    "data": {
        "some_data_produced_by_tt": 42
    }
}

The RS provides on virtual host / the following services:

Service Description
results_store.insert_result Insert a result in the Results Store
results_store.get_result Query the Results Store for results
results_store.delete_result Delete one or more results

Insert Result

Request parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key results_store.insert_result.request The routing key used for the request
reply_to results_store.insert_result.reply The routing key used for the reply
correlation_id str(uuid.uuid4()) The correlation id of the request
content_type application/json The content type

Request body:

See database format described on top.

Reply parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key results_store.insert_result.reply The routing key used for the reply
correlation_id same as request The correlation id of the request
content_type application/json The content type

Reply body:

Key Type Value
ok Boolean true if the result was inserted correctly, false otherwise
error String A message describing the error (present only if ok is false)

Insert Result Request example:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "resources": [
        {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
        {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
    ],
    "owners": [
        "user-01",
        "user-02"
    ],
    "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
    "testing_tool_id": "testing-tool-01-id",
    "timestamp": 1520437111,
    "type": "final",
    "data": {
        "some_data_produced_by_tt": 42
    }
}

Insert Result Reply example:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Get Result

Request parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key results_store.get_result.request The routing key used for the request
reply_to results_store.get_result.reply The routing key used for the reply
correlation_id str(uuid.uuid4()) The correlation id of the request
content_type application/json The content type

Request body:

The body must contain a query matching the MongoDB query language. See examples below.

Reply parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key results_store.get_result.reply The routing key used for the reply
correlation_id same as request The correlation id of the request
content_type application/json The content type

Reply body:

Key Type Value
ok Boolean true if the query was executed correctly,
false otherwise
results Array of Objects The result(s) retrieved by the query.
The schema of one result is the one described
at the top but with one additional key _id,
which is added by MongoDB
error String A message describing the error
(present only if ok is false)

Examples

The following examples assume that the RS contain the three results above.
The two first results are produced in the same session and there are two
involved resources owned by two different users. The third one is produced
in another session and there is one resource owned by one user. The
resource is a new version of the one present in the previous results.
Note that the `_id` key is added by MongoDB as unique reference to that
object.

{
    "_id": {"$oid": "5aa7d1b8dbbb46608189119a"},
    "resources": [
        {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
        {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
    ],
    "owners": [
        "user-01",
        "user-02"
    ],
    "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
    "timestamp": 1520437111,
    "testing_tool_id": "testing-tool-01-id",
    "type": "intermediate",
    "data": {
        "some_data_produced_by_tt": 42
    }
}

{
    "_id": {"$oid": "5aa7d1b8dbbb46608189119b"},
    "resources": [
        {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
        {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
    ],
    "owners": [
        "user-01",
        "user-02"
    ],
    "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
    "timestamp": 1520437120,
    "testing_tool_id": "testing-tool-01-id",
    "type": "final",
    "data": {
        "some_data_produced_by_tt": {
            "some_data": 3.14,
            "other_data": "some_string"
        }
    }
}

{
    "_id": {"$oid": "5aa7d1b8dbbb46608189119c"},
    "resources": [
        {"resource_id": "user-01-resource-id", "resource_version": "1.1"}
    ],
    "owners": [
        "user-01"
    ],
    "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999",
    "timestamp": 1520437199,
    "testing_tool_id": "testing-tool-id-654321",
    "type": "final",
    "data": {
        "some_data_produced_by_tt": 1234567890
    }
}

Example 1: Get results of all resources for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01"
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true,
    "results": [
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119a"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
                {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
            ],
            "owners": [
                "user-01",
                "user-02"
            ],
            "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
            "timestamp": 1520437111,
            "testing_tool_id": "testing-tool-01-id",
            "type": "intermediate",
            "data": {
                "some_data_produced_by_tt": 42
            }
        },
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119b"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
                {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
            ],
            "owners": [
                "user-01",
                "user-02"
            ],
            "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
            "timestamp": 1520437120,
            "testing_tool_id": "testing-tool-01-id",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": {
                    "some_data": 3.14,
                    "other_data": "some_string"
                }
            }
        },
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119c"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.1"}
            ],
            "owners": [
                "user-01"
            ],
            "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999",
            "timestamp": 1520437199,
            "testing_tool_id": "testing-tool-id-654321",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": 1234567890
            }
        }
    ]
}

Example 2: Get results of specific resource for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "resources": {"$elemMatch": {"resource_id": "user-01-resource-id"}}
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true,
    "results": [
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119a"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
                {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
            ],
            "owners": [
                "user-01",
                "user-02"
            ],
            "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
            "timestamp": 1520437111,
            "testing_tool_id": "testing-tool-01-id",
            "type": "intermediate",
            "data": {
                "some_data_produced_by_tt": 42
            }
        },
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119b"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.0"},
                {"resource_id": "user-02-resource-id", "resource_version": "2.0"}
            ],
            "owners": [
                "user-01",
                "user-02"
            ],
            "session_id": "b93884bf-c1b9-40da-892a-389b5e02a196",
            "timestamp": 1520437120,
            "testing_tool_id": "testing-tool-01-id",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": {
                    "some_data": 3.14,
                    "other_data": "some_string"
                }
            }
        },
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119c"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.1"}
            ],
            "owners": [
                "user-01"
            ],
            "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999",
            "timestamp": 1520437199,
            "testing_tool_id": "testing-tool-id-654321",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": 1234567890
            }
        }
    ]
}

Example 3: Get results of specific resource + version for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "resources": {"$elemMatch": {"resource_id": "user-01-resource-id", "resource_version": "1.1"}}
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true,
    "results": [
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119c"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.1"}
            ],
            "owners": [
                "user-01"
            ],
            "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999",
            "timestamp": 1520437199,
            "testing_tool_id": "testing-tool-id-654321",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": 1234567890
            }
        }
    ]
}

Example 4: Get results of specific session for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999"
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true,
    "results": [
        {
            "_id": {"$oid": "5aa7d1b8dbbb46608189119c"},
            "resources": [
                {"resource_id": "user-01-resource-id", "resource_version": "1.1"}
            ],
            "owners": [
                "user-01"
            ],
            "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999",
            "timestamp": 1520437199,
            "testing_tool_id": "testing-tool-id-654321",
            "type": "final",
            "data": {
                "some_data_produced_by_tt": 1234567890
            }
        }
    ]
}

Delete Result

Request parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key results_store.delete_result.request The routing key used for the request
reply_to results_store.delete_result.reply The routing key used for the reply
correlation_id str(uuid.uuid4()) The correlation id of the request
content_type application/json The content type

Request body:

The body is a query as in the case of getting the results. All results matching that query will be deleted.

WARNING: Passing an empty document {} will delete all documents (TBD: disable this case from API).

Reply parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key results_store.delete_result.reply The routing key used for the reply
correlation_id same as request The correlation id of the request
content_type application/json The content type

Reply body:

Key Type Value
ok Boolean true if the result(s) was deleted correctly, false otherwise
error String A message describing the error (present only if ok is false)

Examples

The following examples assume that the RS contain the three results
described in the Get Result section.

Example 1: Delete all results for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01"
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Example 2: Delete all results of specific resource for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "resources": {"$elemMatch": {"resource_id": "user-01-resource-id"}}
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Example 3: Delete all results of specific resource + version for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "resources": {"$elemMatch": {"resource_id": "user-01-resource-id", "resource_version": "1.1"}}
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Example 4: Delete all results of specific session for user-01

Request:

 - - -
ROUTING_KEY: results_store.insert_result.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.insert_result.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owners": "user-01",
    "session_id": "b55554bf-c1b9-44fg-444a-389b5e02a999"
}

Reply:

 - - -
ROUTING_KEY: results_store.insert_result.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Results Store Service (RSS)

( version of the API v.1.0 )

The Results Store Service (RSS) is a F-Interop service that acts as a bridge between a session virtual host and the Results Store. During a session it will receive raw data (results or intermediate results) directly from the Testing Tool. This data is then packaged with session meta-data and sent to the Results Store (RS).

To enable the RSS in a session, its Supervisor configuration template must be added inside the Testing Tool configuration file. See the example.

Supervisor configuration template of RSS

[program:{{ session }}|service_results_store]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service-results-store-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service-results-store-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env RS_AMQP_URL={{ rs_amqp_url }}
    --env RS_AMQP_EXCHANGE={{ rs_amqp_exchange }}
    --rm
    --name="session_{{ session }}-service-results-store"
    service-results-store

Example of Testing Tool configuration using the RSS

[program:{{ session }}|my_testing_tool]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-my_testing_tool-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-my_testing_tool-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --rm
    --name="session_{{ session }}-my_testing_tool"
    my_testing_tool-image-name

[program:{{ session }}|service_results_store]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service-results-store-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service-results-store-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env RS_AMQP_URL={{ rs_amqp_url }}
    --env RS_AMQP_EXCHANGE={{ rs_amqp_exchange }}
    --rm
    --name="session_{{ session }}-service-results-store"
    service-results-store

API Description

This API is used, in principle only by Testing Tools, to insert results in the RS. The schema of the data pushed by TTs is completely free as long it is valid JSON. The RS provides on any session virtual host the following services:

Service Description
results_store.session.report.save Insert a result in the Results Store

API Description: Insert Result (through RSS)

Request parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key results_store.session.report.save.request The routing key used for the request
reply_to results_store.session.report.save.reply The routing key used for the reply
correlation_id str(uuid.uuid4()) The correlation id of the request
content_type application/json The content type

Request body:

Key Type Value
type String The type of the result.
It can be intermediate or final
data Object The data produced by the testing tool as a result.
There is no schema for this object

Reply parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key results_store.session.report.save.reply The routing key used for the reply
correlation_id same as request The correlation id of the request
content_type application/json The content type

Reply body:

Key Type Value
ok Boolean true if the result was inserted correctly, false otherwise
error String A message describing the error (present only if ok is false)

Insert Result Request example:

 - - -
ROUTING_KEY: results_store.session.report.save.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "results_store.session.report.save.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "type": "final",
    "data": {
        "some_key": {
            "another_key": "some_value"
        },
        "yet_another_key": 42
    }
}

Insert Result Reply example:

 - - -
ROUTING_KEY: results_store.session.report.save.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Resource Repository (RR)

( version of the API v.2.0 )

The Resource Repository (RR) is a core F-Interop service that allow to store and retrieve the resources used in the tests.

The Results Store repository can be found here.

API Description

The RR uses MySQL as database. A resource is described by the JSON schema showed in the example. All fields are optional except resource_id and owner_id.

Resource Example

{
    "resource_id": "unique-resource-identifier",
    "privacy_flag": "public",
    "available": true,
    "owner_id": "unique-owner-identifier",
    "ts_creation": 1256953732,
    "ts_update": 1256953732,
    "hardware": {
        "manufacturer": "HW manufacturer",
        "machine_type": "phy",
        "hw_platform": "HW platform (board name)",
        "power_supply": "Power supply",
        "processor": "Processor name",
        "cpu_cache": "CPU cache",
        "ram": "RAM",
        "disk": "Secondary storage info",
        "supported_medias": [
            {
                "name": "Media name",
                "version": "Media version",
                "network_card": "Network card info",
                "mac_address": "aa:bb:cc:dd:ee:ff",
                "if_bandwidth": "Interface bandwidth"
            }
        ],
        "other_hardware_info": [
            {"parameter": "info_key", "value": "info_value"}
        ]
    },
    "fw_os": {
        "name": "OS/Firmware name",
        "version": "OS/Firmware version",
        "ip_address": "IP address",
        "supported_protocols_services": [
            {
                "name": "Protocol name 1",
                "version": "Protocol version 1",
                "port": 123,
                "other_protocol_info": [
                    {"parameter": "info_key", "value": "info_value"}
                ]
            }
        ],
        "other_fw_os_info": [
            {"parameter": "fw_os_info", "value": "fw_os_value"}
        ]
    },
    "software": {
        "name": "SW name",
        "type": "SW type",
        "version": [
            {"version": "1.0"},
            {"version": "1.1"},
            {"version": "2.0"}
        ],
        "role": "client",
        "automatic_flag": "auto",
        "test_environment": [
            {"test_environment": "central_server"},
            {"test_environment": "docker"}
        ],
        "testing_protocol": {
            "name": "Protocol name 2",
            "version": "Protocol version 2",
            "port": 1234,
            "other_protocol_info": [
                {"parameter": "info_key", "value": "info_value"}
            ]
        },
        "testing_tool": [
            {
                "name": "Testing Tool name",
                "version": "Testing Tool version"
            }
        ],
        "other_software_info": [
            {"parameter": "info_key", "value": "info_value"}
        ]
    },
    "location": {
        "latitude": 11.1111,
        "longitude": 22.2222,
        "country": "CH",
        "city": "City",
        "address": "Address",
        "x": 0,
        "y": 0,
        "z": 0
    }
}

The RR provides on virtual host / the following services:

Service Description
resource_repository.insert_resource Insert a resource in the RR
resource_repository.get_resource Query the RR for resources
resource_repository.delete_resource Delete one resource

Insert Resource

Request parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key resource_repository.insert_resource.request The routing key used for the request
reply_to resource_repository.insert_resource.reply The routing key used for the reply
correlation_id str(uuid.uuid4()) The correlation id of the request
content_type application/json The content type

Request body:

A JSON object matching the resource schema defined above. All fields are optional except resource_id and owner_id.

Reply parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key resource_repository.insert_resource.reply The routing key used for the reply
correlation_id same as request The correlation id of the request
content_type application/json The content type

Reply body:

Key Type Value
ok Boolean true if the resource was inserted correctly, false otherwise
error String A message describing the error (present only if ok is false)

Insert Resource Request example:

 - - -
ROUTING_KEY: resource_repository.insert_resource.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "resource_repository.insert_resource.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "resource_id": "unique-resource-identifier",
    "privacy_flag": "public",
    "available": true,
    "owner_id": "unique-owner-identifier",
    "location": {
        "latitude": 46.204391,
        "longitude": 6.143158,
        "country": "CH",
        "city": "Geneva"
    }
}

Insert Resource Reply example:

 - - -
ROUTING_KEY: resource_repository.insert_resource.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Get Resource

Request parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key resource_repository.get_resource.request The routing key used for the request
reply_to resource_repository.get_resource.reply The routing key used for the reply
correlation_id str(uuid.uuid4()) The correlation id of the request
content_type application/json The content type

Request body:

An object matching the structure of a resource object. All fields that are present in the query will be used to match resources. All resources that match all the fields present in the query will be returned (see example).

Reply parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key resource_repository.get_resource.reply The routing key used for the reply
correlation_id same as request The correlation id of the request
content_type application/json The content type

Reply body:

Key Type Value
ok Boolean true if the query was executed correctly,
false otherwise
results Array of Objects The resource(s) retrieved by the query
error String A message describing the error
(present only if ok is false)

Get Resource Request example:

The following example shows how to retrieve all resources owned by
the owner id `unique-owner-identifier`.

 - - -
ROUTING_KEY: resource_repository.get_resource.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "resource_repository.get_resource.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "owner_id": "unique-owner-identifier"
}

Get Resouce Reply example:

 - - -
ROUTING_KEY: resource_repository.get_resource.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true,
    "results": [
        {
            "resource_id": "unique-resource-identifier",
            "privacy_flag": "public",
            "available": true,
            "owner_id": "unique-owner-identifier",
            "location": {
                "latitude": 46.204391,
                "longitude": 6.143158,
                "country": "CH",
                "city": "Geneva"
            }
        }
    ]
}

Delete Resource

Request parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key resource_repository.delete_resource.request The routing key used for the request
reply_to resource_repository.delete_resource.reply The routing key used for the reply
correlation_id str(uuid.uuid4()) The correlation id of the request
content_type application/json The content type

Request body:

Key Type Value
resource_id String The id of the resource
owner_id String The id of the owner user

Reply parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key resource_repository.delete_resource.reply The routing key used for the reply
correlation_id same as request The correlation id of the request
content_type application/json The content type

Reply body:

Key Type Value
ok Boolean true if the result(s) was deleted correctly, false otherwise
error String A message describing the error (present only if ok is false)

Delete Resouce Request example:

 - - -
ROUTING_KEY: rresource_repository.delete_resource.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": "resource_repository.delete_resource.reply",
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "resource_id": "unique-resource-identifier",
    "owner_id": "unique-owner-identifier"
}

Delete Resource Reply example:

 - - -
ROUTING_KEY: resource_repository.delete_resource.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "reply_to": null,
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json"
}
 - - -
BODY: {
    "ok": true
}

Events (core API)

This section describes the format of the messages used in F-Interop.

This section of the documentation is autogenerated by tool

Check out the messages library tool

Version 1.1.0

Orchestrator events

User interface events

ui.user.all.display

Routing key ui.user.all.display
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for displaying Markdown text to user interface

Source code: MsgUiDisplayMarkdownText

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.display
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "8580421b-f568-464e-a01c-989dd25834c6", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "p", 
            "value": "Hello World!"
        }
    ], 
    "level": null, 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "p", 
            "value": "Hello World!"
        }
    ], 
    "tags": {}, 
    "level": null
}

ui.user.all.request

Routing key ui.user.all.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for requesting a text input on UI

Source code: MsgUiRequestTextInput

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.request
 -  -  - 
Message properties: {
    "reply_to": "ui.user.all.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "039ea17d-3b98-4e41-9539-ebc6ccd424c5", 
    "message_id": "039ea17d-3b98-4e41-9539-ebc6ccd424c5", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "text", 
            "name": "input_name"
        }
    ], 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "text", 
            "name": "input_name"
        }
    ], 
    "tags": {}
}

ui.user.all.request

Routing key ui.user.all.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for requesting confirmation button

Source code: MsgUiRequestConfirmationButton

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.request
 -  -  - 
Message properties: {
    "reply_to": "ui.user.all.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "26c87027-6a69-4fa2-8b12-d4bce9f9fddb", 
    "message_id": "26c87027-6a69-4fa2-8b12-d4bce9f9fddb", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "button", 
            "name": "test_button", 
            "value": true
        }
    ], 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "button", 
            "name": "test_button", 
            "value": true
        }
    ], 
    "tags": {}
}

ui.core.session.get.request

Routing key ui.core.session.get.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for requesting session information to UI

Source code: MsgUiRequestSessionConfiguration

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.core.session.get.request
 -  -  - 
Message properties: {
    "reply_to": "ui.core.session.get.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "5549a795-3143-4bb6-83af-b337691466fb", 
    "message_id": "5549a795-3143-4bb6-83af-b337691466fb", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0"
}

ui.core.session.get.reply

Routing key ui.core.session.get.reply
Requirements
Type Event
Pub/Sub UI -> TT
Description: Message for requesting session information to UI

Source code: MsgUiSessionConfigurationReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.core.session.get.reply
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "correlation_id": "127104de-6eaf-450c-9894-ab57423dac0d", 
    "message_id": "ce6d27b7-a579-49e4-b0eb-de37f7ef3c25", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "amqp_url": "amqp://WX9D3L5A:5S68CRDC@mq.dev.f-interop.eu:443/277704a1-03c0-467c-b00d-c984976692d7", 
    "logs": [
        {
            "date": "2018-05-07T12:50:47.224000+00:00", 
            "message": "Session created locally", 
            "type": "info"
        }
    ], 
    "ok": true, 
    "resources": [
        {}
    ], 
    "shared": true, 
    "slice_id": "urn:publicid:IDN+finterop:project1+slice+testing", 
    "start_date": "2018-05-07T12:50:48.128000+00:00", 
    "status": "open", 
    "testSuite": "http://orchestrator.dev.f-interop.eu:8181/tests/f-interop/dummy-tool-shared", 
    "testSuiteType": "interoperability", 
    "users": [
        "federico_sismondiojxu", 
        "myslice", 
        "federicosismondiparu"
    ]
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "status": "open", 
    "testSuite": "http://orchestrator.dev.f-interop.eu:8181/tests/f-interop/dummy-tool-shared", 
    "ok": true, 
    "logs": [
        {
            "date": "2018-05-07T12:50:47.224000+00:00", 
            "message": "Session created locally", 
            "type": "info"
        }
    ], 
    "slice_id": "urn:publicid:IDN+finterop:project1+slice+testing", 
    "_api_version": "1.1.0", 
    "testSuiteType": "interoperability", 
    "amqp_url": "amqp://WX9D3L5A:5S68CRDC@mq.dev.f-interop.eu:443/277704a1-03c0-467c-b00d-c984976692d7", 
    "shared": true, 
    "start_date": "2018-05-07T12:50:48.128000+00:00", 
    "resources": [
        {}
    ], 
    "users": [
        "federico_sismondiojxu", 
        "myslice", 
        "federicosismondiparu"
    ]
}

ui.user.all.request

Routing key ui.user.all.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for file upload request on UI

Source code: MsgUiRequestUploadFile

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.request
 -  -  - 
Message properties: {
    "reply_to": "ui.user.all.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "b7da591f-b651-404c-99c6-2d587d558e03", 
    "message_id": "b7da591f-b651-404c-99c6-2d587d558e03", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "file", 
            "name": "upload a file"
        }
    ], 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "file", 
            "name": "upload a file"
        }
    ], 
    "tags": {}
}

ui.user.all.request

Routing key ui.user.all.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for checkbox request on UI

Source code: MsgUiRequestQuestionCheckbox

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.request
 -  -  - 
Message properties: {
    "reply_to": "ui.user.all.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "8348c8e8-24f9-44c8-bd98-140e5c81bae9", 
    "message_id": "8348c8e8-24f9-44c8-bd98-140e5c81bae9", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "checkbox", 
            "name": "Choice1", 
            "value": 0, 
            "label": "Choice1"
        }, 
        {
            "type": "checkbox", 
            "name": "Choice2", 
            "value": 1, 
            "label": "Choice2"
        }
    ], 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "checkbox", 
            "name": "Choice1", 
            "value": 0, 
            "label": "Choice1"
        }, 
        {
            "type": "checkbox", 
            "name": "Choice2", 
            "value": 1, 
            "label": "Choice2"
        }
    ], 
    "tags": {}
}

ui.user.all.request

Routing key ui.user.all.request
Requirements
Type Event
Pub/Sub TT -> UI
Description: Message for radio request on UI

Source code: MsgUiRequestQuestionRadio

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: ui.user.all.request
 -  -  - 
Message properties: {
    "reply_to": "ui.user.all.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "2526ab40-2906-489b-b188-1005d2fad282", 
    "message_id": "2526ab40-2906-489b-b188-1005d2fad282", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "radio", 
            "name": "True", 
            "value": true
        }, 
        {
            "type": "radio", 
            "name": "False", 
            "value": false
        }
    ], 
    "tags": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "fields": [
        {
            "type": "radio", 
            "name": "True", 
            "value": true
        }, 
        {
            "type": "radio", 
            "name": "False", 
            "value": false
        }
    ], 
    "tags": {}
}

Testing Tool events

testingtool.ready

Routing key testingtool.ready
Requirements TT SHOULD publish event as soon as TT is up and listening on the event bus
Type Event
Pub/Sub Testing Tool -> GUI
Description: Used to indicate to the GUI that testing is ready to start the test suite

Source code: MsgTestingToolReady

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testingtool.ready
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "5a5d99ed-954c-4b99-8b5f-8f3bf8396f8c", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Testing tool READY to start test suite."
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "description": "Testing tool READY to start test suite."
}

session.configuration

Routing key session.configuration
Requirements TT SHOULD listen to event, and configure accordingly
Type Event
Pub/Sub Orchestrator -> Testing Tool
Description: TT SHOULD listen to this message and configure the testsuite correspondingly

Source code: MsgSessionConfiguration

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: session.configuration
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "b46a5a1d-765f-4232-a494-ed5a5eb7d936", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "configuration": {
        "testsuite.testcases": [
            "someTestCaseId1", 
            "someTestCaseId2"
        ]
    }, 
    "session_id": "666", 
    "testing_tools": "f-interop/someTestToolId", 
    "users": [
        "u1", 
        "f-interop"
    ]
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "configuration": {
        "testsuite.testcases": [
            "someTestCaseId1", 
            "someTestCaseId2"
        ]
    }, 
    "testing_tools": "f-interop/someTestToolId", 
    "session_id": "666", 
    "users": [
        "u1", 
        "f-interop"
    ]
}

testingtool.configured

Routing key testingtool.configured
Requirements TT SHOULD publish event once session.configuration message has been processed.
Type Event
Pub/Sub Testing Tool -> Orchestrator, GUI
Description: The goal is to notify orchestrator and other components that the testing tool has been configured

Source code: MsgTestingToolConfigured

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testingtool.configured
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "66275e5b-3ec1-4c2e-a3d0-6cec3c17357a", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Testing tool CONFIGURED", 
    "session_id": null, 
    "testing_tools": "f-interop/interoperability-coap"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "description": "Testing tool CONFIGURED", 
    "session_id": null, 
    "testing_tools": "f-interop/interoperability-coap"
}

testsuite.start

Routing key testsuite.start
Requirements TT SHOULD listen to event and start the test suite right after reception. MsgTestSuiteStarted
Type Event
Pub/Sub GUI -> Testing Tool
Description: tbd

Source code: MsgTestSuiteStart

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.start
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "cad6dc20-1a51-447c-873f-ecd69ec915ee", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Test suite START command"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "description": "Test suite START command"
}

testsuite.testcase.step.stimuli.execute

Routing key testsuite.testcase.step.stimuli.execute
Requirements TT SHOULD publish event
Type Event
Pub/Sub Testing Tool -> GUI
Description:
- Used to indicate to the GUI (or automated-iut) which is the stimuli step to be executed by the user (or
automated-IUT).

Source code: MsgStepStimuliExecute

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.testcase.step.stimuli.execute
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "c13e6e99-ac85-4ae1-bd74-20aaa97b0555", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Please execute TD_COAP_CORE_01_step_01", 
    "node": "coap_client", 
    "node_execution_mode": "user_assisted", 
    "step_id": "TD_COAP_CORE_01_step_01", 
    "step_info": [
        "Client is requested to send a GET request with", 
        "Type = 0(CON)", 
        "Code = 1(GET)"
    ], 
    "step_state": "executing", 
    "step_type": "stimuli", 
    "target_address": null, 
    "testcase_id": null, 
    "testcase_ref": null
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "node": "coap_client", 
    "step_state": "executing", 
    "description": "Please execute TD_COAP_CORE_01_step_01", 
    "_api_version": "1.1.0", 
    "testcase_ref": null, 
    "testcase_id": null, 
    "step_type": "stimuli", 
    "target_address": null, 
    "step_info": [
        "Client is requested to send a GET request with", 
        "Type = 0(CON)", 
        "Code = 1(GET)"
    ], 
    "node_execution_mode": "user_assisted", 
    "step_id": "TD_COAP_CORE_01_step_01"
}

testsuite.testcase.step.stimuli.executed

Routing key testsuite.testcase.step.stimuli.executed
Requirements TT SHOULD listen to event
Type Event
Pub/Sub GUI (or automated-IUT)-> Testing Tool
Description:
- Used to indicate stimuli has been executed by user (and it’s user-assisted iut) or by automated-iut

Source code: MsgStepStimuliExecuted

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.testcase.step.stimuli.executed
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "2a31d848-1689-4c01-9e1c-a87fbbcbcc7e", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Step (stimuli) EXECUTED", 
    "node": "coap_client", 
    "node_execution_mode": "user_assisted"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "node": "coap_client", 
    "node_execution_mode": "user_assisted", 
    "description": "Step (stimuli) EXECUTED"
}

testsuite.testcase.step.verify.execute

Routing key testsuite.testcase.step.verify.execute
Requirements TT SHOULD publish event
Type Event
Pub/Sub Testing Tool -> GUI (or automated-IUT)
Description:
- Used to indicate to the GUI (or automated-iut) which is the verify step to be executed by the user (or
automated-IUT).

Source code: MsgStepVerifyExecute

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.testcase.step.verify.execute
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "474476b9-d879-4332-97d1-a118528bcbe9", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Please execute TD_COAP_CORE_01_step_04", 
    "node": "coap_client", 
    "node_execution_mode": "user_assisted", 
    "response_type": "bool", 
    "step_id": "TD_COAP_CORE_01_step_04", 
    "step_info": [
        "Client displays the received information"
    ], 
    "step_state": "executing", 
    "step_type": "verify", 
    "testcase_id": null, 
    "testcase_ref": null
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "node": "coap_client", 
    "step_state": "executing", 
    "description": "Please execute TD_COAP_CORE_01_step_04", 
    "_api_version": "1.1.0", 
    "testcase_ref": null, 
    "testcase_id": null, 
    "step_type": "verify", 
    "step_info": [
        "Client displays the received information"
    ], 
    "response_type": "bool", 
    "node_execution_mode": "user_assisted", 
    "step_id": "TD_COAP_CORE_01_step_04"
}

testsuite.testcase.step.verify.executed

Routing key testsuite.testcase.step.verify.executed
Requirements TT SHOULD listen to event
Type Event
Pub/Sub GUI (or automated-IUT)-> Testing Tool
Description:
- Message generated by user (GUI or automated-IUT) declaring if the IUT VERIFY verifies the expected behaviour.

Source code: MsgStepVerifyExecuted

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.testcase.step.verify.executed
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "b2aff90a-092e-4eee-a15c-ff2ceb28c3b0", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Step (verify) EXECUTED", 
    "node": "coap_client", 
    "node_execution_mode": "user_assisted", 
    "response_type": "bool", 
    "verify_response": true
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "node": "coap_client", 
    "description": "Step (verify) EXECUTED", 
    "_api_version": "1.1.0", 
    "response_type": "bool", 
    "node_execution_mode": "user_assisted", 
    "verify_response": true
}

testsuite.testcase.verdict

Routing key testsuite.testcase.verdict
Requirements TT SHOULD publish event
Type Event
Pub/Sub Testing Tool -> GUI
Description: Used to indicate to the GUI (or automated-iut) which is the final verdict of the testcase.

Source code: MsgTestCaseVerdict

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.testcase.verdict
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "d839ebb1-f115-4a0d-b799-4c07d7a9755a", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "No interoperability error was detected,", 
    "objective": "Perform GET transaction(CON mode)", 
    "partial_verdicts": [
        [
            "TD_COAP_CORE_01_step_02", 
            null, 
            "CHECK postponed", 
            ""
        ], 
        [
            "TD_COAP_CORE_01_step_03", 
            null, 
            "CHECK postponed", 
            ""
        ], 
        [
            "TD_COAP_CORE_01_step_04", 
            "pass", 
            "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
            ""
        ], 
        [
            "CHECK_1_post_mortem_analysis", 
            "pass", 
            "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
        ], 
        [
            "CHECK_2_post_mortem_analysis", 
            "pass", 
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
        ], 
        [
            "CHECK_3_post_mortem_analysis", 
            "pass", 
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
        ]
    ], 
    "state": "finished", 
    "testcase_id": "TD_COAP_CORE_01", 
    "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01", 
    "verdict": "pass"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "description": "No interoperability error was detected,", 
    "_api_version": "1.1.0", 
    "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01", 
    "testcase_id": "TD_COAP_CORE_01", 
    "partial_verdicts": [
        [
            "TD_COAP_CORE_01_step_02", 
            null, 
            "CHECK postponed", 
            ""
        ], 
        [
            "TD_COAP_CORE_01_step_03", 
            null, 
            "CHECK postponed", 
            ""
        ], 
        [
            "TD_COAP_CORE_01_step_04", 
            "pass", 
            "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
            ""
        ], 
        [
            "CHECK_1_post_mortem_analysis", 
            "pass", 
            "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
        ], 
        [
            "CHECK_2_post_mortem_analysis", 
            "pass", 
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
        ], 
        [
            "CHECK_3_post_mortem_analysis", 
            "pass", 
            "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
        ]
    ], 
    "state": "finished", 
    "verdict": "pass", 
    "objective": "Perform GET transaction(CON mode)"
}

testsuite.report

Routing key testsuite.report
Requirements TT SHOULD publish event
Type Event
Pub/Sub Testing Tool -> GUI
Description: Used to indicate to the GUI (or automated-iut) the final results of the test session.

Source code: MsgTestSuiteReport

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testsuite.report
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "39d709a8-ade1-4f0f-8117-daad33dbed0d", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "tc_results": [
        {
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_01_step_02", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_01_step_03", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_01_step_04", 
                    "pass", 
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
                    ""
                ], 
                [
                    "CHECK_1_post_mortem_analysis", 
                    "pass", 
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
                ], 
                [
                    "CHECK_2_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
                ], 
                [
                    "CHECK_3_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
                ]
            ], 
            "testcase_id": "TD_COAP_CORE_01", 
            "verdict": "pass", 
            "description": "No interoperability error was detected,"
        }, 
        {
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_02_step_02", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_02_step_03", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_02_step_04", 
                    "pass", 
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
                    ""
                ], 
                [
                    "CHECK_1_post_mortem_analysis", 
                    "pass", 
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43213] DELETE /test> Match: CoAP(type=0, code=4)"
                ], 
                [
                    "CHECK_2_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43213] 2.02 Deleted > Match: CoAP(code=66, mid=0xa8cd, tok=b'')"
                ]
            ], 
            "testcase_id": "TD_COAP_CORE_02", 
            "verdict": "pass", 
            "description": "No interoperability error was detected,"
        }
    ]
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "tc_results": [
        {
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_01_step_02", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_01_step_03", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_01_step_04", 
                    "pass", 
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
                    ""
                ], 
                [
                    "CHECK_1_post_mortem_analysis", 
                    "pass", 
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43211] GET /test> Match: CoAP(type=0, code=1)"
                ], 
                [
                    "CHECK_2_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(code=69, mid=0xa8cb, tok=b'', pl=Not(b''))"
                ], 
                [
                    "CHECK_3_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43211] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
                ]
            ], 
            "testcase_id": "TD_COAP_CORE_01", 
            "verdict": "pass", 
            "description": "No interoperability error was detected,"
        }, 
        {
            "partial_verdicts": [
                [
                    "TD_COAP_CORE_02_step_02", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_02_step_03", 
                    null, 
                    "CHECK postponed", 
                    ""
                ], 
                [
                    "TD_COAP_CORE_02_step_04", 
                    "pass", 
                    "VERIFY step: User informed that the information was displayed correclty on his/her IUT", 
                    ""
                ], 
                [
                    "CHECK_1_post_mortem_analysis", 
                    "pass", 
                    "<Frame   3: [bbbb::1 -> bbbb::2] CoAP [CON 43213] DELETE /test> Match: CoAP(type=0, code=4)"
                ], 
                [
                    "CHECK_2_post_mortem_analysis", 
                    "pass", 
                    "<Frame   4: [bbbb::2 -> bbbb::1] CoAP [ACK 43213] 2.02 Deleted > Match: CoAP(code=66, mid=0xa8cd, tok=b'')"
                ]
            ], 
            "testcase_id": "TD_COAP_CORE_02", 
            "verdict": "pass", 
            "description": "No interoperability error was detected,"
        }
    ]
}

testingtool.terminate

Routing key testingtool.terminate
Requirements TT SHOULD listen to event, and handle a gracefully termination of all it’s processes
Type Event
Pub/Sub GUI, (or Orchestrator) -> Testing Tool
Description: Testing tool should stop all it’s processes gracefully.

Source code: MsgTestingToolTerminate

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: testingtool.terminate
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "12f42067-768f-4a05-9b16-3a73b4860084", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "description": "Command TERMINATE testing tool execution"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "description": "Command TERMINATE testing tool execution"
}

sniffing.start.request

Routing key sniffing.start.request
Requirements Testing Tool SHOULD implement (other components should not subscribe to event)
Type Request (service)
Pub/Sub coordination -> sniffing
Description: tbd

Source code: MsgSniffingStart

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: sniffing.start.request
 -  -  - 
Message properties: {
    "reply_to": "sniffing.start.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "06cccbb3-6a24-4c10-9110-2a2c59a502dd", 
    "message_id": "06cccbb3-6a24-4c10-9110-2a2c59a502dd", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "capture_id": "TD_COAP_CORE_01", 
    "filter_if": "tun0", 
    "filter_proto": "udp"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "capture_id": "TD_COAP_CORE_01", 
    "filter_if": "tun0", 
    "filter_proto": "udp"
}

sniffing.stop.request

Routing key sniffing.stop.request
Requirements Testing Tool SHOULD implement (other components should not subscribe to event)
Type Request (service)
Pub/Sub coordination -> sniffing
Description: tbd

Source code: MsgSniffingStop

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: sniffing.stop.request
 -  -  - 
Message properties: {
    "reply_to": "sniffing.stop.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "cdd45f6d-7c03-4625-a466-16d1892a3199", 
    "message_id": "cdd45f6d-7c03-4625-a466-16d1892a3199", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0"
}

sniffing.getcapture.request

Routing key sniffing.getcapture.request
Requirements Testing Tool SHOULD implement (other components should not subscribe to event)
Type Request (service)
Pub/Sub coordination -> sniffing
Description: tbd

Source code: MsgSniffingGetCapture

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: sniffing.getcapture.request
 -  -  - 
Message properties: {
    "reply_to": "sniffing.getcapture.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "62a982a8-bd52-447a-87cf-bc039b799616", 
    "message_id": "62a982a8-bd52-447a-87cf-bc039b799616", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "capture_id": "TD_COAP_CORE_01"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "capture_id": "TD_COAP_CORE_01"
}

sniffing.getlastcapture.request

Routing key sniffing.getlastcapture.request
Requirements Testing Tool SHOULD implement (other components should not subscribe to event)
Type Request (service)
Pub/Sub coordination -> sniffing
Description: tbd

Source code: MsgSniffingGetCaptureLast

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: sniffing.getlastcapture.request
 -  -  - 
Message properties: {
    "reply_to": "sniffing.getlastcapture.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "276773c2-40cc-461d-bede-601c9bff96e5", 
    "message_id": "276773c2-40cc-461d-bede-601c9bff96e5", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0"
}

dissection.dissectcapture.request

Routing key dissection.dissectcapture.request
Requirements Testing Tool SHOULD implement (other components should not subscribe to event)
Type Request (service)
Pub/Sub coordination -> dissection, analysis -> dissection
Description: TBD

Source code: MsgDissectionDissectCapture

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: dissection.dissectcapture.request
 -  -  - 
Message properties: {
    "reply_to": "dissection.dissectcapture.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "86ec1ea6-93c0-41f5-be42-4a4a7e5bce8b", 
    "message_id": "86ec1ea6-93c0-41f5-be42-4a4a7e5bce8b", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "file_enc": "pcap_base64", 
    "filename": "TD_COAP_CORE_01.pcap", 
    "protocol_selection": "coap", 
    "value": "1MOyoQIABAAAAAAAAAAAAMgAAABlAAAAqgl9WK8aBgA7AAAAOwAAAGADPxUAExFAu7sAAAAAAAAAAAAAAAAAAbu7AAAAAAAAAAAAAAAAAALXvBYzABNZUEABcGO0dGVzdMECqgl9WMcaBgCQAAAAkAAAAGAAAAAAaDr//oAAAAAAAAAAAAAAAAAAA7u7AAAAAAAAAAAAAAAAAAGJAAcTAAAAALu7AAAAAAAAAAAAAAAAAAK7uwAAAAAAAAAAAAAAAAACBAgAAAAAAABgAz8VABMRQLu7AAAAAAAAAAAAAAAAAAG7uwAAAAAAAAAAAAAAAAAC17wWMwATWVBAAXBjtHRlc6oJfVjSGgYAOwAAADsAAABgAz8VABMRP7u7AAAAAAAAAAAAAAAAAAG7uwAAAAAAAAAAAAAAAAAC17wWMwATWVBAAXBjtHRlc3TBAg=="
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "protocol_selection": "coap", 
    "filename": "TD_COAP_CORE_01.pcap", 
    "value": "1MOyoQIABAAAAAAAAAAAAMgAAABlAAAAqgl9WK8aBgA7AAAAOwAAAGADPxUAExFAu7sAAAAAAAAAAAAAAAAAAbu7AAAAAAAAAAAAAAAAAALXvBYzABNZUEABcGO0dGVzdMECqgl9WMcaBgCQAAAAkAAAAGAAAAAAaDr//oAAAAAAAAAAAAAAAAAAA7u7AAAAAAAAAAAAAAAAAAGJAAcTAAAAALu7AAAAAAAAAAAAAAAAAAK7uwAAAAAAAAAAAAAAAAACBAgAAAAAAABgAz8VABMRQLu7AAAAAAAAAAAAAAAAAAG7uwAAAAAAAAAAAAAAAAAC17wWMwATWVBAAXBjtHRlc6oJfVjSGgYAOwAAADsAAABgAz8VABMRP7u7AAAAAAAAAAAAAAAAAAG7uwAAAAAAAAAAAAAAAAAC17wWMwATWVBAAXBjtHRlc3TBAg==", 
    "file_enc": "pcap_base64"
}

dissection.dissectcapture.reply

Routing key dissection.dissectcapture.reply
Requirements Testing Tool SHOULD implement (other components should not subscribe to event)
Type Reply (service)
Pub/Sub Dissector -> Coordinator, Dissector -> Analyzer
Description: TBD

Source code: MsgDissectionDissectCaptureReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: dissection.dissectcapture.reply
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "correlation_id": "1e58eab1-f7e2-4a5d-b8f5-bc9edbc1fabb", 
    "message_id": "94691506-af7e-4a6d-992a-4d4752a02534", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "frames": [
        {
            "timestamp": 1464858393.547275, 
            "_type": "frame", 
            "protocol_stack": [
                {
                    "_type": "protocol", 
                    "ProtocolFamily": "0", 
                    "AddressFamily": "2", 
                    "_protocol": "NullLoopback"
                }, 
                {
                    "TotalLength": "41", 
                    "_type": "protocol", 
                    "DestinationAddress": "127.0.0.1", 
                    "Reserved": "0", 
                    "MoreFragments": "0", 
                    "HeaderLength": "5", 
                    "DontFragment": "0", 
                    "HeaderChecksum": "0x0000", 
                    "Version": "4", 
                    "Identification": "0x71ac", 
                    "TypeOfService": "0x00", 
                    "_protocol": "IPv4", 
                    "Protocol": "17", 
                    "TimeToLive": "64", 
                    "SourceAddress": "127.0.0.1", 
                    "Options": "b''", 
                    "FragmentOffset": "0"
                }
            ], 
            "id": 1, 
            "error": null
        }
    ], 
    "frames_simple_text": null, 
    "ok": true, 
    "token": "0lzzb_Bx30u8Gu-xkt1DFE1GmB4"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "frames": [
        {
            "timestamp": 1464858393.547275, 
            "_type": "frame", 
            "protocol_stack": [
                {
                    "_type": "protocol", 
                    "ProtocolFamily": "0", 
                    "AddressFamily": "2", 
                    "_protocol": "NullLoopback"
                }, 
                {
                    "TotalLength": "41", 
                    "_type": "protocol", 
                    "DestinationAddress": "127.0.0.1", 
                    "Reserved": "0", 
                    "MoreFragments": "0", 
                    "HeaderLength": "5", 
                    "DontFragment": "0", 
                    "HeaderChecksum": "0x0000", 
                    "Version": "4", 
                    "Identification": "0x71ac", 
                    "TypeOfService": "0x00", 
                    "_protocol": "IPv4", 
                    "Protocol": "17", 
                    "TimeToLive": "64", 
                    "SourceAddress": "127.0.0.1", 
                    "Options": "b''", 
                    "FragmentOffset": "0"
                }
            ], 
            "id": 1, 
            "error": null
        }
    ], 
    "token": "0lzzb_Bx30u8Gu-xkt1DFE1GmB4", 
    "ok": true, 
    "frames_simple_text": null
}

analysis.interop.testcase.analyze.request

Routing key analysis.interop.testcase.analyze.request
Requirements Testing Tool SHOULD implement (other components should not subscribe to event)
Type Request (service)
Pub/Sub coordination -> analysis
Description:
- Method to launch an analysis from a pcap file or a token if the pcap file has already been provided.
- The method need a token or a pcap_file but doesn’t allow someone to provide both.

Source code: MsgInteropTestCaseAnalyze

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: analysis.interop.testcase.analyze.request
 -  -  - 
Message properties: {
    "reply_to": "analysis.interop.testcase.analyze.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "e1133f12-1dab-4c14-b2ed-cd3698138ef1", 
    "message_id": "e1133f12-1dab-4c14-b2ed-cd3698138ef1", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "file_enc": "pcap_base64", 
    "filename": "TD_COAP_CORE_01.pcap", 
    "protocol": "coap", 
    "testcase_id": "TD_COAP_CORE_01", 
    "testcase_ref": "http://doc.f-interop.eu/tests/TD_COAP_CORE_01", 
    "value": "1MOyoQIABAAAAAAAAAAAAMgAAAAAAAAA"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "protocol": "coap", 
    "_api_version": "1.1.0", 
    "testcase_ref": "http://doc.f-interop.eu/tests/TD_COAP_CORE_01", 
    "testcase_id": "TD_COAP_CORE_01", 
    "value": "1MOyoQIABAAAAAAAAAAAAMgAAAAAAAAA", 
    "filename": "TD_COAP_CORE_01.pcap", 
    "file_enc": "pcap_base64"
}

analysis.interop.testcase.analyze.reply

Routing key analysis.interop.testcase.analyze.reply
Requirements Testing Tool SHOULD implement (other components should not subscribe to event)
Type Reply (service)
Pub/Sub analysis -> coordination
Description:
- The recommended structure for the partial_verdicts field is a list of partial verdicts which complies to:
- each one of those elements of the list correspond to one CHECK or VERIFY steps of the test description
- first value of the list MUST be a “pass”, “fail”, “inconclusive” or eventually “error” partial verdict (
string)
- the second value MUST be a string with a description of partial verdict (intended for the user)
- more values elements MAY be added to the list

Source code: MsgInteropTestCaseAnalyzeReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: analysis.interop.testcase.analyze.reply
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "correlation_id": "953df231-48a8-4215-ace2-140b7e221537", 
    "message_id": "a935b017-40fe-4851-8402-53af99691863", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "analysis_type": "postmortem", 
    "description": "The test purpose has been verified without any fault detected", 
    "ok": true, 
    "partial_verdicts": [
        [
            "pass", 
            "<Frame   1: [127.0.0.1 -> 127.0.0.1] CoAP [CON 43521] GET /test> Match: CoAP(type=0, code=1)"
        ], 
        [
            "pass", 
            "<Frame   2: [127.0.0.1 -> 127.0.0.1] CoAP [ACK 43521] 2.05 Content > Match: CoAP(code=69, mid=0xaa01,                 tok=b'b\\xda', pl=Not(b''))"
        ], 
        [
            "pass", 
            "<Frame   2: [127.0.0.1 -> 127.0.0.1] CoAP [ACK 43521] 2.05 Content >                 Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
        ]
    ], 
    "review_frames": [], 
    "testcase_id": "TD_COAP_CORE_01", 
    "testcase_ref": "http://doc.f-interop.eu/tests/TD_COAP_CORE_01", 
    "token": "0lzzb_Bx30u8Gu-xkt1DFE1GmB4", 
    "verdict": "pass"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "ok": true, 
    "description": "The test purpose has been verified without any fault detected", 
    "_api_version": "1.1.0", 
    "testcase_ref": "http://doc.f-interop.eu/tests/TD_COAP_CORE_01", 
    "testcase_id": "TD_COAP_CORE_01", 
    "review_frames": [], 
    "partial_verdicts": [
        [
            "pass", 
            "<Frame   1: [127.0.0.1 -> 127.0.0.1] CoAP [CON 43521] GET /test> Match: CoAP(type=0, code=1)"
        ], 
        [
            "pass", 
            "<Frame   2: [127.0.0.1 -> 127.0.0.1] CoAP [ACK 43521] 2.05 Content > Match: CoAP(code=69, mid=0xaa01,                 tok=b'b\\xda', pl=Not(b''))"
        ], 
        [
            "pass", 
            "<Frame   2: [127.0.0.1 -> 127.0.0.1] CoAP [ACK 43521] 2.05 Content >                 Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"
        ]
    ], 
    "token": "0lzzb_Bx30u8Gu-xkt1DFE1GmB4", 
    "verdict": "pass", 
    "analysis_type": "postmortem"
}

Results Repository events

Resources Repository events

Visualization tools events

viztool-grafana.init.request

Routing key viztool-grafana.init.request
Requirements Implementing Test Tools should send this at start
Type Event
Pub/Sub Testing Tool -> Visualization Tool
Description: Visualization Tool is waiting for this message
Visualization Tool MUST listen to this
to start init routines

Source code: MsgVizInitRequest

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: viztool-grafana.init.request
 -  -  - 
Message properties: {
    "reply_to": "viztool-grafana.init.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "956eb46c-e161-469c-b427-a1711e4287f9", 
    "message_id": "956eb46c-e161-469c-b427-a1711e4287f9", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0"
}

viztool-grafana.init.reply

Routing key viztool-grafana.init.reply
Requirements Visualization MUST send this after recieving MsgVizInitRequest
Type Event
Pub/Sub Visualization Tool -> Testing Tool
Description: This message contains the URL to access the internal Webserver
Test Tool MUST listen to this
that serves the Grafana Instance

Source code: MsgVizInitReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: viztool-grafana.init.reply
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "correlation_id": "ac0c6803-1606-4f88-9f87-37e2e7107fa5", 
    "message_id": "1e62d419-166b-4aaa-a224-41ddfe507dc6", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "ok": true, 
    "url": "http://url-to-access-grafana:1234"
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "url": "http://url-to-access-grafana:1234", 
    "ok": true
}

viztool-grafana.set_dashboard.request

Routing key viztool-grafana.set_dashboard.request
Requirements
Type Event
Pub/Sub Testing Tool -> Visualization Tool
Description: Visualization Tool uses this message to configure the Dashboard based on the JSON config
- Visualization Tool MUST listen to this.
- Test Tools should send this message after the Visualization Tool has confirmed the initialization

Source code: MsgVizDashboardRequest

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: viztool-grafana.set_dashboard.request
 -  -  - 
Message properties: {
    "reply_to": "viztool-grafana.set_dashboard.reply", 
    "timestamp": 1527154018, 
    "correlation_id": "cb5c4eeb-0797-4922-98d3-44b9cddcf11c", 
    "message_id": "cb5c4eeb-0797-4922-98d3-44b9cddcf11c", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "config": {}
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "config": {}
}

viztool-grafana.set_dashboard.reply

Routing key viztool-grafana.set_dashboard.reply
Requirements
Type Event
Pub/Sub Visualization Tool -> Testing Tool
Description: This message contains the URL to access the internal Webserver that serves the Grafana Instance
- Visualization MUST send this after recieving MsgVizDashboardRequest
- Test Tool MUST listen to this

Source code: MsgVizDashboardReply

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: viztool-grafana.set_dashboard.reply
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "correlation_id": "21e86b23-4d0a-4551-9392-bb653b1e6328", 
    "message_id": "64d4784e-76cd-4699-b3a5-61a217d65626", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "ok": true
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "ok": true
}

viztool-grafana.write_data

Routing key viztool-grafana.write_data
Requirements
Type Event
Pub/Sub Performance Testing Tool -> Visualization
Description:
- Performance Testing Tool SHOULD emit this event periodically
- Visualization Tool MUST listen to this event
- During the test execution, the Performance Testing Tool MUST periodically emit this event carrying current performance statistics/measurements

Source code: MsgVizWrite

  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
Message routing key: viztool-grafana.write_data
 -  -  - 
Message properties: {
    "timestamp": 1527154018, 
    "message_id": "a7968257-9348-49a4-9986-1d291b1f3899", 
    "content_type": "application/json"
}
 -  -  - 
Message body: {
    "_api_version": "1.1.0", 
    "fields": {
        "value": 0
    }, 
    "measurement": "name", 
    "tags": {}, 
    "time": 0
}
 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 {
    "_api_version": "1.1.0", 
    "fields": {
        "value": 0
    }, 
    "tags": {}, 
    "time": 0, 
    "measurement": "name"
}

Container services (CS)

This section introduces the Container Services (CS) and explain how to use them. This feature allows containers like Testing Tools (TT) or Visualization Tools (VT) to expose their own web services through the Session Orchestrator (SO).

URL-port Mappings

When a container, having inside a web server, is spawned on the SO then a particular URL and a port are created. The URL specifies where to find that service on the SO host while the container must use the given port to expose its service inside the SO host. Then an NGINX proxy server will redirect HTTP traffic from that URL to the corresponding port.

For example, if we have two containers service_1 and service_2 both willing to expose their web services, then the SO will create two mappings like:

http(s)://so.f-interop.eu:8282/container-services/session_id/service_1 -> localhost:6123 http(s)://so.f-interop.eu:8282/container-services/session_id/service_2 -> localhost:6124

As a consequence service_1 must use port 6123 while service_2 must use port 6124. Containers are able to know what URL has been assigned to them, this way they can inform other parties about how to reach them. For example they can use GUI API to have their web service embedded as an iframe into the GUI.

URL-port mappings are stored internally by the SO.

Container Template

The supervisor template of a container (stored in the SO) willing to expose a service must follow a few rules. On the right a minimal example.

Minimal example of a container template

[program:{{ session }}|service_1]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service_1-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service_1-stderr.log
command = docker run
    --env SERVICE_URL={{ service_1_url }}
    --rm
    --name="session_{{ session }}-service_1"
    -p {{ service_1_port }}:8080
    service_1-image-name

The variables used to access the given url and port are structured as follows:

Name Value
<program_name>_url The URL to access the service
<program_name>_port The port number to be used internally

Where <program_name> is the name of the supervisor program described in the template but without the prefix {{ session }}|. For example, if the template defines the program as [program:{{ session }}|my_complex_program_name] then <program_name> will correspond to my_complex_program_name. As a consequence <program_name>_url will correspond to my_complex_program_name_url, while <program_name>_port will correspond to my_complex_program_name_port.

The template can, of course, still use the usual variables like amqp_url and amqp_exchange if the container needs to connect to the session AMQP bus.

There can also be more than one service per session in the same config file, just use different valid names.

URLs can also be directly passed to other programs, for example in the case there is a container that needs to communicate with the other container through HTTP. See the examples on the right.

Example 1: Container using amqp_url and amqp_exchange variables

[program:{{ session }}|service_2]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service_2-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service_2-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env SERVICE_URL={{ service_2_url }}
    --rm
    --name="session_{{ session }}-service_2"
    -p {{ service_2_port }}:8080
    service_2-image-name

Example 2: Two containers exposing two different services

[program:{{ session }}|service_1]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service_1-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service_1-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env SERVICE_URL={{ service_1_url }}
    --rm
    --name="session_{{ session }}-service_1"
    -p {{ service_1_port }}:8080
    service_1-image-name

[program:{{ session }}|service_2]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service_2-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service_2-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env SERVICE_URL={{ service_2_url }}
    --rm
    --name="session_{{ session }}-service_2"
    -p {{ service_2_port }}:8080
    service_2-image-name

Example 3: A container tool using the service of the second container

[program:{{ session }}|my_tool]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-my_tool-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-my_tool-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env SERVICE_2_URL={{ service_2_url }}
    --rm
    --name="session_{{ session }}-my_tool"
    my_tool-image-name

[program:{{ session }}|my_service]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-my_service-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-my_service-stderr.log
command = docker run
    --rm
    --name="session_{{ session }}-my_service"
    -p {{ my_service_port }}:8080
    my_service-image-name

Visualization Tools (VT)

This section of the documentation describes the visualization tool’s minimal requirements.

General requirements on Visualization Tools (VT):

Visualization Tools are an implementation of Container Services (CS) and therefore they must respect CS requirements and conventions.

Minimal set of events to handle

There is no minimal set of events to handle.

Reference implementations of VTs:

See viztool-grafana for reference implementation.

Viztool-Grafana

( version of the API v.2.0 )

This tool offers real-time visualization support for Time Series data to F-Interop testing tools. The data is stored internally by using InfluxDB and the visualization is handled by Grafana.

The viztool-grafana repository can be found here.

Using the tool

To use the tool, its Supervisor configuration template must be copied inside the configuration template of the tool that wants to use it. By doing this way, an instance of this visualization tool will be spawned alongside the other tools defined for that session. See the example.

Usage example

[program:{{ session }}|my_testing_tool]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-my_testing_tool-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-my_testing_tool-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --rm
    --privileged=true
    --name="session_{{ session }}-my_testing_tool"
    my_testing_tool

[program:{{ session }}|service_viztool_grafana]
stopsignal=TERM
killasgroup=true
autostart=false
stdout_logfile = %(here)s/logs/{{ session }}-service_viztool_grafana-stdout.log
stderr_logfile = %(here)s/logs/{{ session }}-service_viztool_grafana-stderr.log
command = docker run
    --env AMQP_URL={{ amqp_url }}
    --env AMQP_EXCHANGE={{ amqp_exchange }}
    --env VIZTOOL_URL={{ service_viztool_grafana_url }}
    --env GF_SERVER_ROOT_URL={{ service_viztool_grafana_url }}
    --rm
    --name="session_{{ session }}-service_viztool_grafana"
    -p {{ service_viztool_grafana_port }}:3000
    service-viztool-grafana

API Description

The viztool-grafana provides on the current virtual host the following services:

Service Description
viztool-grafana.init Initialize the tool
viztool-grafana.set_dashboard Set the Grafana dashboard
viztool-grafana.write_data Write data to be visualized by Grafana

API Description: Init

This RPC should be called first and it initializes the tool. It is also used to retrieve the URL used to access the Grafana web server.

Request parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key viztool-grafana.init.request The routing key used for the request
reply_to viztool-grafana.init.reply The routing key used for the reply
correlation_id str(uuid.uuid4()) The correlation id of the request
content_type application/json The content type

Request body:

An empty JSON object (not used).

Reply parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key viztool-grafana.init.reply The routing key used for the reply
correlation_id same as request The correlation id of the request
content_type application/json The content type

Reply body:

Key Type Value
ok Boolean true if the operation succeded, false otherwise
url String The URL to access Grafana (present only if ok is true)
error String A message describing the error (present only if ok is false)

INIT REQUEST EXAMPLE:

 - - -
ROUTING_KEY: viztool-grafana.init.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": "viztool-grafana.init.reply",
    ...
}
 - - -
BODY {}
 - - -

INIT RESPONSE EXAMPLE

 - - -
ROUTING_KEY: viztool-grafana.init.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": null,
    ...
}
 - - -
BODY {
    "ok": true,
    "url": "https://someHost:somePort/path/to/service_viztool_grafana"
}
 - - -

API Description: Set Dashboard

This RPC is called after initialization and it is used to set the dashboard to be used by Grafana to display the data.

Request parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key viztool-grafana.set_dashboard.request The routing key used for the request
reply_to viztool-grafana.set_dashboard.reply The routing key used for the reply
correlation_id str(uuid.uuid4()) The correlation id of the request
content_type application/json The content type

Request body:

Key Type Value
config Object The JSON object representing the Grafana dashboard
to be used.

See Grafana doc. An easy way to get a JSON dashboard is to manually create and edit one in a local Grafana instance and then export it as JSON.

Reply parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key viztool-grafana.set_dashboard.reply The routing key used for the reply
correlation_id same as request The correlation id of the request
content_type application/json The content type

Reply body:

Key Type Value
ok Boolean true if the operation succeded, false otherwise
error String A message describing the error (present only if ok is false)

SET DASHBOARD REQUEST EXAMPLE:

 - - -
ROUTING_KEY: viztool-grafana.set_dashboard.request
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": "viztool-grafana.set_dashboard.reply",
    ...
}
 - - -
BODY {
  "config": {
      "annotations": {
        "list": [
          {
            "builtIn": 1,
            "datasource": "-- Grafana --",
            "enable": true,
            "hide": true,
            "iconColor": "rgba(0, 211, 255, 1)",
            "name": "Annotations & Alerts",
            "type": "dashboard"
          }
        ]
      },
      "editable": true,
      "gnetId": null,
      "graphTooltip": 0,
      "hideControls": false,
      "id": 1,
      "links": [],
      "refresh": "5s",
      "rows": [
        {
          "collapse": false,
          "height": 318,
          "panels": [
            {
              "aliasColors": {
                "cpu.mean": "#65c5db"
              },
              "bars": true,
              "dashLength": 10,
              "dashes": false,
              "datasource": null,
              "fill": 1,
              "id": 1,
              "legend": {
                "avg": false,
                "current": false,
                "max": false,
                "min": false,
                "show": true,
                "total": false,
                "values": false
              },
              "lines": false,
              "linewidth": 1,
              "links": [],
              "nullPointMode": "null",
              "percentage": false,
              "pointradius": 2,
              "points": true,
              "renderer": "flot",
              "seriesOverrides": [],
              "spaceLength": 10,
              "span": 12,
              "stack": false,
              "steppedLine": false,
              "targets": [
                {
                  "dsType": "influxdb",
                  "groupBy": [
                    {
                      "params": [
                        "$__interval"
                      ],
                      "type": "time"
                    },
                    {
                      "params": [
                        "null"
                      ],
                      "type": "fill"
                    }
                  ],
                  "measurement": "cpu",
                  "orderByTime": "ASC",
                  "policy": "default",
                  "refId": "A",
                  "resultFormat": "time_series",
                  "select": [
                    [
                      {
                        "params": [
                          "value"
                        ],
                        "type": "field"
                      },
                      {
                        "params": [],
                        "type": "mean"
                      }
                    ]
                  ],
                  "tags": []
                }
              ],
              "thresholds": [],
              "timeFrom": null,
              "timeShift": null,
              "title": "Demo Panel",
              "tooltip": {
                "shared": true,
                "sort": 0,
                "value_type": "individual"
              },
              "transparent": false,
              "type": "graph",
              "xaxis": {
                "buckets": null,
                "mode": "time",
                "name": null,
                "show": true,
                "values": []
              },
              "yaxes": [
                {
                  "format": "short",
                  "label": null,
                  "logBase": 1,
                  "max": null,
                  "min": null,
                  "show": true
                },
                {
                  "format": "short",
                  "label": null,
                  "logBase": 1,
                  "max": null,
                  "min": null,
                  "show": true
                }
              ]
            }
          ],
          "repeat": null,
          "repeatIteration": null,
          "repeatRowId": null,
          "showTitle": false,
          "title": "Dashboard Row",
          "titleSize": "h6"
        }
      ],
      "schemaVersion": 14,
      "style": "dark",
      "tags": [],
      "templating": {
        "list": []
      },
      "time": {
        "from": "now-5m",
        "to": "now"
      },
      "timepicker": {
        "nowDelay": "",
        "refresh_intervals": [
          "1s",
          "5s",
          "10s",
          "30s",
          "1m",
          "5m",
          "15m",
          "30m",
          "1h",
          "2h",
          "1d"
        ],
        "time_options": [
          "5m",
          "15m",
          "1h",
          "6h",
          "12h",
          "24h",
          "2d",
          "7d",
          "30d"
        ]
      },
      "timezone": "browser",
      "title": "Demo Dashboard",
      "version": 1
    }
}
 - - -

SET DASHBOARD RESPONSE EXAMPLE

 - - -
ROUTING_KEY: viztool-grafana.set_dashboard.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": null,
    ...
}
 - - -
BODY {
    "ok": true
}
 - - -

API Description: Write Data

This method is used to push data into Grafana to be visualized in the previously set dashboard.

Method parameters:

Parameter Value Description
exchange amq.topic The AMQP exchange
routing_key viztool-grafana.write_data The routing key used for the request
content_type application/json The content type

Request body:

The format of the data reflects the Line Protocol specified by InfluxDB (reference). Note that the data you are sending must be compatible with the queries you specify in the dashboard. See Grafana doc.

WRITE DATA EXAMPLE

 - - -
ROUTING_KEY: viztool-grafana.write_data
 - - -
HEADERS: None
 - - -
PROPS: {
    "content_type": "application/json", 
    ...
}
 - - -
BODY {
  "measurement": "cpu",
  "tags": {
    "host": "server01",
    "region": "us-west"
  },
  "fields": {
    "value_1": 0.42,
    "value_2": 42
  }
}

 - - -