NAV
Json Example Python

Introduction

In this doc we present the important information regarding the architecture, the different components and the description of the messages exchanged between components.

Must Reads:

Developers should read public deliverable D1.3 - Initial architecture design before getting started here!

“At the early stage [of an API defintion], agility trumps completeness”.

Joshua J. Bloch (Sun Microsystems and Google)

Architecture

F-Interop’s architecture follows the event bus design pattern: generic messages are routed between the different software components, some of these running at different locations. The F-Interop architecture is composed of different components exchanging messages through an “Event Bus”. All communication is done through this mechanism, including control messages, raw data packets, logs, etc. We use the RabbitMQ as the underlying message-passing mechanism. It acts as a secure message broker between all the components through encrypted channels.

Screenshot

Event Bus

Components of the system communicates with each other using messages. We need message delivery to be:

This architecture enables many independant components to talk to each other and avoid a monolith that would be difficult to manage as the number of components increase.

We use the RabbitMQ as the underlying message-passing mechanism.

It acts as a secure message broker between all the components through encrypted channels. Each message contains a routing key which indicates how to route this message to the relevant input queues of the components. It also shipped with a web interface for monitoring the load and enable easy debugging. Isolation of test session is performed using virtual hosts.

Message types

Messages are of two types: control plane and data plane. Control plane messages relate to the management of an ongoing test session: e.g. start sniffer, signal the start/end of a test case, etc. Data plane messages contains the raw data exchanged between the implementations under test (IUT).

JSON is the data format used for the application messages exchanged in the bus.

AMQP

For producing/consuming events we use the publish/subscribe pattern which implementation is based on topic exchanges of AMQP (more about topic exchanges)

Topic exchanges route messages to one or many queues based on matching between a message routing key and the pattern that was used to bind a queue to an exchange.

Topics/routing keys format

In the scope of a testing session we have two communication planes, the data plane, and the control plane.

how the topics/routing keys are componsed is standardized by F-Interop, by using certain conventions.

and is defined as follows:

Control Plane

Control plane messages relate to the management of an ongoing test session: e.g. start sniffer, signal the start/end of a test case, etc.

For the control plane the topic/routing key construction follow the following structure:

examples:

Data plane

Data plane messages contains the raw data exchanged between the IUTs. In here, topics/routing keys are not built around the concept of the component, but around the concept of channels.

More than one communication channel might be needed per test session, example CoAP TD describes some test cases with a “packet dropper” (GW) between the two IUTs.

client <====channel_01=====> GW <====channel_02=====> server

For the data plane we defined the topics/routing keys as:

data.channel__id.packet.raw

AMQP message attributes

Field Type Description
routing_key string tbd
source_id string a unique ID for the component that published this message such as: amqp://component_id@f-interop.eu
msg_id number a globally unique ID for this message
correlation_id string we include this field whenever the message is a request for an other component’s service or reply a reply to one (either method call or action call), this field MUST be included only if the message is a request or reply.
timestamp number the Posix/Unix time in second when this message was generated by its publisher. Extra care should be given to the timestamp format and its precision.
version number version of message specification (TBD if used or not)
body JSON payload of the message in JSON format
AMQP message structure:
{
    "routing_key": "data.channel_01.packet.raw",
    "src_id": "amqp://agent_id@finterop.org",
    "msg_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "correlation_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "timestamp": 1469806702,
    "payload": {
                    {
                        "_type" : "packet_raw",
                        "mimetype" : "application/octet-stream",
                        "metadata" : [ ],
                        "value" :   "TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyByZWFzb24sIGJ1dCBieSB0aGlz"
                    }

}

Components description

Describe the general arch. The components, and say how they interact with each other. General timeouts mechanisms. Mention that this is important due to the arch’s distributed nature.

TBD

Messaging pattern types used in F-Interop:

As previously mentioned, F-Interop uses an event bus messaging pattern, which is implemented using

Screenshot

Screenshot

Screenshot

Examples

TBD examples describing the three types of interactions.

TBD for components providing services it’s specially important to mention:
- how the routing_keys are formated for the reply and for the request
- what allows developpers to map a request to a reply

Application Layer Messages

Message Header

Example service start sniffer

{
    "routing_key": "control.sniffing.service",
    "src_id": "amqp://coordinator@finterop.org",
    "msg_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "correlation_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "timestamp": 1469806702,
    "payload": {
                      {
                        "_type":"sniffing_service",
                        "value":"start",
                        "filter-if":"lo",
                        "filter-proto":"coap"
                      }
                }
}

Response ok

{

    "routing_key": "control.sniffing.info",
    "src_id": "amqp://sniffer@finterop.org",
    "msg_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "correlation_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "timestamp": 1469806702,
    "payload": {
                     {
                        "_type": "sniffing_info",
                        "value": "started"
                     }
                 }
}

Response fail

{
    "routing_key": "control.sniffing.info",
    "src_id": "amqp://sniffer@finterop.org",
    "msg_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "correlation_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "timestamp": 1469806702,
    "payload": {
                    {
                        "_type": "sniffing_error",
                        "value": "Interface not existent!",
                        "error_code" : 1
                    }
               }

}

Every message in the top-level structure (payload of the MOM message) contains the set of fields:

Field Type Description
_type string basically it describes what the message is about.

Session Orchestrator

The orchestrator plays a purely administrative role: it monitors the users that are connected, activates the rooms currently in use and starts/stops the test sessions.

It is also in charge of provisioning the message broker and updating Technical Overview of F-Interop firewall rules when test sessions are activated. It does so by spawning/killing the processes of the different components connected to the event bus.

Create a new session

PUT http://f-interop.paris.inria.fr/sessions/<session_id>

{
    "users": [
        "a",
        "b",
        "c"
    ],
    "iuts": [
      {
        "id": "http://f-interop.paris.inria.fr/devices/device_a",
        "role": "client",
        "execution_mode": "user-assisted",
        "owner": "a"
      },
      {
        "id": "http://f-interop.paris.inria.fr/devices/device_b",
        "role": "server",
        "execution_mode": "automated",
        "owner": "f-interop"
      }
    ],
    "testing_tools": "http://f-interop.paris.inria.fr/testingtools/f-interop/coap_testing_tools",
    "tests": [
      {
        "testcase_id": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01",
        "settings": {}
      },
      {
        "testcase_id": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_02",
        "settings": {}
      }
    ]
}
import requests

url = "http://f-interop.paris.inria.fr/sessions/<session_id>"

data = {
    "users": [
        "a",
        "b",
        "c"
    ],
    "iuts": [
        {
            "id": "http://f-interop.paris.inria.fr/devices/device_a",
            "role": "client",
            "execution_mode": "user-assisted",
            "owner": "a"
        },
        {
            "id": "http://f-interop.paris.inria.fr/devices/device_b",
            "role": "server",
            "execution_mode": "automated",
            "owner": "f-interop"
        }
    ],
    "testing_tools": "http://f-interop.paris.inria.fr/testingtools/f-interop/coap_testing_tools",
    "tests": [
        {
            "testcase_id": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01",
            "settings": {}
        }
    ]
}

register_response = requests.put(url,
                                 json=data,
                                 auth=("username", "password"))

Get a session

GET http://f-interop.paris.inria.fr/sessions/<session_id>

import requests

url = "http://f-interop.paris.inria.fr/sessions/<session_id>"

register_response = requests.get(url,
                                 auth=("username", "password"))

Start a session

PATCH http://f-interop.paris.inria.fr/sessions/<session_id>

{
  "status": "started"
}
import requests

url = "http://f-interop.paris.inria.fr/sessions/<session_id>"

register_response = requests.patch(url,
                                   json={"status": "started"},
                                   auth=("username", "password"))

Stop a session

PATCH http://f-interop.paris.inria.fr/sessions/<session_id>

{
  "status": "stopped"
}
import requests

url = "http://f-interop.paris.inria.fr/sessions/<session_id>"

register_response = requests.patch(url,
                                   json={"status": "stopped"},
                                   auth=("username", "password"))

Delete a session

DELETE http://f-interop.paris.inria.fr/sessions/<session_id>

import requests

url = "http://f-interop.paris.inria.fr/sessions/<session_id>"

register_response = requests.delete(url,
                                    auth=("username", "password"))

Create a new user

PUT http://f-interop.paris.inria.fr/users/<user_id>

import requests

url = "http://f-interop.paris.inria.fr/users/<user_id>"
register_response = requests.put(url,
                                 json={"password": "foo"})
{
  "password": "foo"
}

Get information about a user

GET http://f-interop.paris.inria.fr/users/<user_id>

import requests

url = "http://f-interop.paris.inria.fr/users/<user_id>"
register_response = requests.get(url)

Delete a user

DELETE http://f-interop.paris.inria.fr/users/<user_id>

import requests

url = "http://f-interop.paris.inria.fr/users/<user_id>"
register_response = requests.delete(url,
                                    auth=("username", "password"))

Post an event on the event bus of a session

HTTP is not designed to handle event flow such as AMQP is. Sending event is supported but the reply will only tell the sender whether or not the event is arrived inside the broker but it won’t contains anything about the effect.

Publish a message to a given exchange. You will need a body looking something like:

{ "properties": {}, "routing_key": "my key", "payload": "my body", "payload_encoding": "string" }

All keys are mandatory. The payload_encoding key should be either “string” (in which case the payload will be taken to be the UTF-8 encoding of the payload field) or “base64” (in which case the payload field is taken to be base64 encoded). If the message is published successfully, the response will look like:

{ "routed": true }

Routed will be true if the message was sent to at least one queue. Please note that the HTTP API is not ideal for high performance publishing; the need to create a new TCP connection for each message published can limit message throughput compared to AMQP or other protocols using long-lived connections.

POST http://f-interop.paris.inria.fr/sessions/<session_id>/publish

{
  "properties": {},
  "routing_key": "my key",
  "payload": "my body",
  "payload_encoding": "string"
}
import requests

url = "http://f-interop.paris.inria.fr/sessions/<session_id>/publish"

register_response = requests.post(url,
                                  json={"password": "foo"},
                                  auth=("username", "password"))

Testing tools

Interactions

The general architecture and interfaces of F-Interop is described in the diagram

However, the documentation hosted in doc.f-interop.eu focus on the description of the following interfaces:

Screenshot

A second level of abstraction is provided for the design of the testing tool in the case of interoperability and conformance testing:

Screenshot

This documentation also specifies the interfaces between the coordinator + orchestrator + tat + sniffer + agents + packet generator which is used by F-Interop team to build out native F-Interop testing tools.

Some components may be used for other testing tools (like performance or privacy testing).

F-Interop recommends the following design shown in the picture above (and also to re-use as many components as possible of the code provided). link_to_source_code Implementors of testing tools may nevertheless implement a different one if they found this doesnt fit their needs.

For doing so, please contact the main F-Interop developers, architect or integrator for studying these needs.

You can find some reference implementations for the testing tools here:

For the time being only the CoAP interoperability testing tool is publicly available from the F-Interop’s the testing tools.

CoAP Testing Tool git repo

For future reference, you can find the complete list of F-Interop public git repositories at: Explore git repos

Note:

Requirements

Testing tools (potentially the coordination module of the testing tool):

Agent

Introduction

The “agent” is a program a user downloads from the F-Interop website, and which allows him/her to connect the IUT to the F-Interop server. Communication between the agent and the server is authenticated and secure. Through the agent, the F-Interop server can (remotely) interact with the IUT, for example by changing its configuration or injecting packets in the link/interface the IUT is listening to. Similarly, the agent reports to F-Interop the data packets sent from the IUT.

Source code

N.B. this is work in progress

agent’s gitlab repo

Routing key

There can be several agents during a session. Therefore, there is a need to address each agent individually. A user can provide such address by specifying a name while connecting. If this name is already in use the connection fail with a message inviting the user to choose a different name.

Embedded services

Agent’s embeded services have two purposes, first, triggering actions in the IUTs, particularly interesting when automating an IUT, and secondly opening virtual interfaces on the machine which runs them which work as a VPN with F-Interop backend, and other IUTs which are part of the session.

F-Interop can send actions to the agent, with the purpose of triggering a particular action. These are refered as “agent hooks”. The following protocols are used for the agent hooks:

For IUTs which “talk” IP , agents can also open a virtual network interface on a PC, and can used to communicate between users and the F-Interop platform over IP.

The contributor implementing an automated-IUT or bot-like IUT MAY use these agent hooks for automating the whole execution of a test. Note that this is the advance way of using F-Interop and is not mandatory to do so for using F-Interop.

Tun interface

Routing key: control.agent.tun.agent_id

Contents Type Description
_type String Services can be one of the following {“tun.start”, “tun.stop”,“tun.restart”} TBD
ipv4_host String e.g. xxx.xxx.xxx.xxx
ipv4_network String
ipv4_netmask String
ipv6_host String
ipv6_prefix String
{
    "_type" : "tun.start",
    "ipv6_host": "::1",
    "ipv6_prefix": "bbbb"
}

Routing key: data.channel_id.packet.raw Typically, a channel_id can be in the form: source|destination

Contents Type Description
data String packet captured

ZMQ

Routing key: control.agent.zmq.agent_id

An agent can open ZMQ socket and send messages over it.

Contents Type Description
zmq_socket_type String [REQ, PUB]
connect path Where to connect to
data String data to send through the ZMQ socket
{
    "_type" : "agent_hook.zmq", 
    "zmq_socket_type": "REQ",
    "connect": "localhost:50000",
    "data": {"sub": ["cmdToMote"]}
}

HTTP

Routing key: control.agent.http.agent_id

An agent can send an HTTP request and send back the answer.

Contents Type Description
verb String [GET, POST, PUT, DELETE]
URL String URL that the agent will target. It has to be accessible from the host where the agent is running.
data JSON JSON to send
{
    "_type" : "agent_hook.http",
    "url": "http://example.com",
    "verb": "GET",
    "data": {"data": "data"}
}

Ping

Routing key: control.agent.ping.agent_id

An agent can send an ICMP echo request (ping) to a given host. On the background, the agent is performing it through the shell command.

** Request format **

Contents Type Description
version String IPv4 is the default. IPv6 is also available
host String my.example.com
payload String payload to send on the ping request
count int Number of packet to send. Default 1
interface String Network interface to use
{
    "_type": "agent_hook.ping",
    "version": "IPv6",
    "host": "bbbb::1",
    "payload": "cafe",
    "count": 1,
    "interface": "tun0"
}

Graphical interface and other visualization tools

BASICS

SERVICES PROVIDED

OTHER MESSAGES PUSHED INTO THE BUS

OTHER MESSAGES CONSUMED FROM THE BUS

Result repository

Resource Repository (RR)

BASICS

The Resource Repository is the central database that contains information about all the resources of the platform.

API - Services and Events

SERVICES:

The Resource Repository provides the following services:

SERVICES DESCRIPTION
resource_repository.insert Message for inserting/updating resources in the RR.
resource_repository.get Message for querying the RR about resources.

for services request (RPC calls):

Parameters Value Description
exchange ‘default’ TBD
routing_key 'control.resource_repository.service’ TBD
reply_to 'control.resource_repository.service.reply’ TBD
correlation_id str(uuid.uuid4()) TBD

for the reply:

Parameters Value Description
exchange 'default’ TBD
routing_key 'control.resource_repository.service.reply’ TBD
correlation_id same as request TBD

EVENTS:

The Resource Repository generates and publishes the following messages:

MESSAGES PUBLISHED DESCRIPTION
control.resource_repository.error Message used to indicate errors on coordiation component. TBD

SERVICES

resource_repository.insert_resource

INSERT RESOURCE REQUEST EXAMPLE:

 - - -
ROUTING_KEY: control.resource_repository.service
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": "control.resource_repository.service.reply",
    ...
}
 - - -
BODY {
    "_type": "resource_repository.insert_resource",
    "resource":{
        "resource_id": "unique-resource-identifier",
        "privacy_flag": "public",
        "available": true,
        "owner_id": "unique-owner-identifier",
        "ts_creation": 1256953732,
        "ts_update": 1256953732,
        "hardware": {
            "manufacturer": "HW manufacturer",
            "machine_type": "phy",
            "hw_platform": "HW platform (board name)",
            "power_supply": "Power supply",
            "processor": "Processor name",
            "cpu_cache": "CPU cache",
            "ram": "RAM",
            "disk": "Secondary storage info",
            "supported_medias": [
                {
                    "name": "Media name",
                    "version": "Media version",
                    "network_card": "Network card info",
                    "mac_address": "aa:bb:cc:dd:ee:ff",
                    "if_bandwidth": "Interface bandwidth"
                }
            ],
            "other_hardware_info": [
                {"parameter": "info_key", "value": "info_value"}
            ]
        },
        "fw_os": {
            "name": "OS/Firmware name",
            "version": "OS/Firmware version",
            "ip_address": "IP address",
            "supported_protocols_services": [
                {
                    "name": "Protocol name 1",
                    "version": "Protocol version 1",
                    "port": 123,
                    "other_protocol_info": [
                        {"parameter": "info_key", "value": "info_value"}
                    ]
                }
            ],
            "other_fw_os_info": [
                {"parameter": "fw_os_info", "value": "fw_os_value"}
            ]
        },
        "software": {
            "name": "SW name",
            "version": "SW version",
            "role": "client",
            "automatic_flag": "auto",
            "test_environment": "central_server",
            "testing_protocol": {
                "name": "Protocol name 2",
                "version": "Protocol version 2",
                "port": 1234,
                "other_protocol_info": [
                    {"parameter": "info_key", "value": "info_value"}
                ]
            },
            "testing_tool": [
                {
                    "name": "Testing Tool name",
                    "version": "Testing Tool version"
                }
            ],
            "other_software_info": [
                {"parameter": "info_key", "value": "info_value"}
            ]
        },
        "location": {
            "latitude": 11,
            "longitude": 22,
            "country": "CH",
            "city": "City",
            "address": "Address",
            "x": 0,
            "y": 0,
            "z": 0
        }
    }
}
 - - -

INSERT RESOURCE RESPONSE EXAMPLE

 - - -
ROUTING_KEY: control.testcoordination.service.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": null,
    ...
}
 - - -
BODY {
    "_type": "testcoordination.testsuite.getstatus",
    "ok": true
}
 - - -

REQUEST

Contents Type Value
“_type” String “resource_repository.insert_resource”
“resource” Object The resource that needs to be inserted/updated. All fields (see example) are required, if they don’t exist then set String fields to “None” and set Integeer fields to 0.

RESPONSE

Contents Type Value
“_type” String “resource_repository.insert_resource”
“ok” Boolean true if the resource was inserted/updated correcly, false otherwise.
“error_message” String A message describing the error. (Present iff ok = false)
“error_code” Integer TBD (Present iff ok = false)

resource_repository.get_resource

GET RESOURCE REQUEST EXAMPLE:

This query will return all public resources that match all the specified
fields. In other words: all public resources that support 802.11n WiFi,
that have CoAP observe testing tool and that are in Switzerland.

 - - -
ROUTING_KEY: control.resource_repository.service
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": "control.resource_repository.service.reply",
    ...
}
 - - -
BODY {
    "_type": "resource_repository.get_resource",
    "query": {
        "hardware": {
            "supported_medias": [
                {
                    "name": "2.4GHz WiFi",
                    "version": "802.11n"
                }
            ]
        },
        "software": {
            "testing_tool": [
                {
                    "name": "CoAP observe",
                    "version": "RFC 7641"
                }
            ]
        },
        "location": {
            "country": "CH"
        }
    }
}
 - - -

GET RESOURCE RESPONSE EXAMPLE

 - - -
ROUTING_KEY: control.testcoordination.service.reply
 - - -
HEADERS: None
 - - -
PROPS: {
    "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", 
    "content_type": "application/json", 
    "reply_to": null,
    ...
}
 - - -
BODY {
    "_type": "resource_repository.get_resource",
    "ok": true,
    "results": [
        {
            "resource_id": "resource-001-unique-id",
            "owner_id": "user-001-unique-id",
            "privacy_flag": "public",
            "available": true,
            "ts_creation": 1481382022000,
            "ts_update": 1481382022000,
            "hardware": {
                "hw_platform": "HW platform info",
                "manufacturer": "Manufacturer info",
                "machine_type": "phy OR vm",
                "power_supply": "Power supply info",
                "processor": "Processor info",
                "cpu_cache": "CPU cache info",
                "ram": "RAM info",
                "disk": "Secondary storage info",
                "other_hw_info": [
                    {"param": "Extra info name", "value": "Extra info value"},
                    ...
                ],
                "supported_medias": [
                    {
                        "name": "2.4GHz WiFi",
                        "version": "802.11n",
                        "network_card": "None",
                        "mac_address": "aa:bb:cc:dd:ee:ff",
                        "if_bandwidth": "150 Mbit/s"
                    },
                    ...
                ]
            },
            "fw_os": {
                "name": "Firmware/Software name",
                "version": "1.2.3",
                "ip_address": "None",
                "supported_protocols_services": [
                    {
                        "name": "CoAP observe",
                        "version": "RFC 7641",
                        "port": 5683,
                        "other_protocol_info": [
                            {"param": "Extra info name", "value": "Extra info value"},
                            ...
                        ]
                    },
                ]
                "other_fw_os_info": [
                    {"parameter": "fw_os_info", "value": "fw_os_value"},
                    ...
                ]
            },
            "software": {
                "name": "SW name",
                "version": "SW version",
                "role": "client",
                "automatic_flag": "auto",
                "test_environment": "central_server",
                "testing_tool": [
                    {
                        "name": "Tool name",
                        "version": "1.2.3"
                    }
                ]
                "testing_protocol": {
                    "name": "CoAP observe",
                    "version": "RFC 7641",
                    "port": 5683,
                    "other_protocol_info": [
                        {"param": "Extra info name", "value": "Extra info value"},
                        ...
                    ]
                },
                "other_software_info": [
                    {"param": "Extra info name", "value": "Extra info value"},
                    ...
                ]
            },
            "location": {
                "country": "CH",
                "city": "Geneva",
                "address": "None",
                "latitude": 0,
                "longitude": 0,
                "x": 0,
                "y": 0,
                "z": 0
            }
        },
        {
            "resource_id": "resource-123-unique-id",
            "owner_id": "user-002-unique-id",
            "privacy_flag": "public",
            "available": true,
            "ts_creation": 1481382022000,
            "ts_update": 1481382022000,
            "hardware": {
                ...
            },
            "fw_os": {
                ...
            },
            "software": {
                ...
            },
            "location": {
                ...
            }
        },
        ...
    ]
}
 - - -

REQUEST

Contents Type Value
“_type” String “resource_repository.get_resource”
“query” Object An object matching the structure of a resource object. All fields that are present in the query will be used to match resources. All resources that match all the fields present in the query will be returned (see example).

RESPONSE

Contents Type Value
“_type” String “resource_repository.get_resource”
“ok” Boolean true if the query was executed correcly, false otherwise.
“results” Array of Objects The list of resources that match the query. An empty list if no results. (Present iff ok = true)
“error_message” String A message describing the error. (Present iff ok = false)
“error_code” Integer TBD (Present iff ok = false)

Test Coordinator

BASICS

The coordinator is the component that glues all parts together. It’s purpose is to coordinate the test cases execution during a test session. It’s a generic executor of the test case, although at the bootstrapping phase it retrieves a test session context from the test orchestrator.

TEDs example
--- !configuration
configuration_id: COAP_CFG_01_v01
uri: http://f-interop.paris.inria.fr/tests/CoAP_CFG_01
nodes:
  - coap_client
  - coap_server
topology:
  - link_id : link_01
    capture_filter: udp port 5683
    nodes:
      - coap_client
      - coap_server
description:
  - node: coap_server
    message:
      - CoAP servers are requested to offer the following resources
      - - /test
        - Default test resource
        - Should not exceed 64bytes
      - - /seg1/seg2/seg3
        - Long path ressource
        - Should not exceed 64bytes
      - - /query
        - Ressource accepting query parameters
        - Should not exceed 64bytes
        (...)

  - node: coap_client
    message:
      - None

--- !testcase
testcase_id: TD_COAP_CORE_01_v01
uri : http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01_v01
configuration: COAP_CFG_01_v01
objective: Perform GET transaction(CON mode)
pre_conditions: Server offers the resource /test with resource content is not empty that handles GET with an arbitrary payload
references: '[COAP] 5.8.1, 1.2, 2.1, 2.2, 3.1'
sequence:
  - step_id: 'TD_COAP_CORE_01_v01_step_01'
    type: stimuli
    iut : coap_client
    description:
      - Client is requested to send a GET request with
      - Type = 0(CON)
      - Code = 1(GET)

  - step_id: TD_COAP_CORE_01_v01_step_02
    type: check
    description:
      - The request sent by the client contains
      - Type=0 and Code=1
      - Client-generated Message ID(\u2794 CMID)
      - Client-generated Token(\u2794 CTOK)
      - Uri-Path option "test"

  - step_id: TD_COAP_CORE_01_v01_step_03
    type: check
    description:
        - Server sends response containing
        - Code = 2.05(Content)
        - Message ID = CMID, Token = CTOK
        - Content-format option
        - Non-empty Payload

  - step_id: TD_COAP_CORE_01_v01_step_04
    type: verify
    iut: coap_client
    description:
        - Client displays the received information

In the case of interoperability testing, it triggers actions in the following components:

This component implementation is protocol agnostic, once instantiated it gets the specific context from the orchestrator and feeds from the test extended descriptions (TED yaml files). TEDs play a very important role in the architecture, they describe in detail each test case execution. TEDs are meant to be easy to write using YAML language, and they are written by the contributors.

TEDs describe the setup of the test, they include a step by step description of each test case. See example for CoAP

API - Services and Events

SERVICES:

Coordinator provides the following services:

SERVICES DESCRIPTION
testcoordination.testsuite.getstatus Message for debugging purposes. The coordination component returns the status of the execution
testcoordination.testsuite.gettestcases Message for requesting the list of test cases included in the test suite.

for services request (RPC calls):

Parameters Value Description
exchange ‘default’ TBD
routing_key 'control.testcoordination.service’ TBD
reply_to 'control.test.service.reply’ TBD
correlation_id str(uuid.uuid4()) TBD

for the reply:

Parameters Value Description
exchange 'default’ TBD
routing_key 'control.testcoordination.service.reply’ TBD
correlation_id same as request TBD

EVENTS:

Coordinator listens and consumes the following messages from the bus:

MESSAGES CONSUMED DESCRIPTION
testcoordination.testsuite.start Message for triggering start of test suite. The command is given by one of the users of the session.
testcoordination.testsuite.abort Message for aborting the ongoing test session.
testcoordination.testcase.skip Message for skipping a test case. Coordinator passes to the next test case if there is any left.
testcoordination.testcase.select Message for selecting next test case to be executed. Allows the user to relaunch an already executed test case.
testcoordination.testcase.start Message for triggering the start of the test case. The command is given by one of the users of the session.
testcoordination.testcase.restart Message for triggering the restart of a test case (TBD any of the participants MAY send this?)
testcoordination.testcase.finish (# TBD this may not be necesary, as we can detect the end of the TC with the Steps info) Message for triggering the end of a test case. This command is given by one of the users of the session.
testcoordination.step.stimuli.executed Message pushed by UI or agent indicating the stimuli was executed.
testcoordination.step.check.response TBD (for step_by_step analysis).
testcoordination.step.verify.response Message pushed by UI or agent providing the response to verify step.

Coordinator generates and publishes the following messages:

MESSAGES PUBLISHED DESCRIPTION
testcoordination.testcase.next Indicates next testcase to be executed
testcoordination.testsuite.finished Indicates there’s no more test cases to execute
testcoordination.step.execute Provides info about next step to execute, message used by user interfaces and automated-IUTs
testcoordination.testcase.verdict Provides final verdict of the test case
testcoordination.error Message used to indicate errors on coordiation component
testcoordination.info Message for diffusing information about the coordination component

SERVICES

testcoordination.testsuite.getstatus

REQUEST:

 - - -
ROUTING_KEY: control.testcoordination.service
 - - -
HEADERS: None
 - - -
PROPS: {"headers": null, "timestamp": null, "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", "expiration": null, "content_encoding": null, "priority": null, "message_id": null, "cluster_id": null, "app_id": null, "user_id": null, "content_type": "application/json", "delivery_mode": null, "reply_to": "control.testcoordination.service.reply"}
 - - -
BODY {"_type": "testcoordination.testsuite.getstatus"}
 - - -

RESPONSE

 - - -
ROUTING_KEY: control.testcoordination.service.reply
 - - -
HEADERS: None
 - - -
PROPS: {"headers": null, "timestamp": null, "correlation_id": "1a58de57-176e-41c4-bca3-e81b968142c9", "expiration": null, "content_encoding": null, "priority": null, "message_id": null, "cluster_id": null, "app_id": null, "user_id": null, "content_type": "application/json", "delivery_mode": null, "reply_to": null}
 - - -
BODY {
        "_type": "testcoordination.testsuite.getstatus",
        "ok": true,
        "status": ["Current test case: TD_COAP_CORE_01_v01, state: executing",
                    "Current step [
                            ('step_id', 'TD_COAP_CORE_01_v01_step_04'),
                            ('step_type', 'verify'),
                            ('step_info', ['Client displays the received information']),
                            ('step_state', 'executing'),
                            ('iut', 'coap_client'),
                            ('iut_mode', 'user_assisted')
                            ]
                   "]
      }
 - - -

REQUEST

Contents Type Value
“_type” String “testcoordination.testsuite.getstatus”

RESPONSE

Contents Type Value
“_type” String type of service requested
“ok” bool true or false value, indicating if the service was processed correcly
“status” Array of String WIP

testcoordination.testsuite.gettestcases

REQUEST:

ROUTING_KEY: control.testcoordination.service
 - - -
HEADERS: None
 - - -
PROPS: {"headers": null, "timestamp": null, "correlation_id": "23a77905-d534-476b-bd14-b14d8dc99124", "expiration": null, "content_encoding": null, "priority": null, "message_id": null, "cluster_id": null, "app_id": null, "user_id": null, "content_type": "application/json", "delivery_mode": null, "reply_to": "control.testcoordination.service.reply"}
 - - -
BODY {"_type": "testcoordination.testsuite.gettestcases"}
 - - -

RESPONSE


 - - -
ROUTING_KEY: control.testcoordination.service.reply
 - - -
HEADERS: None
 - - -
PROPS: {"headers": null, "timestamp": null, "correlation_id": "23a77905-d534-476b-bd14-b14d8dc99124", "expiration": null, "content_encoding": null, "priority": null, "message_id": null, "cluster_id": null, "app_id": null, "user_id": null, "content_type": "application/json", "delivery_mode": null, "reply_to": null}
 - - -
BODY {
        "_type": "testcoordination.testsuite.gettestcases",
        "ok": true,
        "tc_list":
                    [
                        {
                        "testcase_id": "TD_COAP_CORE_01_v01",
                        "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01_v01",
                        "objective": "Perform GET transaction(CON mode)",
                        "state": "executing"
                        },

                        {
                        "testcase_id": "TD_COAP_CORE_02_v01",
                        "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_02",
                        "objective": "Perform DELETE transaction (CON mode)",
                        "state": null
                        },
                        {
                        "testcase_id": "TD_COAP_CORE_03_v01",
                        "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_03",
                        "objective": "Perform PUT transaction (CON mode)",
                        "state": null
                        }
                    ]
      }
 - - -

REQUEST

Contents Type Value
“_type” String type of service requested

RESPONSE

Contents Type Value
“_type” String type of service requested
“ok” bool true or false value, indicating if the service was processed correcly
“tc_list” Array list of test cases implemented in the testing tool

MESSAGES CONSUMED

testcoordination.testsuite.start

 - - -

ROUTING_KEY: control.testcoordination

 - - -

HEADERS: None

 - - -

PROPS: {"user_id": null, "correlation_id": "b35e1d49-facf-4623-af8e-2e069ed53d1d", "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": "control.testcoordination.reply", "content_type": "application/json", "timestamp": null, "expiration": null}

 - - -

BODY {"_type": "testcoordination.testsuite.start"}

 - - -

Contents Type Value
“_type” String “testcoordination.testcase.start”

testcoordination.testsuite.abort

Contents Type Value
“_type” String “testcoordination.testcase.abort”

testcoordination.testcase.skip

Contents Type Value
“_type” String “testcoordination.testcase.skip”

testcoordination.testcase.select

Contents Type Value
“_type” String “testcoordination.testcase.select”

testcoordination.testcase.start

 - - -

ROUTING_KEY: control.testcoordination

 - - -

HEADERS: None

 - - -

PROPS: {"user_id": null, "correlation_id": "8b1b13df-bc31-49be-85f7-1e0058a35e89", "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": "control.testcoordination.reply", "content_type": "application/json", "timestamp": null, "expiration": null}

 - - -

BODY {"_type": "testcoordination.testcase.start"}

 - - -
Contents Type Value
“_type” String “testcoordination.testcase.start”

testcoordination.testcase.restart

Contents Type Value
“_type” String “testcoordination.testcase.restart”

testcoordination.testcase.finish

 - - -

ROUTING_KEY: control.testcoordination

 - - -

HEADERS: None

 - - -

PROPS: {"user_id": null, "correlation_id": null, "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": null, "content_type": "application/json", "timestamp": null, "expiration": null}

 - - -

BODY {"_type": "testcoordination.testcase.finished", "message": "Testcase TD_COAP_CORE_01_v01 finished", "testcase_id": "TD_COAP_CORE_01_v01", "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01_v01", "objective": "Perform GET transaction(CON mode)", "state": "finished"}

 - - -

Contents Type Value
“_type” String “testcoordination.testcase.finish”

testcoordination.step.stimuli.executed

 - - -

ROUTING_KEY: control.testcoordination

 - - -

HEADERS: None

 - - -

PROPS: {"user_id": null, "correlation_id": "f382c0af-4c24-45c5-9fce-e2286fb845d4", "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": "control.testcoordination.reply", "content_type": "application/json", "timestamp": null, "expiration": null}

 - - -

BODY {"_type": "testcoordination.step.stimuli.executed"}

 - - -
Contents Type Value
“_type” String “testcoordination.step.stimuli.executed”

testcoordination.step.check.response

WIP
Contents Type Value
“_type” String “ttestcoordination.step.check.response”

testcoordination.step.verify.response

 - - -

ROUTING_KEY: control.testcoordination

 - - -

HEADERS: None

 - - -

PROPS: {"user_id": null, "correlation_id": "e19f1f3c-622a-4b65-ade9-ebb94a09642b", "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": "control.testcoordination.reply", "content_type": "application/json", "timestamp": null, "expiration": null}

 - - -

BODY {"_type": "testcoordination.step.verify.response", "verify_response": true}

 - - -
Contents Type Value
“_type” String “testcoordination.step.verify.response”
“verify_response” boolean answer to the True or False question

testcoordination.testcase.next

 - - -

ROUTING_KEY: control.testcoordination

 - - -

HEADERS: None

 - - -

PROPS: {"user_id": null, "correlation_id": null, "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": null, "content_type": "application/json", "timestamp": null, "expiration": null}

 - - -

BODY {
    "_type": "testcoordination.testcase.next",
    "message": "Next test case to be executed is TD_COAP_CORE_01_v01",
    "testcase_id": "TD_COAP_CORE_01_v01",
    "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01_v01",
    "objective": "Perform GET transaction(CON mode)",
    "state": null
    }
Contents Type Value
“_type” String “testcoordination.testcase.start”

testcoordination.testsuite.finished

Contents Type Value
“_type” String “testcoordination.testsuite.finished”

MESSAGES PRODUCED

testcoodrination.testcase.configuration

 - - -
ROUTING_KEY: control.testcoordination
 - - -
HEADERS: None
 - - -
PROPS: {"user_id": null, "content_type": "application/json", "delivery_mode": null, "content_encoding": null, "headers": null, "timestamp": null, "correlation_id": null, "reply_to": null, "cluster_id": null, "message_id": null, "app_id": null, "expiration": null, "priority": null}
 - - -
BODY {
        "_type": "testcoordination.testcase.configuration",
        "configuration_id": "COAP_CFG_01_v01",
        "node": "coap_server",
        "message": ["CoAP servers are requested to offer the following resources",
                        ["/test", "Default test resource", "Should not exceed 64bytes"],
                        ["/seg1/seg2/seg3", "Long path ressource", "Should not exceed 64bytes"],
                        ["/query", "Ressource accepting query parameters", "Should not exceed 64bytes"],
                        ["/separate", "Ressource which cannot be served immediately and which cannot be acknowledged in a piggy-backed way", "Should not exceed 64bytes"],
                        ["/large", "Large resource (>1024 bytes)", "shall not exceed 2048bytes"],
                        ["/large_update", "Large resource that can be updated using PUT method (>1024 bytes)", "shall not exceed 2048bytes"],
                        ["/large_create", "Large resource that can be  created using POST method (>1024 bytes)", "shall not exceed 2048bytes"],
                        ["/obs", "Observable resource which changes every 5 seconds", "shall not exceed 2048bytes"],
                        ["/.well-known/core", "CoRE Link Format", "may require usage of Block options"]
                    ]
     }
 - - -
Contents Type Value
“_type” String “testcoordination.testcase.configuration”

testcoordination.step.execute

for a STIMULI step:

 - - -

ROUTING_KEY: control.testcoordination

 - - -

HEADERS: None

 - - -

PROPS: {"user_id": null, "correlation_id": null, "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": null, "content_type": "application/json", "timestamp": null, "expiration": null}

 - - -

BODY {
        "_type": "testcoordination.step.execute",
        "message": "Next test step to be executed is TD_COAP_CORE_01_v01_step_01",
        "step_id": "TD_COAP_CORE_01_v01_step_01",
        "step_type": "stimuli",
        "step_info": ["Client is requested to send a GET request with", "Type = 0(CON)", "Code = 1(GET)"],
        "step_state": "executing",
        "iut": "coap_client",
        "iut_mode": "user_assisted"
     }

 - - -

for a VERIFY step:

 - - -

ROUTING_KEY: control.testcoordination

 - - -

HEADERS: None

 - - -

PROPS: {"user_id": null, "correlation_id": null, "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": null, "content_type": "application/json", "timestamp": null, "expiration": null}

 - - -

BODY {"_type": "testcoordination.step.execute", "message": "Next test step to be executed is TD_COAP_CORE_01_v01_step_04", "step_id": "TD_COAP_CORE_01_v01_step_04", "step_type": "verify", "step_info": ["Client displays the received information"], "step_state": "executing", "iut": "coap_client", "iut_mode": "user_assisted"}

 - - -

Contents Type Value
“_type” String “ttestcoordination.step.execute”
WIP WIP WIP

testcoordination.testcase.verdict

 - - -

ROUTING_KEY: control.testcoordination

 - - -

HEADERS: None

 - - -

PROPS: {"user_id": null, "correlation_id": null, "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": null, "content_type": "application/json", "timestamp": null, "expiration": null}

 - - -

BODY {
        "_type": "testcoordination.testcase.verdict",
        "verdict": "error",
        "description": "Test Analysis Tool returned an empty analysis report",
        "partial_verdicts": [
                                ["TD_COAP_CORE_01_v01_step_02", null, "CHECK postponed", ""],
                                ["TD_COAP_CORE_01_v01_step_03", null, "CHECK postponed", ""],
                                ["TD_COAP_CORE_01_v01_step_04", "pass", "VERIFY step: User informed that the information was displayed correclty on his/her IUT", ""]],
        "testcase_id": "TD_COAP_CORE_01_v01",
        "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01_v01",
        "objective": "Perform GET transaction(CON mode)",
        "state": "finished"
    }

Contents Type Value
“_type” String “testcoordination.testcase.verdict”

testcoordination.error

TBD
Contents Type Value
“_type” String “testcoordination.error”

testcoordination.info

 - - -
ROUTING_KEY: control.analysis.info
 - - -
HEADERS: None
 - - -
PROPS: {"user_id": null, "content_type": "application/json", "delivery_mode": null, "content_encoding": null, "headers": null, "timestamp": null, "correlation_id": null, "reply_to": null, "cluster_id": null, "message_id": null, "app_id": null, "expiration": null, "priority": null}
 - - -
BODY { "_type": "analysis.info", "value": "some info!"}
 - - -
Contents Type Value
“_type” String “testcoordination.info”

testcoordination.testcase.select

Request

{
    "_type" : "testcoordination.testcase.select",
    WIP
}
Contents Type Value
“_type” String “testcoordination.testcase.select”

OTHER MESSAGES PUBLISHED

testcoordination.ready

Signal sent when coordinator is ready

 - - - 
ROUTING_KEY: control.session.bootstrap
 - - - 
HEADERS: None
 - - - 
PROPS: {"content_encoding": null, "expiration": null, "headers": null, "cluster_id": null, "delivery_mode": null, "timestamp": null, "reply_to": null, "user_id": null, "message_id": null, "priority": null, "content_type": "application/json", "correlation_id": null, "app_id": null}
 - - - 
BODY {"message": "test_coordinator is up!", "_type": "testcoordination.ready"}
 - - - 
* * * * * * * * * * * * * * * * * * * * * 

testingtool.ready

Signal sent when all components of testing tool are ready.

 - - - 
ROUTING_KEY: control.session.bootstrap
 - - - 
HEADERS: None
 - - - 
PROPS: {"content_encoding": null, "expiration": null, "headers": null, "cluster_id": null, "delivery_mode": null, "timestamp": null, "reply_to": null, "user_id": null, "message_id": null, "priority": null, "content_type": "application/json", "correlation_id": null, "app_id": null}
 - - - 
BODY {"message": "All testing tool components are ready!", "_type": "testingtool.ready"}
 - - - 
* * * * * * * * * * * * * * * * * * * * * 

OTHER MESSAGES CONSUMED

Sniffer

BASICS

This is the component handling the sniffing of the traffic exchanged between the participants of the test session.

Privacy considerations: After session over, captures files MAY must be sent to the secured database(TBD), then all local files eventually created by the component MUST be deleted.

SERVICES PROVIDED

METHOD DESCRIPTION
sniffing.start Action for starting the capture.
sniffing.stop Action for finishing the capture.
sniffing.getcapture Service for retrieving a capture within the session execution scope.

control.sniffing.service.start

Request format

Example service start sniffer

 - - - 

ROUTING_KEY: control.sniffing.service

 - - - 

HEADERS: None

 - - - 

PROPS: {"user_id": null, "correlation_id": "af1a92dd-9b5e-4301-96b1-359506a83e94", "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": "control.sniffing.service.reply", "content_type": "application/json", "timestamp": null, "expiration": null}

 - - - 

BODY {"_type": "sniffing.start", "capture_id": "TD_COAP_CORE_01", "filter_if": "lo0", "filter_proto": "udp port 5683"}

 - - - 
Contents Type Description
_type string “sniffing_service”
service string Accepted services: { “start”, “stop”,“restart”,“configure” } (REQUIRED)
filter-if string Accepted interfaces: { “data.channel_xx”, “lo”, TBD} (REQUIRED for start and configure)
filter-proto string Filter parameter for tshark (OPTIONAL)
filename string Filename for the capture file. Same to be reused for the getcapture API call (REQUIRED)

Response format

Response format (success)

 - - - 

ROUTING_KEY: control.sniffing.service.reply

 - - - 

HEADERS: None

 - - - 

PROPS: {"user_id": null, "correlation_id": "af1a92dd-9b5e-4301-96b1-359506a83e94", "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": null, "content_type": "application/json", "timestamp": null, "expiration": null}

 - - - 

BODY {"_type": "sniffing.start", "ok": true}

Response format (EXCEPTION)

{
    "routing_key": "control.sniffing.info",
    "src_id": "amqp://sniffer@finterop.org",
    "msg_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "token": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "timestamp": 1469806702,
    "payload": {
                    "_type": "sniffing_error",
                    "value": "",
                    "error_code" : 1,
                    "message": "Interface not existent",
                }

}
Contents Type Description
_type string “sniffing_info” or “sniffing_error”
value string TBD (REQUIRED)
message string description of the services executed or problem encountered in case of error. Intended to be used in GUI & for logging purposes
error_code string TBD (REQUIRED in case of error)

control.sniffing.service.finish

Request format

Request for control.sniffing.service.finish

{

        [
                    "_type":"sniffing_service",
                    "value":"stop",
                    "filter-if":"lo",
        ]

}
Contents Type Description
tbd tbd tbd
tbd tbd tbd
filename string tbd

Response format

Response format (success)

{

        [
            TBD
        ]
}

Response format (fail)

{

            TBD
        ]
}

Response format (exception)

{

        [
            TBD
        ]
}
Contents Type Description
tbd tbd tbd
tbd tbd tbd
tbd tbd tbd

control.sniffing.service.getcapture

Request format

Request for control.sniffing.service.getcapture

 - - - 

ROUTING_KEY: control.sniffing.service

 - - - 

HEADERS: None

 - - - 

PROPS: {"user_id": null, "correlation_id": "0a069462-cb40-42e4-b828-ebf3d828f0fc", "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": "control.sniffing.service.reply", "content_type": "application/json", "timestamp": null, "expiration": null}

 - - - 

BODY {"_type": "sniffing.getcapture", "capture_id": "TD_COAP_CORE_01"}

 - - - 

Service for retrieving a capture within the session execution scope.

Contents Type Description
_type string sniffing.getcapture
capture_id string Capture to be retrieved. If no capture_id provided, then either undergoing test case capture, or last captured test case is returned.

Response format

Response format (success)

 - - - 

ROUTING_KEY: control.sniffing.service.reply

 - - - 

HEADERS: None

 - - - 

PROPS: {"user_id": null, "correlation_id": "0a069462-cb40-42e4-b828-ebf3d828f0fc", "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": null, "content_type": "application/json", "timestamp": null, "expiration": null}

 - - - 

BODY {"_type": "sniffing.getcapture", "ok": true, "file_enc": "pcap_base64", "filename": "TD_COAP_CORE_01.pcap", "value": "1MOyoQIABAAAAAAAAAAAAMgAAAAAAAAA"}

 - - - 

Response format (fail)

{
TBD
}

Response format (exception)

{
    "",
    // ...
    "content":
        [
            TBD
        ]
}
Contents Type Description
tbd tbd tbd
tbd tbd tbd
tbd tbd tbd

OTHER MESSAGES PUSHED INTO THE BUS

control.sniffing.debug

see common events

control.sniffing.info

see common events

control.sniffing.warning

see common events

control.sniffing.error

see common events

OTHER MESSAGES CONSUMED FROM THE BUS

None

Test Analyser (TAT)

BASICS

The TAT provides F-Interop with some protocol analysis services. The TAT is the component that performs the verification of traces during a test session. F-Interop provides TATs for different protocols under test.

The TAT issues three types of verdicts:

The architecture supports TATs which perform

TATs are created both by the F-Interop core team and by external contributors. The F-Interop API specification defines the format of the messages a TAT will receive from the Event Bus, and the format of the messages it will produce

TAT may implement it’s own dissector, and in that case they implement all the services described for the Dissector component.

SERVICES PROVIDED

Services provided by TAT:

METHOD DESCRIPTION
analysis.testcase.analyze Analyze frames from one or more PCAP files and returns the verdict of specified test case.
analysis.testsuite.gettestcases Return the implemented test cases by the testing analysis tool.

analysis.testcase.analyze

Method to launch an analysis from a pcap file or a token if the pcap file has already been provided. The method need a token or a pcap_file but doesn’t allow someone to provide both.

Request analysis.testcase.analyze

 - - - 

ROUTING_KEY: control.analysis.service

 - - - 

HEADERS: None

 - - - 

PROPS: {"user_id": null, "correlation_id": "411d520c-b051-422b-84bc-72c2ab5c49b5", "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": "control.analysis.service.reply", "content_type": "application/json", "timestamp": null, "expiration": null}

 - - - 

BODY {
        "_type": "analysis.testcase.analyze", 
        "testcase_id": "TD_COAP_CORE_01", 
        "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01_v01", 
        "file_enc": "pcap_base64",
        "filename": "TD_COAP_CORE_01.pcap", 
        "value": "1MOyoQIABAAAAAAAAAAAAAAABAAAAAAAGfdPV8tZCAAtAAAALQAAAAIAAABFAAApcawAAEARAAB/AAABfwAAAdYxFjMAFf4oQgGqAWLatHRlc3TBAhn3T1fHrAgAXgAAAF4AAAACAAAARQAAWlLmAABAEQAAfwAAAX8AAAEWM9YxAEb+WWJFqgFi2sAhHpEC/1R5cGU6IDAgKENPTikKQ29kZTogMSAoR0VUKQpNSUQ6IDQzNTIxClRva2VuOiA2MmRh"
     }

 - - - 

Response format when TEST PASSED

 - - -
ROUTING_KEY: control.analysis.service.reply
 - - -
HEADERS: None
 - - -
PROPS: {"delivery_mode": null, "message_id": null, "priority": null, "correlation_id": "7c8ed528-5441-4bbf-8839-dadf661c8181", "timestamp": null, "content_encoding": null, "app_id": null, "content_type": "application/json", "expiration": null, "cluster_id": null, "headers": null, "reply_to": null, "user_id": null}
 - - -
BODY {
        "_type": "analysis.testcase.analyze.verdict",
        "ok": true,
        "verdict": "pass",
        "description": "Some short description of the test case analysis",
        "review_frames": [],
        "partial_verdicts": [
                            ["pass", "<Frame   1: [127.0.0.1 -> 127.0.0.1] CoAP [CON 43521] GET /test> Match: CoAP(type=0, code=1)"],
                            ["pass", "<Frame   2: [127.0.0.1 -> 127.0.0.1] CoAP [ACK 43521] 2.05 Content > Match: CoAP(code=69, mid=0xaa01, tok=b'b\\xda', pl=Not(b''))"],
                            ["pass", "<Frame   2: [127.0.0.1 -> 127.0.0.1] CoAP [ACK 43521] 2.05 Content > Match: CoAP(opt=Opt(CoAPOptionContentFormat()))"]
                            ],
        "token": "NOT YET IMPLEMENTED",
        "testcase_id": "TD_COAP_CORE_01",
        "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01_v01"}
 - - -

Response format when TEST inconclusive

 - - - 

ROUTING_KEY: control.analysis.service.reply

 - - - 

HEADERS: None

 - - - 

PROPS: {"user_id": null, "correlation_id": "411d520c-b051-422b-84bc-72c2ab5c49b5", "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": null, "content_type": "application/json", "timestamp": null, "expiration": null}

 - - - 

BODY {
        "_type": "analysis.testcase.analyze.verdict", 
        "ok": true, 
        "verdict": "inconclusive", 
        "description": "no frames correlation with indicated tescase", 
        "review_frames": [], 
        "partial_verdicts": [], 
        "token": "NOT YET IMPLEMENTED", 
        "testcase_id": "TD_COAP_CORE_01", 
        "testcase_ref": "http://f-interop.paris.inria.fr/tests/TD_COAP_CORE_01_v01"
     }

 - - - 

Response format when TEST error

 - - -

ROUTING_KEY: control.analysis.service.reply

 - - -

HEADERS: None

 - - -

PROPS: {"user_id": null, "correlation_id": "411d520c-b051-422b-84bc-72c2ab5c49b5", "message_id": null, "delivery_mode": null, "cluster_id": null, "content_encoding": null, "headers": null, "app_id": null, "priority": null, "reply_to": null, "content_type": "application/json", "timestamp": null, "expiration": null}
 - - -

BODY {
        "_type": "analysis.testcase.analyze",
        "ok": false
        "message": "Empty PCAP file received",
        "error_code" : "TBD"
      }

 - - -

Request format

Contents Type Description
_type tbd tbd
pcap_file tbd PCAP file format with the frames related to the test case execution between the IUTs (may contain noise frames also, TBD how we treat this issue). Some test cases may require to pass a list of pcap files (PCAP or TOKEN REQUIRED)
token tbd A token associated to a previous analyzed pcap file. Allows to retrieve results and frames without giving the file again (PCAP or TOKEN REQUIRED)
testcase_id tbd Test case id, corresponding to the test description and test cases naming conventions (REQUIRED)

Response format

The response format for this service has several objects as contents. (see example on the right) WIP

Contents Type Description
_type tbd tbd
token Object (token) A token associated to a previous analyzed pcap file. Allows to retrieve results and frames without giving the file again (PCAP or TOKEN REQUIRED)

Defined Objects

verdict WIP (IoP and conformance testing)

Content Type Description
_type String tbd
value String The verdict is one of the following {“inconclusive”, “pass”, “fail”, “error”}
description String The description of verdict result
review_frames [Number] The important frames id associated to the analysis

analysis.testsuite.gettestcases

Method to get the test cases that are implemented in the testing analysis tool.

Request format

** WIP Request control.analysis.service**

{
TBD

}
Contents Type Description
None None None

Response format

Respose (success) WIP

{

TBD
}

Respose (fail) WIP

{
TBD
}
Contents Type Description
list of test cases TBD TBD

OTHER MESSAGES PUSHED INTO THE BUS

OTHER MESSAGES CONSUMED FROM THE BUS

Protocol Dissector

BASICS

SERVICES PROVIDED

METHOD DESCRIPTION
dissection.dissectcapture Dissects frames provided as input in PCAP file, returns a json file containing the different fields of the specified protocol. Generates Token for future interactions.
dissection.getframes Return all the frames -or selection if specified- of a previously dissected file. Token for context identification needs to be provided.

dissection.dissectcapture

Request dissection.dissectcapture

 - - -
ROUTING_KEY: control.dissection.service
 - - -
HEADERS: None
 - - -
PROPS: {"content_encoding": null, "timestamp": null, "user_id": null, "expiration": null, "message_id": null, "priority": null, "headers": null, "delivery_mode": null, "correlation_id": "4afac634-3fd2-434b-b3b2-6c8e2600c889", "content_type": "application/json", "app_id": null, "cluster_id": null, "reply_to": "control.dissection.service.reply"}
 - - -
BODY {
        "_type": "dissection.dissectcapture",
        "value": "1MOyoQIABAAAAAAAAAAAAAAABAAAAAAAGfdPV8tZCAAtAAAALQAAA
            AIAAABFAAApcawAAEARAAB/AAABfwAAAdYxFjMAFf4oQgGqAWLatHRlc3TBAhn3
            T1fHrAgAXgAAAF4AAAACAAAARQAAWlLmAABAEQAAfwAAAX8AAAEWM9YxAEb+WWJ
            FqgFi2sAhHpEC/1R5cGU6IDAgKENPTikKQ29kZTogMSAoR0VUKQpNSUQ6IDQzNT
            IxClRva2VuOiA2MmRh",
        "filename": "TD_COAP_CORE_01.pcap",
        "file_enc": "pcap_base64",
        "protocol_selection": "coap"
     }
 - - -

Response dissection.dissectcapture

 - - -
ROUTING_KEY: control.dissection.service.reply
 - - -
HEADERS: None
 - - -
PROPS: {"content_encoding": null, "timestamp": null, "user_id": null, "expiration": null, "message_id": null, "priority": null, "headers": null, "delivery_mode": null, "correlation_id": "4afac634-3fd2-434b-b3b2-6c8e2600c889", "content_type": "application/json", "app_id": null, "cluster_id": null, "reply_to": null}
 - - -
BODY {

    "_type": "dissection.dissectcapture",
    "ok": true,
    "token": "sPb3J-M4f7c0V0q5NhG58WnsrN0",
    "frames": [
        {
            "_type": "frame",
            "id": 1,
            "timestamp": 1464858393.547275,
            "error": null,
            "protocol_stack": [
                {
                    "_type": "protocol",
                    "_protocol": "NullLoopback",
                    "AddressFamily": "2",
                    "ProtocolFamily": "0"
                },
                {
                    "_type": "protocol",
                    "_protocol": "IPv4",
                    "Version": "4",
                    "HeaderLength": "5",
                    "TypeOfService": "0x00",
                    "TotalLength": "41",
                    "Identification": "0x71ac",
                    "Reserved": "0",
                    "DontFragment": "0",
                    "MoreFragments": "0",
                    "FragmentOffset": "0",
                    "TimeToLive": "64",
                    "Protocol": "17",
                    "HeaderChecksum": "0x0000",
                    "SourceAddress": "127.0.0.1",
                    "DestinationAddress": "127.0.0.1",
                    "Options": "b''"
                  },
                  {
                    "_type": "protocol",
                    "_protocol": "UDP",
                    "SourcePort": "54833",
                    "DestinationPort": "5683",
                    "Length": "21",
                    "Checksum": "0xfe28"
                  },
                  {
                    "_type": "protocol",
                    "_protocol": "CoAP",
                    "Version": "1",
                    "Type": "0",
                    "TokenLength": "2",
                    "Code": "1",
                    "MessageID": "0xaa01",
                    "Token": "b'b\\xda'",
                    "Options": [
                        {
                            "Option": "CoAPOptionUriPath",
                            "Delta": "11",
                            "Length": "4",
                            "Value": "test"
                         },
                         {
                            "Option": "CoAPOptionBlock2",
                            "Delta": "12",
                            "Length": "1",
                            "Number": "0",
                            "M": "0",
                            "SizeExponent": "2"
                          }
                         ],
                    "Payload": "b''"
                  }
                ]
            },
        {
            "_type": "frame",
            "id": 2,
            "timestamp": 1464858393.568519,
            "error": null,
            "protocol_stack":
                [
                    {"_type": "protocol",
                    "_protocol": "NullLoopback",
                    "AddressFamily": "2",
                    (...)
                (..)
            (..)

Request format

Content Type Description
pcap_file Object(pcap_file) PCAP file format. (PCAP or TOKEN REQUIRED)
token Object(token) A token associated to a previous analyzed/dissected pcap file. Allows to retrieve results and frames without giving the file again (PCAP or TOKEN REQUIRED)

Response format

general header | frames | protocols objects (TODO DRY)

Contents Type Description
frames [frame] Returns string representation of the messages in the PCAP file.
token String A token associated to a previous analyzed/dissected pcap file. Allows to retrieve results and frames without giving the file again

dissection.getframes

Method to get the frames of a previously dissected pcap file. If frame_id is provided, return a single frame which id is the one specified.

Request format

Contents Type Description
token string The token provided after the previously dissected pcap file (REQUIRED)
protocol_selection string Protocol name that filters the response.
frame_id number The id of the frame that we want to retrieve (OPTIONAL)

Example when protocol_selection = coap, only frames containing CoAP messages will be returned. If parameter not present or value null, it will return all the frames (OPTIONAL)

Response format

The response for a control.dissection.service.getDissectedFrames is generated encapsulating:

general header | frames | protocols objects

Frame

Contents Type Description
_type String “frame”
id Number The id of the frame
timestamp Number The float representation of the time when this frame got caught
error String A null object if no error occured otherwise the error message itself
protocol_stack List[Protocol] The list of the protocols contained in this frame’s stack

Description here about protocol object

Protocol

Contents Type Description
_type String “protocol”
_protocol String The name of the protocol
[field] String A mapping of string to string, representing the other fields that are parts of a this protocol’s datas and their associated values

Field can be any string which correspond to a protocol field

Response format

{
    //TBD
        [
            {
                "_type": "token",
                "value": "ayVMgJQiQICOCqBKE7pV7qVzU6k="
            },
            {
                "_type": "frame",
                "id": 2,
                "timestamp": 1465477397.7124586,
                "error": "",
                "protocol_stack":
                    [
                        {
                            "_type": "protocol",
                            "_protocol": "proto name",
                            "Field_1_name": "sth",
                            "Field_2_name": "sth_else"
                            //...
                        },
                        //...
                        {
                            "_type": "protocol",
                            "_protocol": "proto name",
                            "Field_1_name": "sth",
                            "Field_2_name": "sth_else"
                            //...
                        }
                    ]
            }
        ]
}

OTHER MESSAGES PUSHED INTO THE BUS

OTHER MESSAGES CONSUMED FROM THE BUS

Packet Generator

BASICS

In some conformance tests, a packet generator component can be used to generate and send packets towards the IUT. The component can for example implement the behavior of a CoAP client when the IUT implements a CoAP server. F-Interop client can purposely generate wrongly formatted messages to verify the correct behavior of the IUT on this situations.

SERVICES PROVIDED

OTHER MESSAGES PUSHED INTO THE BUS

OTHER MESSAGES CONSUMED FROM THE BUS

Other common event

data.channel_id.packet.raw

{
    "routing_key": "data.channel_01.packet.raw",
    "src_id": "amqp://agent_id@finterop.org",
    "msg_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "correlation_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "timestamp": 1469806702,
    "payload": {
                        "_type" : "packet_raw",
                        "mimetype" : "application/octet-stream",
                         "metadata" : [ ],
                         "value" :   "TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyByZWFzb24sIGJ1dCBieSB0aGlz"
               }
}

Sending a packet (raw) into channel xx of the data plane

Contents Type Description
_type String “packet_raw”(REQUIRED)
tbd tbd tbd
tbd tbd tbd
value BASE64 Raw packet captured by the interface in base64 encoding (REQUIRED)

control.component_id.info

(Logging, change of states of a component, anwswer to a service request, etc)

{
    "routing_key": "control.component_id.info",
    "src_id": "amqp://component_id@finterop.org",
    "msg_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "correlation_id": "9b70cd6b-4a2e-45f7-bb28-12cfd9788f16",
    "timestamp": 1469806702,
    "payload": {
                    "_type": "info",
                    "message": "This component X logging a message!"
                }
 }
Contents Type Description
_type String “info”, “warning”, “error”, “debug” (REQUIRED)
error_code number Code number for compute automatically errors (REQUIRED for ERROR type) see error_codes
value String This field is REQUIRED when message is originated by a service request. (REQUIRED)
message String Message to be logged or displayed to the user in GUI components (REQUIRED)

Agent kick-off document

Purpose of the document

With this doc we expect to help people understand what the agent does, and how to deploy it

Objective

Create a sort of IPv6 based VPN between two computers (yes two computers, and not two networks, we are not there yet).

What can I do with that? Send any IPv6 packets from one machine to the other (both ways), e.g. ping6, CoAP ping, CoAP get, etc

Experiment setup

Screenshot

examples for the ipv6 communicating piece of software:

for the lazy ones -> just using native implementation of ping6 would do..

supported OSs

Installation

  1. before anything make sure to have:

    i. if agent running on MacOs:

    tuntap driver needed: brew install Caskroom/cask/tuntap

    ii. python2 -> for agent

    iii. python3 -> packet-router

  2. agent:

    i. download agent code: https://gitlab.distantaccess.com/f-interop-contributors/agent

    ii. read README.md

  3. packet-router

    i. download packet-router code: https://gitlab.distantaccess.com/fsismondi/coap_testing_tool

    ii. don’t read README.md, we just need the packet router component from this repo

  4. you need a rabbit mq session (which is basically a vhost, and users with some permissions on it).

    Either: i. ask a f-interop developer (federico.sismondi@inria.fr,remy.leone@inria.fr) to generate one

    ii. deploy your own rabbit message broker

    iii. use orchestrator API for automatically deploying one <- simpler option if API is up and running

  5. install requirements for agent and packet-router (see requirements.txt in respective dirs)

Running the experiment

0. On both sides:

export AMQP connection variables, the correct ones!! ( see step 3)

e.g.

export AMQP_VHOST='/'

export AMQP_EXCHANGE='default'

export AMQP_USER='walrus'

export AMQP_PASS='somePassword’

export AMQP_SERVER='f-interop.rennes.inria.fr'

1. On agent1 side (inside agent dir):

sudo python -m agent connect --server $AMQP_SERVER --user $AMQP_USER --password $AMQP_PASS --session $AMQP_VHOST --name agent1

2. On agent2 side (inside agent dir):

sudo python -m agent connect --server $AMQP_SERVER --user $AMQP_USER --password $AMQP_PASS --session $AMQP_VHOST --name agent2

4. On any side (inside coap_testing_tool dir):

python3 -m coap_testing_tool.packet_router.packet_router

5. Send message to agents to enable the tun (inside agent dir):

python3 -m utils.bootstrap_agents

NOTE: if warning printed on the output of last command, then something went wrong, probably one of the agents is either not running, or using the wrong amqp params

test

from agent1

ping6 bbbb::02

from agent2

ping6 bbbb::01

or even better, test you coap client-server interaction! sniff with wireshark and verify all traffic exchanged is AMQP traffic

problems running this? comments? contributions? insults?

-> federico.sismondi@inria.fr