API Manual - Forensic SIEM Integration

Retarus Forensic SIEM Integration provides an API allowing customers to receive “events” (logs) which are generated based on the raw data (tracking points) that also form the data basis for our Email Live Search. Customers can subscribe to the specific events they would like to receive and configure their (SIEM) systems to alert or trigger actions based on the data.

Events can be provided for the following Retarus Email Security features (service booking required):

• Retarus AntiVirus MultiScan (In- and Outbound)

• Sandboxing

• Retarus Patient Zero Detection

• Retarus CxO Fraud Detection

• All Inbound emails processed (accepted or rejected) by a Retarus Inbound MTA

• All Outbound emails processed (accepted or rejected) by a Retarus Outbound MTA

The webservice communicates via WebSocket (RFC 6455 (v13)) using the JSON data format. The programming language in which the webservice client is implemented is not restricted by the webservice itself.

Data provided within an event:

• Meta data of the email

• Information regarding which feature the event is for

• Feature-specific data

Throughout this document, “Customer Number” or “CuNo” refers to your Retarus Customer Number.

Endpoint

The webservice is reachable at:
wss://events.retarus.com/email/siem/v1/websocket?channel=default
where default is the channel name created by the client. Valid channel names have a length of 6-32 characters and contain the characters [a-z0-9-]. If the client does not provide a channel name, the service will use default automatically.

A client can open more than one channel using the same token ID. For each channel opened, the client receives a unique set of events. For instance, if the client opens two channels, it receives two sets of unique events.

Authorization

Each connection must be authorized with a Bearer Authorization using a token Retarus has provided to the customer. The client connects to the endpoint by means of HTTP and then sends an upgrade request to WebSocket, using the token ID in a HTTP header.

Example
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJjdXN0b21lciI6IjQ0NDQ0IiwidHlwZXMiOltdLCJ2ZXJzaW9uIjoidjEiLCJ0cyI6IjAifQ.omLdEAWGgoPLctYuJ7APvDkBYRQWthp_4nCS-LCvokU

Retarus is able to configure multiple forms of access (Token IDs) for the webservice. Multiple processes can be operated concurrently by the webservice using multiple Token IDs, making it possible to have a different configuration saved for each process. Additionally, customers can define multiple channel IDs, which enables the parallel consumption of unique events with the same token.

Connection handling

Retarus Forensic SIEM Integration provisions events for those features that have been configured for the customer at the moment they occur. Each event is assigned to the token IDs which are active for the customer.

The connection to the WebSocket is generally initiated by the client application. While the connection is open, all events generated are transferred to the client application.

In case there is no active connection by the client application, Retarus Forensic SIEM Integration keeps the channel active for 60 minutes and caches occurring events. Once the channel itself has been closed, events will be discarded until the customer creates a new connection and reopens the channel.

The connection to the websocket is a long-living connection, therefore closing and reopening the connection for every event is not required and should be avoided.

Retarus will keep connections with no traffic open for at most 5 minutes. It is highly recommended that the client sends a PING control frame every 60 seconds to keep the connection open. Once the connection is closed by the server, the client is required to reconnect automatically.

Client application - code sample

The following abstract shows a simple client application, written in Python. The client application opens a connection and provides the authentication token to the server. Any event received is written by the application to the standard output.

The sample client code does not contain any routines regarding automatic error-handling, subsequent processing of events received or an automatic reconnect process.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import sys
import websocket # pip install websocket-client==0.59.0

ENDPOINT = "wss://events.retarus.com/v1/websocket"
TOKEN = "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJjdXN0b21lciI6IjQ0NDQ0IiwidHlwZXMiOl
tdLCJ2ZXJzaW9uIjoidjEiLCJ0cyI6IjAifQ.omLdEAWGgoPLctYuJ7APvDkBYRQWthp_4nCS-LCvokU"

if __name__ == "__main__":

  def on_open(ws):
    sys.stderr.write("Connection established\n")
    
  def on_message(ws, message):
    sys.stdout.write(f"{message}\n")
    
  def on_error(ws, error):
    sys.stderr.write(f"{error}\n")
    
  def on_close(ws):
    sys.stderr.write("Connection closed\n")
    
  ws = websocket.WebSocketApp(
    ENDPOINT,
    header=["Authorization: Bearer " + TOKEN],
    on_open=on_open,
    on_message=on_message,
    on_error=on_error,
    on_close=on_close,
  )
  ws.run_forever(ping_interval=60)

Event types provided

All events provided by Retarus Forensic SIEM Integration contain a set of defined data and consist of two parts. Firstly, generic information regarding the email which includes sender, recipient, customer ID, time stamp, event type. Secondly, specific details for this type of event. The data format employed is JSON.

The order of fields provided per event is not fixed and may vary.

Generic data fields for each event (this information is provided for any event generated).

AntiVirus MultiScan (Inbound)

Inbound emails are checked for viruses by Retarus MultiScan. If a message contains a virus, an event is provisioned to the client application. In addition to the generic details provided, the event contains the following feature-specific data fields:

Full event in JSON (sample)

{
   "customer":"CuNo",
   "metaData":{
      "details":"EICAR-AV-Test"
   },
   "host":"events.retarus.com",
   "ts":"2018-10-16 14:58:43 +0200",
   "version":"1.0",
   "sourceIp":"xxx.xxx.xxx.xxx",
   "sender":"xxxxxxx@retarus.com",
   "type":"MultiScan",
   "direction":"INBOUND",
   "recipient":"xxxxxxx@retarus.de",
   "mimeId":"<5616dfeid.xxxxxxxxxx@retarus.net>",
   "status":"INFECTED",
   "class":"THREAT",
   "rmxId":"20181016-145842-xxxxxx-xxxxxx-0@mailin27"
}

AntiVirus MultiScan (Outbound)

Messages relayed from customer systems are checked for viruses by Retarus MultiScan. If a message contains a virus, an event is provisioned to the client application. In addition to the generic details provided, the event contains the following feature-specific data fields:

Full event in JSON (sample)

{
   "customer":"CuNo",
   "metaData":{
      "details":"EICAR-AV-Test"
   },
   "host":"events.retarus.com",
   "ts":"2018-10-16 14:58:43 +0200",
   "version":"1.0",
   "sourceIp":"xxx.xxx.xxx.xxx",
   "sender":"xxxxxxx@retarus.com",
   "type":"MultiScan",
   "direction":"OUTBOUND",
   "recipient":"xxxxxxx@retarus.de",
   "mimeId":"<5616dfeid.xxxxxxxxxx@retarus.net>",
   "status":"INFECTED",
   "class":"THREAT",
   "rmxId":"20181016-145842-xxxxxx-xxxxxx-0@mailin27"
}

Sandboxing

For Sandboxing, an event is provisioned at the moment detection has occurred. In addition to the generic details provided, the event contains the following feature-specific data:

Full event in JSON (sample)

{
   "customer":"CuNo",
   "metaData":{
      "hashFunction":"sha256",
      "checksum":"cbfdedf25f7f04daf9d705548cf6b6546d66bc206ea1a166fff15ece9434"
   },
   "host":"events.retarus.com",
   "ts":"2018-10-16 15:03:43 +0200",
   "version":"1.0",
   "sourceIp":"xxx.xxx.xxx.xxx",
   "sender":"xxxxxxx@retarus.com",
   "type":"Sandboxing",
   "direction":"INBOUND",
   "recipient":"xxxxxxx@retarus.de",
   "mimeId":"<37357C96.xxxxxxx@retarus.net>",
   "status":"SUSPICIOUS",
   "class":"THREAT",
   "rmxId":"20181016-145902-xxxxxx-0@mailin08"
}

CxO Fraud Detection

For CxO Fraud Detection, an event is provisioned at the moment detection has occurred. The event doesn’t contain feature-specific data.

Full event in JSON (sample)

{
   "customer":"CuNo",
   "metaData":{
      
   },
   "host":"events.retarus.com",
   "ts":"2018-10-16 14:58:18 +0200",
   "version":"1.0",
   "sourceIp":"xxx.xxx.xxx.xxx",
   "sender":"xxxxxxx@retarus.com",
   "type":"CxO",
   "direction":"INBOUND",
   "recipient":"xxxxxxx@retarus.de",
   "mimeId":"<164D6G96.xxxxxxx@retarus.net>",
   "status":"DETECTED",
   "class":"THREAT",
   "rmxId":"20181016-145817-42ZFjPxxxxxx-0@mailin01"
}

Patient Zero Detection

For Patient Zero Detection, an event is provisioned at the moment detection has occurred. In addition to the generic details provided, the event contains the following feature-specific data fields:

Full event in JSON (sample for a virus finding)

{
   "customer":"CuNo",
   "metaData":{
      "hashFunction":"sha256",
      "threatType":"VIRUS",
      "checksum":"6b84714d0fa8c77d846306f37f4f3135596d34e17dca4f84088195272fd",
      "mimeType":"application\\x-dosexec",
      "details":"EICAR-Test-File"
   },
   "host":"events.retarus.com",
   "ts":"2018-10-16 14:58:56 +0200",
   "version":"1.0",
   "sourceIp":"xxx.xxx.xxx.xxx",
   "sender":"xxxxxx@retarus.de",
   "type":"PZD",
   "direction":"INBOUND",
   "recipient":"xxxxxxx@retarus.de",
   "mimeId":"<56168B42.xxxxxxx@retarus.net>",
   "status":"DETECTED",
   "class":"THREAT",
   "rmxId":"20181016-145852-xxxxxx-xxxxxx-0@mailin01"
}

MTA Inbound

This event is not a threat type event, but it is generated for every single email that the Retarus Inbound MTAs are receiving from the Internet (=accepting or rejecting) and may be used to monitor the amount of inbound emails in order to detect unusual email volumes or get information about their origin. The event does NOT contain any information about the final relay/delivery status of an email.

In addition to the generic details provided, the event contains the following feature-specific data fields:

Full event in JSON (sample for an accepted inbound email)

{
   "customer":"CuNo",
   "metaData":{
      "authentication":{
         "dkim":{
            "status":"dkim=none",
            "details":"dkim=none reason=\"no signature\""
         }
      },
      "transportEncryption":{
         "requested":false,
         "established":false
      },
      "header":{
         "subject":"This is a test mail",
         "from":"sender@example.com"
      },
      "contentEncryption":false
   },
   "host":"events.retarus.com",
   "ts":"2021-07-11 14:58:43 +0200",
   "version":"1.0",
   "sourceIp":"xxx.xxx.xxx.xxx",
   "sender":"xxxxxxx@retarus.com",
   "type":"MTA",
   "subtype":"INCOMING",
   "direction":"INBOUND",
   "recipient":"xxxxxxx@retarus.de",
   "mimeId":"<5616dfeid.xxxxxxxxxx@retarus.net>",
   "status":"ACCEPTED",
   "class":"EVENT",
   "rmxId":"20210711-145842-xxxxxx-xxxxxx-0@mailin27"
}

MTA Outbound

This event is not a threat type event, but it is generated for every single email that the Retarus Outbound MTAs are receiving from the customer systems (=accepting or rejecting) and may be used to monitor the amount of outbound emails in order to detect unusual email volumes. The event does NOT contain any information about the final relay/delivery status of an email.

In addition to the generic details provided, the event contains the following feature-specific data fields:

Full event in JSON (sample for an accepted outbound email with TLS established)

{
   "customer":"CuNo",
   "metaData":{
      "transportEncryption":{
         "requested":true,
         "established":true,
         "protocol":"TLSv1.2",
         "cipherSuite":"ECDHE-RSA-AES128-SHA256 (128/128 bits)"
      },
      "header":{
         "subject":"This is a test mail",
         "from":"sender@example.com"
      }
   },
   "host":"events.retarus.com",
   "ts":"2021-07-11 14:58:43 +0200",
   "version":"1.0",
   "sourceIp":"xxx.xxx.xxx.xxx",
   "sender":"xxxxxxx@retarus.com",
   "type":"MTA",
   "subtype":"INCOMING",
   "direction":"OUTBOUND",
   "recipient":"xxxxxxx@retarus.de",
   "mimeId":"<5616dfeid.xxxxxxxxxx@retarus.net>",
   "status":"ACCEPTED",
   "class":"EVENT",
   "rmxId":"20210711-145842-xxxxxx-xxxxxx-0@mailin27"
}

HTTP status codes