Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage

Database Inserter

The goal of this container is to insert decoded AIS messages, pulled from the message broker, into a database. Pretty simple stuff. It handles creating the connection to the DB and doing bulk inserts periodically. This is more efficient than inserting every message individually.

Input Data

The service connects to RabbitMQ, creates a message queue that accepts messages that contain the relavant routing key. These messages are decoded AIS messages that could fall into one of the many AIS message types.

The DB Sink inserts the messages into a specific DB table depending on AIS message type (the below example is of a type 24 AIS message). Some messages contain position information that goes into the pos_report table while other contain voyage report information, going into the voy_reports table. Some messages contain both sets, or a subset, of the data and are inserted into multiple tables.

{"ais": "!AIVDM,1,1,,B,H7OfI?PPu9<rM8uDp@000000000,2*41\r", 
"header": null, 
"server_time": "2022-08-31T11:35:05.326355", 
"event_time": "2022-08-31T11:35:05.326357", 
"routing_key": "ais.aishub.all", 
"multiline": false, 
"parsed_msg": {"header": null, 
                "footer": null, 
                "event_time": null, 
                "talker": "!AIVDM", 
                "frag_count": "1", 
                "frag_num": "1", 
                "seq_id": "", 
                "radio_chan": "B", 
                "payload": "H7OfI?PPu9<rM8uDp@000000000", 
                "padding": "2", 
                "checksum": "41"}, 
"decoded_msg": {"id": 24, 
                "repeat_indicator": 0, 
                "mmsi": 503028030, 
                "part_num": 0, 
                "name": "HORSN`ROUND"}, 
"source_key": "encoded_ais.aishub.all"}


The service is configured using an environment variable file. An example can be found in ./config/sample.env. This config file is loaded into the docker container on run (generally with docker-compose).

The config file is split up into several different catagories. Below is a short explanation of them.

PROJECT_NAME=ais            - The name of the project, this is prepended to all the containers being run
UID=1000                    - UID and GID of the user that will end up owning files created by the containers

# Database with Postgis and TimescaleDB
TIMESCALE_TAG=1.7.4-pg12          - DB Host and ports
# these should be docker secrets...
POSTGRES_USER=inserter      - DB User details
# RabbitMQ
RABBIT_MSG_PORT=5672        - RabbitMQ connection details
# these should be docker secrets...
# DB Inserter
BIND_TO_KEYS=["ais.imis.sat", "ais.samsa.sat", "ais.tnpa.coastal"]


The system is generally deployed using docker-compose. There is a compose file stored in the repo that shows how to start the container, mount volumes, use config files in the .env file. The steps to get this running on your system is:

  • Clone/Pull the repo to the target machine
  • Copy the ./config/sample.env to .env and edit it to reflect your environment
  • run “docker-compose up –build -d”

That will get the service running on your machine but will be pretty useless without any of the other services. There is work being done on a generic deployment that can be found in the deployment project in the OpenAIS namespace.