Server Startup

This project should be the first stop for a developer setting up a new OpenAIS node or development environement. This docker-compose file sets up a reverse proxy server; Traefik, and a docker management tool; Portainer.

Getting started

This is intended to be run on a machine that already has:

Ubuntu 22.04 (LTS version) installed
Docker CE engine installed.
Docker post installation steps completed.

After having a ready Linux and Docker environment this

Clone repository to a target server
Navigate to folder with docker compose file.
Copy example configuration environment file to the root directory as .env cp example.env .env
Edit the .env file to represent your local environment
Create Docker networks to handle traffic between containers
Run command docker compose up -d --build
Confirm everything launched without errors docker compose logs -f
Configure Portainer
Use Portainer to deploy other stacks in this project

4 - Environment File

The example environment file looks like this:

  TRAEFIK_TAG=v3.0
TRAEFIK_HOST=localhost
TF_USERNAME=rory
# TF_PASSWORD=traefik_for_ais
# Use to hahs password: HASHED_PASSWORD=$(openssl passwd -apr1 $PASSWORD)
TF_HASHED_PASSWORD='$apr1$H3Yv8DqQ$ewZBJpO15.Rv1UH5hINF8.'

# For generating LetsEncrypt certs
LE_EMAIL=spam@oceanmaps.xyz
LE_DOMAIN=oceanmaps.duckdns.org

#----------------------
# Portainer
#----------------------
PORTAINER_TAG=2.19.4
PORTAINER_AGENT_TAG=2.19.4
# This is a hashed_password using:
# https://gist.github.com/deviantony/62c009b41bde5e078b1a7de9f11f5e55
# This is the hash of password '12345'. Not escaping $'s seems okay
PT_PASSWORD_HASH='$2y$05$t/7ADOqxvucEBxj8qjlU6Ov0UEcT9BpiKAun3VvYTrbbhGXfUO4Aa'

There are some conventions share by this config file and others in the OpenAIS project: service specific config variables are in rows near each other, docker images/tag information is at the top of the service group and password/user examples SHOULD be changed.

TRAEFIK_TAG: This referes to the Docker Image Tag associated with the hard coded image in the portainer.yaml file. In this case it refers a tag here
TRAEFIK_HOST: This is the hostname of the machine that the deployment is running on. This can be set to “localhost” but that would result in the LetsEncrypt certificate generation failing. This hostname becomes the suffix for all URL’s pointing to deployed services.
TF_USERNAME: The Username of the default Traefik user.
TF_HASHED_PASSWORD: The apr1 hash of the dummy password ‘12345’ for the above username.
LE_EMAIL: The email address of the SysAdmin responible for monitoring LE certificates.
LE_DOMAIN: The domain to generate LE certificates for. This will probably be the same as the TRAEFIK_HOST in production.
PORTAINER_TAG: This points to here.
PORTAINER_AGENT_TAG: This points to here.
PT_PASSWORD_HASH: This is the MD5 hash of your chosen password. Read more here.

5 - Docker Networks

The various services in the OpenAIS project need to talk to each other but it’s also a good idea not to allow everything to talk to everything else. To connect various services Docker networks need to be created. The various docker networks are:

traefik-public: The network that Traefik will use to connect the various UI’s of the containers to the internet.
database-backend: The network that will be used to connect services to the database. This is NOT a public network.
rabbit-backend: The network that will be used to connect services to the RabbitMQ. This is NOT a public network.
agent-network: The network that will be used to connect Portainer agents to the Portainer master

Before the services stacks are deployed the networks need to be created on the host machine:

docker network create traefik-public
docker network create database-backend
docker network create rabbit-backend
docker network create agent-network

or else just run the create-networks.sh script.

8 - Configure Portainer

The Portainer UI should be available at portainer.TRAEFIK_HOST or localhost.9000.

https://docs.portainer.io/start/install-ce/server/docker/linux Once Portainer is up and running it needs some basic configuration. On first login (admin:12345) the user is prompted to change the default password, this should be done. The next step is to setup a new “environment”. If the machine is a single docker host (no swarm, no kubernetes) then possible options are:

Select “Docker Standalone” and “Start Wizard”
Select “Agent”
Give it a name to identify the host machine/ deployment environment and
give it the address as “port_agent:9001”, the same as specified in the docker-compose file.

The dashboard should now include all the docker containers running on the deployment environment, including Traefik, Portainer and Portainer Agent.

9 - Use Portainer to Deploy Other Stacks

On Portainer navigate to Stacks>Add Stack:

Give it a name
Select “Repository” as build method
Input “https://gitlab.com/openais/shared_services" as the repository URL
Add Environment variables, either one by one or from a file that you’ve already edited to represent your deployment environment.
Deploy the stack

NOTE: Portainer seems to have some issues with building containers that are specified with relative paths. The Jupyter Notebook container has been removed until a workaround can be figured out.

Next Steps

The OpenAIS server should now be ready to run the services specified in the Shared Services repository. After that the AIS specific services can be started from the Deploy repository.

These services are:

Shared Services: Set of services that are not required for OpenAIS operation, but might be useful.
Data Services: Off-the-shelf services that handle the flow and storage of the data.
Deploy: TODO Rename this one. Custom services used to build the data pipelines.