4.1. Installing and running CRATE via Docker

4.1.1. Overview

Docker is a cross-platform system for running applications in “containers”. A computer (or computing cluster) can run lots of containers. They allow applications to be set up in standardized and isolated enviroments, which include their own operating system). The containers then talk to each other, and to their “host” computer, to do useful things.

The core of Docker is called Docker Engine. The Docker Compose tool allows multiple containers to be created, started, and connected together automatically.

CRATE provides an installer script to make installation using Docker easy. The script uses Docker Compose to set up several containers, specifically:

  • a database system, via MySQL on Linux (internal container name mysql);

  • a message queue, via RabbitMQ on Linux (rabbitmq);

  • the CRATE web server itself, offering SSL directly via CherryPy on Linux (crate_server);

  • the CRATE web site back-end (crate_workers);

  • a background task monitor, using Flower (crate_monitor).

  • demonstration source and destination MySQL databases for anonymisation

Additionally, you can run a number of important one-off command using the crate Docker image. Apart from CRATE itself, this image also includes:

4.1.2. Quick start

4.1.2.1. Windows

4.1.2.2. Linux

4.1.2.3. MacOS

4.1.2.4. All platforms

The installer can be run interactively, where you will be prompted to enter settings specific to your CRATE installation. Alternatively you can supply this information by setting environment variables. This is best done by putting the settings in a file and executing them before running the installer (e.g. source ~/my_crate_settings).

Here is an example settings file. See environment_variables for a description of each setting.

export CRATE_DOCKER_CONFIG_HOST_DIR=${HOME}/crate_config
export CRATE_DOCKER_GATE_BIOYODIE_RESOURCES_HOST_DIR=${HOME}/bioyodie_resources
export CRATE_DOCKER_MYSQL_CRATE_USER_PASSWORD=mysqluserpassword
export CRATE_DOCKER_MYSQL_CRATE_ROOT_PASSWORD=mysqlrootpassword
export CRATE_DOCKER_MYSQL_CRATE_HOST_PORT=43306
export CRATE_DOCKER_CRATEWEB_SUPERUSER_USERNAME=admin
export CRATE_DOCKER_CRATEWEB_SUPERUSER_PASSWORD=adminpassword
export CRATE_DOCKER_CRATEWEB_SUPERUSER_EMAIL=admin@example.com
export CRATE_DOCKER_CRATEWEB_USE_HTTPS=1
export CRATE_DOCKER_CRATEWEB_HOST_PORT=8100
export CRATE_DOCKER_CRATEWEB_SSL_CERTIFICATE=${HOME}/certs/crate.localhost.crt
export CRATE_DOCKER_CRATEWEB_SSL_PRIVATE_KEY=${HOME}/certs/crate.localhost.key

To start the installer on all platforms:

curl --location https://github.com/ucam-department-of-psychiatry/crate/releases/latest/download/installer.sh --fail --output crate_docker_installer.sh && chmod u+x crate_docker_installer.sh && ./crate_docker_installer.sh

4.1.3. Environment variables

4.1.3.1. CRATE_DOCKER_CONFIG_HOST_DIR

No default. Must be set.

Path to a directory on the host that contains key configuration files. Don’t use a trailing slash.

Note

Under Windows, don’t use Windows paths like C:\Users\myuser\my_crate_dir. Translate this to Docker notation as /host_mnt/c/Users/myuser/my_crate_dir. As of 2020-07-21, this doesn’t seem easy to find in the Docker docs! Ensure that this path is within the Windows (not WSL2) file system.

4.1.3.2. CRATE_DOCKER_CRATEWEB_CONFIG_FILENAME

Default: crateweb_local_settings.py

Base name of the CRATE web server config file (see CRATE_DOCKER_CONFIG_HOST_DIR).

4.1.3.3. CRATE_DOCKER_CRATEWEB_HOST_PORT

No default: Must be set

The TCP/IP port number on the host computer that CRATE should provide an HTTP or HTTPS (SSL) connection on.

It is strongly recommended that you make all connections to CRATE use HTTPS. The two ways of doing this are:

  • Have CRATE run plain HTTP, and connect it to another web server (e.g. Apache) that provides the HTTPS component.

    • If you do this, you should not expose this port to the “world”, since it offers insecure HTTP.

    • The motivation for this method is usually that you are running multiple web services, of which CRATE is one.

    • We don’t provide Apache within Docker, because the Apache-inside-Docker would only see CRATE, so there’s not much point – you might as well use the next option…

  • Have CRATE run HTTPS directly, by specifying the CRATE_DOCKER_CRATEWEB_SSL_CERTIFICATE and CRATE_DOCKER_CRATEWEB_SSL_PRIVATE_KEY options.

    • This is simpler if CRATE is the only web service you are running on this machine. Use the standard HTTPS port, 443, and expose it to the outside through your server’s firewall. (You are running a firewall, right?)

4.1.3.4. CRATE_DOCKER_CRATEWEB_USE_HTTPS

Access the CRATE web app over HTTPS? (0 = no, 1 = yes) See CRATE_DOCKER_CRATEWEB_HOST_PORT above.

4.1.3.5. CRATE_DOCKER_CRATEWEB_SSL_CERTIFICATE

Default is blank.

Filename for an SSL public certificate for HTTPS. See CRATE_DOCKER_CRATEWEB_HOST_PORT above.

4.1.3.6. CRATE_DOCKER_CRATEWEB_SSL_PRIVATE_KEY

Default is blank.

Filename for an SSL private key file for HTTPS. See CRATE_DOCKER_CRATEWEB_HOST_PORT above.

4.1.3.7. CRATE_DOCKER_CRATEWEB_SUPERUSER_USERNAME

User name for the CRATE administrator, via CRATE’s web application.

4.1.3.8. CRATE_DOCKER_CRATEWEB_SUPERUSER_PASSWORD

Password for the CRATE administrator, via CRATE’s web application.

4.1.3.9. CRATE_DOCKER_CRATEWEB_SUPERUSER_EMAIL

Email address for the CRATE administrator.

4.1.3.10. CRATE_DOCKER_FLOWER_HOST_PORT

Default: 5555

Host port on which to launch the Flower monitor.

4.1.3.11. CRATE_DOCKER_GATE_BIOYODIE_RESOURCES_HOST_DIR

No default. Must be set (even if to a dummy directory).

A directory to be mounted that contains preprocessed UMLS data for the Bio-YODIE NLP tool (which is part of KConnect/SemEHR, and which runs under GATE). (You need to download UMLS data and use the crate_nlp_prepare_ymls_for_bioyodie script to process it. The output directory used with that command is the directory you should specify here.) On Windows, ensure this is within the Windows (not WSL2) file system.

4.1.3.12. CRATE_DOCKER_MYSQL_CRATE_DATABASE_NAME

Default: crate_web_db

Name of the MySQL database to be used for CRATE web site data.

4.1.3.13. CRATE_DOCKER_MYSQL_CRATE_USER_PASSWORD

No default. Must be set during MySQL container creation.

MySQL password for the CRATE database user (whose name is set by CRATE_DOCKER_MYSQL_CRATE_USER_NAME).

Note

This only needs to be set when Docker Compose is creating the MySQL container for the first time. After that, it doesn’t have to be set (and is probably best not set for security reasons!).

4.1.3.14. CRATE_DOCKER_MYSQL_CRATE_USER_NAME

Default: crate_web_user

MySQL username for the main CRATE web user. This user is given full control over the database named in CRATE_DOCKER_MYSQL_CRATE_DATABASE_NAME. See also CRATE_DOCKER_MYSQL_CRATE_USER_PASSWORD.

4.1.3.15. CRATE_DOCKER_MYSQL_CRATE_HOST_PORT

Default: 3306

Port published to the host, giving access to the CRATE MySQL installation. You can use this to allow other software to connect to the CRATE database directly.

This might include using MySQL tools from the host to perform database backups (though Docker volumes can also be backed up in their own right).

The default MySQL port is 3306. If you run MySQL on your host computer for other reasons, this port will be taken, and you should change it to something else.

You should not expose this port to the “outside”, beyond your host.

4.1.3.16. CRATE_DOCKER_MYSQL_CRATE_ROOT_PASSWORD

No default. Must be set during MySQL container creation.

MySQL password for the root user.

Note

This only needs to be set when Docker Compose is creating the MySQL container for the first time. After that, it doesn’t have to be set (and is probably best not set for security reasons!).

4.1.3.17. COMPOSE_PROJECT_NAME

Default: crate

This is the Docker Compose project name. It’s used as a prefix for all the containers in this project.

Todo

fix below here; see CamCOPS help

4.1.4. Tools

All live in the installer directory.

4.1.4.1. enter_crate_container.sh

Starts a container with the CRATE image and runs a Bash shell within it.

Warning

Running a shell within a container allows you to break things! Be careful.

4.1.4.2. start_crate.sh

Shortcut for docker compose up -d. The -d switch is short for --detach (or daemon mode).

4.1.4.3. stop_crate.sh

Shortcut for docker compose down.

4.1.4.4. run_crate_command

This script starts a container with the CRATE image, activates the CRATE virtual environment, and runs a command within it. For example, to explore this container, you can do

./run_crate_command.sh /bin/bash

… which is equivalent to the enter_docker_container script (see above and note the warning).

4.1.5. Development notes