8.1. Web site components

  • A front-end web server such as Apache runs on your server and talks to the world. It serves static files and may run other web sites, but it routes a subset of its traffic to an internal TCP/IP port, where it finds CRATE.

  • CRATE runs and serves content to an internal TCP/IP port.

    • CRATE implements its web front end via Django (which provides a WSGI application) and a “mini” web server like CherryPy or Gunicorn 1 (which makes that WSGI application talk to TCP/IP).

  • CRATE talks to one or more databases (e.g. MySQL, SQL Server). These include research databases, but also a database used by CRATE to store its own data.

  • A separate CRATE subsystem handles back-end tasks, like sending e-mails and processing consent requests (which can be a bit slow).

    • This uses the Celery task queue system. Celery itself is a Python program that talks to a message queue, and can put messages into this queue (CRATE calls Celery) 2, or retrieve them from the queue (Celery calls CRATE) 3 to get jobs done.

    • The actual message queue is managed by a “message broker”, which is software that provides message queues via AMQP; typically this broker is RabbitMQ.

    • You can use Flower to monitor this system (see crate_launch_flower).

  • Some sort of operating system supervisor typically controls the various aspects of CRATE. Under Windows, CRATE provides a Windows service (see CRATE Windows service). Under Linux, supervisor is typically used.

Footnotes

1

Gunicorn runs under Windows only

2

In the source code, look for calls like email_rdbm_task.delay().

3

See crate_anon/crateweb/consent/celery.py.