8.1. Web site components

• A front-end web server such as Apache runs on your server and talks to the world. It serves static files and may run other web sites, but it routes a subset of its traffic to an internal TCP/IP port, where it finds CRATE.

• CRATE runs and serves content to an internal TCP/IP port.

  • CRATE implements its web front end via Django (which provides a WSGI application) and a “mini” web server like CherryPy or Gunicorn [1] (which makes that WSGI application talk to TCP/IP).

• CRATE talks to one or more databases (e.g. MySQL, SQL Server). These include research databases, but also a database used by CRATE to store its own data.

• A separate CRATE subsystem handles back-end tasks, like sending e-mails and processing consent requests (which can be a bit slow).

  • This uses the Celery task queue system. Celery itself is a Python program that talks to a message queue, and can put messages into this queue (CRATE calls Celery) [2], or retrieve them from the queue (Celery calls CRATE) [3] to get jobs done.

  • The actual message queue is managed by a “message broker”, which is software that provides message queues via AMQP; typically this broker is RabbitMQ.

  • You can use Flower to monitor this system (see crate_launch_flower).

• Some sort of operating system supervisor typically controls the various aspects of CRATE. Under Windows, CRATE provides a Windows service (see CRATE Windows service). Under Linux, supervisor is typically used.

---

Footnotes

[1]

Gunicorn runs under Windows only

[2]

In the source code, look for calls like email_rdbm_task.delay().

[3]

See crate_anon/crateweb/consent/celery.py.