8.1. Web site components

  • A front-end web server such as Apache runs on your server and talks to the world. It serves static files and may run other web sites, but it routes a subset of its traffic to an internal TCP/IP port, where it finds CRATE.
  • CRATE runs and serves content to an internal TCP/IP port.
    • CRATE implements its web front end via Django (which provides a WSGI application) and a “mini” web server like CherryPy or Gunicorn [1] (which makes that WSGI application talk to TCP/IP).
  • CRATE talks to one or more databases (e.g. MySQL, SQL Server). These include research databases, but also a database used by CRATE to store its own data.
  • A separate CRATE subsystem handles back-end tasks, like sending e-mails and processing consent requests (which can be a bit slow).
    • This uses the Celery task queue system. Celery itself is a Python program that talks to a message queue, and can put messages into this queue (CRATE calls Celery) [2], or retrieve them from the queue (Celery calls CRATE) [3] to get jobs done.
    • The actual message queue is managed by a “message broker”, which is software that provides message queues via AMQP; typically this broker is RabbitMQ.
    • You can use Flower to monitor this system (see crate_launch_flower).
  • Some sort of operating system supervisor typically controls the various aspects of CRATE. Under Windows, CRATE provides a Windows service (see CRATE Windows service). Under Linux, supervisor is typically used.


[1]Gunicorn runs under Windows only
[2]In the source code, look for calls like email_rdbm_task.delay().
[3]See crate_anon/crateweb/consent/celery.py.