8.3. Web config file

General
Site URL configuration
- DJANGO_SITE_ROOT_ABSOLUTE_URL
- FORCE_SCRIPT_NAME
Site security
Celery configuration
- BROKER_URL
- CELERYBEAT_SCHEDULE
Database configuration
Archive views
Site-specific help
- DATABASE_HELP_HTML_FILENAME
Local file storage
Outgoing e-mail
Research Database Manager (RDBM) settings
Web site administrators
- ADMINS
PDF creation
Consent-for-contact settings
Local information links
Specimen config file

8.3.1. General 

Settings here are a combination of standard Django settings and settings custom to CRATE. Not all standard Django options are described here; see e.g. https://docs.djangoproject.com/en/3.0/topics/settings/.

Defaults are in crate_anon.crateweb.config.settings, which are then overridden as required by your site’s Python config file (in standard Django fashion). The “normal” settings to consider are described below.

8.3.2. Site URL configuration 

8.3.2.1. DJANGO_SITE_ROOT_ABSOLUTE_URL 

type: str

Absolute root URL of the site (e.g. https://mymachine.mydomain for hosting under Apache). Don’t add a trailing slash.

8.3.2.2. FORCE_SCRIPT_NAME 

type: str

Script name to enforce, e.g. /crate for a site being hosted at a non-root location such as https://mymachine.mydomain/crate.

8.3.3. Site security 

See also the site security deployment checklist at https://docs.djangoproject.com/en/1.8/howto/deployment/checklist/.

8.3.3.1. SECRET_KEY 

type: str

Secret key used for the site.

This is a Django setting: see https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-SECRET_KEY.

Use this command to generate a new random secret key:

crate_generate_new_django_secret_key

8.3.3.2. DEBUG 

type: bool

Turn on debugging features? Do not use this for production sites.

This is a Django setting: https://docs.djangoproject.com/en/2.2/ref/settings/#debug.

Debug features by default include:

allowing any site to connect, via ALLOWED_HOSTS;
turning on the Django Debug Toolbar, via DEBUG_TOOLBAR_CONFIG (see https://django-debug-toolbar.readthedocs.io/en/latest/configuration.html)

Note that when you set DEBUG = False, as you should, you must ensure that static files are served properly.

8.3.3.3. CRATE_HTTPS 

type: bool

Default: True.

Require HTTPS, i.e. disallow unencrypted HTTP. This should be True for good security. If you then attempt to access the site via plain HTTP, you will get a “403 Forbidden: CSF verification failed” error.

This is simply a shortcut to some Django settings. If True, the following Django settings are applied (and that’s what has the real effect):

SESSION_COOKIE_SECURE = True (cookies only via HTTPS)
CSRF_COOKIE_SECURE = True (CSRF cookies only via HTTPS)

8.3.3.4. ALLOWED_HOSTS 

This is a Django setting; see https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-ALLOWED_HOSTS.

8.3.4. Celery configuration 

8.3.4.1. BROKER_URL 

type: str

Optionally, this can be overridden. By default, it is amqp://.

Overriding BROKER_URL will allow you to use multiple virtual hosts, to host multiple independent instances of CRATE (in the unlikely event you’d want to!). See https://stackoverflow.com/questions/12209652/multi-celery-projects-with-same-rabbitmq-broker-backend-process
Similarly, override BROKER_URL to improve RabbitMQ security.

8.3.4.2. CELERYBEAT_SCHEDULE 

Schedule back-end (Celery) tasks for specific times. See:

Celery periodic tasks: https://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html
Celery beat_schedule setting: https://docs.celeryproject.org/en/latest/userguide/configuration.html
Via Django: https://stackoverflow.com/questions/37848481/how-to-configure-celerybeat-schedule-in-django-settings/37851090
Renaming from CELERYBEAT_SCHEDULE to beat_schedule (and from BROKER_URL to broker_url from Celery version 4.0, with backwards compatibility): https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings

The typical use is to make CRATE check the primary clinical record regularly – so that, for example, if a patient withdraws their consent, this is processed promptly to create a withdrawal-of-consent letter to relevant researchers. Like this:

from celery.schedules import crontab

# ...

CELERYBEAT_SCHEDULE = {
    'refresh_consent_modes_at_midnight': {
        'task': 'crate_anon.crateweb.consent.tasks.refresh_all_consent_modes',
        'schedule': crontab(minute=0, hour=0),
    },
}

Note

The scheduled tasks will not run unless you start Celery with the beat option - i.e. run crate_launch_celery --command=beat. This is done automatically as part of the Windows service launcher.

Celery picks up these definitions as follows:

crate_anon/crateweb/consent/celery.py sets the environment variable DJANGO_SETTINGS_MODULE, then calls app.config_from_object('django.conf:settings'). That loads django.conf and reads its settings (which loads the user’s Django configuration file).

8.3.5. Database configuration 

8.3.5.1. DATABASES 

This is a Django setting: https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-DATABASES.

You require databases with the following names:

default: the main database used by CRATE to store its information (e.g. users, studies, queries via the web site);
research: the anonymised research database itself;
optionally, one or more secret databases for RID/PID mapping, cross-referenced by RESEARCH_DB_INFO below;
optionally, one or more specific databases representing a copy of a primary clinical records system; their names must be one of those in crate_anon.crateweb.config.constants.ClinicalDatabaseType.DATABASE_CHOICES.

Warning

It is critically important that the connection information you give for the research database (i.e. its user’s access) is read-only for the research databases [1] and has no access whatsoever to secret databases (like the default or “secret” databases) [2]. Researchers are given full ability to execute sql via this connection, and can do so for any databases that the connection permits, not just the one you specify explicitly.

[1] So researchers can’t alter/delete research data.

[2] So researchers can’t see secrets.

8.3.5.2. CLINICAL_LOOKUP_DB 

type: str

Which database (from DATABASES) should be used to look up demographic details?

It must be

one named in crate_anon.crateweb.config.constants.ClinicalDatabaseType.DATABASE_CHOICES;
defined in DATABASES, unless it is dummy_clinical, which is just for testing purposes.

8.3.5.4. RESEARCH_DB_TITLE 

type: str

Research database title (displayed in web site).

8.3.5.5. RESEARCH_DB_INFO 

type: List[Dict[str, Any]]

Defines the research database. This is the setting that allows CRATE to read several arbitrary relational databases, and link them together helpfully for features like the SQL query builder.

Note that all these databases use the DATABASES['research'] connection specified above.

This variable is a list of dictionaries, one per database. Each database dictionary has the following keys, which are defined in crate_anon.crateweb.config.constants.ResearchDbInfoKeys (referred to as RDIKeys in the settings file):

Key	Type	Value
RDIKeys.NAME	`str`	Unique name (for internal referencing)
RDIKeys.DESCRIPTION	`str`	Human-friendly description
RDIKeys.DATABASE	`str`	Database name, as seen by the database engine. For MySQL and PostgreSQL, use a blank string, `''`. For SQL Server, use the database name.
RDIKeys.SCHEMA	`str`	Schema name. For MySQL, use the database (= schema) name. For PostgreSQL, use the schema name (usual default: `'public''`). For SQL Server, use the schema name (usual default: `'dbo'`).
RDIKeys.PID_PSEUDO_FIELD	`str`	String used as the name of a pseudo-field in some “clinician privileged” views, to representing the PID column. Offered in some views to look up patients based on PID, and elsewhere as the label for the PID column.
RDIKeys.MPID_PSEUDO_FIELD	`str`	String used as the name of a pseudo-field in some “clinician privileged” views, to representing the MPID column. Offered in some views to look up patients based on MPID, and elsewhere as the label for the MPID column.
RDIKeys.TRID_FIELD	`str`	Name of the TRID column within your anonymised research database.
RDIKeys.RID_FIELD	`str`	Name of the RID column within your anonymised research database.
RDIKeys.RID_FAMILY	A “truthy” Python value (e.g. sequential integers).	Explained below; used to determine how CRATE cross-links multiple databases.
RDIKeys.MRID_TABLE	`str`	Name of a table within your anonymised research database that contains MRID values (and corresponding RIDs).
RDIKeys.MRID_FIELD	`str`	Name of the MRID column within the `MRID_TABLE` of your anonymised research database.
RDIKeys.PID_DESCRIPTION	`str`	Description of the PID field (e.g. “MyEPR number”).
RDIKeys.MPID_DESCRIPTION	`str`	Description of the MPID field (e.g. “NHS number”).
RDIKeys.RID_DESCRIPTION	`str`	Description of the RID field (e.g. “Research ID (RID; hashed MyEPR number)” or “BRCID”).
RDIKeys.MRID_DESCRIPTION	`str`	Description of the MRID field (e.g. “Master research ID (MRID; hashed NHS number)”).
RDIKeys.TRID_DESCRIPTION	`str`	Description of the TRID field (e.g. “Transient research ID (TRID) for database X”).
RDIKeys.SECRET_LOOKUP_DB	`str`	To look up PID/RID mappings, provide a value that is a database alias from `DATABASES`.
RDIKeys.DEFAULT_DATE_FIELDS	`List[str]`	For the data finder: is there a standard date field (column) for most patient tables? If so, specify one or more column names here. For example, `["document_date", "record_date"]`.
RDIKeys.DATE_FIELDS_BY_TABLE	`Dict[str, str]`	For the data finder: if some tables have their own specific date columns, you can specify these here. If a table appears here, that overrides the values found in `RDIKeys.DEFAULT_DATE_FIELDS`. Example: `{"doc_table": "document_date", "diagnosis_table": "diagnosis_date"}`.
RDIKeys.UPDATE_DATE_FIELD	`str`	Name of a column indicating when the record was last updated in the database.

More on databases and schemas

Under SQL Server, “database” and “schema” are different levels of organization. Specify a schema of "dbo" if you are unsure; this is the default.
Under MySQL, “database” and “schema” mean the same thing. Here, we’ll call this a SCHEMA.
PostgreSQL can only query a single database via a single connection.
The first database/schema in RESEARCH_DB_INFO is the default selected in CRATE’s query builder.

The RID_FAMILY parameter, and how CRATE auto-links tables

CRATE’s front end will automatically join tables, within and across multiple research databases. In summary:

WITHIN a schema, tables will be autojoined on the TRID_FIELD.
ACROSS schemas, tables will be autojoined on the RID_FIELD if they are in the same RID_FAMILY, and on MRID_TABLE.MRID_FIELD otherwise.

In more detail:

The RID_FAMILY is part of the system that CRATE uses to cross-link multiple research databases automatically (for the convenience of researchers using the web front end).

A RID is present in all such research databases. However, different databases may use different RIDs. If two databases use the same RID, they are part of the same RID family (and CRATE will link them on RID). If they are not part of the same RID family, CRATE will link them on MRID instead.

Here’s an example:

Database	PID/RID	MPID/MRID	RID family
DbOne	RiO number	NHS number	1
DbTwo	RiO number	NHS number	1
DbThree	Epic number	NHS number	2
DbFour	SystmOne number	NHS number	3

(In all cases, the RID is assumed to be a hashed version of the PID.)

Here, DbOne and DbTwo share a RID, so are part of the same RID family (and can be linked directly on RID if a query asks for data from both). The others use different RIDs, so are part of separate RID families – cross-linkage requires CRATE to use the MRID as an intermediate in the linking step.

Date/time columns for the data finder

The application of RDIKeys.DATE_FIELDS_BY_TABLE and RDIKeys.DEFAULT_DATE_FIELDS is performed by crate_anon.crateweb.research.research_db_info.SingleResearchDatabase.get_default_date_field().

8.3.5.6. RESEARCH_DB_FOR_CONTACT_LOOKUP 

type: str

Which database (from those defined in RESEARCH_DB_INFO above) should be used to look up patients when contact requests are made?

Give the name attribute of one of the databases in RESEARCH_DB_INFO. Its secret_lookup_db will be used for the actual lookup process.

8.3.5.7. NLP_SOURCEDB_MAP 

type: Dict[str, str]

This is an optional setting.

Used to provide automatic links from results involving CRATE NLP tables. Such tables have standard NLP output columns. When the CRATE web front end detects a table, it tries to provide a hyperlink to the original data, if available. However, database names in these tables (the _srcdb column) are user-defined.

In NLP_SOURCEDB_MAP, you can provide a mapping from _srcdb names to the names of databases in RESEARCH_DB_INFO, and this will enable the auto-linking.

8.3.5.8. RESEARCH_DB_DIALECT 

type: str

For the automatic query generator, we need to know the underlying SQL dialect. Options are

mysql = MySQL
mssql = Microsoft SQL Server

8.3.5.9. DISABLE_DJANGO_PYODBC_AZURE_CURSOR_FETCHONE_NEXTSET 

type: bool

Default: True.

If True, calls crate_anon.crateweb.research.models.hack_django_pyodbc_azure_cursorwrapper() at startup (q.v.).

8.3.6. Archive views 

8.3.6.1. ARCHIVE_TEMPLATE_DIR 

type: str

Optional.

Root directory of the archive template system.

8.3.6.2. ARCHIVE_ROOT_TEMPLATE 

type: str

Optional.

Filename of the archive’s root template. This should be found within ARCHIVE_TEMPLATE_DIR.

8.3.6.3. ARCHIVE_ATTACHMENT_DIR 

type: str

Optional.

Root directory for archive attachments.

8.3.6.4. ARCHIVE_STATIC_DIR 

type: str

Optional.

Root directory for archive static files.

8.3.6.5. ARCHIVE_TEMPLATE_CACHE_DIR 

type: str

Optional.

Directory in which to store compiled versions of the archive templates.

8.3.6.6. ARCHIVE_CONTEXT 

type: Dict[str, Any]

Optional.

A dictionary that forms the basis of the Python context within archive templates. See archive Python context.

8.3.6.7. CACHE_CONTROL_MAX_AGE_ARCHIVE_STATIC 

type: int

Optional; default 0.

Ask client browsers to cache files from the static part of the archive up to this maximum age in seconds. (This sets the Cache-Control: max-age=<seconds> parameter in the HTTP header.)

CRATE will add file modification times to URLs, so setting a long cache expiry time will not prevent automatic reloading if the file changes.

8.3.6.8. CACHE_CONTROL_MAX_AGE_ARCHIVE_ATTACHMENTS 

type: int

Optional; default 0.

As for CACHE_CONTROL_MAX_AGE_ARCHIVE_STATIC, but for attachments within the archive.

8.3.6.9. CACHE_CONTROL_MAX_AGE_ARCHIVE_TEMPLATES 

type: int

Optional; default 0.

As for CACHE_CONTROL_MAX_AGE_ARCHIVE_STATIC, but for calls to render templates.

8.3.7. Site-specific help 

8.3.7.1. DATABASE_HELP_HTML_FILENAME 

type: Optional[str]

If specified, this must be a string that is an absolute filename of trusted HTML that will be provided to the user when they ask for site-specific help on your database structure (see Help on local database structure).

8.3.8. Local file storage 

8.3.8.1. PRIVATE_FILE_STORAGE_ROOT 

type: str

Where should we store binary files uploaded to CRATE (e.g. study leaflets)? Specify a directory name here. You should create this directory (and don’t let it be served by a generic web server that doesn’t check permissions).

8.3.8.2. XSENDFILE 

type: bool

Specify False to serve files via Django (inefficient but useful for testing) or True to serve via Apache with mod_xsendfile (or another web server configured for the X-SendFile directive).

This setting is read by cardinal_pythonlib.django.serve.serve_file(), called by several functions within crate_anon.crateweb.consent.views.

8.3.8.3. MAX_UPLOAD_SIZE_BYTES 

type: int

How big an upload will we accept? Example:

MAX_UPLOAD_SIZE_BYTES = 10 * 1024 * 1024  # 10 Mb

8.3.9. Outgoing e-mail 

8.3.9.1. EMAIL_*

First, there are general settings for sending e-mail from Django; see https://docs.djangoproject.com/en/1.8/ref/settings/#email-backend. Example:

#   default backend:
# EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
#   bugfix for servers that only support TLSv1:
# EMAIL_BACKEND = 'cardinal_pythonlib.django.mail.SmtpEmailBackendTls1'

EMAIL_HOST = 'smtp.somewhere.nhs.uk'
EMAIL_PORT = 587  # usually 25 (plain SMTP) or 587 (STARTTLS)
# ... see https://www.fastmail.com/help/technical/ssltlsstarttls.html
EMAIL_HOST_USER = 'myuser'
EMAIL_HOST_PASSWORD = 'mypassword'
EMAIL_USE_TLS = True
EMAIL_USE_SSL = False

# Who will the e-mails appear to come from?
EMAIL_SENDER = "My NHS Trust Research Database - DO NOT REPLY <noreply@somewhere.nhs.uk>"  # noqa

Then there are some additional custom settings:

8.3.9.2. SAFETY_CATCH_ON 

type: bool

During development, we set this to True to route all consent-related e-mails to the developer, specified by DEVELOPER_EMAIL. Switch SAFETY_CATCH_ON to False for production mode.

8.3.9.3. DEVELOPER_EMAIL 

type: str

E-mail address of a person developing CRATE (for SAFETY_CATCH_ON).

8.3.9.4. VALID_RESEARCHER_EMAIL_DOMAINS 

type: List[str]

List of e-mail domains, such as ["@cpft.nhs.uk"], which are acceptable to you for researchers. If this list is empty, CRATE will send e-mails to researchers (via the e-mail configured in their user settings) without further checks. If it’s not empty, though, CRATE will refuse to send researcher e-mails – which will often contain patient-identifiable information, as part of the consent-to-contact system – unless the researcher’s e-mail domain is in this list. Setting this prevents e-mails going to an inappropriate domain even if the researcher sets their e-mail to something insecure, e.g. someone@hotmail.com.

8.3.10. Research Database Manager (RDBM) settings 

8.3.10.1. RDBM_NAME 

type: str

Name of the RDBM, e.g. “John Smith”.

8.3.10.2. RDBM_TITLE 

type: str

The RDBM’s title, e.g. “Research Database Manager”.

8.3.10.3. RDBM_TELEPHONE 

type: str

The RDBM’s telephone number, which is provided to clinicians and researchers.

8.3.10.4. RDBM_EMAIL 

type: str

E-mail address of the Research Database Manager (RDBM).

8.3.10.5. RDBM_ADDRESS 

type: List[str]

The address of the RDBM (as a list of address lines). This address is used in communication to patients. Example: ["FREEPOST SOMEWHERE_HOSPITAL RESEARCH DATABASE MANAGER"].

8.3.11. Web site administrators 

8.3.11.1. ADMINS 

type: List[Tuple[str, str], ...]

This is a list of (name, email_address) pairs. Software exception reports get sent to these people.

This is a Django setting; see https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-ADMINS.

8.3.12. PDF creation 

8.3.12.1. WKHTMLTOPDF_FILENAME 

type: str

Absolute path to the wkhtmltopdf executable.

You must specify one that incorporates any need for an X Server (not the default /usr/bin/wkhtmltopdf). See https://stackoverflow.com/questions/9604625/. In brief, you can try

WKHTMLTOPDF_FILENAME = ''

to use the default, and if that fails, try

WKHTMLTOPDF_FILENAME = '/usr/bin/wkhtmltopdf'

and that should work if your version of wkhtmltopdf is a “headless” one using “patched Qt”; but if that fails, use

WKHTMLTOPDF_FILENAME = '/path/to/wkhtmltopdf.sh'

where wkhtmltopdf.sh is an executable script (chmod a+x ...) containing:

#!/usr/bin/env bash
xvfb-run --auto-servernum --server-args="-screen 0 640x480x16" \
    /usr/bin/wkhtmltopdf "$@"

For a recent version of wkhtmltopdf, fetch one from http://wkhtmltopdf.org/, e.g. v0.12.4 for your OS. Make sure you use one for “patched Qt”.

8.3.12.2. WKHTMLTOPDF_OPTIONS 

type: Dict[str, str]

Additional dictionary passed via pdfkit to wkhtmltopdf. See https://wkhtmltopdf.org/usage/wkhtmltopdf.txt. Specimen:

WKHTMLTOPDF_OPTIONS = {  # dict for pdfkit
    "page-size": "A4",
    "margin-left": "20mm",
    "margin-right": "20mm",
    "margin-top": "21mm",  # from paper edge down to top of content?
    "margin-bottom": "24mm",  # from paper edge up to bottom of content?
    "header-spacing": "3",  # mm, from content up to bottom of header
    "footer-spacing": "3",  # mm, from content down to top of footer
}

8.3.12.3. PDF_LOGO_ABS_URL 

type: str

Absolute URL to a file on your server containing a logo to be incorporated into PDFs generated by CRATE – typically, your institutional logo.

This URL is read by wkhtmltopdf. Example:

PDF_LOGO_ABS_URL = 'http://localhost/crate_logo'
# ... path on local machine, read by wkhtmltopdf
# Examples:
#   [if you're running a web server] 'http://localhost/crate_logo'
#   [Linux root path] file:///home/myuser/myfile.png
#   [Windows root path] file:///c:/path/to/myfile.png

8.3.12.4. PDF_LOGO_WIDTH 

type: str

Logo width, passed to an <img> tag in the HTML used to build PDF files. Tune this to your logo file (see PDF_LOGO_ABS_URL). Example:

PDF_LOGO_WIDTH = "75%"
# ... must be suitable for an <img> tag, but "150mm" isn't working; "75%" is.

8.3.12.5. TRAFFIC_LIGHT_*

The PDF generator also needs to be able to find the traffic-light icons, on disk (not via your web site), so specify file:// URLs for the following:

TRAFFIC_LIGHT_RED_ABS_URL = 'file:///somewhere/crate_anon/crateweb/static/red.png'  # noqa
TRAFFIC_LIGHT_YELLOW_ABS_URL = 'file:///somewhere/crate_anon/crateweb/static/yellow.png'  # noqa
TRAFFIC_LIGHT_GREEN_ABS_URL = 'file:///somewhere/crate_anon/crateweb/static/green.png'  # noqa

8.3.14. Local information links 

8.3.14.1. CHARITY_URL 

type: str

Absolute URL to an information page about your charity donation system, e.g.

CHARITY_URL = "http://www.cpft.nhs.uk/research.htm"

8.3.14.2. CHARITY_URL_SHORT 

Short “humanized” version of CHARITY_URL, for use as clickable text; e.g.

CHARITY_URL_SHORT = "www.cpft.nhs.uk/research.htm"

8.3.14.3. LEAFLET_URL_CPFTRD_CLINRES_SHORT 

Short “humanized” version of where to click to get the patient leaflet about your research database. (Used by .html templates.)

LEAFLET_URL_CPFTRD_CLINRES_SHORT = "www.cpft.nhs.uk/research.htm > CPFT Research Database"  # noqa

8.3.15. Specimen config file 

To obtain a specimen file, use

crate_print_demo_crateweb_config

Specimen web config:

# **Site-specific Django settings for CRATE web front end.**

# Put the secret stuff here.

# SPECIMEN FILE ONLY - edit to your own requirements.
# IT WILL NOT WORK until you've edited it.

# For help, see
# https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html

import logging
import os
from typing import List, TYPE_CHECKING

# Include the following if you want to use it in CELERYBEAT_SCHEDULE
# from celery.schedules import crontab

from crate_anon.common.constants import mebibytes
from crate_anon.crateweb.config.constants import ResearchDbInfoKeys as RDIKeys

if TYPE_CHECKING:
    from django.http.request import HttpRequest

log = logging.getLogger(__name__)

log.critical(
    "Well done - CRATE has found your crate_local_settings.py file at {}. "
    "However, you need to configure it for your institution's set-up, and "
    "remove this line.".format(os.path.abspath(__file__))
)


# =============================================================================
# Site URL configuration
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

# DJANGO_SITE_ROOT_ABSOLUTE_URL = "http://mymachine.mydomain"  # example for Apache  # noqa
# DJANGO_SITE_ROOT_ABSOLUTE_URL = "http://localhost:8000"  # for the Django dev server  # noqa
DJANGO_SITE_ROOT_ABSOLUTE_URL = "http://mymachine.mydomain"

FORCE_SCRIPT_NAME = ""
# FORCE_SCRIPT_NAME = ""  # example for Apache root hosting
# FORCE_SCRIPT_NAME = "/crate"  # example for CherryPy or Apache non-root hosting  # noqa


# =============================================================================
# Site security
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = "aaaaaaaaaaaaaaaaaa CHANGE THIS! aaaaaaaaaaaaaaaaaa"
# Run crate_generate_new_django_secret_key to generate a new one.

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = False
# ... when False, note that static files must be served properly

CRATE_HTTPS = True  # True: require HTTPS and disallow plain HTTP


# noinspection PyUnusedLocal
def always_show_toolbar(request: "HttpRequest") -> bool:
    return True  # Always show toolbar, for debugging only.


if DEBUG:
    ALLOWED_HOSTS = []  # type: List[str]
    DEBUG_TOOLBAR_CONFIG = {
        "SHOW_TOOLBAR_CALLBACK": always_show_toolbar,
    }
else:
    ALLOWED_HOSTS = ["*"]


# =============================================================================
# Celery configuration
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

BROKER_URL = ""

# =============================================================================
# Database configuration
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

DATABASES = {
    # See https://docs.djangoproject.com/en/1.8/ref/settings/#databases
    # -------------------------------------------------------------------------
    # Django database for web site (inc. users, audit).
    # -------------------------------------------------------------------------
    # Quick SQLite example:
    # 'default': {
    #     'ENGINE': 'django.db.backends.sqlite3',
    #     'NAME': '/home/myuser/somewhere/crate_db.sqlite3',
    # },
    # Quick MySQL example:
    "default": {
        "ENGINE": "django.db.backends.mysql",
        "HOST": "127.0.0.1",  # e.g. 127.0.0.1
        "PORT": 3306,  # local e.g. 3306
        "NAME": "crate_db",
        "USER": "someuser",
        "PASSWORD": "somepassword",
    },
    # -------------------------------------------------------------------------
    # Anonymised research database
    # -------------------------------------------------------------------------
    "research": {
        # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        # IT IS CRITICALLY IMPORTANT THAT THIS CONNECTION (i.e. its user's
        # access) IS READ-ONLY FOR THE RESEARCH DATABASES [1] AND HAS NO
        # ACCESS WHATSOEVER TO SECRET DATABASES (like the 'default' or
        # 'secret' databases) [2]. RESEARCHERS ARE GIVEN FULL ABILITY TO
        # EXECUTE SQL VIA THIS CONNECTION, AND CAN DO SO FOR ANY DATABASES
        # THAT THE CONNECTION PERMITS, NOT JUST THE ONE YOU SPECIFY
        # EXPLICITLY.
        #
        # [1] ... so researchers can't alter/delete research data
        # [2] ... so researchers can't see secrets
        # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        "ENGINE": "django.db.backends.mysql",
        "HOST": "127.0.0.1",  # e.g. 127.0.0.1
        "PORT": 3306,  # local, e.g. 3306
        "NAME": "anonymous_output",  # will be the default database; use None for no default database  # noqa
        "USER": "researcher",
        "PASSWORD": "somepassword",
    },
    # -------------------------------------------------------------------------
    # One or more secret databases for RID/PID mapping
    # -------------------------------------------------------------------------
    "secret_1": {
        "ENGINE": "django.db.backends.mysql",
        "HOST": "127.0.0.1",  # e.g. 127.0.0.1
        "PORT": 3306,
        "NAME": "anonymous_mapping",
        "USER": "anonymiser_system",
        "PASSWORD": "somepassword",
    },
    # -------------------------------------------------------------------------
    # Others, for consent lookup
    # -------------------------------------------------------------------------
    # Optional: 'cpft_crs'
    # Optional: 'cpft_pcmis'
    # Optional: 'cpft_rio_crate'
    # Optional: 'cpft_rio_datamart'
    # Optional: 'cpft_rio_raw'
    # Optional: 'cpft_rio_rcep'
    # ... see ClinicalDatabaseType in crate_anon/crateweb/config/constants.py
}

# Which database should be used to look up demographic details?
CLINICAL_LOOKUP_DB = "dummy_clinical"

# Which database should be used to look up consent modes?
CLINICAL_LOOKUP_CONSENT_DB = "dummy_clinical"

# Research database title (displayed in web site)
RESEARCH_DB_TITLE = "My NHS Trust Research Database"

# Database structure information for CRATE's query builders.
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa
RESEARCH_DB_INFO = [
    {
        # Unique name e.g. "myresearchdb":
        RDIKeys.NAME: "myresearchdb",
        # Human-friendly description e.g. "My friendly research database":
        RDIKeys.DESCRIPTION: "My friendly research database",
        # Database name as seen by the database engine:
        # - BLANK, i.e. "", for MySQL.
        # - BLANK, i.e. "", for PostgreSQL.
        # - The database name, for SQL Server.
        RDIKeys.DATABASE: "",
        # Schema name:
        # - The database=schema name, for MySQL.
        # - The schema name, for PostgreSQL (usual default: "public").
        # - The schema name, for SQL Server (usual default: "dbo").
        RDIKeys.SCHEMA: "dbo",
        # Fields not in the database, but used for SELECT AS statements for
        # some clinician views:
        # e.g. "my_pid_field":
        RDIKeys.PID_PSEUDO_FIELD: "my_pid_field",
        # e.g. "my_mpid_field":
        RDIKeys.MPID_PSEUDO_FIELD: "my_mpid_field",
        # Fields and tables found within the database:
        # e.g. "trid"
        RDIKeys.TRID_FIELD: "trid",
        # e.g. "brcid"
        RDIKeys.RID_FIELD: "brcid",
        # e.g. 1
        RDIKeys.RID_FAMILY: 1,
        # e.g. "patients"
        RDIKeys.MRID_TABLE: "patients",
        # e.g. "nhshash"
        RDIKeys.MRID_FIELD: "nhshash",
        # Descriptions, used for PID lookup and the like
        # e.g. "Patient ID (My ID Num; PID) for database X"
        RDIKeys.PID_DESCRIPTION: "Patient ID (My ID Num; PID) for database X",
        # e.g. "Master patient ID (NHS number; MPID)"
        RDIKeys.MPID_DESCRIPTION: "Master patient ID (NHS number; MPID)",
        # e.g. "Research ID (RID) for database X"
        RDIKeys.RID_DESCRIPTION: "Research ID (RID) for database X",
        # e.g. "Master research ID (MRID)"
        RDIKeys.MRID_DESCRIPTION: "Master research ID (MRID)",
        # e.g. "Transient research ID (TRID) for database X",
        RDIKeys.TRID_DESCRIPTION: (
            "Transient research ID (TRID) for database X"
        ),
        # To look up PID/RID mappings, provide a key for "secret_lookup_db"
        # that is a database alias from DATABASES:
        # e.g. "secret_1"
        RDIKeys.SECRET_LOOKUP_DB: "secret_1",
        # For the data finder: table-specific and default date column names
        RDIKeys.DATE_FIELDS_BY_TABLE: {},
        # e.g. ["default_date_field"]
        RDIKeys.DEFAULT_DATE_FIELDS: ["default_date_field"],
        # Column name giving time that record was updated
        # e.g. "_when_fetched_utc"
        RDIKeys.UPDATE_DATE_FIELD: "_when_fetched_utc",
    },
    # {
    #     RDIKeys.NAME: "similar_database",
    #     RDIKeys.DESCRIPTION: "A database sharing the RID with the first",
    #     RDIKeys.DATABASE: "similar_database",
    #     RDIKeys.SCHEMA: "similar_schema",
    #     RDIKeys.TRID_FIELD: "trid",
    #     RDIKeys.RID_FIELD: "same_rid",
    #     RDIKeys.RID_FAMILY: 1,
    #     RDIKeys.MRID_TABLE: "",
    #     RDIKeys.MRID_FIELD: "",
    #     RDIKeys.PID_DESCRIPTION: "",
    #     RDIKeys.MPID_DESCRIPTION: "",
    #     RDIKeys.RID_DESCRIPTION: "",
    #     RDIKeys.MRID_DESCRIPTION: "",
    #     RDIKeys.TRID_DESCRIPTION: "",
    #     RDIKeys.SECRET_LOOKUP_DB: "",
    #     RDIKeys.DATE_FIELDS_BY_TABLE: {},
    #     RDIKeys.DEFAULT_DATE_FIELDS: [],
    #     RDIKeys.UPDATE_DATE_FIELD: "_when_fetched_utc",
    # },
    # {
    #     RDIKeys.NAME: "different_database",
    #     RDIKeys.DESCRIPTION: "A database sharing only the MRID with the first",  # noqa: E501
    #     RDIKeys.DATABASE: "different_database",
    #     RDIKeys.SCHEMA: "different_schema",
    #     RDIKeys.TRID_FIELD: "trid",
    #     RDIKeys.RID_FIELD: "different_rid",
    #     RDIKeys.RID_FAMILY: 2,
    #     RDIKeys.MRID_TABLE: "hashed_nhs_numbers",
    #     RDIKeys.MRID_FIELD: "nhshash",
    #     RDIKeys.PID_DESCRIPTION: "",
    #     RDIKeys.MPID_DESCRIPTION: "",
    #     RDIKeys.RID_DESCRIPTION: "",
    #     RDIKeys.MRID_DESCRIPTION: "",
    #     RDIKeys.TRID_DESCRIPTION: "",
    #     RDIKeys.SECRET_LOOKUP_DB: "",
    #     RDIKeys.DATE_FIELDS_BY_TABLE: {},
    #     RDIKeys.DEFAULT_DATE_FIELDS: [],
    #     RDIKeys.UPDATE_DATE_FIELD: "_when_fetched_utc",
    # },
]

# Which database (from those defined in RESEARCH_DB_INFO above) should be used
# to look up patients when contact requests are made?
# Give the 'name' attribute of one of the databases in RESEARCH_DB_INFO.
# Its secret_lookup_db will be used for the actual lookup process.
RESEARCH_DB_FOR_CONTACT_LOOKUP = "myresearchdb"

# Definitions of source database names in CRATE NLP tables
NLP_SOURCEDB_MAP = {"SOURCE_DATABASE": "research"}

# For the automatic query generator, we need to know the underlying SQL dialect
# Options are
# - "mysql" => MySQL
# - "mssql" => Microsoft SQL Server
RESEARCH_DB_DIALECT = "mysql"

DISABLE_DJANGO_PYODBC_AZURE_CURSOR_FETCHONE_NEXTSET = True

# =============================================================================
# Archive views
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

# e.g. /home/somewhere/my_archive_templates
ARCHIVE_TEMPLATE_DIR = "/home/somewhere/my_archive_templates"
# e.g. /home/somewhere/my_archive_templates/cache
ARCHIVE_TEMPLATE_CACHE_DIR = "/tmp/somewhere/my_archive_template_cache"
# e.g. /home/somewhere/my_archive_templates/static
ARCHIVE_STATIC_DIR = "/home/somewhere/my_archive_templates/static"
ARCHIVE_ROOT_TEMPLATE = "root.mako"
# e.g. /home/somewhere/my_archive_attachments
ARCHIVE_ATTACHMENT_DIR = "/home/somewhere/my_archive_attachments"
ARCHIVE_CONTEXT = {}
CACHE_CONTROL_MAX_AGE_ARCHIVE_ATTACHMENTS = 0
CACHE_CONTROL_MAX_AGE_ARCHIVE_TEMPLATES = 0
CACHE_CONTROL_MAX_AGE_ARCHIVE_STATIC = 0


# =============================================================================
# Database extra help file
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

# If specified, this must be a string that is an absolute filename of TRUSTED
# HTML that will be included.
DATABASE_HELP_HTML_FILENAME = None


# =============================================================================
# Local file storage (for PDFs etc).
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

# Where should we store the files? Make this directory (and don't let it
# be served by a generic web server that doesn't check permissions).
# e.g. /srv/crate_filestorage
PRIVATE_FILE_STORAGE_ROOT = "/srv/crate_filestorage"

# Serve files via Django (inefficient but useful for testing) or via Apache
# with mod_xsendfile (or other web server configured for the X-SendFile
# directive)?
XSENDFILE = False

# How big will we accept?
MAX_UPLOAD_SIZE_BYTES = mebibytes(10)


# =============================================================================
# Outgoing e-mail
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

# -----------------------------------------------------------------------------
# General settings for sending e-mail from Django
# -----------------------------------------------------------------------------
# https://docs.djangoproject.com/en/1.8/ref/settings/#email-backend

#   default backend:
# EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
#   bugfix for servers that only support TLSv1:
# EMAIL_BACKEND = "cardinal_pythonlib.django.mail.SmtpEmailBackendTls1"

EMAIL_HOST = "smtp.somewhere.nhs.uk"
EMAIL_PORT = 587  # usually 25 (plain SMTP) or 587 (STARTTLS)
# ... see https://www.fastmail.com/help/technical/ssltlsstarttls.html
EMAIL_HOST_USER = "myuser"
EMAIL_HOST_PASSWORD = "mypassword"
EMAIL_USE_TLS = True
EMAIL_USE_SSL = False

# Who will the e-mails appear to come from?
EMAIL_SENDER = "My NHS Trust Research Database - DO NOT REPLY <noreply@somewhere.nhs.uk>"  # noqa

# -----------------------------------------------------------------------------
# Additional settings
# -----------------------------------------------------------------------------
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

# During development, we route all consent-related e-mails to the developer.
# Switch SAFETY_CATCH_ON to False for production mode.
SAFETY_CATCH_ON = True
DEVELOPER_EMAIL = "testuser@somewhere.nhs.uk"

VALID_RESEARCHER_EMAIL_DOMAINS = []  # type: List[str]
# ... if empty, no checks are performed (any address is accepted)


# =============================================================================
# Research Database Manager (RDBM) details
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

RDBM_NAME = "John Doe"
RDBM_TITLE = "Research Database Manager"
RDBM_TELEPHONE = "01223-XXXXXX"
RDBM_EMAIL = "research.database@somewhere.nhs.uk"
RDBM_ADDRESS = [
    "FREEPOST SOMEWHERE_HOSPITAL RESEARCH DATABASE MANAGER"
]  # a list


# =============================================================================
# Administrators/managers to be notified of errors
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

# Exceptions get sent to these people.
ADMINS = [
    ("Mr Administrator", "mr_admin@somewhere.domain"),
]


# =============================================================================
# PDF creation
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

WKHTMLTOPDF_FILENAME = ""
# WKHTMLTOPDF_FILENAME = "/home/rudolf/dev/wkhtmltopdf/wkhtmltox/bin/wkhtmltopdf"  # noqa
# WKHTMLTOPDF_FILENAME = "/usr/bin/wkhtmltopdf"

WKHTMLTOPDF_OPTIONS = {  # dict for pdfkit
    "page-size": "A4",
    "margin-left": "20mm",
    "margin-right": "20mm",
    "margin-top": "21mm",  # from paper edge down to top of content?
    "margin-bottom": "24mm",  # from paper edge up to bottom of content?
    "header-spacing": "3",  # mm, from content up to bottom of header
    "footer-spacing": "3",  # mm, from content down to top of footer
}

PDF_LOGO_ABS_URL = "http://localhost/crate_logo"
# ... path on local machine, read by wkhtmltopdf
# Examples:
#   [if you're running a web server] "http://localhost/crate_logo"
#   [Linux root path] file:///home/myuser/myfile.png
#   [Windows root path] file:///c:/path/to/myfile.png

PDF_LOGO_WIDTH = "75%"
# ... must be suitable for an <img> tag, but "150mm" isn't working; "75%" is.
# ... tune this to your logo file (see PDF_LOGO_ABS_URL)

# The PDF generator also needs to be able to find the traffic-light pictures,
# on disk (not via your web site):
TRAFFIC_LIGHT_RED_ABS_URL = (
    "file:///somewhere/crate_anon/crateweb/static/red.png"
)
TRAFFIC_LIGHT_YELLOW_ABS_URL = (
    "file:///somewhere/crate_anon/crateweb/static/yellow.png"
)
TRAFFIC_LIGHT_GREEN_ABS_URL = (
    "file:///somewhere/crate_anon/crateweb/static/green.png"
)


# =============================================================================
# Consent-for-contact settings
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

# For how long may we contact discharged patients without specific permission?
# Use 0 for "not at all".
PERMITTED_TO_CONTACT_DISCHARGED_PATIENTS_FOR_N_DAYS = 3 * 365

# Donation to charity for clinician response (regardless of the decision):
CHARITY_AMOUNT_CLINICIAN_RESPONSE = 1.0  # in local currency, e.g. GBP

# Note that using headers/footers requires a version of wkhtmltopdf built using
# "patched Qt". See above.
# Fetch one from http://wkhtmltopdf.org/, e.g. v0.12.4 for your OS.
PDF_LETTER_HEADER_HTML = ""
PDF_LETTER_FOOTER_HTML = ""


# =============================================================================
# Local information links
# =============================================================================
# See https://crateanon.readthedocs.io/en/latest/website_config/web_config_file.html  # noqa

CHARITY_URL = "http://www.cpft.nhs.uk/research.htm"
CHARITY_URL_SHORT = "www.cpft.nhs.uk/research.htm"
LEAFLET_URL_CPFTRD_CLINRES_SHORT = (
    "www.cpft.nhs.uk/research.htm > CPFT Research Database"
)

ANONYMISE_API = {
    "HASH_KEY": "aaaa CHANGE THIS! aaaa",
    "ALLOWLIST_FILENAMES": {},
    "DENYLIST_FILENAMES": {},
}