Skip to main content
Version: 2.x (upcoming)

Infrastructure

These files of the docker-deployment repository configure the containers around OpenCDMP: the Docker Compose environment file, PostgreSQL, RabbitMQ, and Elasticsearch. Keycloak has its own page, Keycloak setup.

FileConfigures
.envImage versions, the webapp settings, ports, the host folders of the configuration, logs, and storage, and the Elasticsearch passwords
config-files/postgres/postgres.env and config-files/postgres/init-scripts/The PostgreSQL superuser and the databases created on the first start
config-files/rabbitmq/rabbitmq.envThe RabbitMQ user and virtual host
config-files/elk/elasticsearch/The Elasticsearch node settings and the script that creates the account of the API service
config-files/keycloak/keycloak.envKeycloak; see Keycloak environment variables

The example values are demo defaults. Replace them as described in Before going to production.


Docker Compose environment (.env)​

The .env file sits next to docker-compose.yml and docker-compose.override.yml. Docker Compose reads it to fill in the image versions, the ports, and the host folders of the Compose files, and the webapp container receives the frontend settings from it.

Registries and image versions​

Env VariableDescriptionRequiredDefault ValueExample
GITHUB_REGISTRYRegistry of the OpenCDMP images (API service, webapp, file transformers, evaluator, and Zenodo deposit service), which Compose pulls as <GITHUB_REGISTRY>/opencdmp/<image>. The images are public on Docker Hub; set docker.io.Yesghcr.iodocker.io
DOCKER_REGISTRYRegistry and namespace of the notification and annotation service images, which Compose pulls as <DOCKER_REGISTRY>/java-pomolo-notification-api and <DOCKER_REGISTRY>/java-pomolo-annotation-api. The images are public on Docker Hub under citesa; set docker.io/citesa.Yescrepo.cite.grdocker.io/citesa
PROXY_TAGVersion of the Nginx proxy image.Yes-mainline-alpine
MAIN_APP_TAGVersion of the API service and webapp images.Yes-2.0.0
NOTIFICATION_APP_TAGVersion of the notification service image.Yes-0.0.23
ANNOTATION_APP_TAGVersion of the annotation service image.Yes-0.0.25
DOCX_APP_TAGVersion of the DOCX file transformer image.Yes-2.0.0
JSON_APP_TAGVersion of the RDA JSON file transformer image.Yes-2.0.0
EVALUATOR_APP_TAGVersion of the RDA maDMP evaluator image.Yes-2.0.0
ZENODO_APP_TAGVersion of the Zenodo deposit service image.Yes-2.0.0
POSTGRES_TAGPostgreSQL version.Yes-17-alpine
ELK_TAGElasticsearch version.Yes-9.1.4
KEYCLOAK_TAGKeycloak version.Yes-26.1.0
RABBITMQ_TAGRabbitMQ version.Yes-4.1.4-management-alpine
PDF_SERVICE_TAGVersion of the Gotenberg PDF service.Yes-8.23

The changelog lists the service versions of each release. Check that each tag exists on Docker Hub (for the notification and annotation services, citesa/java-pomolo-notification-api and citesa/java-pomolo-annotation-api) before you pull.

Application settings​

Env VariableDescriptionRequiredDefault ValueExample
PROFILEProfile of the OpenCDMP services. Each service reads app-<profile>.env and the <name>-<profile>.yml overrides from its config folder, so changing it means renaming those files.Yestestproduction
WEBAPP_BASE_URLBase path of the webapp, when it is not served at the root. Commented out in the example.No//opencdmp/
WEBAPP_API_URLAddress of the API, as the browser reaches it.Yes-https://opencdmp.example.org/api/
INSTALLATION_URLAddress of the webapp.Yes-https://opencdmp.example.org/
DEFAULT_CULTUREDefault language of the webapp.Noenen
KEYCLOAK_ADDRESSAddress of Keycloak, including its relative path.Yes-https://opencdmp.example.org/auth/
KEYCLOAK_REALMKeycloak realm.Yes-OpenCDMP
KEYCLOAK_CLIENT_IDKeycloak client of the webapp.Yes-webapp
KEYCLOAK_SCOPEScopes the webapp requests. They must include api and, for comments and notifications, annotation and notification.Noopenid profile email address phone api notification annotation identity_provideropenid profile email api notification annotation identity_provider
LOGGING_ENABLEDEnables the webapp's logging to the browser console.Nofalsetrue

These settings are described in more detail in the frontend configuration.

Ports and paths​

Env VariableDescriptionRequiredDefault ValueExample
SERVER_PORTPort that the file transformers and the evaluator listen on inside their containers.Yes80808080
PROXY_APP_PORTHost port of the Nginx proxy, which serves the application.Yes8180443
POSTGRES_PORTHost port of PostgreSQL, published on 127.0.0.1 only.Yes54335433
CONFIG_FILES_PATHHost folder with the configuration of every service.Yes./config-files/srv/opencdmp/config-files
LOG_FILES_PATHHost folder that receives the log files of every service.Yes./logs/var/log/opencdmp
STORAGE_PATHHost folder with the data of every service (database, search index, uploaded files).Yes./storage/srv/opencdmp/storage

Where a variable has no built-in default, the Default Value column shows the value of the example .env file. The Elasticsearch passwords are also set in .env; see Elasticsearch. The example serves the application on http://opencdmp.localhost:8180/. To change the port or the host name, update the URLs above together with the files under config-files/ and docker-compose.override.yml, which repeat them (see Installation).


Startup order​

The Docker Compose files start the services in dependency order, using health checks, so a single docker compose up -d starts the whole installation:

  1. opencdmp.init prepares the storage and log folders of Elasticsearch and Keycloak, then exits.
  2. Elasticsearch starts and installs its analysis-icu plugin on the first start; opencdmp.elasticsearch.setup then creates the account of the API service and exits.
  3. PostgreSQL, Keycloak (which imports the OpenCDMP realm on the first start), and RabbitMQ start and report healthy.
  4. The API service starts once the above are ready and migrates the database (see Database migrations). The notification and annotation services start once PostgreSQL, Keycloak, and RabbitMQ are ready.
  5. The webapp, the file transformers, the evaluator, the Zenodo deposit service, and the proxy start after the API service.

PostgreSQL​

The postgres.env file​

The file is in config-files/postgres. PostgreSQL reads it only when it creates the database cluster, on the first start with an empty data folder.

Env VariableDescriptionRequiredDefault ValueExample
POSTGRES_USERPostgreSQL superuser.Nopostgrespostgres
POSTGRES_PASSWORDPassword of the superuser. The example repeats it in the database settings of the API, notification, and annotation services and of Keycloak.Yes-<postgres-password>

In the Docker example, PostgreSQL is published on 127.0.0.1:${POSTGRES_PORT} (port 5433 by default), so it is reachable from the host machine only. Its data is stored in ${STORAGE_PATH}/postgres/data.

Initialization scripts​

The scripts in config-files/postgres/init-scripts run in name order when the container starts with an empty data folder. If the database already exists, they do not run again.

The init script 00-db-init.sql only creates the empty databases:

  • opencdmp, shared by the API service, the notification service, and the annotation service
  • keycloak, used by Keycloak

The init scripts do not create the OpenCDMP schema or insert any data. The API service builds the schema, including the tables of the notification, annotation, and KPI services and the default seed data, the first time it starts against the empty opencdmp database, and applies later schema changes at startup. See Database migrations.

note

If your copy of this folder has schema or seed scripts from an earlier Docker example, remove them before you create a new database, so that the API service builds the schema itself.

The example connects every service with the postgres superuser. To give each service its own database user, see Before going to production. The user of the API service must be allowed to create and alter tables in the opencdmp database (see Database user permissions).


RabbitMQ​

The API service publishes events to RabbitMQ, and the notification and annotation services consume them. The rabbitmq.env file is in config-files/rabbitmq. RabbitMQ creates the user on the first start, with an empty data folder (${STORAGE_PATH}/rabbitmq).

Env VariableDescriptionRequiredDefault ValueExample
RABBITMQ_DEFAULT_USERRabbitMQ user, created on the first start. The OpenCDMP services connect with it (RABBIT_USER).Noguestopencdmp
RABBITMQ_DEFAULT_PASSPassword of that user (RABBIT_PASS of the OpenCDMP services).Noguest<rabbitmq-password>
RABBITMQ_DEFAULT_VHOSTDefault virtual host.No//

The services that connect to RabbitMQ set RABBIT_HOST, RABBIT_PORT, RABBIT_USER, and RABBIT_PASS, and the names of their queue and exchange; see the API service, notification service, and annotation service configuration. All of them must use the same exchange (opencdmp_queue in the example).

warning

RabbitMQ normally accepts the guest user only from localhost; the official Docker image lifts that restriction, which is how the example connects with it over the Docker network. Set your own user and password for any real installation.


Elasticsearch​

The API service indexes plans and assets in Elasticsearch for its listings and searches. The Docker example runs a single node with security enabled.

Passwords​

Both Elasticsearch passwords are set in the .env file, and Docker Compose passes them to the containers that need them:

Env VariableDescriptionRequiredDefault ValueExample
ELASTIC_PASSWORDPassword of the built-in elastic superuser. Elasticsearch sets it on the first start; the health check of the container and the setup script sign in with it.Yeselastic<elastic-password>
ELASTIC_API_PASSWORDPassword of the user that the API service connects as. The setup script creates or updates the user with it, and Docker Compose passes it to the API service as ELASTIC_PASS.Yesopencdmp<elastic-api-password>

Elasticsearch applies ELASTIC_PASSWORD only on the first start, with an empty data folder, so set it before that start. To change it later, see Replace the demo secrets.

Node settings​

config-files/elk/elasticsearch/elasticsearch.yml sets a single-node cluster (discovery.type: single-node, cluster opencdmp-cluster), turns on security (xpack.security.enabled: true) with the basic license, and turns off the GeoIP downloader. The opencdmp.elasticsearch service installs the analysis-icu plugin on its first start, because the index analyzers of the API service need it, and sets the heap with ES_JAVA_OPTS (-Xmx1g -Xms1g, half of its 2 GB memory limit). See Size the containers before you change memory settings.

The account of the API service​

The setup script config-files/elk/elasticsearch/index_init.sh creates the role and the user that the API service connects with: the user opencdmp-api-test with the password ELASTIC_API_PASSWORD, allowed on the indexes opencdmp-plans-test and opencdmp-descriptions-test. These values match ELASTIC_USER, ELASTIC_PASS, ELASTIC_PLAN_INDEX, and ELASTIC_DESCRIPTION_INDEX in the backend configuration. Keep the index names of the role and of the API configuration the same.

The index can be rebuilt from the database at any time with Rebuild plan and asset indexes on the Maintenance page.


See also​