Commcare-cloud docs have moved to a new place.

About this changelog

Below are the list of changes (the newest first) to commcare-cloud and CommCareHQ that need to be applied on your environment to keep it up to date.


2022-06-03 Postgres upgrade from 9.6 to 10

This change upgrade PostgreSQL from 9.6 to 10 version. As part of our ongoing effort to keep CommCare HQ up to date with the latest tools and libraries we have updated PostgreSQL from version 9.6 to version 10.

2022-05-25 update-supervisor-confs

Run command to update shell runner scripts for django and celery.

2022-04-19 Update PgBouncer configuration to support multiple processes

This change updates the PgBouncer role to support multiple processes on a single machine, so that more CPU cores can be used (PgBouncer is single-threaded and uses only one CPU core by default).

2022-04-11 Upgrade Node.js to 14.19.1 and npm to 7.24.2

Node.js 12.x LTS is reaching its end of life 30th April 2022, so node and npm must be upgraded on all machines.

2021-11-02 Upgrade CommCare HQ to Python 3.9

Install Python 3.9 and build a new virtualenv for CommCare HQ.

2021-06-09 Migrate forms & cases from Couch to SQL

A series of management commands must be run to check for and migrate domains’ forms & cases from Couch to SQL.

2021-03-23 Optionally configure auditcare db

Some enterprise deployments of CommCare use a backend feature called “auditcare” as part of their audit logging strategy. Auditcare is enabled by default, so it is active unless you went out of your way to disable it when you configured CommCare. Historically it has used CouchDB as its data backend, but it is being switched to use PostgreSQL instead. If you care about this feature then you may want to carefully consider this change log before your next commcare deploy; otherwise you can ignore.

2021-03-16 Update Formplayer request mapping

A small change to the nginx request matching for Formplayer requests that will prevent an additional / from being prepended to the Formplayer requests.

2021-01-11 Dropping support for Python 2

Python 3.6 is supported and now the preferred version to use. In anticipation of dropping Python 2 support, an error will be displayed when running commands with Python 2. Instructions for upgrading to Python 3 are provided in the error message. An option is provided to temporarily revert to Python 2 (see update steps for details).

Python 2 support will end on 2021-03-04.

2021-01-08 Install new apt requirements on machines running commcarehq code

We will be adding SAML 2.0 Single Sign On (SSO) support over the next few months which requires installing new apt packages as dependencies of python requirements.

2020-11-16 update-letsencrypt-to-alternate-chain

On January 11 2021, Let’s Encrypt will change its default certificate chain from using the legacy Identrust root certificate, to its own modern ISRG root certificate. In order to maintain backwards compatibility with existing mobile devices it is necessary to keep using the Identrust certificate chain.

2020-10-22 Update PostgreSQL monit configurations to be version specific

Update PostgreSQL monit configuration files to be version specific and use systemctl

2020-10-14 Run command to update Supervisor configurations

Run management command to remove unused errand-boy processes.

2020-07-23 Run command to clear sensitive information from Django sessions

It is stronly recommended to sanitize legacy Django sessions after upgrading to Django 2.2.

2020-07-09 Switch to new package manager

To stay up to date with javascript package security alerts, we need to make sure we are using a javascript package manager that is supported by Github.

2020-04-16 Update git to the latest version

Due to a high-severity security advisory on the popular version control software program git, observing security best practices dictates upgrading to one of the git versions designated as “Patched” such as 2.26.1.

2020-02-28 Update deploy CommandArgs

We are adding support for deploying from a specific commit hash or tag, replacing the deploy command’s commcare-branch argument with a more general commcare-rev argument.

2020-02-05 ES upgrade from 1.7.6 to 2.4.6

This change upgrade Elasticsearch from 1.7.6 to 2.4.6 version. CommCare HQ releases after April 2, 2020 will not continue to support Elasticsearch 1.7.6, so we strongly recommend applying this change before then.

2019-11-06 Update Formplayer Configuration

Some properties in the Formplayer configuration have changed names.

2019-08-30 Upgrade Sentry

The Sentry SDK used by CommCare HQ has been updated and along with it we have updated the configuration parameters.

2019-08-23 Add deploy Command

In order to provide a consistent user interface while making underlying changes, we are replacing the commcare-cloud <env> fab deploy command with a more concise commcare-cloud <env> deploy command.

2019-08-23 Removing support for Riak CS

We are removing support for deploying Riak CS clusters in commcare-cloud

2019-08-23 Fix python3 virutalenvs (Deprecated)

Update (2019-11-27): This fix is no longer necessary as it has been superceded by changes to the deploy script that make this change automatically if necessary.

This fixes a bug with how python3 virtualenvs were created by ansible. This fix needs to be applied to any machine which has a python3 virtualenv that was created by commcare-cloud.

The fix is also safe to run on all CommCare hosts.

2019-08-21 Move remaining management commands explicitly

This change requires editing app-processes.yml to add some of processes to the management_comamnds section

2019-08-20 Rename management commands explicitly

This change requires editing app-processes.yml to rename some of the processes in the management_comamnds section

2019-07-17 Define management commands explicitly

This change requires changing app-processes.yml to include a list of management comamnds to run

2018-07-18 Upgrade to Python 3

This change installs Python 3.6.8, builds a new virtualenv, and runs CommCare HQ in Python 3.

2019-05-13 Install Pango

This change installs pango and its dependencies for the weasyprint library which has been added as a requirement to commcare-hq for proper pdf printing of unicode fonts

2019-02-26 Fix to restart nginx after every letsencrypt cert auto-renewal

Previously you had to manually restart nginx every time letsencrypt auto-renewed, which was about every two months. We believed we had fixed this with Restart nginx after every letsencrypt cert auto-renewal, but there was an error in our setup at that time that made it not work as intended.

2019-04-05 Update RabbitMQ logging configuration

This change updates the RabbitMQ logging configuration to change the log level from info to warning.

2019-02-27 Remove celery results backend from localsettings

Upgrading to celery 4.x requires removing the dependency on django-celery, which means that its results backend will no longer be available. This removes the django-celery backend as the default from localsettings, so the results backend can be specified by commcare-hq settings instead.

2019-02-26 Split pgbouncer vars from postgresql vars

This change extracts a new role from the existing postgresql role for installing and configuring pgbouncer.

As a result of this change the postgresql.yml environment configuration file needs to be changed to split out the postgresql vars from the pgbouncer vars.

2019-02-27 Only monitor specific RabbitMQ queues

Datadog RabbitMQ monitoring restricts the number of queues it can monitor to 200. To avoid hitting this limit on large scale deployments we limit the queues being monitored to only the primary queues.

2019-02-22 Update supervisor confs to invoke celery directly

Upgrading to celery 4.x requires removing the dependency on django-celery, which means that the celery management command becomes unavailable. This prepares for that by invoking the celery command directly.

2019-02-22 Separate celery datadog http check

This adds a specific http check for the celery check (serverup.txt?only=celery) to datadog. Environments that are not relying on datadog for monitoring can ignore this change.

2019-02-11 Add tag to datadog http checks

This change adds “check_type” tag to the http_check datadog integration. This change applies only to envs using datadog for monitoring.

2019-02-11 Java upgrade for formplayer

Previously, Formplayer was running on Java 7. This change updates us to Java 8 for formplayer.

2019-02-01 Generalize load case from fixture feature

Previously loading a case from a fixture required the fixture to be an attribute. This change allows using non-attributes from the fixture.

2019-01-16 Fix encrypted temp directory permissions

This is a followup to Added encrypted temporary directory in which we introduced an encrypted directory for temp files. In its original implementation, this file was owned by root, and processes were unable to write to it.

This changes the directory to be owned by cchq, allowing our processes to write to the file.

2019-01-02 Restart nginx after every letsencrypt cert auto-renewal

Update 2019-02-26: There was a bug in this fix and it has been superceded by Fix to restart nginx after every letsencrypt cert auto-renewal.

Previously you had to manually restart nginx every time letsencrypt auto-renewed, which was about every two months.

2018-12-15 Blob Metadata Migration - part 2

Form submission attachment metadata is being consolidated in the blob metadata table in SQL. This migration consists of a series of commands that will consolidate the data in your environment.

2018-09-24 Blob Metadata Migration - part 1

Blob metadata needs to be migrated from CouchDB to SQL. This migration consists of a series of commands that will move the data in your environment.

2018-11-26 Reorganize pillows

Pillows read changes from kafka and do various processing such as sending them to elasticsearch, transforming into a UCR table row etc. A doc for same change is read multiple times for each processor, since there are separte pillows for each processor. This is inefficient, so we have combined multiple processors that apply for a given document type (also called KAFKA_TOPIC) such as form/case/user under one pillow. For e.g. A new single case-pillow pillow replaces various old pillows that process case changes such as CaseToElasticsearchPillow, CaseSearchToElasticsearchPillow, ReportCaseToElasticsearchPillow, and kafka-ucr-main etc.

2018-11-20 New Case Importer Celery Queue

Importing cases is often a time-sensitive task, and prolonged backlogs are very visible to users. It will be useful to have a separate queue specifically for case imports, to improve visibility into backups as well as typical runtimes. Additionally, this is a first step towards allocating resources specifically for case imports, should that become necessary.

2018-08-16 Support multiple Kafka brokers

Large scale deployments of CommCare require scaling out Kafka brokers to support the high traffic volume (as well as for high availability). Up until now CommCare has only supported a single broker.

2018-08-16 Add queue for analytics tasks

Tasks for analytics reporting have been separated into a new analytics celery queue.

2018-07-25 Update Supervisor

Ubuntu 14.04 apt-get install supervisor installs supervisor 3.0b. We occasionally have issues that could be related to supervisor, such as processes not stopping correctly. To rule it out as a possible cause, we decided it was better to be on a later version of supervisor, and one that’s not in beta.

2018-07-13 Update supervisor service definitions

There are several CommCare specific processes that are defined in supervisor configuration files. This change decouples the process definitions from code.

2018-06-11 Added encrypted temporary directory

Some of the CommCare processes make use of temporary files to store client data (such as data exports) so in order to keep that data protected we have modified the setup to use an encrypted temporary directory.