CommCare Cloud

Setting up a new CommCareHQ environment on a single machine

This document will walk you through the process of setting up a new monolith server to run CommCareHQ

Prerequisites

Step 1: Prepare system for automated deploy

  1. Enable root login via ssh

    On a standard Ubuntu install, the root user is not enabled or allowed to ssh. The root user will only be used initially, and will then be disabled automatically by the install scripts.

    Make a root password and store it somewhere safe for later reference.

    Set the root password:

     $ sudo passwd root
    

    Enable the root user

     $ sudo passwd -u root
    

    Edit /etc/ssh/sshd_config:

     $ sudo nano /etc/ssh/sshd_config
    

    and add the following line at the bottom:

     PermitRootLogin yes
    

    Next, search for PasswordAuthentication and make sure it’s set to yes

    Then restart SSH:

     $ sudo service ssh reload
    
  2. Install Required Packages

     $ sudo apt update && sudo apt install python-pip sshpass
     $ sudo pip install ansible virtualenv virtualenvwrapper --ignore-installed six
    
  3. Initialize a log file to be used in the installation process.

     $ sudo touch /var/log/ansible.log && sudo chmod 666 /var/log/ansible.log
    

Step 2: Download and configure the commcare-cloud environment

  1. Clone the sample environment into the environments folder:

     $ git clone https://github.com/dimagi/sample-environment.git environments
    

    You can read more about the files contained in this environments folder here.

  2. Next, we’re going to set up an encrypted “ansible vault” file. This will store all the passwords used in this CommCare environment. You’ll need to create a strong password and save it somewhere safe. This is the master password that grants access to the vault. You’ll need it for any future changes to this file, as well as when you deploy or make configuration changes to this machine.

    Encrypt the provided vault file, using that password:

     $ ansible-vault encrypt ~/environments/monolith/vault.yml
    

    More information on ansible vault can be found in the Ansible help pages.

    You can read more about how we use this vault file here.

Add passwords to the vault file

In the vault file, change each field that has the value CHANGE ME to a strong, unique password or a unique, useful username for that service. These usernames and passwords will be used by the commcare-cloud script to create the required database and system users, and no action is needed to create these users yourself.

$ ansible-vault edit ~/environments/monolith/vault.yml

Set the network interface name and ipaddress

  1. Find the name and IP address of the network interface of your machine, and note it down. You can do this by running

     $ ip addr
    

    This will give an output that looks like:

     $ ip addr
     1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
         link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
         inet 127.0.0.1/8 scope host lo
            valid_lft forever preferred_lft forever
         inet6 ::1/128 scope host
            valid_lft forever preferred_lft forever
     2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
         link/ether 08:00:27:48:f5:64 brd ff:ff:ff:ff:ff:ff
         inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic enp0s3
            valid_lft 85228sec preferred_lft 85228sec
         inet6 fe80::a00:27ff:fe48:f564/64 scope link
            valid_lft forever preferred_lft forever
    

    Here, the network interface we are interested in is enp0s3, which has an IP address of 10.0.2.15. Note these values down.

  2. Open the environments/monolith/inventory.ini file in an editor

     $ nano ~/environments/monolith/inventory.ini
    

    Replace the word localhost on the second line with the IP address you found in the previous step.

    Uncomment and set the value of ufw_private_interface to the network interface of your machine that we found in the previous step.

Add a public key and user to commcare-cloud

Even though we will be running all commands locally, we still need to add the user’s public key.

  1. Generate a public key (optional):

    If your user already has an ssh key pair, ignore this step.

     $ ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
    
  2. Add your public key to commcare-cloud by copying it to ~/environments/_authorized_keys

    For example:

     $ cp ~/.ssh/id_rsa.pub ~/environments/_authorized_keys/$(whoami).pub
    
  3. Add your system username to the present section of ~/environments/_users/admins.yml. This username should correspond to the name you’ve used for the public key in the previous step.

     $ nano ~/environments/_users/admins.yml
    

Install commcare-cloud

Install commcare-cloud onto the monolith:

$ git clone https://github.com/dimagi/commcare-cloud.git
$ source commcare-cloud/control/init.sh

and when you see it ask you this:

Do you want to have the CommCare Cloud environment setup on login?
(y/n):

answer with y.

For more information, see Installing commcare-cloud

Run first-time configuration

  1. Configure commcare-cloud

     $ COMMCARE_CLOUD_ENVIRONMENTS=/home/$(whoami)/environments manage-commcare-cloud configure
    
    

    You will see a few prompts that will guide you through the installation. Answer the questions as follows for a standard installation:

     Do you work or contract for Dimagi? [y/N]n
    
     I see you have COMMCARE_CLOUD_ENVIRONMENTS set to /home/{username}/environments in your environment
     Would you like to use environments at that location? [y/N]y
    
  2. As prompted, add the commcare-cloud config to your bash profile to set the correct paths:

     $ echo "source ~/.commcare-cloud/load_config.sh" >> ~/.bash_profile
    
  3. Load the commcare-cloud config so it takes effect immediately:

     $ source ~/.commcare-cloud/load_config.sh
    
  4. Copy the example fab config file:

     $ cp ~/commcare-cloud/src/commcare_cloud/fab/config.example.py ~/commcare-cloud/src/commcare_cloud/fab/config.py
    
  5. Update the known hosts file

     $ commcare-cloud monolith update-local-known-hosts
    

    You may be prompted for the ansible vault password that you entered earlier.

Step 3: Install all the services onto the monolith

In this step, you will be prompted for the vault password from earlier. You will also be prompted for an SSH password. This is the root user’s password. After this step, the root user will not be able to log in via SSH.

$ commcare-cloud monolith deploy-stack --first-time -e 'CCHQ_IS_FRESH_INSTALL=1'
This command will apply without running the check first. Continue? [y/N]y
ansible-playbook /home/{username}/commcare-cloud/src/commcare_cloud/ansible/deploy_stack.yml -i /home/{username}/environments/monolith/inventory.ini -e @/home/{username}/environments/monolith/vault.yml -e @/home/{username}/environments/monolith/public.yml -e @/home/{username}/environments/monolith/.generated.yml --diff --tags=bootstrap-users -u root --ask-pass --vault-password-file=/home/{username}/commcare-cloud/src/commcare_cloud/ansible/echo_vault_password.sh --ask-pass --ssh-common-args -o=UserKnownHostsFile=/home/{username}/environments/monolith/known_hosts
Vault Password for 'monolith': <password from encrypting vault.yml>
SSH password:<root user's password>

This will run a series of ansible commands that will take quite a long time to run.

If there are failures during the install, which may happen due to timing issues, you can continue running the playbook with:

$ commcare-cloud monolith deploy-stack --skip-check -e 'CCHQ_IS_FRESH_INSTALL=1'

Step 4: Deploy CommCare HQ

Deploying CommcareHQ for the first time needs a few things enabled first.

  1. Create kafka topics:

     $ commcare-cloud monolith django-manage create_kafka_topics
    
  2. Create the CouchDB and Elasticsearch indices:

     $ commcare-cloud monolith django-manage preindex_everything
    
  3. Run the deploy command:

     $ commcare-cloud monolith deploy
    

    You can read more about the deploy process in the deploy documentation.

  4. If deploy fails, you can restart where it left off:

     $ commcare-cloud monolith deploy --resume
    

Step 5: Cleanup

CommCare Cloud will no longer need the root user to be accessible via the password. The password can be removed if you wish.

Step 6: Running CommCareHQ

Learning commcare-cloud basics

In general it will be useful to understand all the commands on the commcare-cloud basics page.

Accessing CommCareHQ from a browser

If everything went well, you should now be able to access CommCareHQ from a browser. See the Configuring VirtualBox for testing CommCareHQ page to find the URL which depends on your networking setup.

Troubleshooting first time set up

If you face any issues, it is recommended to review the Troubleshooting first time setup documentation.

Firefighting issues once CommCareHQ is running

You may also wish to look at the Firefighting page which lists out common issues that commcare-cloud can resolve.

If you ever reboot this machine, make sure to follow the after reboot procedure to bring all the services back up, and mount the encrypted drive by running:

$ commcare-cloud monolith after-reboot all

Step 7: Getting started with CommCareHQ

Make a user

If you are following this process, we assume you have some knowledge of CommCareHQ and may already have data you want to migrate to your new cluster. By default, the monolith deploy scripts will be in Enterprise mode, which means there is no sign up screen. You can change this and other settings in the localsettings file, and following the localsettings deploy instructions.

If you want to leave this setting as is, you can make a superuser with:

$ commcare-cloud monolith django-manage make_superuser {email}

where {email} is the email address you would like to use as the username.

Add a new CommCare build

In order to create new versions of applications created in the CommCareHQ app builder, you will need to add the latest CommCare Mobile and CommCare Core builds to your server.

To add a build to your server visit <server url>/builds/edit_menu and follow the instructions under “Import a new build from the build server”