admin/frappe_docker
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

Update site-operations docs

Browse Source
This commit is contained in:
Lev Vereshchagin
2021-12-17 15:06:32 +03:00
parent cf60d6f805
commit ebfee83f9f
1 changed files with 20 additions and 236 deletions
Download Patch File Download Diff File Expand all files Collapse all files

256
docs/site-operations.md
View File

@@ -1,177 +1,39 @@
# Site operations # Site operations
Create and use env file to pass environment variables to containers, ## Setup new site
```sh
source .env
```
Or specify environment variables instead of passing secrets as command arguments. Refer notes section for environment variables required
## Setup New Site
Note: Note:
- Wait for the database service to start before trying to create a new site. - Wait for the `db` service to start and `configurator` to exit before trying to create a new site. Usually this takes up to 10 seconds.
- If new site creation fails, retry after the MariaDB container is up and running.
- If you're using a managed database instance, make sure that the database is running before setting up a new site.
#### MariaDB Site
```sh ```sh
# Create ERPNext site docker-compose exec backend bench new-site <site-name> --mariadb-root-password <db-password> --admin-password <admin-password>
docker run \
-e "SITE_NAME=$SITE_NAME" \
-e "DB_ROOT_USER=$DB_ROOT_USER" \
-e "MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD" \
-e "ADMIN_PASSWORD=$ADMIN_PASSWORD" \
-e "INSTALL_APPS=erpnext" \
-v <project-name>_sites-vol:/home/frappe/frappe-bench/sites \
--network <project-name>_default \
frappe/erpnext-worker:$VERSION new
``` ```
#### PostgreSQL Site If you need to install some app, specify `--install-app`. To see all options, just run `bench new-site --help`.
PostgreSQL is only available v12 onwards. It is NOT available for ERPNext. To create Postgres site (assuming you already use [Postgres compose override](images-and-compose-files.md#overrides)) you need have to do set `root_login` and `root_password` in common config before that:
It is available as part of `frappe/erpnext-worker`. It inherits from `frappe/frappe-worker`.
```sh ```sh
# Create ERPNext site docker-compose exec backend bench set-config -g root_login <root-login>
docker run \ docker-compose exec backend bench set-config -g root_password <root-password>
-e "SITE_NAME=$SITE_NAME" \
-e "DB_ROOT_USER=$DB_ROOT_USER" \
-e "POSTGRES_HOST=$POSTGRES_HOST" \
-e "POSTGRES_PASSWORD=$POSTGRES_PASSWORD" \
-e "ADMIN_PASSWORD=$ADMIN_PASSWORD" \
-v <project-name>_sites-vol:/home/frappe/frappe-bench/sites \
--network <project-name>_default \
frappe/erpnext-worker:$VERSION new
``` ```
Environment Variables needed: Also command is slightly different:
- `SITE_NAME`: name of the new site to create. Site name is domain name that resolves. e.g. `erp.example.com` or `erp.localhost`.
- `DB_ROOT_USER`: MariaDB/PostgreSQL Root user.
- `MYSQL_ROOT_PASSWORD`: In case of the MariaDB docker container use the one set in `MYSQL_ROOT_PASSWORD` in previous steps. In case of a managed database use the appropriate password.
- `MYSQL_ROOT_PASSWORD_FILE` - When the MariaDB root password is stored using docker secrets.
- `ADMIN_PASSWORD`: set the administrator password for the new site.
- `ADMIN_PASSWORD_FILE`: set the administrator password for the new site using docker secrets.
- `INSTALL_APPS=erpnext`: available only in erpnext-worker and erpnext containers (or other containers with custom apps). Installs ERPNext (and/or the specified apps, comma-delinieated) on this new site.
- `FORCE=1`: optional variable which force installation of the same site.
Environment Variables for PostgreSQL only:
- `POSTGRES_HOST`: host for PostgreSQL server
- `POSTGRES_PASSWORD`: Password for `postgres`. The database root user.
Notes:
- To setup existing frappe-bench deployment with default database as PostgreSQL edit the common_site_config.json and set `db_host` to PostgreSQL hostname and `db_port` to PostgreSQL port.
- To create new frappe-bench deployment with default database as PostgreSQL use `POSTGRES_HOST` and `DB_PORT` environment variables in `erpnext-python` service instead of `MARIADB_HOST`
## Add sites to proxy
Change `SITES` variable to the list of sites created encapsulated in backtick and separated by comma with no space. e.g. `` SITES=`site1.example.com`,`site2.example.com` ``.
Reload variables with following command.
```sh ```sh
docker-compose --project-name <project-name> up -d docker-compose exec backend bench new-site <site-name> --db-type postgres --admin-password <admin-password>
``` ```
## Backup Sites ## Push backup to S3 storage
Environment Variables We have the script that helps to push latest backup to S3.
- `SITES` is list of sites separated by `:` colon to migrate. e.g. `SITES=site1.domain.com` or `SITES=site1.domain.com:site2.domain.com` By default all sites in bench will be backed up.
- `WITH_FILES` if set to 1, it will backup user-uploaded files.
- By default `backup` takes mariadb dump and gzips it. Example file, `20200325_221230-test_localhost-database.sql.gz`
- If `WITH_FILES` is set then it will also backup public and private files of each site as uncompressed tarball. Example files, `20200325_221230-test_localhost-files.tar` and `20200325_221230-test_localhost-private-files.tar`
- All the files generated by backup are placed at volume location `sites-vol:/{site-name}/private/backups/*`
```sh ```sh
docker run \ docker-compose exec backend push_backup.py --site-name <site-name> --bucket <bucket> --region-name <region> --endpoint-url <endpoint-url> --aws-access-key-id <access-key> --aws-secret-access-key <secret-key>
-e "SITES=site1.domain.com:site2.domain.com" \
-e "WITH_FILES=1" \
-v <project-name>_sites-vol:/home/frappe/frappe-bench/sites \
--network <project-name>_default \
frappe/erpnext-worker:$VERSION backup
``` ```
The backup will be available in the `sites-vol` volume. Note that you can restore backup only manually.
## Push backup to s3 compatible storage
Environment Variables
- `BUCKET_NAME`, Required to set bucket created on S3 compatible storage.
- `REGION`, Required to set region for S3 compatible storage.
- `ACCESS_KEY_ID`, Required to set access key.
- `SECRET_ACCESS_KEY`, Required to set secret access key.
- `ENDPOINT_URL`, Required to set URL of S3 compatible storage.
- `BUCKET_DIR`, Required to set directory in bucket where sites from this deployment will be backed up.
- `BACKUP_LIMIT`, Optionally set this to limit number of backups in bucket directory. Defaults to 3.
```sh
docker run \
-e "BUCKET_NAME=backups" \
-e "REGION=region" \
-e "ACCESS_KEY_ID=access_id_from_provider" \
-e "SECRET_ACCESS_KEY=secret_access_from_provider" \
-e "ENDPOINT_URL=https://region.storage-provider.com" \
-e "BUCKET_DIR=frappe-bench" \
-v <project-name>_sites-vol:/home/frappe/frappe-bench/sites \
--network <project-name>_default \
frappe/frappe-worker:$VERSION push-backup
```
Note:
- Above example will backup files in bucket called `backup` at location `frappe-bench-v13/site.name.com/DATE_TIME/DATE_TIME-site_name_com-{filetype}.{extension}`,
- example DATE_TIME: 20200325_042020.
- example filetype: database, files or private-files
- example extension: sql.gz or tar
## Restore backups
Environment Variables
- `MYSQL_ROOT_PASSWORD` or `MYSQL_ROOT_PASSWORD_FILE`(when using docker secrets), Required to restore mariadb backups.
- `BUCKET_NAME`, Required to set bucket created on S3 compatible storage.
- `ACCESS_KEY_ID`, Required to set access key.
- `SECRET_ACCESS_KEY`, Required to set secret access key.
- `ENDPOINT_URL`, Required to set URL of S3 compatible storage.
- `REGION`, Required to set region for s3 compatible storage.
- `BUCKET_DIR`, Required to set directory in bucket where sites from this deployment will be backed up.
```sh
docker run \
-e "MYSQL_ROOT_PASSWORD=admin" \
-e "BUCKET_NAME=backups" \
-e "REGION=region" \
-e "ACCESS_KEY_ID=access_id_from_provider" \
-e "SECRET_ACCESS_KEY=secret_access_from_provider" \
-e "ENDPOINT_URL=https://region.storage-provider.com" \
-e "BUCKET_DIR=frappe-bench" \
-v <project-name>_sites-vol:/home/frappe/frappe-bench/sites \
-v ./backups:/home/frappe/backups \
--network <project-name>_default \
frappe/frappe-worker:$VERSION restore-backup
```
Note:
- Volume must be mounted at location `/home/frappe/backups` for restoring sites
- If no backup files are found in volume, it will use s3 credentials to pull backups
- Backup structure for mounted volume or downloaded from s3:
- /home/frappe/backups
- site1.domain.com
- 20200420_162000
- 20200420_162000-site1_domain_com-\*
- site2.domain.com
- 20200420_162000
- 20200420_162000-site2_domain_com-\*
## Edit configs ## Edit configs
@@ -179,12 +41,11 @@ Editing config manually might be required in some cases,
one such case is to use Amazon RDS (or any other DBaaS). one such case is to use Amazon RDS (or any other DBaaS).
For full instructions, refer to the [wiki](<https://github.com/frappe/frappe/wiki/Using-Frappe-with-Amazon-RDS-(or-any-other-DBaaS)>). Common question can be found in Issues and on forum. For full instructions, refer to the [wiki](<https://github.com/frappe/frappe/wiki/Using-Frappe-with-Amazon-RDS-(or-any-other-DBaaS)>). Common question can be found in Issues and on forum.
`common_site_config.json` or `site_config.json` from `sites-vol` volume has to be edited using following command: `common_site_config.json` or `site_config.json` from `sites` volume has to be edited using following command:
```sh ```sh
docker run \ docker run --rm -it \
-it \ -v <project-name>_sites:/sites \
-v <project-name>_sites-vol:/sites \
alpine vi /sites/common_site_config.json alpine vi /sites/common_site_config.json
``` ```
@@ -195,8 +56,7 @@ Instead of `alpine` use any image of your choice.
For socketio and gunicorn service ping the hostname:port and that will be sufficient. For workers and scheduler, there is a command that needs to be executed. For socketio and gunicorn service ping the hostname:port and that will be sufficient. For workers and scheduler, there is a command that needs to be executed.
```shell ```shell
docker exec -it <project-name>_erpnext-worker-d \ docker-compose exec backend healthcheck.sh --ping-service mongodb:27017
docker-entrypoint.sh doctor -p postgresql:5432 --ping-service mongodb:27017
``` ```
Additional services can be pinged as part of health check with option `-p` or `--ping-service`. Additional services can be pinged as part of health check with option `-p` or `--ping-service`.
@@ -204,86 +64,10 @@ Additional services can be pinged as part of health check with option `-p` or `-
This check ensures that given service should be connected along with services in common_site_config.json. This check ensures that given service should be connected along with services in common_site_config.json.
If connection to service(s) fails, the command fails with exit code 1. If connection to service(s) fails, the command fails with exit code 1.
## Frappe internal commands using bench helper ---
To execute commands using bench helper. For reference of commands like `backup`, `drop-site` or `migrate` check [official guide](https://frappeframework.com/docs/v13/user/en/bench/frappe-commands) or run:
```shell
docker run \
-v <project-name>_sites-vol:/home/frappe/frappe-bench/sites \
--network <project-name>_default \
--user frappe \
frappe/frappe-worker:$VERSION bench --help
```
Example command to clear cache
```shell
docker run \
-v <project-name>_sites-vol:/home/frappe/frappe-bench/sites \
--network <project-name>_default \
--user frappe \
frappe/frappe-worker:$VERSION bench --site erp.mysite.com clear-cache
```
Notes:
- Use it to install/uninstall custom apps, add system manager user, etc.
- To run the command as non root user add the command option `--user frappe`.
## Delete/Drop Site
#### MariaDB Site
```sh ```sh
# Delete/Drop ERPNext site docker-compose exec backend bench --help
docker run \
-e "SITE_NAME=$SITE_NAME" \
-e "DB_ROOT_USER=$DB_ROOT_USER" \
-e "MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD" \
-v <project-name>_sites-vol:/home/frappe/frappe-bench/sites \
--network <project-name>_default \
frappe/erpnext-worker:$VERSION drop
``` ```
#### PostgreSQL Site
```sh
# Delete/Drop ERPNext site
docker run \
-e "SITE_NAME=$SITE_NAME" \
-e "DB_ROOT_USER=$DB_ROOT_USER" \
-e "POSTGRES_PASSWORD=$POSTGRES_PASSWORD" \
-v <project-name>_sites-vol:/home/frappe/frappe-bench/sites \
--network <project-name>_default \
frappe/erpnext-worker:$VERSION drop
```
Environment Variables needed:
- `SITE_NAME`: name of the site to be deleted. Site name is domain name that resolves. e.g. `erp.example.com` or `erp.localhost`.
- `DB_ROOT_USER`: MariaDB/PostgreSQL Root user.
- `MYSQL_ROOT_PASSWORD`: Root User password for MariaDB.
- `FORCE=1`: optional variable which force deletion of the same site.
- `NO_BACKUP=1`: option variable to skip the process of taking backup before deleting the site.
Environment Variables for PostgreSQL only:
- `POSTGRES_PASSWORD`: Password for `postgres`. The database root user.
## Migrate Site
```sh
# Migrate ERPNext site
docker run \
-e "MAINTENANCE_MODE=1" \
-v <project-name>_sites-vol:/home/frappe/frappe-bench/sites \
-v <project-name>_assets-vol:/home/frappe/frappe-bench/sites/assets \
--network <project-name>_default \
frappe/erpnext-worker:$ERPNEXT_VERSION migrate
```
Environment Variables needed:
- `MAINTENANCE_MODE`: If set to `1`, this will ensure the bench is switched to maintenance mode during migration.
- `SITES`: Optional list of sites to be migrated, separated by colon (`:`). e.g. `erp.site1.com:erp.site2.com`. If not used, all sites will be migrated by default.