Global refactoring (#617)
* Rename `bench-build` target to `bench` in bake file * Update bake file and break everything * Rename docker-compose.yml to compose.yml to avoid conflicting on `docker buildx bake` * Fix groups in bake file * Update frappe-worker * Update frappe-nginx, erpnext-nginx * Remove old erpnext images * Update frappe-socketio * Fix develop frappe-nginx build on linux/arm64 * Update dockerignore * Update gitignore * Update gitignore * Update .env files * Update installation (overrides) * Update tests * Fix image names * Update compose * Update get-latest-tags * Update CI * Setup and remove .env on tests * Add build bench workflow * Add triggers to main workflow * Add release helm job * Use reusable workflows * Rollback * Print configuration before running tests * Show tests/.env * Revert "Show tests/.env" This reverts commit4bc3bdebaf. * Fix ci image versions * Remove `frappe-` prefix in build directories * Move requirements-dev.txt * Fix image name in CI * Update gitignore * Update pre-commit config * Drop `version:` in compose files * Add push-backup * Fix postgres CI test * Change .yml to .yaml in compose file to follow compose-spec * Remove prettierignore * Fix dockerignore * Change .yml to .yaml in compose file to follow compose-spec * Don't depend on boto3 while testing (do it in backend) * Update erpnext example version * Don't fail ping on URLError * Move assets volume to main compose file * Fix type annotations for v12 * Fix postgres ci override in tests * Fix spaces in socketio * Reorder stages in nginx image, improve perfomance * Remove unused todo * Optimize worker build * Install Node in worker image * Add 502 error page * Remove unused quiet-pull in tests * Add configurator service to dynamically set common config * Remove unused compose.ci-postgres.yml * Use Python for configurator service: faster and more robust * Add TODO.md * Use python script to get latest tags in CI * Clean up nginx dockerfile * Remove VOLUME declaration https://stackoverflow.com/a/55052682 * Add custom app example * Remove pwd for now * Remove pwd for now * Use jq for parsing config in healthcheck * Take advantage of yaml lang: add defaults in compose file. Also require env vars * Fix CI * Use resusable workflow * Update * Move release_helm job to main.yml * Rename docker-build to docker-build-push * Rename main to build_stable * Rename bench targets * Remove quotes from docker-build-push inputs * Update build develop * Remove HELM_DEPLOY_KEY secret from docker-build-push * Add job names * Remove build_bench workflow * Update version input description in docker-build-push * Print .env in tests, if version is develop, change to latest (for tag) * Fix env setup * Uncomment tests * Parse and set short tags from git tag in bake file * Move devcontainer settings to devcontainer.json * Add db command notice * Fix CI? * Fix inconsistencies in development readme * Remove pwd for now * Remove custom apps for production instruction * Update todos * Add docs for images and compose files * Add variables docs and allow custom frappe site name header * Add notice about internal environment variables * Update site-operations docs * Update todos * Add Overrides header in images-and-compose-files * Update todos * Remove extra docs * Don't log requests in worker image (nginx already does that) * Remove default value of FRAPPE_SITE_NAME_HEADER in example.env * Use file that consistent in v12, v13 and develop to check /assets * Fix paths in CI * Update todos * Remove TODO.md * Update tests/_check_backup_files.py Co-authored-by: Revant Nandgaonkar <revant.one@gmail.com> * Change variables MINIO_ACCESS_KEY and MINIO_SECRET_KEY to S3_ACCESS_KEY, S3_SECRET_KEY in tests * Fix S3 test * Use `nginxinc/nginx-unprivileged` instead of `nginx` image Also use Ngnix 1.20 instead of unstable 1.21 * Fix https override * Update Dockerfile * Mount assets to backend service in read only mode * Touch .build (#307), use scripts from nginx image to generate config and touch .build * Update example env after building stable images * Touch `.build` on develop image (untill https://github.com/frappe/frappe/issues/15396 is resolved) * Add `make` to worker build deps for linux/arm64 * Fix update example.env job * Fix .build creation on develop branch * Move bench CI to different file This way workflow runs only on PRs that relevant to bench build * Fix app name in custom app example * Update erpnext and frappe versions in example.env * Don't install `svg-sprite` and `sass` node modules in nginx image on linux/arm64 (https://github.com/frappe/frappe/pull/15275) * docs: README and docs * docs: add link to site operations from docker swarm * ci: fix tests as per changes to compose.yaml * docs: move wiki articles to docs * docs: fix add custom domain * docs: fix patch code from images * fix: do not expose port 80 for old images * fix: custom domain labels to frontend container/service * Add missing descriptions to envs in example.env * Fix redis depends_on * Fix docker compose in tests when not running on TTY * Set -T flag in `docker compose exec` only if not tty * Run pre-commit on docs * Remove postgres healthcheck (it gets overriden by mariadb) * Refactor test * Update workflow names * Add pip to dependabot config * docs: backup and push (#19) * Beautify changes by @revant (#20) * feat: add gevent to worker image * feat: real_ip configuration for nginx * Return `healthcheck.sh` just for tests Co-authored-by: Lev Vereshchagin <mail@vrslev.com> * Make pretend bench catch unknown commands (closes #666) * Remove debug print in push-backup * Fix typing issues in push-backup * Update file keys in push-backups: from abs path to <site>/<file> * Refactor push-backup * Move gevent installation in Frappe step * Don't pin boto stubs requirement * Cache pip deps on build * Update example env versions * Refactor check backup files * Fix backup test * Fix backup test * Rename build/ dir to images/ * Rename build/ dir to images/ * Fix /build -> /images in docs * Update example.env * Use reusable workflow in frappe user instead of vrslev * Fix compose`s `project` option in docs (https://github.com/frappe/frappe_docker/pull/617#issuecomment-1065178792) * Add note about project option in site-operations doc * Update example env * Rename build arg `USERNAME` to `REGISTRY_USER` * Allow https proxy to access Docker socket * Revert "Use reusable workflow in frappe user instead of vrslev" This reverts commit6062500d0d. * Revert "Revert "Use reusable workflow in frappe user instead of vrslev"" This reverts commit4680d18ff8. Co-authored-by: Revant Nandgaonkar <revant.one@gmail.com>
This commit is contained in:
Lev
303
docker-bake.hcl
303
docker-bake.hcl
@@ -1,11 +1,22 @@
|
||||
# Docker Buildx Bake build definition file
|
||||
# Reference: https://github.com/docker/buildx/blob/master/docs/reference/buildx_bake.md
|
||||
|
||||
variable "REGISTRY_USER" {
|
||||
default = "frappe"
|
||||
}
|
||||
|
||||
variable "FRAPPE_VERSION" {
|
||||
default = "develop"
|
||||
}
|
||||
|
||||
variable "ERPNEXT_VERSION" {
|
||||
default = "develop"
|
||||
}
|
||||
|
||||
# Bench image
|
||||
|
||||
target "bench" {
|
||||
context = "build/bench"
|
||||
context = "images/bench"
|
||||
target = "bench"
|
||||
tags = ["frappe/bench:latest"]
|
||||
}
|
||||
@@ -18,255 +29,67 @@ target "bench-test" {
|
||||
# Main images
|
||||
# Base for all other targets
|
||||
|
||||
target "frappe-nginx" {
|
||||
dockerfile = "build/frappe-nginx/Dockerfile"
|
||||
group "frappe" {
|
||||
targets = ["frappe-worker", "frappe-nginx", "frappe-socketio"]
|
||||
}
|
||||
|
||||
group "erpnext" {
|
||||
targets = ["erpnext-worker", "erpnext-nginx"]
|
||||
}
|
||||
|
||||
group "default" {
|
||||
targets = ["frappe", "erpnext"]
|
||||
}
|
||||
|
||||
function "tag" {
|
||||
params = [repo, version]
|
||||
result = [
|
||||
# If `version` param is develop (development build) then use tag `latest`
|
||||
"${version}" == "develop" ? "${REGISTRY_USER}/${repo}:latest" : "${REGISTRY_USER}/${repo}:${version}",
|
||||
# Make short tag for major version if possible. For example, from v13.16.0 make v13.
|
||||
can(regex("(v[0-9]+)[.]", "${version}")) ? "${REGISTRY_USER}/${repo}:${regex("(v[0-9]+)[.]", "${version}")[0]}" : "",
|
||||
]
|
||||
}
|
||||
|
||||
target "default-args" {
|
||||
args = {
|
||||
FRAPPE_VERSION = "${FRAPPE_VERSION}"
|
||||
ERPNEXT_VERSION = "${ERPNEXT_VERSION}"
|
||||
# If `ERPNEXT_VERSION` variable contains "v12" use Python 3.7. Else — 3.9.
|
||||
PYTHON_VERSION = can(regex("v12", "${ERPNEXT_VERSION}")) ? "3.7" : "3.9"
|
||||
}
|
||||
}
|
||||
|
||||
target "frappe-worker" {
|
||||
dockerfile = "build/frappe-worker/Dockerfile"
|
||||
}
|
||||
|
||||
target "frappe-socketio" {
|
||||
dockerfile = "build/frappe-socketio/Dockerfile"
|
||||
}
|
||||
|
||||
target "erpnext-nginx" {
|
||||
dockerfile = "build/erpnext-nginx/Dockerfile"
|
||||
inherits = ["default-args"]
|
||||
context = "images/worker"
|
||||
target = "frappe"
|
||||
tags = tag("frappe-worker", "${FRAPPE_VERSION}")
|
||||
}
|
||||
|
||||
target "erpnext-worker" {
|
||||
dockerfile = "build/erpnext-worker/Dockerfile"
|
||||
inherits = ["default-args"]
|
||||
context = "images/worker"
|
||||
target = "erpnext"
|
||||
tags = tag("erpnext-worker", "${ERPNEXT_VERSION}")
|
||||
}
|
||||
|
||||
|
||||
# Helpers
|
||||
|
||||
target "develop-args" {
|
||||
args = {
|
||||
GIT_BRANCH = "develop"
|
||||
IMAGE_TAG = "develop"
|
||||
}
|
||||
target "frappe-nginx" {
|
||||
inherits = ["default-args"]
|
||||
context = "images/nginx"
|
||||
target = "frappe"
|
||||
tags = tag("frappe-nginx", "${FRAPPE_VERSION}")
|
||||
}
|
||||
|
||||
function "set_develop_tags" {
|
||||
params = [repo]
|
||||
result = ["${repo}:latest", "${repo}:edge", "${repo}:develop"]
|
||||
target "erpnext-nginx" {
|
||||
inherits = ["default-args"]
|
||||
context = "images/nginx"
|
||||
target = "erpnext"
|
||||
tags = tag("erpnext-nginx", "${ERPNEXT_VERSION}")
|
||||
}
|
||||
|
||||
# NOTE: Variable are used only for stable builds
|
||||
variable "GIT_TAG" {} # git tag, e.g. v13.15.0
|
||||
variable "GIT_BRANCH" {} # git branch, e.g. version-13
|
||||
variable "VERSION" {} # Frappe and ERPNext version, e.g. 13
|
||||
|
||||
target "stable-args" {
|
||||
args = {
|
||||
GIT_BRANCH = "${GIT_BRANCH}"
|
||||
IMAGE_TAG = "${GIT_BRANCH}"
|
||||
# ERPNext build fails on v12
|
||||
# TODO: Remove PYTHON_VERSION argument when v12 will stop being supported
|
||||
PYTHON_VERSION = "${VERSION}" == "12" ? "3.7" : "3.9"
|
||||
}
|
||||
}
|
||||
|
||||
function "set_stable_tags" {
|
||||
# e.g. base_image:v13.15.0, base_image:v13, base_image:version-13
|
||||
params = [repo]
|
||||
result = ["${repo}:${GIT_TAG}", "${repo}:v${VERSION}", "${repo}:${GIT_BRANCH}"]
|
||||
}
|
||||
|
||||
target "test-erpnext-args" {
|
||||
args = {
|
||||
IMAGE_TAG = "test"
|
||||
DOCKER_REGISTRY_PREFIX = "localhost:5000/frappe"
|
||||
}
|
||||
}
|
||||
|
||||
function "set_local_test_tags" {
|
||||
params = [repo]
|
||||
result = ["localhost:5000/${repo}:test"]
|
||||
}
|
||||
|
||||
function "set_test_tags" {
|
||||
params = [repo]
|
||||
result = ["${repo}:test"]
|
||||
}
|
||||
|
||||
|
||||
# Develop images
|
||||
|
||||
target "frappe-nginx-develop" {
|
||||
inherits = ["frappe-nginx", "develop-args"]
|
||||
tags = set_develop_tags("frappe/frappe-nginx")
|
||||
}
|
||||
|
||||
target "frappe-worker-develop" {
|
||||
inherits = ["frappe-worker", "develop-args"]
|
||||
tags = set_develop_tags("frappe/frappe-worker")
|
||||
}
|
||||
|
||||
target "frappe-socketio-develop" {
|
||||
inherits = ["frappe-socketio", "develop-args"]
|
||||
tags = set_develop_tags("frappe/frappe-socketio")
|
||||
}
|
||||
|
||||
target "erpnext-nginx-develop" {
|
||||
inherits = ["erpnext-nginx", "develop-args"]
|
||||
tags = set_develop_tags("frappe/erpnext-nginx")
|
||||
}
|
||||
|
||||
target "erpnext-worker-develop" {
|
||||
inherits = ["erpnext-worker", "develop-args"]
|
||||
tags = set_develop_tags("frappe/erpnext-worker")
|
||||
}
|
||||
|
||||
group "frappe-develop" {
|
||||
targets = ["frappe-nginx-develop", "frappe-worker-develop", "frappe-socketio-develop"]
|
||||
}
|
||||
|
||||
group "erpnext-develop" {
|
||||
targets = ["erpnext-nginx-develop", "erpnext-worker-develop"]
|
||||
}
|
||||
|
||||
# Test develop images
|
||||
|
||||
target "frappe-nginx-develop-test-local" {
|
||||
inherits = ["frappe-nginx-develop"]
|
||||
tags = set_local_test_tags("frappe/frappe-nginx")
|
||||
}
|
||||
|
||||
target "frappe-worker-develop-test-local" {
|
||||
inherits = ["frappe-worker-develop"]
|
||||
tags = set_local_test_tags("frappe/frappe-worker")
|
||||
}
|
||||
|
||||
target "frappe-socketio-develop-test-local" {
|
||||
inherits = ["frappe-socketio-develop"]
|
||||
tags = set_local_test_tags("frappe/frappe-socketio")
|
||||
}
|
||||
|
||||
target "frappe-nginx-develop-test" {
|
||||
inherits = ["frappe-nginx-develop"]
|
||||
tags = set_test_tags("frappe/frappe-nginx")
|
||||
}
|
||||
|
||||
target "frappe-worker-develop-test" {
|
||||
inherits = ["frappe-worker-develop"]
|
||||
tags = set_test_tags("frappe/frappe-worker")
|
||||
}
|
||||
|
||||
target "frappe-socketio-develop-test" {
|
||||
inherits = ["frappe-socketio-develop"]
|
||||
tags = set_test_tags("frappe/frappe-socketio")
|
||||
}
|
||||
|
||||
target "erpnext-nginx-develop-test" {
|
||||
inherits = ["erpnext-nginx-develop", "test-erpnext-args"]
|
||||
tags = set_test_tags("frappe/erpnext-nginx")
|
||||
}
|
||||
|
||||
target "erpnext-worker-develop-test" {
|
||||
inherits = ["erpnext-worker-develop", "test-erpnext-args"]
|
||||
tags = set_test_tags("frappe/erpnext-worker")
|
||||
}
|
||||
|
||||
group "frappe-develop-test-local" {
|
||||
targets = ["frappe-nginx-develop-test-local", "frappe-worker-develop-test-local", "frappe-socketio-develop-test-local"]
|
||||
}
|
||||
|
||||
group "frappe-develop-test" {
|
||||
targets = ["frappe-nginx-develop-test", "frappe-worker-develop-test", "frappe-socketio-develop-test"]
|
||||
}
|
||||
|
||||
group "erpnext-develop-test" {
|
||||
targets = ["erpnext-nginx-develop-test", "erpnext-worker-develop-test"]
|
||||
}
|
||||
|
||||
|
||||
# Stable images
|
||||
|
||||
target "frappe-nginx-stable" {
|
||||
inherits = ["frappe-nginx", "stable-args"]
|
||||
tags = set_stable_tags("frappe/frappe-nginx")
|
||||
}
|
||||
|
||||
target "frappe-worker-stable" {
|
||||
inherits = ["frappe-worker", "stable-args"]
|
||||
tags = set_stable_tags("frappe/frappe-worker")
|
||||
}
|
||||
|
||||
target "frappe-socketio-stable" {
|
||||
inherits = ["frappe-socketio", "stable-args"]
|
||||
tags = set_stable_tags("frappe/frappe-socketio")
|
||||
}
|
||||
|
||||
target "erpnext-nginx-stable" {
|
||||
inherits = ["erpnext-nginx", "stable-args"]
|
||||
tags = set_stable_tags("frappe/erpnext-nginx")
|
||||
}
|
||||
|
||||
target "erpnext-worker-stable" {
|
||||
inherits = ["erpnext-worker", "stable-args"]
|
||||
tags = set_stable_tags("frappe/erpnext-worker")
|
||||
}
|
||||
|
||||
group "frappe-stable" {
|
||||
targets = ["frappe-nginx-stable", "frappe-worker-stable", "frappe-socketio-stable"]
|
||||
}
|
||||
|
||||
group "erpnext-stable" {
|
||||
targets = ["erpnext-nginx-stable", "erpnext-worker-stable"]
|
||||
}
|
||||
|
||||
# Test stable images
|
||||
|
||||
target "frappe-nginx-stable-test-local" {
|
||||
inherits = ["frappe-nginx-stable"]
|
||||
tags = set_local_test_tags("frappe/frappe-nginx")
|
||||
}
|
||||
|
||||
target "frappe-worker-stable-test-local" {
|
||||
inherits = ["frappe-worker-stable"]
|
||||
tags = set_local_test_tags("frappe/frappe-worker")
|
||||
}
|
||||
|
||||
target "frappe-socketio-stable-test-local" {
|
||||
inherits = ["frappe-socketio-stable"]
|
||||
tags = set_local_test_tags("frappe/frappe-socketio")
|
||||
}
|
||||
|
||||
target "frappe-nginx-stable-test" {
|
||||
inherits = ["frappe-nginx-stable"]
|
||||
tags = set_test_tags("frappe/frappe-nginx")
|
||||
}
|
||||
|
||||
target "frappe-worker-stable-test" {
|
||||
inherits = ["frappe-worker-stable"]
|
||||
tags = set_test_tags("frappe/frappe-worker")
|
||||
}
|
||||
|
||||
target "frappe-socketio-stable-test" {
|
||||
inherits = ["frappe-socketio-stable"]
|
||||
tags = set_test_tags("frappe/frappe-socketio")
|
||||
}
|
||||
|
||||
target "erpnext-nginx-stable-test" {
|
||||
inherits = ["erpnext-nginx-stable", "test-erpnext-args"]
|
||||
tags = set_test_tags("frappe/erpnext-nginx")
|
||||
}
|
||||
|
||||
target "erpnext-worker-stable-test" {
|
||||
inherits = ["erpnext-worker-stable", "test-erpnext-args"]
|
||||
tags = set_test_tags("frappe/erpnext-worker")
|
||||
}
|
||||
|
||||
group "frappe-stable-test-local" {
|
||||
targets = ["frappe-nginx-stable-test-local", "frappe-worker-stable-test-local", "frappe-socketio-stable-test-local"]
|
||||
}
|
||||
|
||||
group "frappe-stable-test" {
|
||||
targets = ["frappe-nginx-stable-test", "frappe-worker-stable-test", "frappe-socketio-stable-test"]
|
||||
}
|
||||
|
||||
group "erpnext-stable-test" {
|
||||
targets = ["erpnext-nginx-stable-test", "erpnext-worker-stable-test"]
|
||||
target "frappe-socketio" {
|
||||
inherits = ["default-args"]
|
||||
context = "images/socketio"
|
||||
tags = tag("frappe-socketio", "${FRAPPE_VERSION}")
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user