Add Docker Secrets Support for Database Password & fixed missing boto3 dependency in tests (#1657)

* Added overrider to use docker secrets for mariadb password

* typo in overrider file name

* typo in overrider mariadb-secrets

* typo in overriider mariadb-secrets

* Secrets enviroment variable override

* secrets need different enviroment variable name

* no - for env variables

* Updated deprecated MinIO Env Variables

* refactored to be more robust and better error handling

* temprary debugging changes

* Revert "temprary debugging changes"

This reverts commit d01931064d.

* Revert "refactored to be more robust and better error handling"

This reverts commit fe508668a8.

* Revert "Updated deprecated MinIO Env Variables"

This reverts commit 76e66b5262.

* manually added boto3 before running _create_bucket

* lint formatting

docs/environment-variables.md

@@ -25,6 +25,10 @@ Frappe framework release. You can find all releases [here](https://github.com/fr
 
 Password for MariaDB (or Postgres) database.
 
+### `DB_PASSWORD_SECRETS_FILE`
+
+Path to the db_password.txt file. Set only if you use docker secrets for the database password (use `overrides/compose.mariadb-secrets.yaml`)
+
 ### `DB_HOST`
 
 Hostname for MariaDB (or Postgres) database. Set only if external service for database is used or the container can not be reached by its service name (db) by other containers.

example.env

@@ -4,6 +4,9 @@ ERPNEXT_VERSION=v15.70.2
 
 DB_PASSWORD=123
 
+#Only if you use docker secrets for the db password
+DB_PASSWORD_SECRETS_FILE=
+
 # Only if you use external database
 DB_HOST=
 DB_PORT=

overrides/compose.mariadb-secrets.yaml

@@ -0,0 +1,13 @@
+services:
+  db:
+    environment:
+      MYSQL_ROOT_PASSWORD: !reset null
+      MYSQL_ROOT_PASSWORD_FILE: /run/secrets/db_password
+    healthcheck:
+      test: mysqladmin ping -h localhost --password="$(cat /run/secrets/db_password)"
+    secrets:
+      - db_password
+
+secrets:
+  db_password:
+    file: ${DB_PASSWORD_SECRETS_FILE:?No db secret file set}

overrides/compose.mariadb.yaml

@@ -10,7 +10,7 @@ services:
   db:
     image: mariadb:10.6
     healthcheck:
-      test: mysqladmin ping -h localhost --password=${DB_PASSWORD}
+      test: mysqladmin ping -h localhost --password=${DB_PASSWORD:-123}
       interval: 1s
       retries: 20
     restart: unless-stopped
@@ -20,7 +20,7 @@ services:
       - --skip-character-set-client-handshake
       - --skip-innodb-read-only-compressed # Temporary fix for MariaDB 10.6
     environment:
-      MYSQL_ROOT_PASSWORD: ${DB_PASSWORD:?No db password set}
+      MYSQL_ROOT_PASSWORD: ${DB_PASSWORD:-123}
     volumes:
       - db-data:/var/lib/mysql
 

tests/conftest.py

@@ -151,6 +151,7 @@ def s3_service(python_path: str, compose: Compose):
     subprocess.check_call(cmd)
 
     compose("cp", "tests/_create_bucket.py", "backend:/tmp")
+    compose.exec("backend", "bench", "pip", "install", "boto3~=1.34.143")
     compose.exec(
         "-e",
         f"S3_ACCESS_KEY={access_key}",