没有不值得去解决的问题,也没有不值得去学习的技术!

Automatically Run App Initialization and Database Migration After Rancher Container Upgrades Using Environment Variables

If the environment variable PCS_API_CFG_MIGRATE is already set, and its value is: true, then execute the following commands in sequence: php /sobey/www/pcs-api/yii migrate --migrationPath=@yii/log/migrations/ --interactive=0, php /sobey/www/pcs-api/yii migrate --interactive=0

1. With the current upgrade process, after the Rancher container is upgraded, you enter the Docker container and run the following commands in sequence from the command line:

Bash

// 开发环境 / 测试环境
php /sobey/www/pcs-api/init --env=Development --overwrite=All
// 演示环境 / 生产环境
php /sobey/www/pcs-api/init --env=Production --overwrite=All
// 数据库迁移(日志的数据库模式,仅在第一次安装时运行)
php /sobey/www/pcs-api/yii migrate --migrationPath=@yii/log/migrations/ --interactive=0
// 数据库迁移(MySQL 5.7.7 之前的版本:innodb_large_prefix=1、innodb_file_format=BARRACUDA)
php /sobey/www/pcs-api/yii migrate --interactive=0


2. The problem with the current upgrade process is that after a Rancher container upgrade, operations staff must paste and run at least 2 commands (the 2nd and 4th) in every container. It is easy to miss a container, which leads to program execution errors, and it also involves a certain amount of manual effort. I plan to use a SHELL script to run the command lines automatically, ensuring that every container initializes successfully while saving the operations staff’s workload.

3. Create \build\c_files\config\init\pcs-api_init.sh

Note:
If the environment variable PCS_API_CFG_ENV is not set, its default value is set to: dev
If the environment variable PCS_API_CFG_ENV is already set, and its value is: dev, then execute the command: php /sobey/www/pcs-api/init –env=Development –overwrite=All; if its value is: prod, then execute the command: php /sobey/www/pcs-api/init –env=Production –overwrite=All; otherwise output: please set environment variable PCS_API_CFG_ENV dev or prod
If the environment variable PCS_API_CFG_MIGRATE is not set, its default value is set to: false
If the environment variable PCS_API_CFG_MIGRATE is already set, and its value is: true, then execute the following commands in sequence: php /sobey/www/pcs-api/yii migrate –migrationPath=@yii/log/migrations/ –interactive=0, php /sobey/www/pcs-api/yii migrate –interactive=0; if its value is: false, then output: running without db migrate; otherwise output: please set environment variable PCS_API_CFG_MIGRATE true or false

ShellScript

#!/bin/bash
env | grep PCS_API_CFG_ENV || export PCS_API_CFG_ENV="dev"
if [[ $PCS_API_CFG_ENV == "dev" ]]
then
    php /sobey/www/pcs-api/init --env=Development --overwrite=All
elif [[ $PCS_API_CFG_ENV == "prod" ]]
then
    php /sobey/www/pcs-api/init --env=Production --overwrite=All
else
    echo "please set environment variable PCS_API_CFG_ENV dev or prod"
fi
env | grep PCS_API_CFG_MIGRATE || export PCS_API_CFG_MIGRATE="false"
if [[ $PCS_API_CFG_MIGRATE == "true" ]]
then
    php /sobey/www/pcs-api/yii migrate --migrationPath=@yii/log/migrations/ --interactive=0
    php /sobey/www/pcs-api/yii migrate --interactive=0
elif [[ $PCS_API_CFG_MIGRATE == "false" ]]
then
    echo "running without db migrate"
else
    echo "please set environment variable PCS_API_CFG_MIGRATE true or false"
fi

4. Delete all tables in the development environment database. Leave the environment variables PCS_API_CFG_ENV and PCS_API_CFG_MIGRATE unset to test whether the default values take effect when the environment variables are not set. The result matches expectations.
Note:
If the environment variable PCS_API_CFG_ENV is not set, its default value is set to: dev, and the command is executed: php /sobey/www/pcs-api/init –env=Development –overwrite=All, as shown in Figure 1

If the environment variable PCS_API_CFG_ENV is not set, its default value is set to: dev, and the command is executed: php /sobey/www/pcs-api/init --env=Development --overwrite=All
Figure 1

If the environment variable PCS_API_CFG_MIGRATE is not set, its default value is set to: false, and it outputs: running without db migrate, as shown in Figure 2

If the environment variable PCS_API_CFG_MIGRATE is not set, its default value is set to: false, and it outputs: running without db migrate
Figure 2

5. Delete all tables in the development environment database. Set the environment variable PCS_API_CFG_ENV to: dev, and set the environment variable PCS_API_CFG_MIGRATE to: true, to test whether the corresponding commands are executed when the environment variables are set. The result matches expectations, as shown in Figure 3

Delete all tables in the development environment database. Set the environment variable PCS_API_CFG_ENV to: dev, and set the environment variable PCS_API_CFG_MIGRATE to: true, to test whether the corresponding commands are executed when the environment variables are set. The result matches expectations.
Figure 3

Note:
If the environment variable PCS_API_CFG_ENV is already set, and its value is: dev, then execute the command: php /sobey/www/pcs-api/init –env=Development –overwrite=All
If the environment variable PCS_API_CFG_MIGRATE is already set, and its value is: true, then execute the following commands in sequence: php /sobey/www/pcs-api/yii migrate –migrationPath=@yii/log/migrations/ –interactive=0, php /sobey/www/pcs-api/yii migrate –interactive=0, as shown in Figure 4 and Figure 5

If the environment variable PCS_API_CFG_MIGRATE is already set, and its value is: true, then execute the following commands in sequence: php /sobey/www/pcs-api/yii migrate --migrationPath=@yii/log/migrations/ --interactive=0, php /sobey/www/pcs-api/yii migrate --interactive=0
Figure 4

 

Check the database; the migration commands have been successfully executed
Figure 5

6. Delete all tables in the development environment database. Set the environment variable PCS_API_CFG_ENV to: prod, and set the environment variable PCS_API_CFG_MIGRATE to: false, to test whether the corresponding commands are executed when the environment variables are set. The result matches expectations, as shown in Figure 6 and Figure 7

Delete all tables in the development environment database. Set the environment variable PCS_API_CFG_ENV to: prod, and set the environment variable PCS_API_CFG_MIGRATE to: false
Figure 6
Delete all tables in the development environment database. Set the environment variable PCS_API_CFG_ENV to: prod, and set the environment variable PCS_API_CFG_MIGRATE to: false, to test whether the corresponding commands are executed when the environment variables are set. The result matches expectations.
Figure 7

Note:
If the environment variable PCS_API_CFG_ENV is already set, and its value is: prod, then execute the command: php /sobey/www/pcs-api/init –env=Production –overwrite=All
If the environment variable PCS_API_CFG_MIGRATE is already set, and its value is: false, then output: running without db migrate

7. Delete all tables in the development environment database. Set the environment variable PCS_API_CFG_ENV to an empty value, and set the environment variable PCS_API_CFG_MIGRATE to an empty value, to test whether the corresponding prompt is output when the environment variables are set but their values do not meet the specifications. The result matches expectations, as shown in Figure 8 and Figure 9

Delete all tables in the development environment database. Set the environment variable PCS_API_CFG_ENV to an empty value, and set the environment variable PCS_API_CFG_MIGRATE to an empty value
Figure 8
Delete all tables in the development environment database. Set the environment variable PCS_API_CFG_ENV to an empty value, and set the environment variable PCS_API_CFG_MIGRATE to an empty value, to test whether the corresponding prompt is output when the environment variables are set but their values do not meet the specifications. The result matches expectations.
Figure 9

Note:
If the environment variable PCS_API_CFG_ENV is already set but invalid, output: please set environment variable PCS_API_CFG_ENV dev or prod
If the environment variable PCS_API_CFG_MIGRATE is already set but invalid, output: please set environment variable PCS_API_CFG_MIGRATE true or false

8. Delete all tables in the development environment database. Set the environment variable PCS_API_CFG_ENV to: dev, and set the environment variable PCS_API_CFG_MIGRATE to: true. Set the value of PCS_API_CFG_DB_PASSWORD to an incorrect password to test the scenario where an error occurs while executing the corresponding commands. The result matches expectations: the container upgrade fails, as shown in Figure 10

Delete all tables in the development environment database. Set the environment variable PCS_API_CFG_ENV to: dev, and set the environment variable PCS_API_CFG_MIGRATE to: true. Set the value of PCS_API_CFG_DB_PASSWORD to an incorrect password to test the scenario where an error occurs while executing the corresponding commands. The result matches expectations: the container upgrade fails.
Figure 10

Note:
If the environment variable PCS_API_CFG_ENV is already set, and its value is: dev, then execute the command: php /sobey/www/pcs-api/init –env=Development –overwrite=All
If the environment variable PCS_API_CFG_MIGRATE is already set, and its value is: true, then execute the following commands in sequence: php /sobey/www/pcs-api/yii migrate –migrationPath=@yii/log/migrations/ –interactive=0, php /sobey/www/pcs-api/yii migrate –interactive=0. Checking the container logs shows an incorrect database password, as shown in Figure 11

If the environment variable PCS_API_CFG_MIGRATE is already set, and its value is: true, then execute the following commands in sequence: php /sobey/www/pcs-api/yii migrate --migrationPath=@yii/log/migrations/ --interactive=0, php /sobey/www/pcs-api/yii migrate --interactive=0. Checking the container logs shows an incorrect database password.
Figure 11

9. The Rancher environment variables are described as follows. At present, if the machine executing the database migration fails to upgrade, problems will persist even if the other machines upgrade successfully. This issue is postponed for now. As shown in Figure 12

The Rancher environment variables are described as follows. At present, if the machine executing the database migration fails to upgrade, problems will persist even if the other machines upgrade successfully. This issue is postponed for now.
Figure 12
Plaintext
	PCS_API_CFG_ENV=dev # Environment, dev: development environment / test environment; prod: staging environment / production environment, default: dev
	PCS_API_CFG_MIGRATE=false # Whether to execute database migration (for single-machine deployment, it needs to be set to true; for cluster deployment, only one machine needs to be set to true, and the other machines need to be set to false), true: yes; false: no, default: false

10. In another project, when upgrading the container in Rancher, an error occurred while automatically executing the database migration command: Exception ‘yii\db\Exception’ with message ‘SQLSTATE[HY000] [2002] php_network_getaddresses: getaddrinfo failed: Name or service not known’, as shown in Figure 13

In another project, when upgrading the container in Rancher, an error occurred while automatically executing the database migration command: Exception 'yii\db\Exception' with message 'SQLSTATE[HY000] [2002] php_network_getaddresses: getaddrinfo failed: Name or service not known'
Figure 13
Plaintext
2019/1/22 下午3:49:43CHANNEL_PUB_API_CFG_MIGRATE=true
2019/1/22 下午3:49:43Yii Migration Tool (based on Yii v2.0.15.1)
2019/1/22 下午3:49:43
2019/1/22 下午3:49:43Exception 'yii\db\Exception' with message 'SQLSTATE[HY000] [2002] php_network_getaddresses: getaddrinfo failed: Name or service not known'
2019/1/22 下午3:49:43
2019/1/22 下午3:49:43in /sobey/www/channel-pub-api/vendor/yiisoft/yii2/db/Connection.php:624
2019/1/22 下午3:49:43
2019/1/22 下午3:49:43Stack trace:
2019/1/22 下午3:49:43#0 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/db/Connection.php(996): yii\db\Connection->open()
2019/1/22 下午3:49:43#1 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/db/Connection.php(983): yii\db\Connection->getMasterPdo()
2019/1/22 下午3:49:43#2 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/db/Command.php(253): yii\db\Connection->getSlavePdo()
2019/1/22 下午3:49:43#3 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/db/Command.php(1143): yii\db\Command->prepare(true)
2019/1/22 下午3:49:43#4 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/db/Command.php(399): yii\db\Command->queryInternal('fetchAll', NULL)
2019/1/22 下午3:49:43#5 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/db/mysql/Schema.php(312): yii\db\Command->queryAll()
2019/1/22 下午3:49:43#6 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/db/mysql/Schema.php(125): yii\db\mysql\Schema->findColumns(Object(yii\db\TableSchema))
2019/1/22 下午3:49:43#7 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/db/Schema.php(744): yii\db\mysql\Schema->loadTableSchema('CHANNEL_PUB_API...')
2019/1/22 下午3:49:43#8 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/db/Schema.php(194): yii\db\Schema->getTableMetadata('{{%migration}}', 'schema', true)
2019/1/22 下午3:49:43#9 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/console/controllers/MigrateController.php(210): yii\db\Schema->getTableSchema('{{%migration}}', true)
2019/1/22 下午3:49:43#10 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/console/controllers/BaseMigrateController.php(875): yii\console\controllers\MigrateController->getMigrationHistory(NULL)
2019/1/22 下午3:49:43#11 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/console/controllers/BaseMigrateController.php(166): yii\console\controllers\BaseMigrateController->getNewMigrations()
2019/1/22 下午3:49:43#12 [internal function]: yii\console\controllers\BaseMigrateController->actionUp(0)
2019/1/22 下午3:49:43#13 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
2019/1/22 下午3:49:43#14 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/base/Controller.php(157): yii\base\InlineAction->runWithParams(Array)
2019/1/22 下午3:49:43#15 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/console/Controller.php(148): yii\base\Controller->runAction('', Array)
2019/1/22 下午3:49:43#16 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/base/Module.php(528): yii\console\Controller->runAction('', Array)
2019/1/22 下午3:49:43#17 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/console/Application.php(180): yii\base\Module->runAction('migrate', Array)
2019/1/22 下午3:49:43#18 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/console/Application.php(147): yii\console\Application->runAction('migrate', Array)
2019/1/22 下午3:49:43#19 /sobey/www/channel-pub-api/vendor/yiisoft/yii2/base/Application.php(386): yii\console\Application->handleRequest(Object(yii\console\Request))
2019/1/22 下午3:49:43#20 /sobey/www/channel-pub-api/yii(23): yii\base\Application->run()
2019/1/22 下午3:49:43#21 {main}

11. \build\c_files\config\init\channel-pub-api_init.sh, the SHELL script is as follows:

ShellScript

#!/bin/bash
env | grep CHANNEL_PUB_API_CFG_ENV || export CHANNEL_PUB_API_CFG_ENV="dev"
if [[ $CHANNEL_PUB_API_CFG_ENV == "dev" ]]
then
    php /sobey/www/channel-pub-api/init --env=Development --overwrite=All
elif [[ $CHANNEL_PUB_API_CFG_ENV == "prod" ]]
then
    php /sobey/www/channel-pub-api/init --env=Production --overwrite=All
else
    echo "please set environment variable CHANNEL_PUB_API_CFG_ENV dev or prod"
fi
env | grep CHANNEL_PUB_API_CFG_MIGRATE || export CHANNEL_PUB_API_CFG_MIGRATE="false"
if [[ $CHANNEL_PUB_API_CFG_MIGRATE == "true" ]]
then
    php /sobey/www/channel-pub-api/yii migrate --migrationPath=@yii/log/migrations/ --interactive=0
    php /sobey/www/channel-pub-api/yii migrate --interactive=0
    php /sobey/www/channel-pub-api/yii qq-tp-app/init-sync
    php /sobey/www/channel-pub-api/yii qq-cw-app/init-sync
    php /sobey/www/channel-pub-api/yii weibo-weibo-connect-web-app/init-sync
elif [[ $CHANNEL_PUB_API_CFG_MIGRATE == "false" ]]
then
    echo "running without db migrate"
else
    echo "please set environment variable CHANNEL_PUB_API_CFG_MIGRATE true or false"
fi

12. The upgrade failed and rolled back to the old version. Manually executing the commands all succeeded.

13. Comparing the differences between the two projects, the reason is the difference in the execution order of the files in the \build\c_files\config\init directory. To ensure that the command line executes successfully, you must ensure that the pcs-api_init.sh/channel-pub-api_init.sh file runs after the config0.sh file executes. However, in channel-pub-api, channel-pub-api_init.sh executes before config0.sh, as shown in Figure 14

Comparing the differences between the two projects, the reason is the difference in the execution order of the files in the \build\c_files\config\init directory. To ensure that the command line executes successfully, you must ensure that the pcs-api_init.sh/channel-pub-api_init.sh file runs after the config0.sh file executes. However, in channel-pub-api, channel-pub-api_init.sh executes before config0.sh.
Figure 14

14. In both projects, rename pcs-api_init.sh/channel-pub-api_init.sh to console_init.sh to ensure that the console_init.sh file runs after the config0.sh file executes, as shown in Figure 15

In both projects, rename pcs-api_init.sh/channel-pub-api_init.sh to console_init.sh to ensure that the console_init.sh file runs after the config0.sh file executes.
Figure 15

15. After upgrading pcs-api, checking the logs shows the result matches expectations, as shown in Figure 16

After upgrading pcs-api, checking the logs shows the result matches expectations.
Figure 16

16. After upgrading channel-pub-api, checking the logs shows the result matches expectations, as shown in Figure 17

After upgrading channel-pub-api, checking the logs shows the result matches expectations.
Figure 17

关于作者 · 持续学习与技术实践

我是一名拥有 15 年以上开发经验的技术从业者,自 2013 年起持续运营个人技术博客,记录工作与个人项目中遇到的真实问题,以及解决问题过程中的探索、实践和思考。

除了技术写作,我也在持续开发和维护自己的独立项目,探索新技术和新工具在实际场景中的应用。我始终相信,没有不值得去解决的问题,也没有不值得去学习的技术。

如果这篇文章对你有所帮助,欢迎浏览博客中的其他文章,也欢迎交流技术经验与想法。

关于我  ·  GitHub  ·  邮件联系