1. When upgrading the container in Rancher for the first time, the database migration executed repeatedly, causing an error and resulting in an upgrade failure. See Figure 1.
*** applying m210106_122427_add_columns_to_task_table
> add column article_type_id integer NOT NULL DEFAULT 0 COMMENT '文章类型ID' AFTER `type` to table {{%task}} ...Exception: SQLSTATE[42S21]: Column already exists: 1060 Duplicate column name 'article_type_id'
The SQL being executed was: ALTER TABLE `cpa_task` ADD `article_type_id` int(11) NOT NULL DEFAULT 0 COMMENT '文章类型ID' AFTER `type` (/mcloud/www/ccp_api/vendor/yiisoft/yii2/db/Schema.php:678)
#0 /mcloud/www/ccp_api/vendor/yiisoft/yii2/db/Command.php(1304): yii\db\Schema->convertException(Object(PDOException), 'ALTER TABLE `cp...')
#1 /mcloud/www/ccp_api/vendor/yiisoft/yii2/db/Command.php(1099): yii\db\Command->internalExecute('ALTER TABLE `cp...')
#2 /mcloud/www/ccp_api/vendor/yiisoft/yii2/db/Migration.php(377): yii\db\Command->execute()
#3 /mcloud/www/ccp_api/console/migrations/m210106_122427_add_columns_to_task_table.php(15): yii\db\Migration->addColumn('{{%task}}', 'article_type_id', Object(yii\db\mysql\ColumnSchemaBuilder))
#4 /mcloud/www/ccp_api/vendor/yiisoft/yii2/db/Migration.php(114): m210106_122427_add_columns_to_task_table->safeUp()
#5 /mcloud/www/ccp_api/vendor/yiisoft/yii2/console/controllers/BaseMigrateController.php(744): yii\db\Migration->up()
#6 /mcloud/www/ccp_api/vendor/yiisoft/yii2/console/controllers/BaseMigrateController.php(202): yii\console\controllers\BaseMigrateController->migrateUp('m210106_122427_...')
#7 [internal function]: yii\console\controllers\BaseMigrateController->actionUp(0)
#8 /mcloud/www/ccp_api/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#9 /mcloud/www/ccp_api/vendor/yiisoft/yii2/base/Controller.php(181): yii\base\InlineAction->runWithParams(Array)
#10 /mcloud/www/ccp_api/vendor/yiisoft/yii2/console/Controller.php(184): yii\base\Controller->runAction('', Array)
#11 /mcloud/www/ccp_api/vendor/yiisoft/yii2/base/Module.php(534): yii\console\Controller->runAction('', Array)
#12 /mcloud/www/ccp_api/vendor/yiisoft/yii2/console/Application.php(181): yii\base\Module->runAction('migrate', Array)
#13 /mcloud/www/ccp_api/vendor/yiisoft/yii2/console/Application.php(148): yii\console\Application->runAction('migrate', Array)
#14 /mcloud/www/ccp_api/vendor/yiisoft/yii2/base/Application.php(392): yii\console\Application->handleRequest(Object(yii\console\Request))
#15 /mcloud/www/ccp_api/yii(23): yii\base\Application->run()
#16 {main}
*** failed to apply m210106_122427_add_columns_to_task_table (time: 0.006s)
2. I confirmed that the database structure had not been modified manually. I then analyzed the container’s health checks. Requests are sent periodically to the container to detect its health status. Under the default settings, readiness and liveness use the same configuration parameters. For containers with a long application initialization time, the readiness probe timeout needs to be increased. Preliminary analysis conclusion: Because the database migration took too long to execute, the container initialization time was prolonged, causing the health check to fail and the container to restart repeatedly, which in turn caused the database migration to execute repeatedly. I decided to increase the readiness probe timeout. The time to wait before starting the probe was changed from 10 seconds to 100 seconds. See Figure 2.
3. During the initial deployment, the error still occurred: ReplicaSet “api-77dfcc7cfb” has timed out progressing.; Deployment does not have minimum availability. See Figure 3.
4. I checked the logs, and the database migration was still executing repeatedly. I examined the table structure in MySQL and the records in the migration table, and determined that the migration record m210402_052030_add_columns_to_channel_app_source_table did not exist in the migration table, but the table structure in MySQL already reflected the result of executing this migration. See Figure 4.
m210402_052030_add_columns_to_channel_app_source_table
*** applying m201105_020453_add_article_read_count_column_to_channel_app_task_table
> add column article_read_count integer NOT NULL DEFAULT 0 COMMENT '文章的阅读数' AFTER `have_pub_number` to table {{%channel_app_task}} ...Exception: SQLSTATE[42S21]: Column already exists: 1060 Duplicate column name 'article_read_count'
The SQL being executed was: ALTER TABLE `cpa_channel_app_task` ADD `article_read_count` int(11) NOT NULL DEFAULT 0 COMMENT '文章的阅读数' AFTER `have_pub_number` (/mcloud/www/ccp_api/vendor/yiisoft/yii2/db/Schema.php:678)
#0 /mcloud/www/ccp_api/vendor/yiisoft/yii2/db/Command.php(1304): yii\db\Schema->convertException(Object(PDOException), 'ALTER TABLE `cp...')
#1 /mcloud/www/ccp_api/vendor/yiisoft/yii2/db/Command.php(1099): yii\db\Command->internalExecute('ALTER TABLE `cp...')
#2 /mcloud/www/ccp_api/vendor/yiisoft/yii2/db/Migration.php(377): yii\db\Command->execute()
#3 /mcloud/www/ccp_api/console/migrations/m201105_020453_add_article_read_count_column_to_channel_app_task_table.php(15): yii\db\Migration->addColumn('{{%channel_app_...', 'article_read_co...', Object(yii\db\mysql\ColumnSchemaBuilder))
#4 /mcloud/www/ccp_api/vendor/yiisoft/yii2/db/Migration.php(114): m201105_020453_add_article_read_count_column_to_channel_app_task_table->safeUp()
#5 /mcloud/www/ccp_api/vendor/yiisoft/yii2/console/controllers/BaseMigrateController.php(744): yii\db\Migration->up()
#6 /mcloud/www/ccp_api/vendor/yiisoft/yii2/console/controllers/BaseMigrateController.php(202): yii\console\controllers\BaseMigrateController->migrateUp('m201105_020453_...')
#7 [internal function]: yii\console\controllers\BaseMigrateController->actionUp(0)
#8 /mcloud/www/ccp_api/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#9 /mcloud/www/ccp_api/vendor/yiisoft/yii2/base/Controller.php(181): yii\base\InlineAction->runWithParams(Array)
#10 /mcloud/www/ccp_api/vendor/yiisoft/yii2/console/Controller.php(184): yii\base\Controller->runAction('', Array)
#11 /mcloud/www/ccp_api/vendor/yiisoft/yii2/base/Module.php(534): yii\console\Controller->runAction('', Array)
#12 /mcloud/www/ccp_api/vendor/yiisoft/yii2/console/Application.php(181): yii\base\Module->runAction('migrate', Array)
#13 /mcloud/www/ccp_api/vendor/yiisoft/yii2/console/Application.php(148): yii\console\Application->runAction('migrate', Array)
#14 /mcloud/www/ccp_api/vendor/yiisoft/yii2/base/Application.php(392): yii\console\Application->handleRequest(Object(yii\console\Request))
#15 /mcloud/www/ccp_api/yii(23): yii\base\Application->run()
#16 {main}
*** failed to apply m201105_020453_add_article_read_count_column_to_channel_app_task_table (time: 0.006s)
0 from 27 migrations were applied.
Migration failed. The rest of the migrations are canceled.
5. The time to wait before starting the probe was changed from 10 seconds to 100 seconds. See Figure 5.
