Implement asynchronous execution of tasks based on Yii2 queue extension in Yii2 advanced template
1. Install the Yii2 queue extension and execute the following commands
composer require --prefer-dist yiisoft/yii2-queue
2. Check composer.json, the installation has been successful
"require": {
"yiisoft/yii2-queue": "^2.1"
},
3. Register multiple queue components to the console, edit \console\config\main.php
"require": {
"yiisoft/yii2-queue": "^2.1"
},
4. Configuration of multiple queue components, editing \environments\dev\common\config\main-local.php, \environments\prod\common\config\main-local.php
'copyAssetQueue' => [ // 复制资源文件队列
'class' => 'yii\queue\redis\Queue',
'redis' => 'redis', // Redis 连接组件或它的配置
'channel' => 'cpa:queue:copy:asset', // 队列键前缀
'ttr' => 10 * 60, // 作业处理的最长时间，单位(秒)
'as log' => 'yii\queue\LogBehavior',
],
'qqCwVideoMultipartUploadQueue' => [ // 企鹅号的内容网站应用的视频文件分片上传队列
'class' => 'yii\queue\redis\Queue',
'redis' => 'redis', // Redis 连接组件或它的配置
'channel' => 'cpa:queue:qq:cw:video:multipart:upload', // 队列键前缀
'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
'as log' => 'yii\queue\LogBehavior',
],
'pubArticleQueue' => [ // 发布文章队列
'class' => 'yii\queue\redis\Queue',
'redis' => 'redis', // Redis 连接组件或它的配置
'channel' => 'cpa:queue:pub:article', // 队列键前缀
'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
'as log' => 'yii\queue\LogBehavior',
],
'sourceCallbackQueue' => [ // 来源回调队列
'class' => 'yii\queue\redis\Queue',
'redis' => 'redis', // Redis 连接组件或它的配置
'channel' => 'cpa:queue:source:callback', // 队列键前缀
'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
'as log' => 'yii\queue\LogBehavior',
],
5. Define the basic class of each task sent to the queue, create a new \common\jobs\job.php
<?php
/**
* Created by PhpStorm.
* User: terryhong
* Date: 2018/9/7
* Time: 下午4:54
*/
namespace common\jobs;
use yii\base\BaseObject;
use yii\queue\JobInterface;
/**
* 队列的任务的基础类
*
* @author Qiang Wang <shuijingwanwq@163.com>
* @since 1.0
*/
class Job extends BaseObject implements JobInterface
{
public function execute($queue){
}
}
6. Each task sent to the queue should be defined as a separate class. Based on the official example, if you need to download and save a file, the class might look like this
<?php
/**
* Created by PhpStorm.
* User: Qiang Wang
* Date: 2018/10/18
* Time: 10:39
*/
namespace common\jobs;
/**
* 下载并保存一个文件
*
* @author Qiang Wang <shuijingwanwq@163.com>
* @since 1.0
*/
class DownloadJob extends Job
{
public $url;
public $file;
public function execute($queue)
{
file_put_contents($this->file, file_get_contents($this->url));
}
}
7. Define the corresponding download class in the QQ application, create a new \QQ\Jobs\DownloadJob.php
<?php
/**
* Created by PhpStorm.
* User: Qiang Wang
* Date: 2018/10/19
* Time: 10:28
*/
namespace qq\jobs;
/**
* 下载并保存一个文件
*
* @author Qiang Wang <shuijingwanwq@163.com>
* @since 1.0
*/
class DownloadJob extends \common\jobs\DownloadJob
{
}
8. Test the job status, edit \qq\rests\qq_cw_app\indexaction.php
<?php
/**
* @link http://www.yiiframework.com/
* @copyright Copyright (c) 2008 Yii Software LLC
* @license http://www.yiiframework.com/license/
*/
namespace qq\rests\qq_cw_app;
use Yii;
use qq\jobs\DownloadJob;
use yii\data\ActiveDataProvider;
use yii\base\InvalidConfigException;
/**
*
* For more details and usage information on IndexAction, see the [guide article on rest controllers](guide:rest-controllers).
*
* @author Qiang Wang <shuijingwanwq@163.com>
* @since 1.0
*/
class IndexAction extends \yii\rest\IndexAction
{
/**
* Prepares the data provider that should return the requested collection of the models.
* @return ActiveDataProvider
* @throws InvalidConfigException if the configuration is invalid.
*/
protected function prepareDataProvider()
{
// 将作业推送到队列并获得其ID
$id = Yii::$app->copyAssetQueue->push(new DownloadJob([
'url' => 'http://wjdev2.chinamcloud.com:8661/robots.txt',
'file' => './qq/runtime/robots.txt',
]));
// 这个作业等待执行
var_dump(Yii::$app->copyAssetQueue->isWaiting($id));
// Worker 从队列获取作业，并执行它
var_dump(Yii::$app->copyAssetQueue->isReserved($id));
// Worker 作业执行完成
var_dump(Yii::$app->copyAssetQueue->isDone($id));
echo $id;
exit;
}
}
9. Open the URL:http://api.channel-pub-api.localhost/qq/v1/qq-cw-apps?group_id=spider, print the result, it is found that the task is not executed
bool(true)
bool(false)
bool(false)
1
10. Check redis, the queue has been stored, as shown in Figure 1
600;O:19:"qq\jobs\DownloadJob":2:{s:3:"url";s:45:"http://wjdev2.chinamcloud.com:8661/robots.txt";s:4:"file";s:23:"./qq/runtime/robots.txt";}
11. The info command prints information about the queue status, and a task status is waiting
.\yii copy-asset-queue/info
Jobs
- waiting: 1
- delayed: 0
- reserved: 0
- done: 0
12. The run command obtains and executes the tasks in the loop until the queue is empty, as shown in Figure 2
.\yii copy-asset-queue/run --verbose=1 --isolate=1 --color=0
.\yii copy-asset-queue/info
2018-10-19 11:07:30 [pid: 36016] - Worker is started
2018-10-19 11:07:30 [2] qq\jobs\DownloadJob (attempt: 1, pid: 36016) - Started
2018-10-19 11:07:30 [2] qq\jobs\DownloadJob (attempt: 1, pid: 36016) - Done (0.255 s)
2018-10-19 11:07:30 [pid: 36016] - Worker is stopped (0:00:00)
Jobs
- waiting: 0
- delayed: 0
- reserved: 0
- done: 1
13. Check redis, key: cpa:queue:copy:asset.messages, cpa:queue:copy:asset.waiting has been deleted, as shown in Figure 3
14. The file \qq\runtime\robots.txt has been saved, as shown in Figure 4
15. In order to make the development process more friendly, you can add a panel to the Yii2 debugging module. The panel displays a list of counters and queue tasks. edit \environments\dev\api\config\main-local.php, \environments\dev\back nd\config\main-local.php, \environments\dev\frontend\config\main-loca L.php, \environments\dev\qq\config\main-local.php, \environments\dev\ rpc\config\main-local.php, \environments\dev\wx\config\main-local.php
if (!YII_ENV_TEST) {
// configuration adjustments for 'dev' environment
$config['bootstrap'][] = 'debug';
$config['modules']['debug'] = [
'class' => 'yii\debug\Module',
'panels' => [
'queue' => 'yii\queue\debug\Panel',
],
];
$config['bootstrap'][] = 'gii';
$config['modules']['gii'] = [
'class' => 'yii\gii\Module',
];
}
16. Check the Yii2 debugging module, the Queue panel has been added, as shown in Figure 5
17. The local environment is Windows 10, and it is ready to be deployed to development, testing, demonstration, and production environments based on Supervisor. The latter four are all CentOS 7, and supervisor is the process monitor of Linux. It will automatically start your console process.
\build\c_files\etc\supervisord.d\yii-copy-asset-queue-worker.ini
[program:yii-copy-asset-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-copy-asset-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-copy-asset-queue-worker-stdout.log
\build\c_files\etc\supervisord.d\yii-qq-cw-video-multipart-upload-queue-worker.ini
[program:yii-qq-cw-video-multipart-upload-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii qq-cw-video-multipart-upload-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stdout.log
\Build\c_files\etc\supervisord.d\yii-pub-article-queue-worker.ini
[program:yii-pub-article-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii pub-article-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-pub-article-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-pub-article-queue-worker-stdout.log
\build\c_files\etc\supervisord.d\yii-source-callback-queue-worker.ini
[program:yii-source-callback-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii source-callback-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-source-callback-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-source-callback-queue-worker-stdout.log
18. After the container is upgraded, execute the command, as shown in Figure 6
php /sobey/www/channel-pub-api/init --env=Development --overwrite=All
php /sobey/www/channel-pub-api/yii migrate --interactive=0
19. Open the URL:http://wjdev2.chinamcloud.com:8661/qq/v1/qq-cw-apps?group_id=spider, print the result, it is found that the task is not executed
bool(true)
bool(false)
bool(false)
1
20. The info command prints information about the queue status, a task status is waiting
php /sobey/www/channel-pub-api/yii copy-asset-queue/info
Jobs
- waiting: 1
- delayed: 0
- reserved: 0
- done: 0
21. Manually run the queue/listen command to listen and process queue tasks
php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0
2018-10-19 15:56:04 [pid: 1151] - Worker is started
2018-10-19 15:56:05 [5] qq\jobs\DownloadJob (attempt: 1, pid: 1151) - Started
2018-10-19 15:56:05 [5] qq\jobs\DownloadJob (attempt: 1, pid: 1151) - Error (0.031 s)
> yii\base\ErrorException: file_put_contents(./qq/runtime/robots.txt): failed to open stream: No such file or directory
^C2018-10-19 15:56:13 [pid: 1151] - Worker is stopped (0:00:09)
22. Test the job status, edit \qq\rests\qq_cw_app\indexaction.php, set file to absolute path
// 将作业推送到队列并获得其ID
$id = Yii::$app->copyAssetQueue->push(new DownloadJob([
'url' => 'http://wjdev2.chinamcloud.com:8661/robots.txt',
'file' => '/sobey/www/channel-pub-api/qq/runtime/robots.txt',
]));
23. After upgrading to the development environment, manually run the queue/listen command to listen and process the queue task. The queue task is executed successfully, but the supervisor configuration has not taken effect, and the tasks in the queue have not been automatically processed.
php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0
2018-10-19 17:04:57 [pid: 1717] - Worker is started
2018-10-19 17:04:57 [10] qq\jobs\DownloadJob (attempt: 1, pid: 1717) - Started
2018-10-19 17:04:57 [10] qq\jobs\DownloadJob (attempt: 1, pid: 1717) - Done (0.021 s)
^C2018-10-19 17:05:01 [pid: 1717] - Worker is stopped (0:00:04)
24. The local environment is Windows 10, and it is ready to be deployed to development, testing, demonstration, and production environments based on Supervisor. The latter four are all CentOS 7, and supervisor is the process monitor of Linux. It will automatically start your console process. The value of StartSecs is adjusted to 0
\build\c_files\etc\supervisord.d\yii-copy-asset-queue-worker.ini
[program:yii-copy-asset-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-copy-asset-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-copy-asset-queue-worker-stdout.log
\build\c_files\etc\supervisord.d\yii-qq-cw-video-multipart-upload-queue-worker.ini
[program:yii-qq-cw-video-multipart-upload-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii qq-cw-video-multipart-upload-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stdout.log
\Build\c_files\etc\supervisord.d\yii-pub-article-queue-worker.ini
[program:yii-pub-article-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii pub-article-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-pub-article-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-pub-article-queue-worker-stdout.log
\build\c_files\etc\supervisord.d\yii-source-callback-queue-worker.ini
[program:yii-source-callback-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii source-callback-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-source-callback-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-source-callback-queue-worker-stdout.log
25. The supervisor configuration has taken effect, and the tasks in the queue will be automatically processed, /sobey/www/channel-pub-api/qq/runtime/robots.txt The generation time of the file is in the update, as shown in Figure 7
