在 Yii2 高级模板中基于 Yii2 队列扩展实现异步执行任务

1、安装 Yii2 队列扩展，执行以下命令

composer require --prefer-dist yiisoft/yii2-queue

2、查看 composer.json，已经安装成功

    "require": {
        "yiisoft/yii2-queue": "^2.1"
    },

3、把多个队列组件注册到控制台，编辑 \console\config\main.php

    "require": {
        "yiisoft/yii2-queue": "^2.1"
    },

4、多个队列组件的配置，编辑 \environments\dev\common\config\main-local.php、\environments\prod\common\config\main-local.php

        'copyAssetQueue' => [ // 复制资源文件队列
            'class' => 'yii\queue\redis\Queue',
            'redis' => 'redis', // Redis 连接组件或它的配置
            'channel' => 'cpa:queue:copy:asset', // 队列键前缀
            'ttr' => 10 * 60, // 作业处理的最长时间，单位(秒)
            'as log' => 'yii\queue\LogBehavior',
        ],
        'qqCwVideoMultipartUploadQueue' => [ // 企鹅号的内容网站应用的视频文件分片上传队列
            'class' => 'yii\queue\redis\Queue',
            'redis' => 'redis', // Redis 连接组件或它的配置
            'channel' => 'cpa:queue:qq:cw:video:multipart:upload', // 队列键前缀
            'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
            'as log' => 'yii\queue\LogBehavior',
        ],
        'pubArticleQueue' => [ // 发布文章队列
            'class' => 'yii\queue\redis\Queue',
            'redis' => 'redis', // Redis 连接组件或它的配置
            'channel' => 'cpa:queue:pub:article', // 队列键前缀
            'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
            'as log' => 'yii\queue\LogBehavior',
        ],
        'sourceCallbackQueue' => [ // 来源回调队列
            'class' => 'yii\queue\redis\Queue',
            'redis' => 'redis', // Redis 连接组件或它的配置
            'channel' => 'cpa:queue:source:callback', // 队列键前缀
            'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
            'as log' => 'yii\queue\LogBehavior',
        ],

5、定义每个被发送到队列的任务的基础类，新建 \common\jobs\Job.php

<?php
/**
 * Created by PhpStorm.
 * User: terryhong
 * Date: 2018/9/7
 * Time: 下午4:54
 */
namespace common\jobs;

use yii\base\BaseObject;
use yii\queue\JobInterface;

/**
 * 队列的任务的基础类
 *
 * @author Qiang Wang <shuijingwanwq@163.com>
 * @since 1.0
 */class Job extends BaseObject implements JobInterface
{
    public function execute($queue){

    }
}

6、每个被发送到队列的任务应该被定义为一个单独的类。基于官方示例，如果您需要下载并保存一个文件，该类可能看起来如下

<?php
/**
 * Created by PhpStorm.
 * User: Qiang Wang
 * Date: 2018/10/18
 * Time: 10:39
 */
namespace common\jobs;

/**
 * 下载并保存一个文件
 *
 * @author Qiang Wang <shuijingwanwq@163.com>
 * @since 1.0
 */class DownloadJob extends Job
{
    public $url;
    public $file;

    public function execute($queue)
    {
        file_put_contents($this->file, file_get_contents($this->url));
    }
}

7、在 qq 应用中定义对应的下载类，新建 \qq\jobs\DownloadJob.php

<?php
/**
 * Created by PhpStorm.
 * User: Qiang Wang
 * Date: 2018/10/19
 * Time: 10:28
 */
namespace qq\jobs;

/**
 * 下载并保存一个文件
 *
 * @author Qiang Wang <shuijingwanwq@163.com>
 * @since 1.0
 */class DownloadJob extends \common\jobs\DownloadJob
{

}

8、作业状态的测试，编辑 \qq\rests\qq_cw_app\IndexAction.php

<?php
/**
 * @link http://www.yiiframework.com/
 * @copyright Copyright (c) 2008 Yii Software LLC
 * @license http://www.yiiframework.com/license/
 */
namespace qq\rests\qq_cw_app;

use Yii;
use qq\jobs\DownloadJob;
use yii\data\ActiveDataProvider;
use yii\base\InvalidConfigException;

/**
 *
 * For more details and usage information on IndexAction, see the [guide article on rest controllers](guide:rest-controllers).
 *
 * @author Qiang Wang <shuijingwanwq@163.com>
 * @since 1.0
 */class IndexAction extends \yii\rest\IndexAction
{

    /**
     * Prepares the data provider that should return the requested collection of the models.
     * @return ActiveDataProvider
     * @throws InvalidConfigException if the configuration is invalid.
     */    protected function prepareDataProvider()
    {
        // 将作业推送到队列并获得其ID
        $id = Yii::$app->copyAssetQueue->push(new DownloadJob([
            'url' => 'http://wjdev2.chinamcloud.com:8661/robots.txt',
            'file' => './qq/runtime/robots.txt',
        ]));
        // 这个作业等待执行
        var_dump(Yii::$app->copyAssetQueue->isWaiting($id));

        // Worker 从队列获取作业，并执行它
        var_dump(Yii::$app->copyAssetQueue->isReserved($id));

        // Worker 作业执行完成
        var_dump(Yii::$app->copyAssetQueue->isDone($id));
        echo $id;
        exit;

    }
}

9、打开网址：http://api.channel-pub-api.localhost/qq/v1/qq-cw-apps?group_id=spider ，打印结果，发现任务并未执行

bool(true)
bool(false)
bool(false)
1

10、查看 Redis ，队列已经存入，如图1

图1

600;O:19:"qq\jobs\DownloadJob":2:{s:3:"url";s:45:"http://wjdev2.chinamcloud.com:8661/robots.txt";s:4:"file";s:23:"./qq/runtime/robots.txt";}

11、info 命令打印关于队列状态的信息，一个任务状态为等待

.\yii copy-asset-queue/info

Jobs
- waiting: 1
- delayed: 0
- reserved: 0
- done: 0

12、run 命令获取并执行循环中的任务，直到队列为空，如图2

图2

.\yii copy-asset-queue/run --verbose=1 --isolate=1 --color=0
.\yii copy-asset-queue/info

2018-10-19 11:07:30 [pid: 36016] - Worker is started
2018-10-19 11:07:30 [2] qq\jobs\DownloadJob (attempt: 1, pid: 36016) - Started
2018-10-19 11:07:30 [2] qq\jobs\DownloadJob (attempt: 1, pid: 36016) - Done (0.255 s)
2018-10-19 11:07:30 [pid: 36016] - Worker is stopped (0:00:00)

Jobs
- waiting: 0
- delayed: 0
- reserved: 0
- done: 1

13、查看 Redis ，键：cpa:queue:copy:asset.messages、cpa:queue:copy:asset.waiting 均已经被删除，如图3

图3

14、文件 \qq\runtime\robots.txt 已经被保存，如图4

图4

15、为了使开发过程更加友好，您可以向 Yii2 调试模块添加一个面板。面板显示计数器和队列任务列表。编辑 \environments\dev\api\config\main-local.php、\environments\dev\backend\config\main-local.php、\environments\dev\frontend\config\main-local.php、\environments\dev\qq\config\main-local.php、\environments\dev\rpc\config\main-local.php、\environments\dev\wx\config\main-local.php

if (!YII_ENV_TEST) {
    // configuration adjustments for 'dev' environment
    $config['bootstrap'][] = 'debug';
    $config['modules']['debug'] = [
        'class' => 'yii\debug\Module',
        'panels' => [
            'queue' => 'yii\queue\debug\Panel',
        ],
    ];

    $config['bootstrap'][] = 'gii';
    $config['modules']['gii'] = [
        'class' => 'yii\gii\Module',
    ];
}

16、查看 Yii2 调试模块，已经增加 queue 面板，如图5

图5

17、本地环境是 Windows 10，准备基于 Supervisor 部署至开发、测试、演示、生产环境上，后四者皆是 Centos 7，Supervisor 是Linux的进程监视器。它会自动启动您的控制台进程。

\build\c_files\etc\supervisord.d\yii-copy-asset-queue-worker.ini

[program:yii-copy-asset-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-copy-asset-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-copy-asset-queue-worker-stdout.log

\build\c_files\etc\supervisord.d\yii-qq-cw-video-multipart-upload-queue-worker.ini

[program:yii-qq-cw-video-multipart-upload-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii qq-cw-video-multipart-upload-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stdout.log

\build\c_files\etc\supervisord.d\yii-pub-article-queue-worker.ini

[program:yii-pub-article-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii pub-article-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-pub-article-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-pub-article-queue-worker-stdout.log

\build\c_files\etc\supervisord.d\yii-source-callback-queue-worker.ini

[program:yii-source-callback-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii source-callback-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-source-callback-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-source-callback-queue-worker-stdout.log

18、容器升级完毕后，执行命令，如图6

图6

php /sobey/www/channel-pub-api/init --env=Development --overwrite=All
php /sobey/www/channel-pub-api/yii migrate --interactive=0

19、打开网址：http://wjdev2.chinamcloud.com:8661/qq/v1/qq-cw-apps?group_id=spider ，打印结果，发现任务并未执行

bool(true)
bool(false)
bool(false)
1

20、info 命令打印关于队列状态的信息，一个任务状态为等待

php /sobey/www/channel-pub-api/yii copy-asset-queue/info

Jobs
- waiting: 1
- delayed: 0
- reserved: 0
- done: 0

21、手动运行 queue/listen 命令监听和处理队列任务

php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0

2018-10-19 15:56:04 [pid: 1151] - Worker is started
2018-10-19 15:56:05 [5] qq\jobs\DownloadJob (attempt: 1, pid: 1151) - Started
2018-10-19 15:56:05 [5] qq\jobs\DownloadJob (attempt: 1, pid: 1151) - Error (0.031 s)
> yii\base\ErrorException: file_put_contents(./qq/runtime/robots.txt): failed to open stream: No such file or directory
^C2018-10-19 15:56:13 [pid: 1151] - Worker is stopped (0:00:09)

22、作业状态的测试，编辑 \qq\rests\qq_cw_app\IndexAction.php，设置 file 为绝对路径

        // 将作业推送到队列并获得其ID
        $id = Yii::$app->copyAssetQueue->push(new DownloadJob([
            'url' => 'http://wjdev2.chinamcloud.com:8661/robots.txt',
            'file' => '/sobey/www/channel-pub-api/qq/runtime/robots.txt',
        ]));

23、升级至开发环境后，手动运行 queue/listen 命令监听和处理队列任务，队列任务执行成功，但是 Supervisor 配置未生效，未自动处理队列中的任务，一直处于等待中

php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0

2018-10-19 17:04:57 [pid: 1717] - Worker is started
2018-10-19 17:04:57 [10] qq\jobs\DownloadJob (attempt: 1, pid: 1717) - Started
2018-10-19 17:04:57 [10] qq\jobs\DownloadJob (attempt: 1, pid: 1717) - Done (0.021 s)
^C2018-10-19 17:05:01 [pid: 1717] - Worker is stopped (0:00:04)

24、本地环境是 Windows 10，准备基于 Supervisor 部署至开发、测试、演示、生产环境上，后四者皆是 Centos 7，Supervisor 是Linux的进程监视器。它会自动启动您的控制台进程。startsecs 的值调整为 0

\build\c_files\etc\supervisord.d\yii-copy-asset-queue-worker.ini

[program:yii-copy-asset-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-copy-asset-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-copy-asset-queue-worker-stdout.log

\build\c_files\etc\supervisord.d\yii-qq-cw-video-multipart-upload-queue-worker.ini

[program:yii-qq-cw-video-multipart-upload-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii qq-cw-video-multipart-upload-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stdout.log

\build\c_files\etc\supervisord.d\yii-pub-article-queue-worker.ini

[program:yii-pub-article-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii pub-article-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-pub-article-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-pub-article-queue-worker-stdout.log

\build\c_files\etc\supervisord.d\yii-source-callback-queue-worker.ini

[program:yii-source-callback-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii source-callback-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-source-callback-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-source-callback-queue-worker-stdout.log

25、Supervisor 配置已生效，会自动处理队列中的任务，/sobey/www/channel-pub-api/qq/runtime/robots.txt 文件的生成时间时刻在更新中，如图7

图7

永夜

Next 在 Yii2 高级模板中基于 Yii2 队列扩展实现异步执行任务，附加事件处理器，在(每次成功执行作业后、在作业执行期间发生未捕获的异常时) »

Previous « 执行 .\nginx -s reload 时，提示：nginx: [emerg] host not found in upstream "api.aims-api.localhost" in C:/nginx-1.10.1/conf/vhosts/aims.conf:14 的分析解决

在 Yii2 高级模板中基于 Yii2 队列扩展实现异步执行任务

Recent Posts

Laravel 9 中的 Redis 门面 hset 不允许传递多个字段/值

在 Google Chrome 中由于网站升级，打开空白，仅在无痕模式中可正常打开的解决

在 https://tinys.pl/cn/swap 上用 SOL 购买得到 NUTS，在欧易 Web3 钱包中显示数量为 0 的排查分析

在 Laravel 9 中，基于 Spatie\QueryBuilder\QueryBuilder 来生成查询 SQL，同一字段可支持：partial、exact

基于 PHP-Xlswriter 导出 Excel 文件时，设置日期列的格式

在 OneKey 钱包中，发送 Optimism 网络中的 USDT 的流程

在 Yii2 高级模板中基于 Yii2 队列扩展实现异步执行任务

Related Post

Recent Posts

Laravel 9 中的 Redis 门面 hset 不允许传递多个字段/值

在 Google Chrome 中由于网站升级，打开空白，仅在无痕模式中可正常打开的解决

在 https://tinys.pl/cn/swap 上用 SOL 购买得到 NUTS，在 欧易 Web3 钱包 中显示数量为 0 的排查分析

在 Laravel 9 中，基于 Spatie\QueryBuilder\QueryBuilder 来生成查询 SQL，同一字段可支持：partial、exact

基于 PHP-Xlswriter 导出 Excel 文件时，设置日期列的格式

在 OneKey 钱包中，发送 Optimism 网络中的 USDT 的流程

在 https://tinys.pl/cn/swap 上用 SOL 购买得到 NUTS，在欧易 Web3 钱包中显示数量为 0 的排查分析