没有不值得去解决的问题,也没有不值得去学习的技术!

Split a Queue Job in Laravel 6 When the Maximum Tries Cannot Exceed 1

When the newly split queue job fails, the original queue job has attempted 1 time, and the new queue job has attempted 3 times. This matches expectations

1. In Laravel 6, when the maximum number of attempts for a queue job is greater than 1, it fails on the second run: must be an instance of App\Models\Theme, null given. The reason is that the queue job does not support being retried. See Figure 1.

In Laravel 6, when the maximum number of attempts for a queue job is greater than 1, it fails on the second run: must be an instance of App\Models\Theme, null given. The reason is that the queue job does not support being retried
Figure 1
PHP

    /**
     * 任务可以尝试的最大次数。
     *
     * @var int
     */
    public $tries = 2;

Plaintext

Symfony\Component\Debug\Exception\FatalThrowableError: Argument 1 passed to Modules\ThemeStoreDb\ThemeSetting\ThemeSettingMigration::migrateBladeSettingsData() must be an instance of App\Models\Theme, null given, called in E:\wwwroot\object\Modules\ThemeStoreDb\Handlers\ThemeHandler.php on line 129 in E:\wwwroot\object\Modules\ThemeStoreDb\ThemeSetting\ThemeSettingMigration.php:42
Stack trace:
#0 E:\wwwroot\object\Modules\ThemeStoreDb\Handlers\ThemeHandler.php(129): Modules\ThemeStoreDb\ThemeSetting\ThemeSettingMigration->migrateBladeSettingsData(NULL, Object(Modules\ThemeStoreDb\Entities\ThemeInstallationTask))
#1 E:\wwwroot\object\Modules\ThemeStoreDb\Handlers\ThemeHandler.php(64): Modules\ThemeStoreDb\Handlers\ThemeHandler->migrateSettingsData(Object(Modules\ThemeStoreDb\Entities\ThemeInstallation), Object(Modules\ThemeStoreDb\Entities\ThemeInstallationTask), Array)
#2 E:\wwwroot\object\Modules\ThemeStoreDb\Jobs\ThemeInstallationJob.php(203): Modules\ThemeStoreDb\Handlers\ThemeHandler->migrateThemeSettings(Object(Modules\ThemeStoreDb\Entities\ThemeInstallation), Object(Modules\ThemeStoreDb\Entities\ThemeInstallationTask))
#3 E:\wwwroot\object\Modules\ThemeStoreDb\Jobs\ThemeInstallationJob.php(90): Modules\ThemeStoreDb\Jobs\ThemeInstallationJob->migrateThemeSettings()
#4 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Container\BoundMethod.php(36): Modules\ThemeStoreDb\Jobs\ThemeInstallationJob->handle()
#5 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Container\Util.php(37): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#6 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Container\BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#7 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Container\BoundMethod.php(37): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#8 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Container\Container.php(590): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#9 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Bus\Dispatcher.php(94): Illuminate\Container\Container->call(Array)
#10 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Pipeline\Pipeline.php(130): Illuminate\Bus\Dispatcher->Illuminate\Bus\{closure}(Object(Modules\ThemeStoreDb\Jobs\ThemeInstallationJob))
#11 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Pipeline\Pipeline.php(105): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Modules\ThemeStoreDb\Jobs\ThemeInstallationJob))
#12 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Bus\Dispatcher.php(98): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#13 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Queue\CallQueuedHandler.php(83): Illuminate\Bus\Dispatcher->dispatchNow(Object(Modules\ThemeStoreDb\Jobs\ThemeInstallationJob), false)
#14 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Pipeline\Pipeline.php(130): Illuminate\Queue\CallQueuedHandler->Illuminate\Queue\{closure}(Object(Modules\ThemeStoreDb\Jobs\ThemeInstallationJob))
#15 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Pipeline\Pipeline.php(105): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Modules\ThemeStoreDb\Jobs\ThemeInstallationJob))
#16 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Queue\CallQueuedHandler.php(85): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#17 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Queue\CallQueuedHandler.php(59): Illuminate\Queue\CallQueuedHandler->dispatchThroughMiddleware(Object(Illuminate\Queue\Jobs\RedisJob), Object(Modules\ThemeStoreDb\Jobs\ThemeInstallationJob))
#18 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Queue\Jobs\Job.php(88): Illuminate\Queue\CallQueuedHandler->call(Object(Illuminate\Queue\Jobs\RedisJob), Array)
#19 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Queue\Worker.php(368): Illuminate\Queue\Jobs\Job->fire()
#20 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Queue\Worker.php(314): Illuminate\Queue\Worker->process('redis', Object(Illuminate\Queue\Jobs\RedisJob), Object(Illuminate\Queue\WorkerOptions))
#21 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Queue\Worker.php(134): Illuminate\Queue\Worker->runJob(Object(Illuminate\Queue\Jobs\RedisJob), 'redis', Object(Illuminate\Queue\WorkerOptions))
#22 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Queue\Console\WorkCommand.php(112): Illuminate\Queue\Worker->daemon('redis', 'default', Object(Illuminate\Queue\WorkerOptions))
#23 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Queue\Console\WorkCommand.php(96): Illuminate\Queue\Console\WorkCommand->runWorker('redis', 'default')
#24 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Container\BoundMethod.php(36): Illuminate\Queue\Console\WorkCommand->handle()
#25 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Container\Util.php(37): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#26 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Container\BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#27 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Container\BoundMethod.php(37): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#28 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Container\Container.php(590): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#29 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Console\Command.php(134): Illuminate\Container\Container->call(Array)
#30 E:\wwwroot\object\vendor\symfony\console\Command\Command.php(255): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#31 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Console\Command.php(121): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#32 E:\wwwroot\object\vendor\symfony\console\Application.php(1009): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#33 E:\wwwroot\object\vendor\symfony\console\Application.php(273): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Queue\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#34 E:\wwwroot\object\vendor\symfony\console\Application.php(149): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#35 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Console\Application.php(93): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#36 E:\wwwroot\object\vendor\laravel\framework\src\Illuminate\Foundation\Console\Kernel.php(131): Illuminate\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#37 E:\wwwroot\object\artisan(37): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#38 {main}

2. During the first run, the queue job deleted records from some database tables. When it runs for the second time, the deleted records are needed again, which causes the second run to fail. This compatibility issue is difficult to handle, so I ultimately decided to split the queue job.

3. I split the task step that needs to be retried into a separate queue job and set its maximum number of attempts to greater than 1. The original queue job has a maximum number of attempts equal to 1 to avoid errors on a second run.

PHP

    /**
     * 任务可以尝试的最大次数。
     *
     * @var int
     */
    public $tries = 1;

	// 推送任务到队列
	UploadThemeAssetToCdnJob::dispatch($this->themeInstallationTask, $this->absolutePath, $this->destination);

PHP

    /**
     * 任务可以尝试的最大次数。
     *
     * @var int
     */
    public $tries = 3;

4. When the newly split queue job fails, the original queue job has attempted 1 time, and the new queue job has attempted 3 times. This matches expectations. See Figure 2.

When the newly split queue job fails, the original queue job has attempted 1 time, and the new queue job has attempted 3 times. This matches expectations
Figure 2
Plaintext

PS E:\wwwroot\object> php artisan queue:work
[2023-04-06 17:01:50][FSn8E0VIfENgDzP2CydEHcP4ZKdub5vT] Processing: Modules\ThemeStoreDb\Jobs\ThemeInstallationJob
[2023-04-06 17:02:39][FSn8E0VIfENgDzP2CydEHcP4ZKdub5vT] Processed:  Modules\ThemeStoreDb\Jobs\ThemeInstallationJob
[2023-04-06 17:02:39][Tf9DYc8hDoW6f3GVH2889yb9ZA1jSyWX] Processing: Modules\ThemeStoreDb\Jobs\UploadThemeAssetToCdnJob
[2023-04-06 17:02:50][Tf9DYc8hDoW6f3GVH2889yb9ZA1jSyWX] Processing: Modules\ThemeStoreDb\Jobs\UploadThemeAssetToCdnJob
[2023-04-06 17:02:50][Tf9DYc8hDoW6f3GVH2889yb9ZA1jSyWX] Processing: Modules\ThemeStoreDb\Jobs\UploadThemeAssetToCdnJob
[2023-04-06 17:02:50][Tf9DYc8hDoW6f3GVH2889yb9ZA1jSyWX] Failed:     Modules\ThemeStoreDb\Jobs\UploadThemeAssetToCdnJob

5. After 3 consecutive queue jobs all succeed, each queue job has attempted 1 time. This matches expectations. See Figure 3.

After 3 consecutive queue jobs all succeed, each queue job has attempted 1 time. This matches expectations
Figure 3
Plaintext

PS E:\wwwroot\object> php artisan queue:work
[2023-04-06 17:21:13][mJUCYgune7X37F4pBxbBTfqvKzOTi1I4] Processing: Modules\ThemeStoreDb\Jobs\ThemeInstallationJob
[2023-04-06 17:21:56][mJUCYgune7X37F4pBxbBTfqvKzOTi1I4] Processed:  Modules\ThemeStoreDb\Jobs\ThemeInstallationJob
[2023-04-06 17:21:56][245oFV2O4NuYfD81b9gwKE4rnG6iuSCa] Processing: Modules\ThemeStoreDb\Jobs\UploadThemeAssetToCdnJob
[2023-04-06 17:22:02][245oFV2O4NuYfD81b9gwKE4rnG6iuSCa] Processed:  Modules\ThemeStoreDb\Jobs\UploadThemeAssetToCdnJob
[2023-04-06 17:22:02][htBRuBPcfiKFXebjeCLuk8Vf6Lx55I5I] Processing: Modules\ThemeStoreDb\Jobs\ThemeInstallationedJob
[2023-04-06 17:22:04][htBRuBPcfiKFXebjeCLuk8Vf6Lx55I5I] Processed:  Modules\ThemeStoreDb\Jobs\ThemeInstallationedJob

关于作者 · 持续学习与技术实践

我是一名拥有 15 年以上开发经验的技术从业者,自 2013 年起持续运营个人技术博客,记录工作与个人项目中遇到的真实问题,以及解决问题过程中的探索、实践和思考。

除了技术写作,我也在持续开发和维护自己的独立项目,探索新技术和新工具在实际场景中的应用。我始终相信,没有不值得去解决的问题,也没有不值得去学习的技术。

如果这篇文章对你有所帮助,欢迎浏览博客中的其他文章,也欢迎交流技术经验与想法。

关于我  ·  GitHub  ·  邮件联系