1. Laravel 8 natively implemented exponential backoff in queue jobs. Reference: https://learnku.com/laravel/t/50086
2. Reference: http://snags88.github.io/backoff-strategy-for-laravel-jobs . Backoff Strategy for Laravel Jobs.
3. In modern applications, when something goes wrong, the application attempts to retry the code block that caused the problem. For example, if you are using Gmail and lose your internet connection, the site will try to reconnect to the Google servers. You might notice that on the first attempt, it tries to reconnect immediately. Then it waits a few seconds and retries. Then it waits 10-15 seconds and retries. Then it waits 30-40 seconds, and so on. What you are experiencing is called a backoff strategy for retrying failed tasks. As you can imagine, adopting this strategy for queued jobs would be great, especially for jobs that handle third-party APIs. If a third party experiences an outage, it might take some time to recover, so employing a backoff strategy is beneficial. Sadly, Laravel 6 jobs do not have a built-in backoff strategy. However, through the failed job handling it exposes to us, we can easily write our own retriable jobs. The specific backoff strategy we will implement is called an exponential backoff strategy, because we increase the backoff exponentially (i.e., 2, 4, 8, 16, 32, etc.).
4. There is an existing Job with a maximum of 3 attempts. I decided to implement exponential backoff on top of this. Existing code implementation:
class ThemeAssetUploadJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, ThemeAssetGlobPatternTrait;
/**
* 任务可以尝试的最大次数。
*
* @var int
*/
public $tries = 3;
/**
* 任务失败的处理过程
*
* @param Exception $exception
* @return void
*/
public function failed(Exception $exception)
{
$themeManager = new ThemeManager();
$themeManager->themeInstallationJobFailed($exception, $this->themeInstallationTask, self::$absolutePath, self::$destination);
}
}
5. Create RetriableJob.php
<?php
namespace Modules\ThemeStoreDb\Jobs;
use Exception;
use Illuminate\Foundation\Bus\PendingDispatch;
class RetriableJob
{
public $tries = 1; // 覆盖默认值并确保我们不会自动重新排队
public $currentRetryCount = 1;
public $maxRetries = 3; // 3, 9, 27 seconds
public $backoffFactor = 3;
/**
* 任务失败的处理过程
*
* @param Exception $exception
* @return PendingDispatch|void
*/
public function failed(Exception $exception)
{
if ($this->currentRetryCount <= $this->maxRetries) {
$this->delay(now()->addSeconds($this->backoffFactor ** $this->currentRetryCount));
$this->currentRetryCount += 1;
return dispatch($this);
}
}
}6. Edit ThemeAssetUploadJob.php to inherit from RetriableJob, and add logging in the handle() method to record the final number of executions.
class ThemeAssetUploadJob extends RetriableJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, ThemeAssetGlobPatternTrait;
/**
* 任务可以尝试的最大次数。
*
* @var int
*/
public $tries = 1;
/**
* Execute the job.
*
* @return void
* @throws OverflowException
*/
public function handle()
{
Log::info(date("Y-m-d H:i:s"));
aa();
}
/**
* 任务失败的处理过程
*
* @param Exception $exception
* @return void
*/
public function failed(Exception $exception)
{
parent::failed($exception);
$themeManager = new ThemeManager();
$themeManager->themeInstallationJobFailed($exception, $this->themeInstallationTask, self::$absolutePath, self::$destination);
}
}
7. Check the queue job execution status. After the first failure of ThemeAssetUploadJob, it retried 3 more times in total. The intervals were 3, 9, and 27 seconds, respectively. This matches expectations. See Figure 1.
PS E:\wwwroot\object> php artisan queue:work
[2023-04-07 11:26:38][9Eq40nHhxmivDgMMdlFWue9CPLVtgXlB] Processing: Modules\ThemeStoreDb\Jobs\ThemeInstallationJob
[2023-04-07 11:27:20][9Eq40nHhxmivDgMMdlFWue9CPLVtgXlB] Processed: Modules\ThemeStoreDb\Jobs\ThemeInstallationJob
[2023-04-07 11:27:20][QxjbrKqslQ31DHW4feCXGmFGwg2LNU1t] Processing: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:27:22][QxjbrKqslQ31DHW4feCXGmFGwg2LNU1t] Failed: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:27:25][lbE1431gMXnH5W5gXXmfRbI3fnlumTFt] Processing: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:27:25][lbE1431gMXnH5W5gXXmfRbI3fnlumTFt] Failed: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:27:34][WCq0aET7JOPjiHGxtbJEEt5p4BGrEpko] Processing: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:27:34][WCq0aET7JOPjiHGxtbJEEt5p4BGrEpko] Failed: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:28:01][NhYilL7Uus8oBRNgsxFzrB0x1ZKkl40o] Processing: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:28:01][NhYilL7Uus8oBRNgsxFzrB0x1ZKkl40o] Failed: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
8. Check the generated log records again for further confirmation. It executed 4 times in total. The intervals were 5, 9, and 27 seconds, respectively. This matches expectations. See Figure 2.
[2023-04-07 11:27:20] local.INFO: 2023-04-07 11:27:20
[2023-04-07 11:27:25] local.INFO: 2023-04-07 11:27:25
[2023-04-07 11:27:34] local.INFO: 2023-04-07 11:27:34
[2023-04-07 11:28:01] local.INFO: 2023-04-07 11:28:01
9. However, the expectation is that the $themeManager->themeInstallationJobFailed() method should only be executed after the 4th failure (i.e., when it truly fails). But I found that the $themeManager->themeInstallationJobFailed() method was executed after every failure. Edit RetriableJob.php. See Figure 3.
<?php
namespace Modules\ThemeStoreDb\Jobs;
use Exception;
use Illuminate\Foundation\Bus\PendingDispatch;
class Retriable
{
public $tries = 1; // 覆盖默认值并确保我们不会自动重新排队
public $currentRetryCount = 1;
public $maxRetries = 3; // 3, 9, 27 seconds
public $backoffFactor = 3;
/**
* 任务失败的处理过程
*
* @param Exception $exception
* @return PendingDispatch|false
*/
public function failed(Exception $exception)
{
if ($this->currentRetryCount <= $this->maxRetries) {
$this->delay(now()->addSeconds($this->backoffFactor ** $this->currentRetryCount));
$this->currentRetryCount += 1;
return dispatch($this);
}
return false;
}
}10. Edit ThemeAssetUploadJob.php to inherit from RetriableJob.
/**
* 任务失败的处理过程
*
* @param Exception $exception
* @return void
*/
public function failed(Exception $exception)
{
if (!parent::failed($exception)) {
$themeManager = new ThemeManager();
$themeManager->themeInstallationJobFailed($exception, $this->themeInstallationTask);
}
}
11. The $themeManager->themeInstallationJobFailed() method is only executed after the 4th failure (i.e., when it truly fails). This matches expectations. See Figure 4.
