Implementation of Exponential Backoff on Retries in Laravel 6
1. In Laravel 8, the exponential fallback in the queue task is natively realized. Reference:https://learnku.com/laravel/t/50086
2. Reference:http://snags88.github.io/backoff-strategy-for-laravel-jobs. backoff strategy for laravel jobs.
3. In modern applications, when there is a problem, you will try to retry the block of code that caused the problem. For example, if you are using Gmail and you lose your internet connection, the website will try to reconnect to the Google server. You may notice that it will try to reconnect immediately on the first attempt. Then it will wait for a few seconds and then try again. Then wait 10-15 seconds, then try again. Then wait for another 30-40 seconds, and so on. What you’ve experienced is called a backoff strategy for retrying a failed task. It is conceivable that this strategy would be great for queuing jobs, especially for jobs that handle third-party APIs. If a third party fails, it may take some time to get back to normal, so it is beneficial to use a backward strategy. Sadly, there is no built-in backoff strategy in Laravel 6’s job. However, through the processing of our failed jobs that it exposes to us, we can easily write our own retryable jobs. The specific backoff strategy we enforce is called the exponential backoff strategy, as we add back off (i.e. 2, 4, 8, 16, 32, etc.) exponentially.
4. There is a job, the maximum number of attempts to try is 3 times. Decide to achieve exponential fallback on this basis. Existing code implementation
class ThemeAssetUploadJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, ThemeAssetGlobPatternTrait;
/**
* 任务可以尝试的最大次数。
*
* @var int
*/
public $tries = 3;
/**
* 任务失败的处理过程
*
* @param Exception $exception
* @return void
*/
public function failed(Exception $exception)
{
$themeManager = new ThemeManager();
$themeManager->themeInstallationJobFailed($exception, $this->themeInstallationTask, self::$absolutePath, self::$destination);
}
}
5. Create a new RetriableJob.php
<?php
namespace Modules\ThemeStoreDb\Jobs;
use Exception;
use Illuminate\Foundation\Bus\PendingDispatch;
class RetriableJob
{
public $tries = 1; // 覆盖默认值并确保我们不会自动重新排队
public $currentRetryCount = 1;
public $maxRetries = 3; // 3, 9, 27 seconds
public $backoffFactor = 3;
/**
* 任务失败的处理过程
*
* @param Exception $exception
* @return PendingDispatch|void
*/
public function failed(Exception $exception)
{
if ($this->currentRetryCount <= $this->maxRetries) {
$this->delay(now()->addSeconds($this->backoffFactor ** $this->currentRetryCount));
$this->currentRetryCount += 1;
return dispatch($this);
}
}
}
6. Edit themeAssetUploadJob.php, inherit to the RetriableJob, and add a log in the handle() method to record the final number of executions
class ThemeAssetUploadJob extends RetriableJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, ThemeAssetGlobPatternTrait;
/**
* 任务可以尝试的最大次数。
*
* @var int
*/
public $tries = 1;
/**
* Execute the job.
*
* @return void
* @throws OverflowException
*/
public function handle()
{
Log::info(date("Y-m-d H:i:s"));
aa();
}
/**
* 任务失败的处理过程
*
* @param Exception $exception
* @return void
*/
public function failed(Exception $exception)
{
parent::failed($exception);
$themeManager = new ThemeManager();
$themeManager->themeInstallationJobFailed($exception, $this->themeInstallationTask, self::$absolutePath, self::$destination);
}
}
7. Check the execution of the queue task. The ThemeAssetUploadJob retweeted a total of 3 more times after the first failure. The interval time is: 3, 9, and 27, respectively. in line with expectations. as shown in Figure 1
Figure 1
PS E:\wwwroot\object> php artisan queue:work
[2023-04-07 11:26:38][9Eq40nHhxmivDgMMdlFWue9CPLVtgXlB] Processing: Modules\ThemeStoreDb\Jobs\ThemeInstallationJob
[2023-04-07 11:27:20][9Eq40nHhxmivDgMMdlFWue9CPLVtgXlB] Processed: Modules\ThemeStoreDb\Jobs\ThemeInstallationJob
[2023-04-07 11:27:20][QxjbrKqslQ31DHW4feCXGmFGwg2LNU1t] Processing: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:27:22][QxjbrKqslQ31DHW4feCXGmFGwg2LNU1t] Failed: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:27:25][lbE1431gMXnH5W5gXXmfRbI3fnlumTFt] Processing: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:27:25][lbE1431gMXnH5W5gXXmfRbI3fnlumTFt] Failed: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:27:34][WCq0aET7JOPjiHGxtbJEEt5p4BGrEpko] Processing: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:27:34][WCq0aET7JOPjiHGxtbJEEt5p4BGrEpko] Failed: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:28:01][NhYilL7Uus8oBRNgsxFzrB0x1ZKkl40o] Processing: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
[2023-04-07 11:28:01][NhYilL7Uus8oBRNgsxFzrB0x1ZKkl40o] Failed: Modules\ThemeStoreDb\Jobs\ThemeAssetUploadJob
8. Check the generated log records again to confirm again. 4 times in total. The interval time is: 5, 9 and 27 respectively. in line with expectations. as shown in Figure 2
Figure 2
[2023-04-07 11:27:20] local.INFO: 2023-04-07 11:27:20
[2023-04-07 11:27:25] local.INFO: 2023-04-07 11:27:25
[2023-04-07 11:27:34] local.INFO: 2023-04-07 11:27:34
[2023-04-07 11:28:01] local.INFO: 2023-04-07 11:28:01
9. However, it is expected that the $themeManager->ThemeInstallationJobFailed() method will only be executed after the 4th failure (that is, when the real failure) is executed. However, it was found that after each failure, the $themeManager->themeInstallationJobFailed() method is executed. Edit RetriableJob.php . as shown in Figure 3
Figure 3
<?php
namespace Modules\ThemeStoreDb\Jobs;
use Exception;
use Illuminate\Foundation\Bus\PendingDispatch;
class Retriable
{
public $tries = 1; // 覆盖默认值并确保我们不会自动重新排队
public $currentRetryCount = 1;
public $maxRetries = 3; // 3, 9, 27 seconds
public $backoffFactor = 3;
/**
* 任务失败的处理过程
*
* @param Exception $exception
* @return PendingDispatch|false
*/
public function failed(Exception $exception)
{
if ($this->currentRetryCount <= $this->maxRetries) {
$this->delay(now()->addSeconds($this->backoffFactor ** $this->currentRetryCount));
$this->currentRetryCount += 1;
return dispatch($this);
}
return false;
}
}
10. Edit ThemeAssetUploadJob.php, inherit to RetriableJob.
/**
* 任务失败的处理过程
*
* @param Exception $exception
* @return void
*/
public function failed(Exception $exception)
{
if (!parent::failed($exception)) {
$themeManager = new ThemeManager();
$themeManager->themeInstallationJobFailed($exception, $this->themeInstallationTask);
}
}
11. The $ThemeManager->ThemeInstallationJobFailed() method is only executed after the 4th failure (that is, when it really fails). in line with expectations. as shown in Figure 4
Figure 4
![使用 Composer 从 Yii 2.0.x 升级时，报错： [ErrorException] Undefined variable: nameMatchesRequiredPackage](https://www.shuijingwanwq.com/wp-content/uploads/2019/02/1-520x245.png)