The investigation and analysis of the page of the theme editor that is inconsistent with the actual page is not consistent with the actual page

1. The current implementation principle of the theme editor is: when a configuration item in the editor changes, the temporary storage API of the backend will be requested to cache the data to Redis. When the user clicks the save button, the save API of the backend is requested, and the data will be read from Redis and stored in MySQL.

2. When the user edits some configuration items quickly, the page data of the theme editor may be inconsistent with the actual page. as shown in Figure 1

3. Check the order of the network requests, and the last time to request the temporary storage API of the backend, and determine that it is before the save API of the request backend. Then it can be inferred that when the save API of the request backend and the backend read data from Redis, the last time the temporary storage API of the request backend has not saved the data to Redis At this time, the last time the data is read. Examples are as follows, time is calculated in milliseconds:

Initiation time of the last requested backend of the last request: 2022/11/22 10:38:49.200
Initiation time of the save API of the request backend: 2022/11/22 10:38:49.300
The Scratch API of the last request backend to cache the data to Redis when: 2022/11/22 10:38:49.800
The save API of the request backend reads the data stored in Redis from Redis when stored to MySQL: 2022/11/22 10:38:49.600

4. View the response time point of the last request backend of the last request in the network request: TUE, 22 Nov 2022 02:38:49 GMT. as shown in Figure 2

5. View the response time point of the save API of the request backend in the network request: TUE, 22 Nov 2022 02:38:49 GMT. as shown in Figure 3

6. Check the duration of the last request backend in the network request: 686.69 milliseconds. as shown in Figure 4

7. View the duration of the request to save the backend of the request in the network request: 380.19 milliseconds. as shown in Figure 5

8. At this stage, there are 2 schemes. The first solution is front-end processing. When the user clicks the save button, only after all requests to the back-end of the back-end storage API end, the storage API of the back-end is requested. The second scenario is back-end processing, and only after all requesting backend’s staging APIs cache the data to Redis, the back-end save API reads data from Redis and is stored in MySQL.

9. In the end, it was decided to adopt the second scheme, which was handled by the back end. (1) When the backend receives a request to temporarily store the API, add 1 to the key in the current session: update_count, and when the data is cached to redis, the update_count is minus 1. When the backend receives the request to save the API, it determines the key in the current session: update_count, if it is greater than 0, continue to wait, and the key will be judged in the current session again during the waiting period. : update_count, after the judgment reaches 3 times, continue to execute. Note: 1 second = 1000 ms; 1 ms = 1000 microseconds. (2) Read data from Redis and store it into MySQL.

10. The implementation in the temporary storage API, based on the Redis command: incr and decr are implemented as follows

    /**
     * 将 更新计数 中储存的数字值增一
     * @param string $sessionId
     * @return int
     */
    public static function incrUpdateCount(string $sessionId): int
    {
        $key = self::getUpdateCountKey($sessionId);
        $incr = Redis::connection('cache')->incr($key);
        Redis::connection('cache')->expire($key, ThemePreviewInterface::TTL);
        return $incr;
    }

    /**
     * 将 更新计数 中储存的数字值减一
     * @param string $sessionId
     * @return int
     */
    public static function decrUpdateCount(string $sessionId): int
    {
        $key = self::getUpdateCountKey($sessionId);
        $decr = Redis::connection('cache')->decr($key);
        Redis::connection('cache')->expire($key, ThemePreviewInterface::TTL);
        return $decr;
    }

11. Save the implementation in the API, based on the while judgment, a total of 3 judgments

    /**
     * 获取 更新计数 中储存的数字值
     * @param string $sessionId
     * @return mixed
     */
    public static function getUpdateCount(string $sessionId)
    {
        return Redis::connection('cache')->get(self::getUpdateCountKey($sessionId));
    }

	$i = 1;
	while (ThemePreview::getUpdateCount($sessionId) !== null && ThemePreview::getUpdateCount($sessionId) > 0 && $i <= 3) {
		file_put_contents(storage_path() . '/logs/Publish-' . microtime(true) . '-' . mt_rand()  . '.txt', print_r($i, true), FILE_APPEND | LOCK_EX);
		usleep(500000); // 延迟执行 0.5 秒
		$i++;
	}

12. The output file is as follows. 1669187163.3346 – 1669187162.8234 = 0.5112. 1669187162.8234 – 1669187162.3034 = 0.52. The creation time difference is roughly 0.5 seconds, in line with expectations. as shown in Figure 8

Publish-1669187162.3034-776928128.txt
Publish-1669187162.8234-549007488.txt
Publish-1669187163.3346-885156477.txt

13. The local environment reappears and cannot be reproduced, but it is determined that the file is generated: publish-1669188133.9627-720653875.txt, its value is 1. Indicates that a wait was performed in the Save API. as shown in Figure 9

14. View the duration of the last request to the backend of the last request in the network request: 2.79 seconds, because the local environment performance is not good. As shown in Figure 10

15. View the duration of the request to save the backend of the request in the network request: 3.37 seconds, which is longer than the duration of the temporary storage API. as shown in Figure 11

16. However, this scheme still has a certain probability of losing data. If the temporary storage interface arrives at the server later than the time when the save interface is completed. The final decision is made to refer to the implementation of Shopify.

17. Refer to Shopify’s theme editor. View the save API of its request backend, and the request and response variables are as follows. as shown in Figure 12

{
  "eventMetadata": {
    "dirtyTransitionCountInSession": 13,
    "durationSeconds": 0,
    "lastViewport": "desktop",
    "pageType": "JSON"
  },
  "sessionId": "etPGCT4PQr7YyCr87svY9KJz",
  "resourceHashes": [
    {
      "hashValue": "fe8062f520ac1ab9be9a336c76d9f77517551223",
      "resourceId": "gid://shopify/OnlineStoreTheme/130440429753"
    },
    {
      "hashValue": "17183337545547076327",
      "resourceId": "gid://shopify/OnlineStoreThemeJsonTemplate/index?theme_id=130440429753"
    }
  ],
  "ignoreConflicts": null
}

{
    "data": {
        "onlineStoreThemeEditorSessionPublish": {
            "conflicts": null,
            "userErrors": [],
            "updatedHash": {
                "resourceId": "gid:\/\/shopify\/OnlineStoreTheme\/130440429753",
                "hashValue": "fe8062f520ac1ab9be9a336c76d9f77517551223",
                "__typename": "OnlineStoreResourceHash"
            },
            "__typename": "OnlineStoreThemeEditorSessionPublishPayload"
        }
    },
    "extensions": {
        "cost": {
            "requestedQueryCost": 10,
            "actualQueryCost": 10,
            "throttleStatus": {
                "maximumAvailable": 1200.0,
                "currentlyAvailable": 1190,
                "restoreRate": 60.0
            }
        }
    }
}

18. Preliminary speculation should be related to HashValue, and save operations are only performed when the HashValue of the data in the cache is equal to the HashValue in the request parameter. The duration of the storage API is nearly 1.5 seconds. as shown in Figure 13

19. The final design is the following scheme. In the temporary storage interface, the hashValue is calculated based on the content of the request body, and it is saved to the Redis key. Then in the save interface, compare it with the hashValue saved to Redis based on the hashValue in the request parameter, and save it if it is equal. If it is not equal, but there is a corresponding HashValue under IgnoreConflicts, it can also be saved. If it is not equal, and there is no corresponding HashValue under IgnoreConflicts, it will not be saved, and the returned CONFLICTS contains the corresponding HashValue.

20. Existing requests and responses. Figure 14

21. Implementation in Staging API

    /**
     * 获取更新时的哈希的缓存键
     * @param string $sessionId
     * @param string $resourceId
     * @return string
     */
    private static function getUpdateHashKey(string $sessionId, string $resourceId): string
    {
        return ThemePreviewInterface::THEME_EDITOR_SESSION_PREFIX . $sessionId . ':' . $resourceId . ':' . ThemePreviewInterface::UPDATE_HASH_KEY;
    }

    /**
     * 保存更新时的哈希
     * @param string $sessionId
     * @param string $resourceId
     * @param string $hashValue
     * @return bool
     */
    public static function saveUpdateHash(string $sessionId, string $resourceId, string $hashValue): bool
    {
        return self::getCacheTags($sessionId)->put(self::getUpdateHashKey($sessionId, $resourceId), $hashValue, ThemePreviewInterface::TTL);
    }

    /**
     * 获取更新时的哈希
     * @param string $sessionId
     * @param string $resourceId
     * @return mixed
     */
    public static function getUpdateHash(string $sessionId, string $resourceId)
    {
        return self::getCacheTags($sessionId)->get(self::getUpdateHashKey($sessionId, $resourceId));
    }

    /**
     * 判断哈希值是否与缓存中的相等
     * @param string $sessionId
     * @param string $resourceId
     * @param string $hashValue
     * @return bool
     */
    public static function isHashEq(string $sessionId, string $resourceId, string $hashValue): bool
    {
        return self::getUpdateHash($sessionId, $resourceId) == $hashValue;
    }

	$variables = json_encode($args);
	ThemePreview::saveUpdateHash($sessionId, $templateBasename, sha1($variables));

22. Save the implementation in the API

    public function __invoke($rootValue, array $args, GraphQLContext $context, ResolveInfo $resolveInfo)
    {
        $conflicts = null;
        $ignoreResourceIds = data_get($args, 'ignoreConflicts.*.resourceId');
        $hashValue = $themeId;
        foreach ($args['resourceHashes'] as $resourceHash) {
            if ($this->isConflict($sessionId, $resourceHash, $ignoreResourceIds)) {
                $conflicts[] = [
                    'resourceId' => $resourceHash['resourceId'],
                    'hashValue' => $resourceHash['hashValue']
                ];
                continue;
            }

            if ($resourceHash['resourceId'] == $themeId) {
                $hashValue = $resourceHash['hashValue'];
                $this->themeConfigLoader->publishSettingsData();
            } else {
                $this->rawSchemaLoaderAdapter->publishTemplateSettings($resourceHash['resourceId']);
            }
        }

        return [
            'conflicts' => $conflicts,
            'session' => ['sessionId' => $sessionId],
            'updatedHash' => [
                'resourceId' => $themeId,
                'hashValue' => $hashValue
            ],
            'publishedAt' => $publishedAt
        ];
    }

    /**
     * 判断哈希值是否冲突
     * @param string $sessionId
     * @param array $resourceHash
     * @param ?array $ignoreResourceIds
     * @return bool
     */
    private function isConflict(string $sessionId, array $resourceHash, ?array $ignoreResourceIds): bool
    {
        if (!ThemePreview::isHashEq($sessionId, $resourceHash['resourceId'], $resourceHash['hashValue'])) {
            if (!is_array($ignoreResourceIds) || !in_array($resourceHash['resourceId'], $ignoreResourceIds)) {
                return true;
            }
        }
        return false;
    }

23. The first request, when HashValue is hit, the response list of the response is NULL. At this time, the front end is no longer processed. as shown in Figure 15

24. The first request, when HashValue misses, the response list of the response is not null. as shown in Figure 16

25. When the first request, the conflicting list of the response is not NULL, the front-end needs a second request at this time, and the request parameter does not change. as shown in Figure 17

26. The second request, when HashValue is hit, the response list of the response is NULL. At this time, the front end is no longer processed.

27. When the conflict list of the response is not NULL for the second request, the front-end needs a third request at this time, and the value of the request parameter IgnoreConflicts is consistent with the resourceHashes. as shown in Figure 18

28. Reference:https://www.shuijingwanwq.com/2022/12/08/7260/. In Laravel 6 and Lighthouse 5, the parameter $args of the method __invoke is inconsistent with the front-end request parameters. The staging interface is finally implemented as follows

ThemePreview::saveUpdateHash($sessionId, $templateBasename, sha1($context->request()->getContent()));