1. At 19:36:00 on 2017-06-18, when the match started, the application response time was 51.8 ms, which is normal, as shown in Figure 1.
2. During the entire football match, the response time increased linearly. From 20:30:00 to 20:40:00 on 2017-06-18, the response time dropped because the match was in halftime, and the match data sent to the server remained basically unchanged, resulting in simple processing logic, as shown in Figure 2.
3. At 21:25:00 on 2017-06-18, when the match was about to end, the application response time was 2320 ms. The average response time required by the business target is 150 ms, so the performance failed to meet the business requirements, as shown in Figure 3.
4. In the APM Transaction trace, it can be found that in a request with a response time of 5710 ms, Redis-related operations consumed 4390 ms, accounting for 77%. The httpclient called another interface, which also used Redis. Therefore, the analysis result indicates that Redis is the performance bottleneck, as shown in Figure 4.
5. In the Trace details of the APM Transaction trace, it was found that shujujiexi\models\redis\GamePlayerData::gamePlayerDataByShuJuJieXi accounted for 57.25% of the time consumed, as shown in Figure 5.
6. Continuing to expand until it could not be expanded further, yii\redis\Connection::parseResponse was found, and its running mechanism needs to be analyzed, as shown in Figure 6.
7. In Alibaba Cloud monitoring, at 19:36:00 on 2017-06-18, when the match started, the Redis CPU usage was 1.9%, as shown in Figure 7.
8. During the entire football match, in Alibaba Cloud monitoring, the Redis CPU usage increased linearly. From 20:30:00 to 20:40:00 on 2017-06-18, the Redis CPU usage dropped because the match was in halftime, and the match data sent to the server remained basically unchanged, resulting in simple processing logic. The entire curve is directly proportional to the response time, as shown in Figure 8.
9. In Alibaba Cloud monitoring, at 21:27:00 on 2017-06-18, when the match was about to end, the Redis CPU usage was 100%, as shown in Figure 9.
10. In Alibaba Cloud monitoring, at 21:28:00 on 2017-06-18, when the match was about to end, the number of used Redis connections reached a peak of 54, as shown in Figure 10.
11. Now we need to analyze what exactly caused the Redis CPU usage to reach 100%. In the APM Transaction trace, it was found that yii\redis\ActiveRecord::deleteAll accounted for 78% of the time consumed, as shown in Figure 11.
12. In the Trace details of the APM Transaction trace, it was found that Application code (in yii\redis\ActiveRecord::deleteAll) consumed the most time. It can basically be confirmed that ActiveRecord-related operations are the root cause of the Redis CPU usage reaching 100%, as shown in Figure 12.
13. In the production environment, we prepared to implement two versions of a typical request. The ActiveRecord-related operations in the first version are basically query and update, while the second version’s are basically query and insert. This was to confirm whether an excessive number of keys in a specific ActiveRecord model would increase the Redis CPU usage. We first cleared Redis, and then the scheduled task generated Redis data with 3170 keys, as shown in Figure 13.
14. In the production environment, after the first version of the request ran for 2 hours, the Redis data had 3220 keys, an increase of only 50 keys, as shown in Figure 14.
15. In the production environment, during the entire execution of the typical request, in Alibaba Cloud monitoring, the Redis CPU usage peaked at 2.6%. After 16:50:00 on 2017-06-27, the Redis CPU usage maintained at 0.6% because the match data sent to the server remained basically unchanged, resulting in simple processing logic, as shown in Figure 15.
16. In the production environment, during the entire execution of the typical request, the response time remained basically stable, with a maximum response time of 266 ms. The average response time required by the business target for this request in the production environment is 200 ms, so the performance meets the business requirements. After 16:50:00 on 2017-06-27, the response time dropped because the match data sent to the server remained basically unchanged, resulting in simple processing logic, as shown in Figure 16.
17. First, we cleared Redis, and then the scheduled task generated Redis data with 2909 keys, as shown in Figure 17.
18. In the production environment, after the second version of the request ran for 2 hours, the Redis data had 89840 keys, an increase of 86931 keys, as shown in Figure 18.
19. The number of keys in the Redis model operated on by the second version of the request was 79527, as shown in Figure 19.
20. In the production environment, for the second version of the request, during the entire execution of the typical request, in Alibaba Cloud monitoring, the Redis CPU usage peaked at 66.8%, as shown in Figure 20.
21. Based on the current analysis results, it can be confirmed that when the number of keys in a model operated on by ActiveRecord becomes too large, the Redis CPU usage will increase significantly. However, whether this is caused by queries or inserts needs further confirmation. The code is shown in Figure 21.
22. We modified the query approach, changing count(‘id’) to exists(), which means changing from querying all keys that meet the conditions in the model to querying just 1 key that meets the conditions. The code is shown in Figure 22.
23. Without clearing Redis, in the production environment, the number of keys in the Redis model operated on by the third version of the request was 81277, as shown in Figure 23.
24. In the production environment, for the third version of the request, during the entire execution of the typical request, in Alibaba Cloud monitoring, the Redis CPU usage peaked at 11.8%, as shown in Figure 24.
25. Based on the current analysis results, it can be confirmed that when the number of keys in a model operated on by ActiveRecord becomes too large, the Redis CPU usage will increase significantly, and this is most closely related to queries. However, the performance comparison between queries and inserts needs further confirmation. In the code, we commented out the queries and kept only the insert operations, as shown in Figure 25.
26. Without clearing Redis, in the production environment, the number of keys in the Redis model operated on by the fourth version of the request was 81947, as shown in Figure 26.
27. In the production environment, for the fourth version of the request, during the entire execution of the typical request, in Alibaba Cloud monitoring, the Redis CPU usage peaked at 2.2%, as shown in Figure 27.
28. Change count() to exists() to improve query performance, as shown in Figure 28.
29. We refactored some ActiveRecord implementations into native Redis command implementations. During the entire football match, in Alibaba Cloud monitoring, the Redis CPU usage peaked at 21.1%, so the response time was normal, as shown in Figure 29.
30. Conclusion: For Yii 2’s ActiveRecord, the ranking of related methods regarding Redis CPU computing pressure is: insert() < one()/exists() < count()/all(). When the number of keys in a specific Redis model exceeds 10000, if you need to perform a query (with a where condition) operation, it is recommended not to use ActiveRecord, as shown in Figure 30.
