Table of Contents
Introduction
Web performance optimization services, or web performance optimizers (WPOs) for short, play a critical role in today’s web ecosystem by improving page load speed and saving network traffic. However, WPOs are known for introducing visual distortions that disrupt the users’ web experience. Sadly, visual distortions are notoriously hard to analyze, test, and debug, due to their subjective measure, dynamic content, and sophisticated WPO implementations.
This paper presents Vetter, a novel and effective system that automatically tests and debugs visual distortions. Its key idea is to reason about the morphology of web pages, which describes the topological forms and scale-free geometrical structures of visual elements. Vetter efficiently calculates morphology and comparatively analyzes the morphologies of web pages before and after a WPO, which acts as a differential test oracle. Such morphology analysis enables Vetter to detect visual distortions accurately and reliably. Vetter further diagnoses the detected visual distortions to pinpoint the root causes in WPOs’ source code. This is achieved by morphological causal inference, which localizes the offending visual elements that trigger the distortion and maps them to the corresponding code. We applied Vetter to four representative WPOs. Vetter discovers 21 unknown defects responsible for 98% visual distortions; 12 of them have been confirmed and 5 have been fixed.
Keywords: Web Performance Optimization; Web Page Distortion; Visual-Aware Testing and Debugging
Data and Code Release
Data Release. We collected Chrome’s invocation logs of SKPaint APIs when visiting Alexa top and bottom 2,500 websites on Dec. 9th, 2021, which are available in the Google Drive. Besides, we have made part of our dataset available in Google Drive (the remaining part will be publicly available when the paper is published).
Code Release. Currently we are scrutinizing the codebase to avoid possible anonymity violation. To this end, we will release Vetter’s source code in a module-by-module manner as soon as we have finished examining a module. The codebase of Vetter is organized as follows.
Vetter
|---- Vetter_Testing
|---- Morph_Instantiation_Minification
|---- MorphSIM_Calculation
|---- Vetter_Debugging
|---- Morph_Causal_Inference
|---- Causality_Informed_Code_Analysis
The released part can be found here.
Defects Found
Below, we list all the defects we have found for four representative WPOs: Compy, Ziproxy, Fawkes, and Siploader. Note that for Ziproxy, we mail the developers of Ziproxy with the defects we have found (together with our suggested fixes) through their official channels, but have not received the reply yet.
Compy
| Index | Description | Issue/PR NO. | Current State |
|---|---|---|---|
| 1 | Compy goes wrong when compressing some JPG/PNG images, which makes the images unable to load. | Issue-63 & PR-70 | Confirmed & Fixed |
| 2 | Compy fails to parse the compressed images. | Issue-64 | Reported |
| 3 | Compy can’t deal with the websocket, which fails some interaction tasks like chatrooms and online services. | Issue-65 | Reported |
| 4 | Compy may block the redirecting process of some websites. | Issue-66 & PR-68 | Confirmed & Fixed |
| 5 | Compy can’t support GIF images. | PR-70 | Confirmed & Fixed |
Ziproxy
| Index | Description | Issue/PR NO. | Current State |
|---|---|---|---|
| 1 | Ziproxy goes wrong when compressing some contexts, which makes the original contexts become messy code. | - | Waiting For Reply |
| 2 | Ziproxy disturbs the loading sequence of JS files, leading to loading failure of web pages. | - | Waiting For Reply |
| 3 | Ziproxy cannot handle GIF files, leading to image display error transcoding. | - | Waiting For Reply |
| 4 | Ziproxy causes conflicting fields in response header. | - | Waiting For Reply |
Fawkes
| Index | Description | Issue/PR NO. | Current State |
|---|---|---|---|
| 1 | Fawkes can’t handle some elements whose innerText has multiple lines. | Issue-14 | Reported |
| 2 | Fawkes mistakenly selects elements in template HTML | Issue-13 | Reported |
SipLoader
| Index | Description | Issue/PR NO. | Current State |
|---|---|---|---|
| 1 | SipLoader cannot track dependencies triggered by CSS files. | Issue-1 | Confirmed |
| 2 | SipLoader cannot handle dependency loops among resources. | Issue-2 | Confirmed |
| 3 | SipLoader cannot request cross-origin resources. | Issue-3 | Confirmed |
| 4 | Disordered page loading of websites with multiple HTML files. | Issue-4 | Confirmed |
| 5 | “404 Not Found” error when loading websites with multiple HTML files. | Issue-5 | Confirmed |
| 6 | SipLoader cannot handle some dynamic resources. | Issue-6 | Confirmed |
| 7 | A problem related to Chromium CDP used by SipLoader. | Issue-7 | Confirmed |
| 8 | CSS abormality of some websites. | Issue-8 & PR-9 | Confirmed & Fixed |
| 9 | SipLoader fails to rewrite web page objects compressed by brotli. | Issue-10 & PR-12 | Confirmed & Fixed |
| 10 | SipLoader cannot distinguish between data URIs and real URLs in CSS files. | Issue-11 & PR-9 | Confirmed & Fixed |