Strengthen requestAnimationFrame() spec for VSYNC / refresh rate matching in HTML 5.2

UPDATE: Change committed on #785. Please see #375 instead (rewritten/newer) for further standardization efforts on this.

PROBLEM

• requestAnimationFrame() is not sufficiently rigidly enough defined.

The older W3C "Animation Timing" spec had a better definition (https://www.w3.org/TR/animation-timing/) Section 5 says "The expectation is that the user agent will run tasks from the animation task source at at a regular interval matching the display's refresh rate. Running tasks at a lower rate can result in animations not appearing smooth. Running tasks at a higher rate can cause extra computation to occur without a user-visible benefit."
-- However the newer W3C HTML 5.1 no longer correctly describes the desire to match requestAnimationFrame() exactly to the refresh rate.

DETAIL

• Browser support of refresh rates are mostly consistent except for a few browsers:
  http://www.testufo.com/browser.html
• Some browsers use an arbitrary hardcoded framerate limits (e.g. 105fps), such as found in Microsoft Internet Explorer:
  https://connect.microsoft.com/IE/feedback/details/794072/internet-explorer-animations-fails-on-120hz-computer-monitors-works-at-60hz-75hz-100hz?siteID=rGMTN56tf_w-1PqoBW8wrx7DKpzuXQ.Wbg
• A listing of many problems with requestAnimationFrame() timing:
  http://www.vsynctester.com/howtocomputevsync.html and at http://www.testufo.com/browser.html
• Firefox issue where it intentionally computes incorrect VSYNC times:
  http://www.vsynctester.com/firefoxisbroken.html
• Differing amounts of time spent inside requestAnimationFrame() can create jerkiness/stutters or lower frame rates despite using only a fraction of CPU. There were some benchmarks done:
  http://www.blurbusters.com/blur-busters-120hz-web-browser-tests/
  Stutter began to occur in some browsers even at 10% CPU, while Chrome remained stutter-resistant all the way to 75% CPU used up (e.g. 12ms inside requestAnimationFrame() during a 1/60sec period). This disparity is unreasonable.

WHY IT IS IMPORTANT

• Games
• High-refresh-rate monitors (120Hz+ monitors, often found in GSYNC / FreeSync gaming monitors)
• Animations, motion tests, etc.
• Virtual reality. Note that Oculus/Rift is 90Hz

EASY SOLUTIONS

• Discourage hardcoding of numbers such as "60Hz" or arbitrary maximum framerates such as "105". This does a disservice to high-refresh-rate monitors and future HFR displays (including future 4K120Hz and 8K120Hz standards)
• Strengthen specification to make synchronization to display refresh rate a "MUST" instead of a "RECOMMENDED"
• Strengthen specification to make it return exact vsync time rather than now(). Chrome is the only one that does it correctly.
• [This lineitem has been moved to VSYNC API: Support for VSYNC OFF / 120Hz+ / variable refresh displays / etc #375] Strengthen specification to reduce allowable jitter, to permit smoother animations, given the increasingly real-time nature of websites, games, VR, etc. Things should be able to be stutter/jank-free all the way to ~50%-75% CPU used up inside requestAnimationFrame()
• [This lineitem has been moved to VSYNC API: Support for VSYNC OFF / 120Hz+ / variable refresh displays / etc #375] Add options of flexibility such as to maximize smoothness (when fluidity is priority) versus minimizing lag (when minimum latency is priority).

COMPLIANCE TESTING WEBSITES FOR requestAnimationFrame()

• http://www.vsynctester.com (click the gear icon for lots of tweakable settings)
• http://www.testufo.com/#test=animation-time-graph&easteregg=1

SUPERSEDES (IN PART): #422, #375, #159

[chaals]

@plehegar assigning you to take a first look as the person who might know best. Feel free to pass on, or others please fell free to take it if PLH is busy.

[duckware]

Work in this area is needed. Please do address the issues raised.

• rAF callback time arg: Setting the time argument to 'now' adds no value. Because of this, web browser vendors have already started to switch over to passing in the actual vsync time (Chrome / Firefox) as the argument to the rAF callback (which actually does add a lot of value). However, Chrome is currently the only browser that does this correctly. Firefox actually passes in a faked time (http://www.vsynctester.com/firefoxisbroken.html). Please strengthen the specification to make the rAF time argument be the exact vsync time.

As just as a very interesting FYI. On Windows, all rAF animations in all web browsers have a two frame input lag caused by how all browsers interact with the GPU. Apparently what is happening is that vsync triggers the rAF callback, but 'what was drawn' is not presented until the next vsync, but at that point, it is too late to make it to the screen (because the Windows OS composites). What was drawn in the rAF callback actually makes it to the screen one entire frame later. It sure would be nice if web browser vendors took notice, and (under Windows), presented what was drawn in a rAF callback immediately, instead of waiting for the next vsync (which then delays an entire frame).

The two biggies are likely:

1. [Easy spec change] Strengthen specification to make synchronization to display refresh rate a "MUST" instead of a "RECOMMENDED"
2. [Easy spec change] Discourage hardcoding of numbers such as "60Hz" or arbitrary maximum framerates such as "105"

Currently, Microsoft's EDGE browser doesn't run at 100Hz nor 120Hz -- in fact, it supports fewer refresh rates than Microsoft's Internet Explorer. The developers at Microsoft should get a BENQ XL2720Z which can support custom refresh rates in 1Hz increments from 50Hz through 144Hz, to do validation testing.

Chrome does it almost perfectly (even 240fps @ 240Hz! Video https://www.youtube.com/watch?v=oJakpZbvm3I#t=4m0s -- this is a YouTube video of www.testufo.com/frameskipping in Chrome browser on a 240Hz monitor!

Opera / FireFox / Safari can do any refresh rate too (Albiet Chrome is the clear winner at the moment), as they ran successfully at 120fps@120Hz on a 120Hz monitor.

Microsoft IE 10/11 goes up to 105fps (my tests apppear to show a hardlimit of 105, no matter how slow or fast CPU or GPU is, even on a GEFORCE GTX TITAN). Microsoft EDGE doesn't even do 100fps@100Hz nor 120fps@120Hz.

FireFox Update: I am pleased that since the earlier 120Hz Browser Tests done -- FireFox has dramatically improved fluidity (elimination of jank/judder) and it now stays smooth even up to around ~70% of a refresh cycle spent inside requestAnimationFrame() at least on Windows systems.

EDGE Update: Alas, Microsoft Edge 15 in the Windows Creator's Update, still does not work correctly. It does not remove its own internal artifical cap on the rate of requestAnimationFrame(), still running at 60fps at 120Hz -- and 72fps at 144Hz.

There is a huge growth in gaming monitors, and there are now over 100 models of gaming monitors on the market (GSYNC and FreeSync). They are becoming more common, with several models now available at ordinary places like the Staples office-supplies store, Best Buy, etc.

I see that W3C Editor's Draft (18 April 2017), section 7.1.4.2 Processing Model, uses this text:

NOTE: Whether a top-level browsing context would benefit from having its rendering updated depends on various factors, such as the update frequency. For example, if the browser is attempting to achieve a 60 Hz refresh rate, then these steps are only necessary every 60th of a second (about 16.7ms). If the browser finds that a top-level browsing context is not able to sustain this rate, it might drop to a more sustainable 30Hz for that set of Documents, rather than occasionally dropping frames. (This specification does not mandate any particular model for when to update the rendering.) Similarly, if a top-level browsing context is in the background, the user agent might decide to drop that page to a much slower 4Hz, or even less.

There appears to be fluff words ("....would benefit from having its rendering updated depends on various factors...") that could easily be shortened with some good English rewriting.

Implied assumptions Being Made We're running on battery power
Also, I observe an excessive focus on concerns of saving battery power, i.e. mobile devices -- when it should be also factored in that modern browsers on desktop GPUs (NVIDIA/AMD) can run in excess of >250fps and even 2000fps (screenshot!) while using less than 20% CPU power. There is a false preconception that browsers are unable to run at high framerates. Also desktop computers are not battery-powered. There are now over 100+ high-Hz gaming monitors on the market.

120Hz mobile phones
While most browsing worldwide now occurs on mobile devices, there are now certain mobile phones starting to experiment with 120Hz (e.g. Sony Xperia X Premium) -- so this needs to be considered.

Wording doesn't mention matching framerate to refresh rate
Also, the browser often runs its update (e.g. scrollbar, 120fps video playback) frequency differently from the frequency of requestAnimationFrame -- even when less than 5% of CPU is used. Microsoft EDGE manages to run testufo.com at 72fps using only 0.8% of CPU, and doesn't bother using the extra 0.8% of CPU needed (total 1.6% CPU) to run at 144fps (at 144Hz) on my AC-powered non-battery-limited non-performance-limited 5-year-old mid-range desktop computer. Why does EDGE rudely limit its requestAnimationFrame() rate to 72 callbacks per second, even though it can use its scrollbar scrolling at 144 frames per second? IE/EDGE plays back 120fps videos in

Wording is Problematic for Input Lag Reducing Edge Cases
Also, sometimes, lag-reducing advocates recommend running at an uncapped rate on high-performance computers to minimize input lag. For example, rendering at 1000fps can reduce the input-latency to 1ms per frame, e.g. between the mouse click (of a 1000Hz mouse) and the actual display of the result. If updating only 60 times a second at 60Hz, this can still be a full 16.7ms delay between the timing of a mouse click (if it occurs right after the input read, keyboard or mouse) and the actual action. This is why many gamers (especially in eSports gaming). .... There is even a setting in chrome://flags (in Chrome) to uncap the update rate, and enable the browser to update in excess of 500 frames per second, for special experimential cases. Ideally, in a perfect world, there should be an W3C-compliant browser API to control this (e.g. turn VSYNC ON versus VSYNC OFF). Obviously, this is an edge case, but this illustrates the problematic wording of saying "....these steps are only necessary every 60th of a second..."

Wording should not dictate a preference of framerate-slowdown algorithm. This should be developer choice
Yes, sometimes this is best. But even for mainstream situations, blanket-recommending 60fps suddenly go to 30fps is very problematic. Sometimes this is actually better, but in many cases, this suddenly adds input lag that throws off aiming in certain kinds of gaming. In many use cases, like brief framerate slowdowns in video games, it's preferable to go in a sequence 60-60-60-60-58-55-50-54-57-59-60-60-60fps -- during brief performance issues, like a momentary complex moment in a video game. A gradual slowdown to 55fps has less input-lag than suddenly going to 30fps, for HTML gaming. However, at other times, it's definitely preferable to go to 30fps than 60fps. In a perfect ideal world, this is a developer-specified behavior (e.g. HTML5 API to specify preferred update-rate degradation behavior). But in the abscence of this, it's not W3C's place to suggest one-size-fits-all specific update-rate degradation behavior.
Also, I've developed simple HTML games before -- Here's a simplified mathematical example: Trying archery at a moving-target of 960 pixels/second, 60fps is 16 pixel steps per frame (combined with 1/60sec lag) and 30fps is 32 pixel steps per frame (combined with 2/60sec lag). If you aim your arrow at the moment after an unexpected 30fps slowdown due to performance -- your archery-shoot (aimed during 60fps) may have an unintentional browser-enforced "+2/60sec" lag modifier instead of a "+1/60sec" lag modifier. Your arrow misses by 16 pixels because of a 1/60sec time delta of aimed button press versus aimed moving target. In this situation, sudden 60-to-30fps slowdown created an unexpected input lag that threw off your aiming. Remember, human reaction times are typically 100-200ms, and if a browser slowdown occurs during this time window, your aiming is off because of an unwanted mandated browser half-framerate slowdown (i.e. a mere millisecond missing a 60fps deadline due to performance, enforces another forced 1/60sec wait before displaying the frame (if browser doing a mandated 60-to-30 slowdown) -- unwanted sudden lag change). As a result, for this specific HTML5 game, you want to override that, and enable the option of "play at maximum framerate as browser performance allows, without half-framerate slowdowns". In this case, user experience improves during this specific particular use-case of "usually 60fps with a few slowdowns". While many use cases are more complex than this, hopefully this simplified example helps developers understand. One has to realize the "60-to-30-slowdown" technique can be great but it is not a one-size-fits-all-apps solution.

As a long-term future plan, consider the eSports use case for a future VSYNC OFF mode
Chrome can do it via a flag, "--disable-gpu-vsync" but this should be accessible in JavaScript full-screen gaming applications. Many people falsely claim that running framerates above refreshrate has no benefit. There is a side consideration of input lag. This is why VSYNC OFF is favoured by many competition gamers -- the latency between a button press or key press, and the screen update. Future standards ideally should make framerate-degrade behaviour a developer-specifiable API behavior instead (along with things like future "VSYNC OFF" API support for future uncapped-1000fps "ultralow-lag" games in web browsers -- scientifically benchmarked, 1000fps@60Hz has approximately 1/2 of a 60Hz refresh cycle less lag than 60fps@60Hz -- due to fresher delivery of fresher frame right before a fixed refresh cycle interval (closer to last-minute WDM compositing) or even splicing into the current refresh cycle (tearline effect in exchange for lower latency) that some games are able to do -- competitive eSports game players (on average) tend to prefer this (and may someday be important for any future browser-spaced eSports-league type games where latency is numero uno, with Olympics-style "millisecond-close" finish line reaction times -- eSports video gaming competitions has reached almost $1 billion dollars in 2016, and slated to cross that threshold in 2017. By making browsers more open and friendly to future developer API control of preferred framerate-limiting behaviour ("VSYNC ON drop to half framerate when performance constrained", "VSYNC ON drop framerate incrementally when constrained", "VSYNC OFF uncapped ultralow-lag mode running screen updates & callbacks at maximum performance rate", etc) -- this makes web browsers potentially more compatible with future eSports programming and special niches. Chrome has a hidden VSYNC OFF mode in chrome://flags that has to be manually accessed -- www.testufo.com has successfully run in Chrome browser at >500fps-1000fps on new NVIDIA GPUs with this mode is toggled. In high speed video input-latency tests, VSYNC OFF (framerate uncapped to ~1000fps / 1ms rendertimes) can have ~8ms less input lag than VSYNC ON at 60Hz (roughly half of a 60Hz refresh cycle less). Although visual updates aren't smoother with framerates running higher than refresh rate -- due to less time delta between final frame & the timing of monitor refresh -- there are input latency reductions in button-to-pixels response. So, this is not 100% advantage-less, and is useful in very niche situations. This is not of importance to most people, but it illustrates another reason not to arbitrarily specify a framerate behavior (e.g. recommending against framerates higher than refreshrate, e.g. recommending a 60-to-30 sudden slowdown) and leave that to developer choice.
EDIT: A few years ago, before EDGE came out, Microsoft once responded to an email of mine saying browsers cannot do 120fps. Hogwash! Here's Google Chrome running at 2000fps on a 5-year-old c omputer (two thousand frames per second) running "chrome.exe --disable-gpu-vsync" mode, using less than 20% of CPU. Modern GPUs are very fast.

The mention of milliseconds (about 16.7ms) may be counterproductive
The technique that many browsers use to synchronize to the display frequency is via a VSYNC trigger -- so browser engines don't measure time between refresh cycles (e.g. 16.7ms timer is STRONGLY discouraged -- as it's not synchronized, to refresh -- some systems are running at 59.94Hz, others 60Hz, and others another fraction thereof, such as 60.031Hz)... If you load a Custom Resolution Utility on a PC, you'll often notice that the computer's refresh rate is not always an exact integer. Avoiding mention of milliseconds is recommended, to repeat the mistake that some old web browsers did to use a timer rather than use VSYNC. The operating system's window compositing manager always synchronizes to the VSYNC too...

Consider future variable refresh rate standards
There are now variable-refresh-rate monitors on the market (GSYNC, FreeSync) that dynamically changes their refresh rate every single refresh cycle. There is now an upcoming HDTV standard for variable refresh rates coming to big-screen TVs too, not just computer monitors. Microsoft is adding variable refresh rate support to the next XBox video game consule. For example, if motion runs at 58 or 57 frames per second, the display is running at 58Hz or 57Hz, and motion remains stutter-free (example simulated animation of stutter-free changes in framerate -- 100% stutterfree changes in framerates! -- this animation of stutterfree framerate changes is accomplished via frame interpolation but VRR displays do this natively). It is possible within 5 or 10 years, W3C may need to have a migration path to variable refresh rates permitting "Variable refresh rate displays may mean that browser vendors may decide to support them, updating the display at the same rate as the browser update rate of the top-level browsing context". Often on such VRR monitors, an OpenGL glFlush() command delivers the frame directly to the monitor (immediately refreshing the monitor, rather than waiting for refresh cycle). It would go to surmise, that this could happen with WebGL, and could happen with browser update contexts -- that a commit-to-display instantly activates a display refresh cycle, rather than buffers for next fixed refresh cycle. This is done at the API level, window compositing manager, or DirectX / OpenGL, etc.

However, matching update frequency to refresh rate (Or vice versa, in the case of VRR displays), is definitely indeed the recommended method for the smoothest, jank-free judder-free operation.

The existing W3C HTML 5.2 DRAFT 7 wording is very clearly written flawed with more than half-a-dozen ugly implied (accidental) assumptions (see below newer post for bullet list), so I propose a rewording of that particular paragraph.

[EDIT: Proposed update text has been moved to new post below]