Gpu submit time frequency boosting
Download as PPTX, PDF0 likes61 views
The document discusses improvements in GPU frequency scaling for workloads, emphasizing the effectiveness of a dynamic voltage and frequency scaling (DVFS) mechanism that adjusts GPU frequency based on load. It highlights the challenges of aggressive tuning leading to higher reactiveness and constant GPU overpowering, particularly in bursty workloads like camera post-processing and edge detection. Performance data shows that optimized frequency scaling can lead to substantial improvements in performance per watt across various use cases.
Gpu submit time frequency boosting
- 1. IMPROVING GPU FREQUENCY SCALING FOR GPU WORKLOADS
- 2. TYPICAL DVFS BASED GPU BOOST MECHANISM • GPU frequency boosting wired through the devfreq governor • Monitors GPU busyness and tries to keep current load under given target load by adjusting gpu frequency with tunables like settling time, bias, damp and rampdown_delay • Basically boost_freq = bias * freq * (load - target)/target • Ideal for sustained loads and burstiness within high load window • Too aggressive tunings lead to higher reactiveness • However also leads to constant gpu overpowering • For e.g. too low target_load or high rampdown_delay
- 3. PROBLEM • Low latency VR use cases typically present repetitive & bursty GPU workloads • Need is guaranteed GPU horsepower exactly when workload gets scheduled • Load quickly gets degenerated (but high chance of repeating) - so frequency needs to quickly fall down (and ramp up back) • Typical use cases exhibiting this kind of burstiness are camera post processing, edge detection, atw... • Slower response time associated with current governor in ramping up frequency clearly shows up with overall low perf/watt
- 4. JUST IN (SUBMIT) TIME FREQ SCALING • Density of work submission (unit time) forms basis of GPU load • Delay (order of ms) in submit to governor’s load visibility • Translates to latency in effective gpu frequency boost • Short boost pulse in submit code path takes care of ramp up latency • Inherently makes frequency follow workload • Increased chances of governor now seeing lower load and pulling frequency down • Effective gpu freq comes down to fmax@vmin for profiled use cases (presenting better perf/watt)
- 5. PERF/POWER DATA ACROSS USE CASES GPU intensive section (ms) Avg GPU Busyness Avg GPU Frequency (Mhz) Avg GPU Power (mW) Avg (VDD_IN) Total Power (mW) % Perf/Watt Increase Pupil Detection (with JIT scaling) Edge Detection 11.004 34 497 471 5488 99.623182 Pupil Detection (with default scaling) 21.158 182 293 421 5286 Passthrough camera (with JIT scaling) Camera to Display (e2e) 40.599 219 596 856 7591 4.5763017 Passthrough camera (with default scaling) 45.466 590 283 837 8129 Passthrough camera (with max gpu) 40.025 153 1331 1377 8677
- 6. PUPIL DETECTION WITH CURRENT FREQ SCALING Avg Max Min GPU intensive code latency ( in ms) 21.158 843.41 7.289 GPU Busyness 182 401 57 GPU frequency (in Mhz) 293 595 109 GPU Power (in mW) 421 534 152
- 7. PUPIL DETECTION WITH JIT FREQ SCALING Avg Max Min GPU intensive code latency (in ms) 11.004 957.52 5890 GPU Busyness 34 504 10 GPU frequency (in Mhz) 497 790 109 GPU Power (in mW) 471 610 152
- 8. PASSTHROUGH WITH DEFAULT FREQ SCALING Avg Max Min GPU intensive code latency (in ms) 45.466 82.425 33.461 GPU Busyness 590 946 173 GPU frequency (in Mhz) 283 693 109 GPU Power (in mW) 837 838 761
- 9. PASSTHROUGH WITH JIT FREQ SCALING Avg Max Min GPU intensive code latency (in ms) 40.599 63.626 31.690 GPU Busyness 219 390 67 GPU frequency (in Mhz) 596 790 303 GPU Power (in mW) 856 914 762