SlideShare a Scribd company logo
CPU Idle Loop Rework
Rafael J. Wysocki
Intel Open Source Technology Center
September 26, 2018
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 1 / 19
Introduction CPU Idle Time Concept
Terminology: CPUs = Logical CPUs
CPU 0
CPU 1
CPU 2
CPU 3
CPU 0
CPU 1
CPU 0
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 2 / 19
Introduction CPU Idle Time Concept
CPUs: Busy Vs Idle
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 3 / 19
Introduction CPU Idle States
CPU Idle States Properties
Time [a. u.]
Power [a. u.]
Entry ExitMR
Residency
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 4 / 19
Introduction CPU Idle States
CPU Idle States Example: C-states On Intel Hardware
MWAIT <HINT>
CPU 0
CPU 1
CPU 2
CPU 3
CPU 4
CPU 5
CPU 6
CPU 7
C6C2
C3 C3
C6C6
C6 C6
CC2 CC3
CC6 CC6
PC2
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 5 / 19
CPU Idle Time Management Original Design and Problem
High-level CPU Idle Time Management Control Flow
CPU scheduler
Select state
Estimate idle time
CPU is idle
Enter state
Governor
YES
NO
menu
ladder
ACPI idle
intel_idle
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 6 / 19
CPU Idle Time Management Original Design and Problem
Complication: CPU Scheduler Tick Timer
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 7 / 19
CPU Idle Time Management Original Design and Problem
Original Idle Loop Design Issue
CPU scheduler
Try to stop tick
need_resched()?
YES
NO
Select state
Estimate idle time
Governor
Restart tick
Enter state
Interrupt
May be too short!
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 8 / 19
CPU Idle Time Management Original Design and Problem
Short Idle Duration Prediction Problem
WIN LOSS
LOSS LOSS
Short Idle
Short Idle
Long Idle
Long Idle
Predicted
Actual
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 9 / 19
CPU Idle Time Management Original Design and Problem
Idle Governor Problem: Two Categories Of Information
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 10 / 19
CPU Idle Time Management Original Design and Problem
The menu Idle Governor
Next Timer Event
CPU scheduler
Utilization Information
NOHZ Infrastructure
Idle Time Range
Latency LimitPredicted Idle Duration
Idle State
Previous Idle Duration
Wakeup Statistics
Timers
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 11 / 19
CPU Idle Time Management Idle Loop Rework
Redesigned Idle Loop (Linux* 4.17 And Later)
CPU scheduler
Try to stop tick
need_resched()?
NO
Select state
Estimate idle time
Governor
Restart tick
Enter state
Interrupt
Stop tick or
tick stopped?
YES
NO
Tick stopped?
NO
YES
* Other names and brands may be claimed as the property of others
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 12 / 19
CPU Idle Time Management Idle Loop Rework
Short Idle Duration Prediction Problem Mitigated
WIN LOSS
Neutral WIN
Short Idle
Short Idle
Long Idle
Long Idle
Predicted
Actual
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 13 / 19
CPU Idle Time Management Idle Loop Rework
Idle Power (Intel OTC Server Power Lab)
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 14 / 19
CPU Idle Time Management Idle Loop Rework
Idle Power (Intel OTC Server Power Lab) Continued
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 15 / 19
CPU Idle Time Management Idle Loop Rework
Idle Power (Intel OTC Server Power Lab) Continued
Before After (different scale)
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 16 / 19
Questions?
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 17 / 19
Resources
References
Linux Kernel Bug 199227, Idle loop ordering issue (https://bugzilla.kernel.org/show_bug.cgi?id=199227).
T. Ilsche, M. H¨ahnel, R. Sch¨ohne, M. Bielert, D. Hackenberg,
Powernightmares: The Challenge of Efficiently Using Sleep States on Multi-Core Systems
(https://tu-dresden.de/zih/forschung/ressourcen/dateien/projekte/haec/powernightmares.pdf?lang=en).
R. J. Wysocki, Power Management Challenges in Linux (https:
//www.linuxplumbersconf.org/2017/ocw//system/presentations/4652/original/linux_pm_challenges.pdf).
R. J. Wysocki, PM Infrastructure in the Linux Kernel – Current Status and Future
(https://events.linuxfoundation.org/sites/events/files/slides/kernel_PM_infra_0.pdf).
J. Corbet, The cpuidle subsystem (http://lwn.net/Articles/384146/).
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 18 / 19
Disclaimer
Intel technologies’ features and benefits depend on system configuration
and may require enabled hardware, software or service activation.
Performance varies depending on system configuration. No computer
system can be absolutely secure. Check with your system manufacturer or
retailer or learn more at www.intel.com.
Intel and the Intel logo are trademarks of Intel Corporation in the U.S.
and/or other countries. *Other names and brands may be claimed as the
property of others.
c Intel Corporation
Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 19 / 19

More Related Content

PDF
Linux Performance Profiling and Monitoring
PDF
Kernel Recipes 2019 - Suricata and XDP
PDF
RxNetty vs Tomcat Performance Results
PDF
Build enterprise wireless with CAPsMAN
PPTX
Linux networking
PDF
Priority Inversion on Mars
ODP
Sockets and Socket-Buffer
PDF
Solaris Kernel Debugging V1.0
Linux Performance Profiling and Monitoring
Kernel Recipes 2019 - Suricata and XDP
RxNetty vs Tomcat Performance Results
Build enterprise wireless with CAPsMAN
Linux networking
Priority Inversion on Mars
Sockets and Socket-Buffer
Solaris Kernel Debugging V1.0

What's hot (20)

PPTX
HTML5 Real Time and WebSocket Code Lab (SFHTML5, GTUGSF)
PDF
Linux Systems Performance 2016
PPTX
Ansible MySQL MHA
PDF
Hinemosのすゝめ(監視編)
PPT
PDF
IIJmio meeting 16 「通信速度」に影響を与える要素とは
PPTX
Linux の hugepage の開発動向
PDF
Zabbix + GLPI: Como estas duas ferramentas podem otimizar seus recursos
PDF
jemalloc 세미나
PDF
私とmysqlとROLE
PDF
自宅ラック勉強会#3
PDF
DS-LiteをFreeBSDで使う
PPTX
NW入門
PDF
Python パッケージの影響を歴史から理解してみよう!
PDF
Meet cute-between-ebpf-and-tracing
PDF
EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Data...
PPTX
Juniper SRX Quickstart 12.1R3 by Thomas Schmidt
PDF
ネットワークでなぜ遅延が生じるのか
PDF
The Internals of "Hello World" Program
PPTX
HTTP2 最速実装 〜入門編〜
HTML5 Real Time and WebSocket Code Lab (SFHTML5, GTUGSF)
Linux Systems Performance 2016
Ansible MySQL MHA
Hinemosのすゝめ(監視編)
IIJmio meeting 16 「通信速度」に影響を与える要素とは
Linux の hugepage の開発動向
Zabbix + GLPI: Como estas duas ferramentas podem otimizar seus recursos
jemalloc 세미나
私とmysqlとROLE
自宅ラック勉強会#3
DS-LiteをFreeBSDで使う
NW入門
Python パッケージの影響を歴史から理解してみよう!
Meet cute-between-ebpf-and-tracing
EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Data...
Juniper SRX Quickstart 12.1R3 by Thomas Schmidt
ネットワークでなぜ遅延が生じるのか
The Internals of "Hello World" Program
HTTP2 最速実装 〜入門編〜
Ad

Similar to Kernel Recipes 2018 - CPU Idle Loop Rework - Rafael J. Wysocki (20)

PDF
BKK16-203 Irq prediction or how to better estimate idle time
PDF
SoC Idling for unconf COSCUP 2016
PDF
BKK16-410 SoC Idling & CPU Cluster PM
PDF
Controlling processor c-state_usage_in_linux_nov2012
PPT
Os Vandeven
PDF
LCA13: CPUIDLE: One driver to rule them all?
PPTX
Intern presentation nicolechiou_2018_vareximaging_sanitized
PDF
LCA14: LCA14-306: CPUidle & CPUfreq integration with scheduler
PDF
Analyze and optimize Android apps power consumption
PDF
Kernel Recipes 2016 - Understanding a Real-Time System (more than just a kernel)
PDF
Q2.12: Idle Power States Nomenclature
PPTX
Improving Xen idle power efficiency
PDF
When the OS gets in the way
PDF
Q2.12: Idling ARMs in a busy world: Linux Power Management for ARM Multiclust...
PDF
semaphore & mutex.pdf
PPT
atom-imp-concept of hardware tools in ECE.ppt
PDF
Tommaso Cucinotta - Low-latency and power-efficient audio applications on Linux
PDF
LCA13: Who Disturbs My Slumber
PDF
Linux power management: are you doing it right?
PDF
LCU14 206- Tools to Analyse Scheduling Behaviour and Its Impact on Power Mana...
BKK16-203 Irq prediction or how to better estimate idle time
SoC Idling for unconf COSCUP 2016
BKK16-410 SoC Idling & CPU Cluster PM
Controlling processor c-state_usage_in_linux_nov2012
Os Vandeven
LCA13: CPUIDLE: One driver to rule them all?
Intern presentation nicolechiou_2018_vareximaging_sanitized
LCA14: LCA14-306: CPUidle & CPUfreq integration with scheduler
Analyze and optimize Android apps power consumption
Kernel Recipes 2016 - Understanding a Real-Time System (more than just a kernel)
Q2.12: Idle Power States Nomenclature
Improving Xen idle power efficiency
When the OS gets in the way
Q2.12: Idling ARMs in a busy world: Linux Power Management for ARM Multiclust...
semaphore & mutex.pdf
atom-imp-concept of hardware tools in ECE.ppt
Tommaso Cucinotta - Low-latency and power-efficient audio applications on Linux
LCA13: Who Disturbs My Slumber
Linux power management: are you doing it right?
LCU14 206- Tools to Analyse Scheduling Behaviour and Its Impact on Power Mana...
Ad

More from Anne Nicolas (20)

PDF
Kernel Recipes 2019 - Driving the industry toward upstream first
PDF
Kernel Recipes 2019 - No NMI? No Problem! – Implementing Arm64 Pseudo-NMI
PDF
Kernel Recipes 2019 - Hunting and fixing bugs all over the Linux kernel
PDF
Kernel Recipes 2019 - Metrics are money
PDF
Kernel Recipes 2019 - Kernel documentation: past, present, and future
PDF
Embedded Recipes 2019 - Knowing your ARM from your ARSE: wading through the t...
PDF
Kernel Recipes 2019 - GNU poke, an extensible editor for structured binary data
PDF
Kernel Recipes 2019 - Analyzing changes to the binary interface exposed by th...
PDF
Embedded Recipes 2019 - Remote update adventures with RAUC, Yocto and Barebox
PDF
Embedded Recipes 2019 - Making embedded graphics less special
PDF
Embedded Recipes 2019 - Linux on Open Source Hardware and Libre Silicon
PDF
Embedded Recipes 2019 - From maintaining I2C to the big (embedded) picture
PDF
Embedded Recipes 2019 - Testing firmware the devops way
PDF
Embedded Recipes 2019 - Herd your socs become a matchmaker
PDF
Embedded Recipes 2019 - LLVM / Clang integration
PDF
Embedded Recipes 2019 - Introduction to JTAG debugging
PDF
Embedded Recipes 2019 - Pipewire a new foundation for embedded multimedia
PDF
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all started
PDF
Kernel Recipes 2019 - Marvels of Memory Auto-configuration (SPD)
PDF
Kernel Recipes 2019 - Formal modeling made easy
Kernel Recipes 2019 - Driving the industry toward upstream first
Kernel Recipes 2019 - No NMI? No Problem! – Implementing Arm64 Pseudo-NMI
Kernel Recipes 2019 - Hunting and fixing bugs all over the Linux kernel
Kernel Recipes 2019 - Metrics are money
Kernel Recipes 2019 - Kernel documentation: past, present, and future
Embedded Recipes 2019 - Knowing your ARM from your ARSE: wading through the t...
Kernel Recipes 2019 - GNU poke, an extensible editor for structured binary data
Kernel Recipes 2019 - Analyzing changes to the binary interface exposed by th...
Embedded Recipes 2019 - Remote update adventures with RAUC, Yocto and Barebox
Embedded Recipes 2019 - Making embedded graphics less special
Embedded Recipes 2019 - Linux on Open Source Hardware and Libre Silicon
Embedded Recipes 2019 - From maintaining I2C to the big (embedded) picture
Embedded Recipes 2019 - Testing firmware the devops way
Embedded Recipes 2019 - Herd your socs become a matchmaker
Embedded Recipes 2019 - LLVM / Clang integration
Embedded Recipes 2019 - Introduction to JTAG debugging
Embedded Recipes 2019 - Pipewire a new foundation for embedded multimedia
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all started
Kernel Recipes 2019 - Marvels of Memory Auto-configuration (SPD)
Kernel Recipes 2019 - Formal modeling made easy

Recently uploaded (20)

PPTX
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
PDF
Designing Intelligence for the Shop Floor.pdf
PDF
Nekopoi APK 2025 free lastest update
PDF
iTop VPN Free 5.6.0.5262 Crack latest version 2025
PPTX
Introduction to Artificial Intelligence
PDF
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
PPTX
CHAPTER 2 - PM Management and IT Context
PDF
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
PDF
Softaken Excel to vCard Converter Software.pdf
PDF
Digital Systems & Binary Numbers (comprehensive )
PDF
Digital Strategies for Manufacturing Companies
PPTX
assetexplorer- product-overview - presentation
PDF
PTS Company Brochure 2025 (1).pdf.......
PPTX
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises
PPTX
Odoo POS Development Services by CandidRoot Solutions
PDF
Navsoft: AI-Powered Business Solutions & Custom Software Development
PDF
top salesforce developer skills in 2025.pdf
PDF
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
PDF
wealthsignaloriginal-com-DS-text-... (1).pdf
PPTX
Why Generative AI is the Future of Content, Code & Creativity?
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
Designing Intelligence for the Shop Floor.pdf
Nekopoi APK 2025 free lastest update
iTop VPN Free 5.6.0.5262 Crack latest version 2025
Introduction to Artificial Intelligence
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
CHAPTER 2 - PM Management and IT Context
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
Softaken Excel to vCard Converter Software.pdf
Digital Systems & Binary Numbers (comprehensive )
Digital Strategies for Manufacturing Companies
assetexplorer- product-overview - presentation
PTS Company Brochure 2025 (1).pdf.......
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises
Odoo POS Development Services by CandidRoot Solutions
Navsoft: AI-Powered Business Solutions & Custom Software Development
top salesforce developer skills in 2025.pdf
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
wealthsignaloriginal-com-DS-text-... (1).pdf
Why Generative AI is the Future of Content, Code & Creativity?

Kernel Recipes 2018 - CPU Idle Loop Rework - Rafael J. Wysocki

  • 1. CPU Idle Loop Rework Rafael J. Wysocki Intel Open Source Technology Center September 26, 2018 Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 1 / 19
  • 2. Introduction CPU Idle Time Concept Terminology: CPUs = Logical CPUs CPU 0 CPU 1 CPU 2 CPU 3 CPU 0 CPU 1 CPU 0 Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 2 / 19
  • 3. Introduction CPU Idle Time Concept CPUs: Busy Vs Idle Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 3 / 19
  • 4. Introduction CPU Idle States CPU Idle States Properties Time [a. u.] Power [a. u.] Entry ExitMR Residency Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 4 / 19
  • 5. Introduction CPU Idle States CPU Idle States Example: C-states On Intel Hardware MWAIT <HINT> CPU 0 CPU 1 CPU 2 CPU 3 CPU 4 CPU 5 CPU 6 CPU 7 C6C2 C3 C3 C6C6 C6 C6 CC2 CC3 CC6 CC6 PC2 Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 5 / 19
  • 6. CPU Idle Time Management Original Design and Problem High-level CPU Idle Time Management Control Flow CPU scheduler Select state Estimate idle time CPU is idle Enter state Governor YES NO menu ladder ACPI idle intel_idle Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 6 / 19
  • 7. CPU Idle Time Management Original Design and Problem Complication: CPU Scheduler Tick Timer Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 7 / 19
  • 8. CPU Idle Time Management Original Design and Problem Original Idle Loop Design Issue CPU scheduler Try to stop tick need_resched()? YES NO Select state Estimate idle time Governor Restart tick Enter state Interrupt May be too short! Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 8 / 19
  • 9. CPU Idle Time Management Original Design and Problem Short Idle Duration Prediction Problem WIN LOSS LOSS LOSS Short Idle Short Idle Long Idle Long Idle Predicted Actual Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 9 / 19
  • 10. CPU Idle Time Management Original Design and Problem Idle Governor Problem: Two Categories Of Information Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 10 / 19
  • 11. CPU Idle Time Management Original Design and Problem The menu Idle Governor Next Timer Event CPU scheduler Utilization Information NOHZ Infrastructure Idle Time Range Latency LimitPredicted Idle Duration Idle State Previous Idle Duration Wakeup Statistics Timers Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 11 / 19
  • 12. CPU Idle Time Management Idle Loop Rework Redesigned Idle Loop (Linux* 4.17 And Later) CPU scheduler Try to stop tick need_resched()? NO Select state Estimate idle time Governor Restart tick Enter state Interrupt Stop tick or tick stopped? YES NO Tick stopped? NO YES * Other names and brands may be claimed as the property of others Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 12 / 19
  • 13. CPU Idle Time Management Idle Loop Rework Short Idle Duration Prediction Problem Mitigated WIN LOSS Neutral WIN Short Idle Short Idle Long Idle Long Idle Predicted Actual Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 13 / 19
  • 14. CPU Idle Time Management Idle Loop Rework Idle Power (Intel OTC Server Power Lab) Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 14 / 19
  • 15. CPU Idle Time Management Idle Loop Rework Idle Power (Intel OTC Server Power Lab) Continued Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 15 / 19
  • 16. CPU Idle Time Management Idle Loop Rework Idle Power (Intel OTC Server Power Lab) Continued Before After (different scale) Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 16 / 19
  • 17. Questions? Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 17 / 19
  • 18. Resources References Linux Kernel Bug 199227, Idle loop ordering issue (https://bugzilla.kernel.org/show_bug.cgi?id=199227). T. Ilsche, M. H¨ahnel, R. Sch¨ohne, M. Bielert, D. Hackenberg, Powernightmares: The Challenge of Efficiently Using Sleep States on Multi-Core Systems (https://tu-dresden.de/zih/forschung/ressourcen/dateien/projekte/haec/powernightmares.pdf?lang=en). R. J. Wysocki, Power Management Challenges in Linux (https: //www.linuxplumbersconf.org/2017/ocw//system/presentations/4652/original/linux_pm_challenges.pdf). R. J. Wysocki, PM Infrastructure in the Linux Kernel – Current Status and Future (https://events.linuxfoundation.org/sites/events/files/slides/kernel_PM_infra_0.pdf). J. Corbet, The cpuidle subsystem (http://lwn.net/Articles/384146/). Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 18 / 19
  • 19. Disclaimer Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at www.intel.com. Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. c Intel Corporation Rafael J. Wysocki CPU Idle @ Kernel Recipes 2018 September 26, 2018 19 / 19