Computer Architecture and Organization
- 1. Computer Architecture and Organization Lecture Series for VIT, Chennai Jaiber John, Intel 2022 - 2023
- 2. Module 7 Energy Efficient Architectures Challenges and Designs
- 3. Contents ● CMOS/Power basics ○ Power issues ● Power management techniques ○ Hardware ○ Software ● Low power architectures
- 4. What consumes power in Computer? Power draw in a sample desktop and monitor in idle mode Power draw in a sample Laptop
- 5. Check your computer power usage ● Use Apps like “LocalCooling”, “HWMonitor” on Windows HWMonitor App LocalCooling App
- 6. CPU Power model ● Power vs. Energy ○ Power – Energy converted or transferred per unit of time ● What happens to the energy consumed by CPU? CPU Signals/Buses Clocks Voltage Signals/Buses ?? Heat
- 8. CMOS Power dissipation CMOS – n-type + p-type MOSFET - Very high leakage at < 22nm - So, 5 – 22 nm -> FinFET (non-planar) - < 5 nm -> RibbonFET CMOS Power Dissipation
- 9. CMOS Power Dissipation ● Static power dissipation ○ Mostly due to Leakage ○ Occurs when all inputs are held at some valid logic level and the circuit is not in charging states ● Dynamic Power Dissipation ○ Due to the current that flows only when the transistors of the devices are switching from one logic state to another. ○ Depends on frequency and the type of internal node CMOS Power Dissipation
- 10. Static and Dynamic Power vs Frequency ● Frequency impacts dynamic power ● Static power does not vary as much with frequency
- 11. Processor Power Delivery ● The power delivery network (PDN) supplies reliable power to all transistors ● Challenge: ○ Deliver with excellent efficiency ○ Swiftly respond to changes in power draw. Power Delivery System
- 12. Processor Cooling ● Cooling is very important for CPU ● Types ○ Active (Fan) ○ Fanless cooling ○ Liquid Cooling Fan + Heatsink Fanless cooling Liquid cooling
- 13. TDP, Energy, Power, Turbo ● TDP (Thermal Design Power) ○ The maximum amount of heat generated by a computer chip that the cooling system in a computer is designed to dissipate under any workload ● Max or Peak Power ○ Maximum power that can be consumed by CPU, at highest frequency of operation ● Turbo Mode ○ Boost frequency – overclock for short period of time to handle bust of workload activity
- 14. Power Management
- 15. Contents ● Hardware Power management techniques ○ Clock, Power gating ○ DVFS ○ ACPI ● Software ○ Idle vs active state management ○ Frequency governors (Linux)
- 16. Hardware Power management techniques ● Clock gating ○ Clock gating is a popular power management technique used in many synchronous circuits for reducing dynamic power dissipation, by removing the clock signal when the circuit is not in use or ignores clock signal Simple Clock Gating
- 17. Power Gating ● Reduce power consumption, by shutting off the current to blocks of the circuit that are not in use ● Gate controlled by: ○ software – using driver software. ○ hardware – using hardware timers. ○ By using a dedicated power gating controller in the design. Power Gating in CMOS
- 18. Clock gating vs Power gating Clock gating Power gating Clock gating reduces dynamic power by reducing switching frequency. Power gating reduces dynamic and static (leakage) power. Clock gating is simpler to implement. Power gating requires additional circuits to maintain states. Power gating is more efficient at saving power. Both require extensive verification to cover corner cases.
- 19. Clock gating vs Power gating
- 20. DVFS – Dynamic Voltage and Frequency Scaling ● Dynamic power (switching power) = C * V2 * A * f ○ where C is the capacitance being switched per clock cycle ○ V is voltage ○ A is the Activity Factor ● Voltage has higher impact (since V2), controlled by VRs ● Frequency – controlled by the PLL (Phase locked loop) ● Disadvantages: ○ Adds complexity to design and verification ○ Introduces performance fluctuations ○ If not done correctly, can end up using more energy
- 21. Power Control Unit ● PCU - Tiny microcontrollers within the processor or SoC ● Monitors sensors, and controls power gating and voltage regulation ● Why? Since power control done directly by OS is complicated Sample Power Control Unit
- 22. ACPI – Advanced Configuration and Power Interface ● ACPI - Open standard, implemented in firmware (BIOS) of most x86 systems ● Abstracts hardware configurations and presents to OS ● OSPM – OS-directed Power Manager controls power states ● OSPM calls ACPI to manage power states OSPM ACPI
- 23. ACPI Power States ● Instead of directly controlling Voltage or frequency, ACPI provides ‘states’ ● Performance states ○ P0, P1, P2, P3.. ● Power states ○ C0, C1, C2, C3.. ● Device states ○ D0, D1, D2, D3.. Example: P-states (V vs. f) of a processor
- 24. ACPI State Machine • Actual power management is quite complicated • Involves Firmware, Operating System and device drivers
- 25. Power Management – Software level ● OS plays a major role in power management ● It is usually the ‘brain’ behind power management control ● OS has knowledge of task activity, priority, etc ● OS uses system provided knobs (e.g., ACPI) to set power states Applications OS Scheduler Policy CPU Device drivers IO Devices
- 26. Active and Idle states ● Generally, system state can be: ○ Active – when tasks are running. Performance state can be adjusted. ○ Idle/Sleep – no tasks available to run. Power states can be adjusted. Can easily wake up to active state. ○ Suspend – Suspend to RAM or Suspend to disk (hibernation). System state is maintained. Very low power required. Takes some time to wake up. ○ Off – System powered off, no states maintained. Need to start from Power-on.
- 27. CPU Frequency Scaling Algorithms (Linux) Performance Always chooses the highest performance state. Highest performance but low power savings Powersave Always chooses the lowest performance state. Low performance but offers high power savings Schedutil Chooses performance states to improve task scheduling efficiency Ondemand Chooses performance states based on CPU load. Higher load -> boost performance and vice versa Conservative Chooses performance based on CPU load like Ondemand, but does not change processor states so aggressively but in small steps Algorithms that control the frequency scaling are called ‘governors’, they implement the PM policy
- 28. Power saving design approaches ● What policy or algorithm would you devise to: ○ Optimize for thermal efficiency (control temperature) in handheld device? ○ Optimize for power efficiency in real time applications? ○ Optimize for maximum performance in a supercomputer? ○ Can predictive (ML techniques) be used to optimize power and performance? ○ Can application specific scheduling be helpful? ■ Ex: Database server vs Network server vs DL inference server ○ Design optimal scheduling for smartphones – ex. Android? ■ Only one foreground app runs, other apps frozen in background
- 30. Contents ● What are Low Power systems? ● RISC vs. CISC microarchitecture ● Strategies for low power/energy designs ○ SoC and Integration ○ Special purpose processors ○ Big.LITTLE architecture ○ Efficient Hardware designs ○ Energy aware scheduling
- 31. Power consumption of various devices
- 32. Low Power Designs ● Roughly consume less than 5-10W of power ● Finds usage in IoT, embedded, mobile devices, smartwatches,.. ● Primarily dominated by microcontrollers, ARM/RISC-V designs ● SoC designs contribute to power efficiency ● Low power designs tend to be custom designs Various factors from hardware (circuit/gate) design to software scheduling play a key role
- 33. Processors – Features and Power Name Architecture Features Power Arduino Uno (microcontroller) RISC 8-bit 20MHz, USART, SPI, I2C 0.29 W Raspberry Pi (education, robotics) ARM 64-bit SoC 4-core, 1.5GHz, iGPU, WiFI, BT, HDMI 6 W Shakti (embedded) RISC-V 64-bit 2-16 cores, 1.2GHz 5-10 W * Qualcomm Snapdragon 8 G2 (for smartphones) ARM 64-bit, big.LITTLE, SoC 4-core, 3.2GHz, iGPU, WiFi, modem, BT, ISP, Video, Audio 7.5 W Apple M2 (for MacBook Air) ARM 64-bit, big.LITTLE, SoC 8-core, 3.48GHz, iGPU, neural engine, ISP 20 W Intel AlderLake (Laptop, desktop) x64, Hybrid 16-core, 5.5GHz, iGPU, PCIe, WiFI, AVX, 125 W Intel SaphireRapids (Server) X64 60-cores, several accelerators 350 W
- 34. RISC vs. CISC – power perspective RISC (ARM, RISC-V, ..) CISC (x86) Designed for lower power Designed for higher performance Fixed width instructions, easier to decode, Variable length complex instructions, need dedicated circuits to decode Lower clock speed, Lesser transistors Higher clock speed More transistors, more legacy features Easier to pipeline OOO, speculative execution optional Out of order, speculative execution, branch prediction, pipelining complicated Can be “fanless” – less power for cooling Requires fans, additional power needed for cooling
- 35. Strategy #1 for Low Power: SoC (Integration of IP) ● SoC – System on Chip ● Integrating multiple components –> processor, memory, controllers into a single Chip ● Big reduction in interconnect (bus) power ● Power or Clock gate entire IP blocks ● On chip memory reduces memory accesses and lowers power Sample SoC
- 36. Strategy #2 – Special purpose processors ● General idea is to offload processing from CPU to special purpose processors to optimize power Examples of special purpose processors: ● DSP – Digital Signal Processors ● ASIC – Application Specific Integrated Circuits ● FPGA – Field Programmable Gate Arrays ● TPU – Tensor Processing Unit ● IPU – Image Processing Unit
- 37. Strategy #3 – big.LITTLE architecture
- 38. Strategy #3: contd.. ● Combines high performance + low power cores into single chip ● High-performance cores used for power-intensive tasks, like gaming ● Lower-performance cores are used for less power-intensive tasks, such as web browsing and social media. ● By using the appropriate core for a specific task, the processor can reduce power consumption and extend battery life. ● Operating system dynamically switches between the high- performance and low-performance cores based on the workload. ● Commonly used in smartphones and tablets, embedded systems and IoT devices where power efficiency is a key concern.
- 39. Strategy #4: Efficient HW design ● Various circuit level techniques ● Logic Gate design ○ MOSFT vs FinFET ● Fabrication technology ○ 22nm vs 14nm ● Power aware optimization ○ Netlist simulation and synthesis ● Power gating, Clock gating, DVFS, etc.. HW power saving techniques vs. usage
- 40. Strategy #5: Custom Instructions ● Design custom instructions that can perform a set of operations efficiently than general purpose CPU instructions ● Example: Vector instructions, SIMD, cryptographic functions ● Energy can be saved, though power readings could be higher Example: Energy savings (~35%) with SIMD
- 41. Strategy #6: Energy aware scheduling ● Performed by the OS Scheduler ● Scheduler aware of system topology ● Primary methods: ○ Task placement ○ Task migration ● Used in embedded real time systems ● Example 1: Power aware routing in reconfigurable architectures ● Example 2: Linux Energy Aware Scheduler (EAS)
- 42. Strategy #6: contd.. ● Energy Aware Scheduler (EAS) in Linux
Editor's Notes
- #29: https://www.kernel.org/doc/html/latest/admin-guide/pm/cpufreq.html https://www.kernel.org/doc/html/v4.14/driver-api/pm/devices.html