Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers
0 likes847 views
The document discusses strategies for improving the performance of direct-mapped caches by introducing a small fully-associative cache and implementing prefetch buffers. It covers the types of cache misses (conflict, capacity, and compulsory) and methods to reduce them through techniques like miss caching, victim caching, and various prefetch algorithms. The overall goal is to enhance cache efficiency, thereby improving the performance of advanced processors impacted by increasing miss costs.
Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers
- 1. Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch BuffersEng. MshariAlabdulkarim
- 5. Reducing Capacity and Compulsory Misses.Direct-Mapped Cache PerformanceIntroduction (1):Goal: Improve the performance of caches.Why it is important to enhance the performance of the cache?Because it has a dramatic effect on the performance of advanced processors.The miss cost has been increasing in the last decade because of:The cycle time has been decreasing much faster than main memory access time.
- 6. The average number of machines cycles per instruction has also been decreasing.Direct-Mapped Cache PerformanceIntroduction (2):VAX was an instruction set architecture (ISA) developed by Digital Equipment Corporation (DEC) in the mid-1970s.
- 7. VAX 11/780 is a complex instruction set computer (CISC) machine.
- 8. VAX-11/780 was the first model of VAX machines, and it has been introduced on October 25, 1977.
- 9. It has been used for a while as a baseline in CPU benchmarks because its speed was about one MIPS.Direct-Mapped Cache PerformanceIntroduction (3):The increasing cost of cache misses:
- 10. Direct-Mapped Cache PerformanceBaseline Design (1):
- 11. Direct-Mapped Cache PerformanceBaseline Design (2):Chip Specifications:The cycle time of this chip is 3 to 8 times longer than the instruction issue rate.
- 12. The first & second level caches are assumed to be direct-mapped ⟹ “fastest effective access time”.
- 13. The data cache can be either write-through or write-back. How to obtain?123Very fast on-chip clockIssuing many inst. per cycleUsing higher speed technology for the processor chip
- 14. Direct-Mapped Cache PerformanceBaseline Design (3):System Parameters:Instruction issue rate = 1000 MIPS.
- 15. There are two separate first-level caches: instruction and data caches.
- 16. The size of the first-level caches is 4KB (with 16B lines).
- 17. The size of the second-level is 1MB (with 128B lines).
- 18. The miss penalties are assumed to be 24 instruction times for the first level and 320 instruction times for the second level.Direct-Mapped Cache PerformanceBaseline Design (4):Test program characteristics
- 19. Direct-Mapped Cache PerformanceBaseline Design (5):Baseline system first-level cache miss rates
- 20. Direct-Mapped Cache PerformanceBaseline Design (6):Performance lost in memory hierarchyBaseline design performanceNet performance of the system
- 21. Direct-Mapped Cache PerformanceReducing Conflict Misses (1):Caches MissesCapacityCoherenceCompulsoryConflictConflict misses can be reduced using either miss caching or victim caching.Direct-Mapped Cache PerformanceReducing Conflict Misses (2):Conflict misses: are misses that would not occur if cache was fully–associative, and had LRU replacement.
- 22. Compulsory misses: are misses required in any cache organization because they are the first references to an instruction or piece of data.
- 23. Capacity misses: occur when the cache size is not sufficient to hold data between references.
- 24. Coherence misses: are misses that occur as a result of invalidation to preserve multiprocessor cache consistency.Direct-Mapped Cache PerformanceReducing Conflict Misses (3):Percentage of conflict misses, 4K I and D caches, 16B lines
- 25. Direct-Mapped Cache PerformanceReducing Conflict Misses (4):Miss Caching:We can add associativity to a direct-mapped cache by placing a small miss cache on-chip between a first-level cache and the access port to the second-level cache.
- 26. Miss cache: is a small fully-associative cache containing on the order of two to five cache lines of data.
- 27. Misses in the cache that hit in the miss cache have only a one cycle miss penalty, as opposed to a many cycle miss penalty without the miss cache.Direct-Mapped Cache PerformanceReducing Conflict Misses (5):Miss cache organization
- 28. Direct-Mapped Cache PerformanceReducing Conflict Misses (6):How miss cache works?When a miss occurs, data is returned to the direct mapped cache and the miss cache under it, where it replaces the least recently used item.
- 29. Each time the upper cache is probed, the miss cache is probed as well.
- 30. If a miss occurs in the upper cache but the address hits in the miss cache, then the direct-mapped cache can be reloaded in the next cycle from the miss cache.Direct-Mapped Cache PerformanceReducing Conflict Misses (7):Conflict misses removed by miss caching
- 31. Direct-Mapped Cache PerformanceReducing Conflict Misses (8):Victim Caching:How it works?When a miss occurs, the fully-associative cache will be loaded with the victim line from the direct-mapped cache.
- 32. In the case of a miss in the direct-mapped cache that hits in the victim cache, the contents of the direct-mapped cache line and the matching victim cache line are swapped.Direct-Mapped Cache PerformanceReducing Conflict Misses (9):Victim cache organization
- 33. Direct-Mapped Cache PerformanceReducing Conflict Misses (10):Conflict misses removed by victim caching
- 34. Direct-Mapped Cache PerformanceReducing Conflict Misses (11):Victim cache performance with varying direct-mapped data cache size
- 35. Direct-Mapped Cache PerformanceReducing Conflict Misses (12):Victim cache performance with varying data cache line size4KB direct-mapped cacheDirect-Mapped Cache PerformanceReducing Capacity and Compulsory Misses (1):Compulsory misses are misses required in any cache organization because they are the first references to a piece of data.
- 36. Capacity misses occur when the cache size is not sufficient to hold data between references.
- 37. One way of reducing the number of capacity and compulsory misses is to use prefetch techniques such as longer cache line sizes or prefetching methods.Direct-Mapped Cache PerformanceReducing Capacity and Compulsory Misses (2):Prefetch algorithms:1. Prefetch always: prefetches after every reference.2. Prefetch on miss: on a miss, always fetches the next line.3. Tagged prefetch: When a block is prefetched, its tag bit is set to zero. Each time a block is used its tag bit is set to one. When a block undergoes a zero to one transition its successor block is prefetched.
- 38. Direct-Mapped Cache PerformanceReducing Capacity and Compulsory Misses (3):Limited time for prefetch
- 39. Direct-Mapped Cache PerformanceReducing Capacity and Compulsory Misses (4):Stream Buffers:Goal: start the prefetch before a tag transition can take place.
- 40. Direct-Mapped Cache PerformanceReducing Capacity and Compulsory Misses (5):How stream buffer works?When a miss occurs, the stream buffer begins prefetching successive lines starting at the miss target.
- 41. As each prefetch request is sent out, the tag for the address is entered into the stream buffer, and the available bit is set to false.
- 42. When the prefetch data returns it is placed in the entry with its tag and the available bit is set to true.
- 43. If a reference misses in the cache but hits in the buffer the cache can be reloaded in a single cycle from the stream buffer.
- 44. When a line is moved from a stream buffer to the cache, the entries in the stream buffer can shift up by one and a new successive address is fetched.Direct-Mapped Cache PerformanceReducing Capacity and Compulsory Misses (6):Sequential stream buffer performance4-entry I-stream buffer backing 4KB I-cache with 16B lines.
- 45. 4-entry D-stream buffer backing 4KB D-cache with 16B lines.Direct-Mapped Cache PerformanceReducing Capacity and Compulsory Misses (6):Multi-Way Stream Buffers:The single stream buffer could remove 72% of the instruction cache misses, but it could only remove 25% of the datacache misses.
- 46. One reason for this is that data references tend to consist of interleaved streams of data from different sources.Direct-Mapped Cache PerformanceReducing Capacity and Compulsory Misses (7):
- 47. Direct-Mapped Cache PerformanceReducing Capacity and Compulsory Misses (8):Four-way stream buffer performance
- 48. Direct-Mapped Cache PerformanceReducing Capacity and Compulsory Misses (9):Stream buffer performance vs. cache size
- 49. Direct-Mapped Cache PerformanceReducing Capacity and Compulsory Misses (10):Stream buffer performance vs. line size
- 50. Direct-Mapped Cache PerformanceSystem performance with victim cache and stream buffers
- 51. Direct-Mapped Cache PerformanceReference:N. P Jouppi, “Improving direct-mapped cache performance by the addition of a small fully-associative cache and prefetch buffers,” in Computer Architecture, 1990. Proceedings., 17th Annual International Symposium on, 2002, 364–373.
- 52. Direct-Mapped Cache PerformanceThank you for your attention