Case Study of End to End Formal Verification Methodology
- 2. Methodology for verifying a block completely using formal methods • Alternative for traditional UVM based IP verification environment • Avoid high-cost development of custom agents, sequence libs, checkers for UVM test benches with custom interfaces • UVM agents or VIPs are replaced by “proof-kits” IP functionality is fully described using formal properties • All properties derived from spec and reviewed with arch/design teams • Includes covers, black-box and white-box assertions, for all control/data/debug interfaces Work can start after specifications are ready. • First bugs are available almost immediately after the first RTL drop Registers are checked with assertions generated from an IPXACT XML source IP is refined as assertions are implemented & proofs are debugged IP is fully verified when all the assertions and proof kits are proven
- 3. • Experiment with end-to-end formal methods to achieve same quality as traditional UVM test benches, especially for certain types of control- path designs • Reduce verification effort by deploying a formal verification only environment due to resource/schedule pressures on IP verification • Develop an end-to-end formal verification methodology for custom IP blocks • Identify areas of improvement with the EDA vendors on metrics used for formal end-to-end IP verification signoff
- 4. Benefits • Early feedback on RTL due to formal methodology • Exhaustive verification compared to simulation based bug-hunting methodologies. Multiple SEED regressions not required to close coverage • Shorter DV Cycle compared to UVM • Portability of assertions to other verification environments (subsystem/chip level test benches) • Uses proof-kits, which are lower cost (effort) than traditional UVM VIP Challenges • Relating formal verification “metrics” to traditional metrics used for simulation based test benches (e.g. code/functional coverage) • Reviews were more time consuming to ensure the completeness of the assertions and assumptions to correctly constrain the design • Partitioning of proofs to achieve a reasonable run time • Trade-offs between automatic vs manual generation of assertions
- 5. Interrupt Controller (INTC) Maskable Level Interrupts InterruptRouting Interrupt Bit-Vectors Bus Slave Maskable Output Interrupts MMIO
- 6. Crossbar (XB) Unit0 Unit 2 Bank Modules Unit1
- 7. Hierarchical Strategy • When an IP has repetitive components, total runtime can be reduced by separately verifying the IP’s sub-blocks and connections. Proofs can focus on unique interactions and ignore repeated subblocks, or repeated subblocks can be blackboxed with additional constraints. Divide and Conquer • Dividing large bodies of assertions into separate jobs, keeping existing limits, abstractions, et al. the same between jobs, yielded a reduction in total runtime. Abstractifying • Large families of assertions to be reduced to equivalent, single assertions with limited addition of Verilog in the glue logic. This reduction in property count significantly improved runtime, despite checking identical functionality. Traditional Strategies to Improve Formal Verification • Tool specific optimizations • Constraining inputs
- 9. 70 57 72 17 25 240 0 23 0 20 40 60 80 100 120 140 INTC XBAR Average-UVM-Simple Effort Comparison - PersonDays Effort - RTL 0.5 Effort - RTL 0.8 Effort - RTL 1.0 8.46 3.40 2.79 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 INTC XBAR Average-UVM-Simple Total Bugs found by DV per 1K LOC-DV 16 3 20 0 5 10 15 20 25 INTC XBAR Average-UVM-Simple Time to First Design Bug (First Bug - First Verif Checkin) Calendar Days Item INTC XBAR Average-UVM-Simple Effort - RTL 0.5 70 57 72 Effort - RTL 0.8 17 25 24 Effort - RTL 1.0 0 0 23 LOC-RTL 21000 4500 3100 LOC-DV 1300 5000 5350 Total Bugs found by IP Verif 11 17 17 Gate Count (Est) 835734 60186 58502 Escaped Bugs from L1 1* 1 1
- 10. Shallow Cone of Logic • Do any possible states of the IP require more cycles to reach, than can be tolerated in FE2E? High Symmetry • The IP reuses sub-blocks or has repeating patterns Low Code Complexity • If the IP can reasonably be fully specified in SVA, it is likely a good candidate for FE2E Avoid Blocks with Known Large State Spaces • Signal Processing Blocks
- 11. Make Formal coverage reporting more like simulation based metrics • Add features to improve compatibility with existing coverage tracking. e.g. merging reports from hierarchical and partitioned FE2E environments. Provide tools to help appraise how fit an IP is for FE2E Automate specific methods to reduce runtime that are normally done manually • Automate Divide and Conquer • Provide suggestions to user to blackbox repeated modules Warnings for properties which are anticipated to not converge in a reasonable amount of time based on certain heuristics Provide more proofkits for standard interfaces
- 12. • End-To-End Formal IP Verification is well suited for certain types of symmetric and control path intensive designs. • FE2E is worth the savings in verification effort & resources, compared to a traditional simulation based approach • We achieve the same quality of verification with FE2E as with traditional UVM testbenches. • The main challenge is to determine if an IP can be verified using this methodology and manage the proof run times.