1. The Challenge of Hardware Benchmarking
In machine learning research, performance is typically measured using statistical accuracy indices (such as F1 scores or BLEU parameters) evaluated against static training sets. But in safety-critical systems like hardware engineering, statistical averages are functionally meaningless. A layout generator that averages 95% routing success across a hundred test cases can still fail catastrophically on the single test case that controls a motor driver VCC pin.
We propose a different methodology: the Verification Benchmark Framework.
Rather than focusing on layout speed or statistical “success scores,” this framework evaluates the quality of the verification system itself, measuring the depth, completeness, and auditability of the design trail.
2. Core Evaluation Metrics
[ REQUIREMENT INPUTS ]
│
▼
[ Traceability Coverage ] ──► Are requirements mapped to nets?
│
▼
[ Constraint Coverage ] ──► Do parameters contain physical bounds?
│
▼
[ Verification Completeness ] ─► Have deterministic solvers checked nets?
│
▼
[ Review Resolution Rate ] ──► Are overrides resolved and documented?2.1 Traceability Coverage ($C_T$)
Traceability coverage measures the ratio of machine-readable requirements to physical nets or footprints on the board stackup. It is defined as:
$$C_T = \frac{R_m}{R_t}$$
Where:
- $R_m$ is the count of requirement specifications explicitly mapped to design nets.
- $R_t$ is the total count of requirement specifications identified in inputs documents.
A complete design must achieve $C_T = 1.0$. Any unmapped requirement indicates a verification gap.
2.2 Constraint Coverage ($C_C$)
Constraint coverage audits the count of active constraints compiled in the layout. For every mapped net, $C_C$ measures whether minimum/maximum current boundaries, temperature limits, and copper clearance limits are defined.
A high $C_C$ ensures that geometric checks have a physical rule basis, preventing the layout router from operating on unconstrained geometric defaults.
2.3 Verification Completeness ($V_{comp}$)
Verification completeness tracks the percentage of nets that have undergone mathematical calculations using deterministic solvers.
For high-speed or high-power tracks, $V_{comp}$ checks:
- Has the net trace width been verified using IPC-2152 thermal solvers?
- Has spacing clearance been audited against IPC-2221 guidelines?
- Has loop inductance been simulated or checked?
2.4 Review Resolution Rate ($R_{resolved}$)
Reviews must be trackable. This metric measures the ratio of review findings to resolution outcomes:
$$R_{resolved} = \frac{F_r + F_a}{F_t}$$
Where:
- $F_r$ represents review findings resolved by modifying layout coordinates or components.
- $F_a$ represents review findings overridden with explicit engineering justification.
- $F_t$ represents the total count of review findings raised during checks.
For a release state package to generate, the system requires $R_{resolved} = 1.0$ for all high-severity items.
3. Verification Integrity Score
By multiplying these metrics, we establish the Verification Integrity Score (VIS):
$$VIS = C_T \times C_C \times V_{comp} \times R_{resolved}$$
The VIS acts as OmeraCode’s primary index of design assurance. A design package with a high VIS guarantees that design outputs are not generative guesses, but a mathematically closed, audited engineering system.