AI models are becoming increasingly capable of designing CAD parts and routing printed circuit boards, raising a more difficult question for engineers: What do current benchmarks actually prove about an AI model’s engineering ability?
Hardware FYI argues that benchmarks should be viewed as rungs on a ladder rather than definitive tests of hardware design competence. New evaluations such as CAD Arena and EEBench can measure important capabilities, but completing benchmark tasks is different from producing hardware that survives manufacturing and performs reliably.
The first rung measures whether an AI model can operate engineering software such as CAD, CAM, and CAE tools. Frontier models are already showing considerable progress here. CAD Arena, for example, evaluates factors including model correctness and whether generated feature trees remain editable. These capabilities are relatively straightforward and inexpensive to test.
The second rung asks whether AI can judge the quality of its own design in a way comparable to a human engineer. This is harder because engineering requires reasoning across both digital and physical domains. Designers must consider geometry, tolerances, manufacturability, and iteration. Although AI models have improved their understanding of three-dimensional relationships, significant limitations remain.
The third and most demanding rung moves beyond digital design. An AI system must take a design through manufacturing, assembly, testing, and iteration, then verify that the finished hardware meets its intended specifications. Such benchmarks would be slower and more expensive to run, but they would provide stronger evidence of practical engineering capability.
The article cautions against dismissing existing benchmarks simply because they do not measure the entire engineering process. Each benchmark provides evidence about a specific capability. AI has not yet reached the point where engineers can casually prompt complete, production-ready hardware into existence, but its growing competence in individual engineering tasks deserves attention.
