Spec-Based Testing

A separate verification agent or context uses maintained intended behavior to generate and execute checks against running software, APIs, or interfaces where practical. Playwright is one possible browser tool, not a mandatory SHAPE dependency. The verifier records passes, failures, blockers, and ambiguities. Expected results must not be inferred solely from implemented code. Passing tests generated from inaccurate intent cannot prove fitness for stakeholder needs; product inspection and UAT remain essential.

The verifier needs the relevant specifications, enough legitimate context to understand intended behavior, and authorized means to inspect or exercise the application. It should not treat the implementation's current behavior as proof of what the specification means. Where the specification is ambiguous, that ambiguity should be reported, not silently resolved as a new requirement.

For each meaningful requirement, derive representative positive, negative, boundary, and permission-related checks as appropriate. Exercise user interactions, application interfaces, or other externally observable behavior where feasible. Identify which requirements were tested, what evidence was observed, and which checks could not be completed. Existing unit, integration, and end-to-end tests remain valuable; independent spec-based verification complements them.

A useful report distinguishes Pass, Fail, Blocked, and Needs clarification. A pass identifies the behavior observed and its link to an expected requirement. A fail records the mismatch and how to reproduce it. Blocked identifies an environment, data, or access limitation. Needs clarification identifies a specification that cannot support a defensible judgment.

Findings return to the Product Engineer, who decides whether the software must change, the specification must be clarified, or a limitation requires another specialist or stakeholder decision. Corrections are tested again. The specification remains the reference for intended behavior; accepted changes to that intention must be recorded rather than making tests conform silently to an accidental implementation.

A verification agent can fail to notice a problem, misread a requirement, or generate incomplete test coverage. A perfectly passing specification may also omit an important business need. Security, performance, accessibility, and regulatory assurances may require specialist techniques beyond feature-level verification. Spec-Based Testing does not replace architect review, UX judgment, or UAT.

The Run Spec-Based Testing practitioner guide provides a repeatable workflow and example result record. The Reference Project demonstrates what a small specification-to-test trace looks like.