Run Spec-Based Testing

Spec-Based Testing evaluates the application against recorded expected behavior through a verification agent or context separate from the coding agent's implementation context. It provides an additional check on implementation assumptions and makes verification results traceable to specifications.

Responsible participants: Independent verification agent/context, coordinated by the Product Engineer.
Inputs: Committed relevant specification corpus, authorized access to the application and test environment, known constraints, available representative data.
Outputs: Requirement-to-check mapping, observed outcomes, reproducible failures, blocked or ambiguous cases, and follow-up dispositions.

A separate verification agent or context uses maintained intended behavior to generate and execute checks against running software, APIs, or interfaces where practical. Playwright is one possible browser tool, not a mandatory SHAPE dependency. The verifier records passes, failures, blockers, and ambiguities. Expected results must not be inferred solely from implemented code. Passing tests generated from inaccurate intent cannot prove fitness for stakeholder needs; product inspection and UAT remain essential.

1. Establish an independent context. Start verification from the committed specifications rather than the coding agent's self-description of completed work. The verifier should not use existing behavior as the definition of correctness.

2. Identify relevant requirements. Read the relevant feature specs and cross-referenced shared behavior. For an enhancement, include requirements likely to be affected by the change. Note requirements that are too ambiguous to support a defensible test.

3. Derive checks. Produce checks that cover important normal behavior, validation and rejection paths, boundaries, permission differences, and state transitions when relevant. Link checks to stable specification identifiers or clearly identifiable conditions.

4. Exercise the real behavior. Use the authorized test environment to operate the relevant interface or API and observe the result. Where a requirement cannot be exercised, record the reason rather than inventing a pass. Static inspection alone should not be described as full behavioral verification.

5. Record outcomes and evidence. Use four clear dispositions: Pass, Fail, Blocked, or Needs clarification. A useful failure states which requirement was violated, what was expected, what occurred, and how to reproduce it. Evidence can take the form permitted by the project's tools and privacy constraints.

6. Triage with the Product Engineer. Determine whether each issue is an implementation defect, a missing test capability, an ambiguous specification, or a legitimate change in accepted intent. Refer business rule decisions to their owners. Update specifications when accepted intent changes.

7. Correct and rerun. Implement approved corrections and repeat affected checks, along with pertinent regression checks. Preserve the distinction between a requirement that passed and one that has not yet been exercised.

8. Prepare the verification summary. State the scope checked, outstanding failures or blockers, limitations of coverage, and any unresolved clarification. Make this available for final architect review or the applicable enhancement-cycle review and UAT preparation.

This table illustrates disposition; its sample outcomes are fictional, not evidence from a real implementation.

Meaningful requirements have documented dispositions; failures and ambiguities are addressed or explicitly carried forward under established acceptance policy; test limitations are transparent; and changes have received appropriate retesting. The report does not overstate the assurance it provides.

Asking the implementation agent to certify its own output without an independent source of intent; accepting a generated list of tests that were never run; silently treating blocked checks as passes; editing specifications to match accidental behavior; and presenting test success as a replacement for UAT or specialist risk assurance.