{ "title": "From intent to evidence", "lessons": [ { "id": "intent", "title": "Start with expected behavior", "concept": "A requirement says what should happen and under which conditions. An observation says what happened in one implementation. Keep those statements distinguishable.", "product": "Describe who benefits, a normal example, a boundary, and what is still undecided. Record the reason for a decision in Canon and link to the affected LawSpec units.", "programmer": "Read existing code without rewriting it during discovery. Cite functions and tests as observed evidence; mark inferred intent and open product questions in a reviewable draft.", "example": "Observed: refunds currently round down. Question: is that the intended policy for every currency?", "question": "Does an existing test make rounding down a product requirement?", "answer": "It establishes current expectations in that test. A product decision must establish the intended policy and scope.", "links": [] }, { "id": "domains", "title": "Make valid inputs explicit", "concept": "Types describe the values a behavior applies to. Refinements restrict that domain. Indexing relates a value to a measurable property such as length.", "product": "Ask whether zero, missing, duplicate, expired, and out-of-range inputs are valid. Decide whether rejection is an expected outcome or the input is outside the operation’s domain.", "programmer": "Use existing indexed types and refinements. An index below a length can be expressed by an integer refinement. An empty domain needs evidence of emptiness; failing to generate a candidate does not establish it.", "example": "An item position i must satisfy 0 <= i and i < n. When n = 0, no such position exists.", "question": "Should a generator that tried 100 candidates and found none disable the operation?", "answer": "Only proven emptiness disables it. Bounded rejection is incomplete and must remain visible.", "links": [ [ "Idris Data.Fin", "https://idris-lang.org/Idris2/base/docs/Data.Fin.html" ] ] }, { "id": "properties", "title": "Choose properties that constrain behavior", "concept": "A useful property rules out plausible wrong behavior. Scott Wlaschin’s property heuristics help find these constraints; LawSpec uses its existing laws, definitions, and harnesses.", "product": "Try a deliberately unhelpful implementation. Returning an empty list may satisfy “the result is sorted” while losing every input. Add a structural property preserving the elements.", "programmer": "Combine ordering with permutation preservation. Use round trips for encoders, idempotence for normalization, invariants for stable facts, and independent reference models for complex behavior. These are property choices, not new grammar aliases.", "example": "normalize(normalize(x)) = normalize(x) is idempotence. decode(encode(x)) = x is a round trip.", "question": "Can an encoder and decoder with the same bug pass a round-trip property?", "answer": "Yes. Add independently known encodings and boundary cases to constrain their shared interpretation.", "links": [ [ "Property-based testing series", "https://fsharpforfunandprofit.com/series/property-based-testing/" ] ] }, { "id": "malice", "title": "Try malicious compliance", "concept": "Imagine the weakest implementation that satisfies your current properties. This is Wlaschin’s Enterprise Developer From Hell: a way to expose missing requirements.", "product": "If a delivery system promises “never deliver late,” never delivering satisfies that wording. Ask for existence, eventual outcomes, failures, and deadlines explicitly.", "programmer": "Use a constant implementation or deliberate mutant as a negative control. Pair safety with liveness when appropriate. Do not treat agreement with a reference as independent evidence that the reference captures the product intent.", "example": "A price calculator that always returns zero satisfies “price is nonnegative.”", "question": "What else should constrain that calculator?", "answer": "Known examples, quantity and tax relationships, rounding rules, and independently specified totals narrow the allowed behavior. Choose properties based on the product, not a fixed checklist.", "links": [ [ "Talk, video and slides", "https://fsharpforfunandprofit.com/pbt/" ] ] }, { "id": "generators", "title": "Generate interesting cases", "concept": "A generator chooses which valid cases a sampled test explores. Its distribution determines the bugs the test has an opportunity to find.", "product": "Ask to see categories, not just a test count: empty and nonempty, duplicates, boundaries, invalid requests with modeled outcomes, and rare combinations.", "programmer": "Construct values in the domain where possible. Inspect coverage and classification. Wlaschin’s follow-up series shows why random inputs alone can miss the categories that matter. Bounded generation failure remains incomplete.", "example": "Ten thousand ordinary payments may never exercise a minimum-value refund or a currency boundary.", "question": "Do more random tests guarantee that a rare category was covered?", "answer": "No. Measure the category and generate it deliberately when its behavior matters.", "links": [ [ "Return of the EDFH series", "https://fsharpforfunandprofit.com/series/return-of-the-edfh/" ] ] }, { "id": "shrinking", "title": "Read the smallest disagreement", "concept": "Shrinking searches for a smaller counterexample that still violates the property and stays in its domain. The result is a debugging aid, not automatically the unique root cause.", "product": "Review whether the counterexample exposes a product ambiguity, a specification error, or an implementation defect. A known discrepancy records a mismatch for review.", "programmer": "Replay the seed, keep the original failure, and shrink while checking validity. For stateful sequences, deleting or changing an earlier command requires revalidating the affected suffix from the reference state.", "example": "Removing a push may invalidate a later pop or make an item position out of range.", "question": "Should an invalid shortened sequence be reported as an application defect?", "answer": "No. Reject or regenerate the invalid suffix before executing the candidate against the application.", "links": [] }, { "id": "models", "title": "Generate from the reference state", "concept": "A model connects implementation commands to reference transitions. At each step, valid commands and their arguments depend on the current abstract reference state.", "product": "Specify the expected transition and observable result. Keep model assumptions visible. Passing model conformance does not establish that the reference faithfully models the outside world.", "programmer": "Follow PropEr’s command(state), precondition, next_state, and shrinking precedent. Generate from the abstract reference state and its indices, then recompute after each transition. Shared models stay unindexed; OTP gen_statem integration is separate work.", "example": "After adding two items, choose a position in [0, 2). After clearing, the select command has an empty position domain.", "question": "Why must generation avoid reading the system under test’s state?", "answer": "A faulty implementation could otherwise influence which commands are selected and hide its own defects.", "links": [ [ "PropEr proper_statem", "https://proper.hexdocs.pm/proper_statem.html" ] ] }, { "id": "evidence", "title": "Say what the evidence establishes", "concept": "Definition validation, sampled testing, finite exhaustive checking, solver discharge, and proof checking establish different claims. Keep their outcomes explicit.", "product": "Ask what domain was covered, which implementation ran, and what remains unknown. not-verified, unsupported, incomplete, pending, and known-discrepancy require interpretation; none should become a passing claim.", "programmer": "Mandatory definition validation runs independently of selected law layers. Obligation generation creates a goal; solver execution attempts it; proof checking validates a proof artifact. LawSpec’s Z3 results trust translation and Z3; there is no Lean or Rocq kernel backend integrated by these workflows.", "example": "No counterexample in 100 runs is sampled evidence. All states of a finite model explored is exhaustive within that model. A solver timeout is incomplete.", "question": "Does generating a Why3-style verification condition mean the program was proved?", "answer": "No. The obligation must be discharged, and the evidence must say how and what was trusted.", "links": [ [ "SPARK contracts", "https://docs.adacore.com/spark2014-docs/html/ug/en/source/how_to_write_subprogram_contracts.html" ], [ "Why3 VC generation", "https://why3.org/doc/vcgen.html" ], [ "Lean verification conditions", "https://lean-lang.org/doc/reference/latest/The--mvcgen--tactic/Verification-Conditions/" ], [ "Rocq induction", "https://docs.rocq-prover.org/master/refman/proofs/writing-proofs/reasoning-inductives.html" ] ] }, { "id": "workflow", "title": "Change decisions and behavior together", "concept": "A feature is coordinated work linked to Canon decisions, LawSpec behavior, implementation, and tests. Git supplies authorship and history.", "product": "For an addition, decide the behavior and open questions. For a change, supersede prior decisions. For a removal, preserve the decision record and explain why the behavior was withdrawn.", "programmer": "Use lawspec project init or adopt, then status and validate. Add a target with lawspec init --target. Keep rationale once in Canon, linking affected units; use ADRs only for distinct architecture choices costly to reverse. MCP remains read-only check/list/explain.", "example": "DEC-refund-policy links payment.refunds and the implementation/tests. A later decision supersedes it without deleting it.", "question": "Should this workflow create a separate stored feature registry?", "answer": "No. Canon decisions anchor the work and link the affected units; another registry would duplicate state and rationale.", "links": [] } ], "examples": [ { "id": "identity", "title": "An unchanged value", "source": "unit learn.identity\n\nfunction keep (x :: Int8) -> Int16 is x + 0 end function\n\nlaw unchanged is\n definition: for all (x :: Int8) => keep x = x end for all end definition\nend law\n\nend unit\n", "exercise": "Change x + 0 to x + 1. The definition remains valid, but the law has counterexamples." }, { "id": "round-trip", "title": "A round trip", "source": "unit learn.roundtrip\nfunction encode (x :: Bool) -> Int8 is if x then 1 else 0 end if end function\nfunction decode (x :: Int8) -> Bool is x > 0 end function\nlaw roundtrip is\n definition: for all (x :: Bool) => decode (encode x) = x end for all end definition\nend law\nend unit\n", "exercise": "Swap the 1 and 0 in encode. Then consider why changing both functions together could hide a shared bug." }, { "id": "idempotence", "title": "A stable normalization", "source": "unit learn.idempotence\nfunction normalize (x :: Int8) -> Int8 is if x < 0 then 0 else x end if end function\nlaw stable is\n definition: for all (x :: Int8) => normalize (normalize x) = normalize x end for all end definition\nend law\nend unit\n", "exercise": "Try a function that always returns zero. It still satisfies idempotence. What other behavior should your product require?" } ] }