Fix bulk update mode with sequential snapshots (#33)

* Fix bulk update mode with sequential snapshots

Previously, when a test method contained multiple expect blocks in bulk
update mode, if an earlier snapshot passed, the test would immediately
fail with "snapshot matched, but we are in updating mode" and prevent
later snapshots from being processed for update.

This change implements deferred validation:
- Passing snapshots in bulk mode are registered instead of failing immediately
- Tests continue execution to process all snapshots
- Validation is deferred until bulk update completion
- Only fails if ALL snapshots passed (no updates needed)

Fixes the issue where tests with mixed passing/failing snapshots
couldn't be bulk updated.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

.config/dotnet-tools.json

@@ -13,6 +13,12 @@
       "commands": [
         "fsharp-analyzers"
       ]
+    },
+    "woofware.nunittestrunner": {
+      "version": "0.3.4",
+      "commands": [
+        "woofware.nunittestrunner"
+      ]
     }
   }
-}
+}

.github/workflows/dotnet.yaml

@@ -249,12 +249,17 @@ jobs:
       - name: Identify `dotnet`
         id: dotnet-identify
         run: nix develop --command bash -c 'echo "dotnet=$(which dotnet)" >> $GITHUB_OUTPUT'
+      - name: Obtain NuGet key
+        uses: NuGet/login@d22cc5f58ff5b88bf9bd452535b4335137e24544
+        id: login
+        with:
+            user: ${{ secrets.NUGET_USER }}
       - name: Publish to NuGet
         id: publish-success
         uses: G-Research/common-actions/publish-nuget@2b7dc49cb14f3344fbe6019c14a31165e258c059
         with:
           package-name: WoofWare.Expect
-          nuget-key: ${{ secrets.NUGET_API_KEY }}
+          nuget-key: ${{ steps.login.outputs.NUGET_API_KEY }}
           nupkg-dir: packed/
           dotnet: ${{ steps.dotnet-identify.outputs.dotnet }}
 

CLAUDE.md

@@ -0,0 +1,90 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+WoofWare.Expect is an F# expect/snapshot testing library (similar to Jest snapshots). The project consists of two main components:
+
+- **WoofWare.Expect**: Core library that provides the `expect` computation expression and snapshot comparison functionality
+- **WoofWare.Expect.Test**: Test suite using NUnit that demonstrates and validates the library functionality
+
+## Build and Development Commands
+
+This project uses Nix for development environment management. All commands should be run within the Nix development shell:
+
+```bash
+# Restore dependencies
+nix develop --command dotnet restore
+
+# Build the project
+nix develop --command dotnet build --configuration Release
+
+# Run tests
+nix develop --command dotnet test
+
+# Pack NuGet package
+nix develop --command dotnet pack --configuration Release
+```
+
+### Code Formatting and Analysis
+
+```bash
+# Format F# code (via Nix)
+nix run .#fantomas -- .
+
+# Check formatting without modifying files
+nix run .#fantomas -- --check .
+
+# Run F# analyzers
+nix run .#fsharp-analyzers -- --project ./WoofWare.Expect/WoofWare.Expect.fsproj --analyzers-path ./.analyzerpackages/g-research.fsharp.analyzers/*/
+```
+
+### Single Test Execution
+
+NUnit's filtering is pretty borked.
+You can't apply filters that contain special characters in the test name (like a space character).
+You have to do e.g. `FullyQualifiedName~singleword` rather than `FullyQualifiedName~single word test`, but this only works on tests whose names are single words to begin with.
+
+Instead of running `dotnet test`, you can perform a build (`dotnet build`) and then run `dotnet woofware.nunittestrunner WoofWare.Expect.Test/bin/Debug/net9.0/WoofWare.Expect.Test.dll`.
+This is an NUnit test runner which accepts a `--filter` arg that takes the same filter syntax as `dotnet test`, but actually parses it correctly: test names can contain spaces.
+(The most foolproof way to provide test names to WoofWare.NUnitTestRunner is by XML-encoding: e.g. `FullyQualifiedName="MyNamespace.MyTestsClass<ParameterType1%2CParameterType2>.MyTestMethod"`. The `~` query operator is also supported.)
+
+## Architecture
+
+### Core Components
+
+- **Builder.fs**: Contains the `ExpectBuilder` computation expression that powers the `expect` syntax
+- **Domain.fs**: Core types including `CallerInfo`, `ExpectState`, and `SnapshotValue`
+- **SnapshotUpdate.fs**: Handles updating snapshot files when tests fail
+- **AstWalker.fs**: Uses Fantomas.FCS to parse F# source and locate/update snapshot strings
+- **Diff.fs**: Provides Patience and Myers diff algorithms for readable test output
+- **Dot.fs**: ASCII art rendering of dot files (requires `graph-easy`)
+
+### Key Design Patterns
+
+- **Computation Expression**: The library is built around F#'s computation expression syntax with `expect { }` blocks
+- **Source Code Manipulation**: Uses F# compiler services to locate and update snapshot strings in source files
+- **Dual Mode Operation**: Tests can run in assertion mode (normal) or update mode (to fix failing snapshots)
+
+### Snapshot Types
+
+- `snapshot`: Plain text comparison using `ToString()`
+- `snapshotJson`: JSON serialization with configurable options
+- `snapshotList`: Formatted list comparison
+- `snapshotThrows`: Exception expectation testing
+- `withFormat`: Custom formatting functions
+- `withJsonSerializerOptions`: Custom JSON serialization
+- `withJsonDocOptions`: Custom JSON parsing (e.g., for comments)
+
+### Test Snapshot Management
+
+- Individual updates: Add `'` to `expect` (making it `expect'`), run test to update, then remove `'`
+- Bulk updates: Use `GlobalBuilderConfig.enterBulkUpdateMode()` in test setup and `GlobalBuilderConfig.updateAllSnapshots()` in teardown
+
+## Important Notes
+
+- Use verbatim string literals for snapshots - the update mechanism requires this
+- Avoid format strings (`$"..."`) or string concatenation in snapshot definitions
+- The project follows strict F# conventions with warnings as errors
+- All commits go through comprehensive CI including formatting, analysis, and testing

README.md

@@ -211,4 +211,4 @@ Observe the `OneTimeSetUp` which sets global state to enter "bulk update" mode,
 
 # Licence
 
-MIT.
+WoofWare.Expect is licensed to you under the MIT licence, a copy of which can be found at [LICENSE](./LICENSE).

WoofWare.Expect.Test/WoofWare.Expect.Test.fsproj

@@ -1,4 +1,4 @@
-<Project Sdk="Microsoft.NET.Sdk">
+<Project Sdk="Microsoft.NET.Sdk">
 
     <PropertyGroup>
       <TargetFramework>net9.0</TargetFramework>
@@ -37,14 +37,13 @@
     </ItemGroup>
 
     <ItemGroup>
-        <PackageReference Include="ApiSurface" Version="4.1.20" />
+        <PackageReference Include="ApiSurface" Version="5.0.1" />
         <PackageReference Include="FsUnit" Version="7.0.1" />
         <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1"/>
         <PackageReference Include="NUnit" Version="4.3.2"/>
         <PackageReference Include="NUnit3TestAdapter" Version="5.0.0"/>
-      <!-- TODO: when ApiSurface accepts https://github.com/G-Research/ApiSurface/pull/116, upgrade these -->
-        <PackageReference Include="System.IO.Abstractions" Version="4.2.13" />
-        <PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="4.2.13" />
+        <PackageReference Include="System.IO.Abstractions" Version="22.0.15" />
+        <PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="22.0.15" />
     </ItemGroup>
 
     <ItemGroup>

WoofWare.Expect/AstWalker.fs

@@ -205,9 +205,15 @@ module internal AstWalker =
             | ParsedInput.SigFile _ -> failwith "unexpected: signature files can't contain expressions"
 
         // Find the closest match
-        results
-        |> Seq.filter (fun loc ->
-            loc.KeywordRange.StartLine <= lineNumber
-            && lineNumber <= loc.KeywordRange.EndLine
-        )
-        |> Seq.exactlyOne
+        let matches =
+            results
+            |> List.filter (fun loc ->
+                loc.KeywordRange.StartLine <= lineNumber
+                && lineNumber <= loc.KeywordRange.EndLine
+            )
+
+        match matches with
+        | [] ->
+            failwith
+                $"Unexpectedly failed to locate snapshot keyword %s{methodName} at line %i{lineNumber} of file %s{infoFilePath}. Please report this along with the contents of the file."
+        | m :: _ -> m

WoofWare.Expect/Builder.fs

@@ -598,10 +598,13 @@ type ExpectBuilder (mode : Mode) =
         match CompletedListSnapshotGeneric.passesAssertion state with
         | None ->
             match mode, GlobalBuilderConfig.isBulkUpdateMode () with
-            | Mode.Update, _
-            | _, true ->
+            | Mode.Update, _ ->
                 failwith
-                    "Snapshot assertion passed, but we are in snapshot-updating mode. Use the `expect` builder instead of `expect'` to assert the contents of a single snapshot; disable `GlobalBuilderConfig.bulkUpdate` to move back to assertion-checking mode."
+                    "Snapshot assertion passed, but we are in snapshot-updating mode. Use the `expect` builder instead of `expect'` to assert the contents of a single snapshot."
+            | _, true ->
+                // In bulk update mode, register passing tests instead of failing immediately
+                // This allows tests with multiple snapshots to continue processing
+                GlobalBuilderConfig.registerPassingTest state.Caller
             | _ -> ()
         | Some (expected, actual) -> raiseError expected actual
 
@@ -646,10 +649,13 @@ type ExpectBuilder (mode : Mode) =
         match CompletedSnapshotGeneric.passesAssertion state with
         | None ->
             match mode, GlobalBuilderConfig.isBulkUpdateMode () with
-            | Mode.Update, _
-            | _, true ->
+            | Mode.Update, _ ->
                 failwith
-                    "Snapshot assertion passed, but we are in snapshot-updating mode. Use the `expect` builder instead of `expect'` to assert the contents of a single snapshot; disable `GlobalBuilderConfig.bulkUpdate` to move back to assertion-checking mode."
+                    "Snapshot assertion passed, but we are in snapshot-updating mode. Use the `expect` builder instead of `expect'` to assert the contents of a single snapshot."
+            | _, true ->
+                // In bulk update mode, register passing tests instead of failing immediately
+                // This allows tests with multiple snapshots to continue processing
+                GlobalBuilderConfig.registerPassingTest state.Caller
             | _ -> ()
         | Some (expected, actual) -> raiseError expected actual
 

WoofWare.Expect/Config.fs

@@ -12,6 +12,8 @@ module GlobalBuilderConfig =
 
     let private allTests : ResizeArray<CompletedSnapshot> = ResizeArray ()
 
+    let private passingTests : ResizeArray<CallerInfo> = ResizeArray ()
+
     let internal isBulkUpdateMode () : bool =
         lock locker (fun () -> bulkUpdate.Value > 0)
 
@@ -34,18 +36,27 @@ module GlobalBuilderConfig =
             )
 
     /// <summary>
-    /// Clear the set of failing tests registered by any previous bulk-update runs.
+    /// Clear the set of failing and passing tests registered by any previous bulk-update runs.
     /// </summary>
     ///
     /// <remarks>
     /// You probably don't need to do this, because your test runner is probably tearing down
     /// anyway after the tests have failed; this is mainly here for WoofWare.Expect's own internal testing.
     /// </remarks>
-    let clearTests () = lock locker allTests.Clear
+    let clearTests () =
+        lock
+            locker
+            (fun () ->
+                allTests.Clear ()
+                passingTests.Clear ()
+            )
 
     let internal registerTest (toAdd : CompletedSnapshot) : unit =
         lock locker (fun () -> allTests.Add toAdd)
 
+    let internal registerPassingTest (caller : CallerInfo) : unit =
+        lock locker (fun () -> passingTests.Add caller)
+
     /// <summary>
     /// For all tests whose failures have already been registered,
     /// transform the files on disk so that the failing snapshots now pass.
@@ -60,8 +71,14 @@ module GlobalBuilderConfig =
             locker
             (fun () ->
                 let allTests = Seq.toArray allTests
+                let passingTestsArray = Seq.toArray passingTests
 
                 try
+                    // Check if we only had passing tests in bulk update mode - this should be an error
+                    if allTests.Length = 0 && passingTestsArray.Length > 0 then
+                        failwith
+                            "All snapshot assertions passed, but bulk-update mode was enabled. Disable bulk-update mode by not calling `GlobalBuilderConfig.enterBulkUpdateMode` to return to normal assertion-checking mode."
+
                     SnapshotUpdate.updateAll allTests
                 finally
                     // double acquiring of reentrant lock is OK, we're not switching threads

WoofWare.Expect/WoofWare.Expect.fsproj

@@ -3,7 +3,7 @@
     <PropertyGroup>
       <TargetFramework>netstandard2.0</TargetFramework>
       <GenerateDocumentationFile>true</GenerateDocumentationFile>
-      <Authors>Patrick Stevens</Authors>
+      <Authors>PatrickStevens</Authors>
       <Copyright>Copyright (c) Patrick Stevens 2025</Copyright>
       <Description>Snapshot/expect testing framework for F#</Description>
       <RepositoryType>git</RepositoryType>

nix/deps.json

@@ -1,8 +1,8 @@
 [
   {
     "pname": "ApiSurface",
-    "version": "4.1.20",
-    "hash": "sha256-koWgO9FC9ax+Ij56ug8kxeyknl0yhLqnNLOUdxtqqo4="
+    "version": "5.0.1",
+    "hash": "sha256-0GMXEMFgWbbE2OGxW+6h4zGgQHg+IZy1aI13Dn97xSU="
   },
   {
     "pname": "fantomas",
@@ -69,11 +69,6 @@
     "version": "1.1.1",
     "hash": "sha256-8hLiUKvy/YirCWlFwzdejD2Db3DaXhHxT7GSZx/znJg="
   },
-  {
-    "pname": "Microsoft.NETCore.Platforms",
-    "version": "2.0.0",
-    "hash": "sha256-IEvBk6wUXSdyCnkj6tHahOJv290tVVT8tyemYcR0Yro="
-  },
   {
     "pname": "Microsoft.NETCore.Targets",
     "version": "1.1.0",
@@ -141,33 +136,33 @@
   },
   {
     "pname": "NuGet.Common",
-    "version": "6.13.2",
-    "hash": "sha256-ASLa/Jigg5Eop0ZrXPl98RW2rxnJRC7pbbxhuV74hFw="
+    "version": "6.14.0",
+    "hash": "sha256-jDOwt3veI1GSG8CfBnf2+dJxD3E/Nmlc+vHtD4J76Ms="
   },
   {
     "pname": "NuGet.Configuration",
-    "version": "6.13.2",
-    "hash": "sha256-z8VW1YdRDanyyRTDYRvRkSv/XPR3c/hMM1y8cNNjx0Y="
+    "version": "6.14.0",
+    "hash": "sha256-1PN9s6fhCw3wd2260U6hQ4vG3jIvcG8GIn1oQgxMXA0="
   },
   {
     "pname": "NuGet.Frameworks",
-    "version": "6.13.2",
-    "hash": "sha256-caDyc+WgYOo43AUTjtbP0MyvYDb6JweEKDdIul61Cac="
+    "version": "6.14.0",
+    "hash": "sha256-3ViM3R1ucQMEL2hQYsivT86kI6veMQK2xDsiAcFcVQk="
   },
   {
     "pname": "NuGet.Packaging",
-    "version": "6.13.2",
-    "hash": "sha256-lhO+SFwIYZ4aPHxIGm5ubkkE2a5Ve2xgtroRbNh7hpw="
+    "version": "6.14.0",
+    "hash": "sha256-Yafbnxs3maj55bJ1oKQiZ0QkntFUzXdhorL94YEUOhY="
   },
   {
     "pname": "NuGet.Protocol",
-    "version": "6.13.2",
-    "hash": "sha256-5lnAHHZjy7A4vgv65AeBAs64mSNpuoUjxW3HnrMpuzY="
+    "version": "6.14.0",
+    "hash": "sha256-uLDKfs+QN1MdnuQtTES8qfNzzsmYKM6XB9pwJc4G+eo="
   },
   {
     "pname": "NuGet.Versioning",
-    "version": "6.13.2",
-    "hash": "sha256-gmpyBpKnt+GHqgx/2uFKp+J2csbxEAy1E7WdVT117sw="
+    "version": "6.14.0",
+    "hash": "sha256-DqdOJgsphKxSvqB8b60zNPCaiLfbiu3WnUJ/90feLrY="
   },
   {
     "pname": "NUnit",
@@ -224,25 +219,15 @@
     "version": "6.0.0",
     "hash": "sha256-KaMHgIRBF7Nf3VwOo+gJS1DcD+41cJDPWFh+TDQ8ee8="
   },
-  {
-    "pname": "System.Formats.Asn1",
-    "version": "8.0.1",
-    "hash": "sha256-may/Wg+esmm1N14kQTG4ESMBi+GQKPp0ZrrBo/o6OXM="
-  },
   {
     "pname": "System.IO.Abstractions",
-    "version": "4.2.13",
-    "hash": "sha256-nkC/PiqE6+c1HJ2yTwg3x+qdBh844Z8n3ERWDW8k6Gg="
+    "version": "22.0.15",
+    "hash": "sha256-2deBvDALOzd+BAnhdbnR7ZPjChE71HPv7w61/2tfYOg="
   },
   {
     "pname": "System.IO.Abstractions.TestingHelpers",
-    "version": "4.2.13",
-    "hash": "sha256-WGGatXlgyROnptdw0zU3ggf54eD/zusO/fvtf+5wuPU="
-  },
-  {
-    "pname": "System.IO.FileSystem.AccessControl",
-    "version": "4.5.0",
-    "hash": "sha256-ck44YBQ0M+2Im5dw0VjBgFD1s0XuY54cujrodjjSBL8="
+    "version": "22.0.15",
+    "hash": "sha256-JRm8yApCvhB/cvkPcm3+SKURhVB+ykF1u3IrxSJ7CLQ="
   },
   {
     "pname": "System.IO.Pipelines",
@@ -294,11 +279,6 @@
     "version": "6.1.0",
     "hash": "sha256-NyqqpRcHumzSxpsgRDguD5SGwdUNHBbo0OOdzLTIzCU="
   },
-  {
-    "pname": "System.Security.AccessControl",
-    "version": "4.5.0",
-    "hash": "sha256-AFsKPb/nTk2/mqH/PYpaoI8PLsiKKimaXf+7Mb5VfPM="
-  },
   {
     "pname": "System.Security.Cryptography.Pkcs",
     "version": "6.0.4",
@@ -309,11 +289,6 @@
     "version": "4.4.0",
     "hash": "sha256-Ri53QmFX8I8UH0x4PikQ1ZA07ZSnBUXStd5rBfGWFOE="
   },
-  {
-    "pname": "System.Security.Principal.Windows",
-    "version": "4.5.0",
-    "hash": "sha256-BkUYNguz0e4NJp1kkW7aJBn3dyH9STwB5N8XqnlCsmY="
-  },
   {
     "pname": "System.Text.Encodings.Web",
     "version": "9.0.0",
@@ -338,5 +313,30 @@
     "pname": "System.Threading.Tasks.Extensions",
     "version": "4.5.4",
     "hash": "sha256-owSpY8wHlsUXn5xrfYAiu847L6fAKethlvYx97Ri1ng="
+  },
+  {
+    "pname": "TestableIO.System.IO.Abstractions",
+    "version": "22.0.15",
+    "hash": "sha256-6YwnBfAnsxM0lEPB2LOFQcs7d1r7CyqjDEmvUBTz+X0="
+  },
+  {
+    "pname": "TestableIO.System.IO.Abstractions.TestingHelpers",
+    "version": "22.0.15",
+    "hash": "sha256-xEmfPBCtVVLc7K494cUuPXIYbUc/GPQlFC7UkDXP2jM="
+  },
+  {
+    "pname": "TestableIO.System.IO.Abstractions.Wrappers",
+    "version": "22.0.15",
+    "hash": "sha256-KoGuXGzecpf4rTmEth4/2goVFFR9V2aj+iibfZxpR7U="
+  },
+  {
+    "pname": "Testably.Abstractions.FileSystem.Interface",
+    "version": "9.0.0",
+    "hash": "sha256-6JW+qDtqQT9StP4oTR7uO0NnmVc2xcjSZ6ds2H71wtg="
+  },
+  {
+    "pname": "WoofWare.NUnitTestRunner",
+    "version": "0.3.4",
+    "hash": "sha256-OaBYMEAXUDiz9ei2/Zg4Q1A8BNDK1oaMB44uVz4UG/0="
   }
 ]