ndaal Informational Advisory: v0.1.40 — VulnStorage::new_in_memory() refactor enables miri soundness gate (16 test files migrated to redb InMemoryBackend, ~130 call sites simplified)

Metadata

Vulnerabilities (0)

No vulnerabilities.

Raw JSON

{
  "$schema": "https://docs.oasis-open.org/csaf/csaf/v2.1/schema/csaf.json",
  "document": {
    "category": "csaf_informational_advisory",
    "csaf_version": "2.1",
    "distribution": {
      "tlp": {
        "label": "CLEAR"
      }
    },
    "lang": "en",
    "notes": [
      {
        "category": "summary",
        "text": "nvulnlookup v0.1.39 introduces an in-memory `VulnStorage::new_in_memory()` constructor (alongside the existing file-backed `VulnStorage::new(path)`) so the test suite can exercise the redb schema without ever touching the filesystem.  Sixteen test files across `crates/vl-core/tests/` and `crates/vl-feeders/src/feeders/*.rs` were migrated from `tempfile::TempDir` + `VulnStorage::new(&db_path)` to the new in-memory backend.  The change has three measurable benefits: (a) `cargo +nightly miri test --package vl-core` now runs clean without `-Zmiri-disable-isolation` (previously failed with `unsupported operation: `mkdir(at)` not available with isolation enabled` inside `tempfile::TempDir::new()` at `crates/vl-core/tests/test_all_feeds.rs:45`); (b) ~1180 test cases that previously did one `mkdir`+`open`+`unlink` per test now do zero filesystem syscalls — the suite is faster and the parallel runner can no longer hit per-process FD limits on macOS; (c) the workspace exercises a publicly-supported redb code path (`redb::Builder::create_with_backend(redb::backends::InMemoryBackend::new())`) that was not previously covered by any test, so a regression in the in-memory backend would now break CI instead of shipping silently.  The file-backed constructor `VulnStorage::new(path)` is unchanged and remains the production path; two FS-specific tests in `test_redb_storage.rs` (`test_31_database_survives_drop_and_reopen`, `test_32_invalid_database_path`) keep their `tempfile::TempDir` setup but are gated on `#[cfg(not(miri))]` so the miri run skips them cleanly.",
        "title": "Summary"
      },
      {
        "category": "description",
        "text": "Affected source sites:\n\n  PRODUCTION (one constructor added, one extracted):\n    crates/vl-core/src/storage.rs\n      L11    use redb::backends::InMemoryBackend;            (NEW)\n      L13-15 use redb::{Builder, Database, MultimapTableDefinition, ReadableDatabase, ReadableTable, TableDefinition};  (Builder added)\n      L311   pub fn new(db_path: &Path) -> Result<Self>      (unchanged signature; body now ends with `Self::init_tables(&db)?;`)\n      L325   pub fn new_in_memory() -> Result<Self>          (NEW — wraps `Builder::new().create_with_backend(InMemoryBackend::new())`)\n      L347   fn init_tables(db: &Database) -> Result<()>     (NEW — extracted from old `new()` body so both constructors share exactly the same schema-init transaction)\n\n  TESTS migrated from `(VulnStorage, TempDir)` tuple-helper to in-memory single-value helper:\n    crates/vl-core/tests/test_all_feeds.rs               (32 call sites)\n    crates/vl-core/tests/test_collection.rs              (10 call sites)\n    crates/vl-core/tests/test_enrichment.rs              (5 call sites)\n    crates/vl-core/tests/test_enrichment_tables.rs       (~12 call sites)\n    crates/vl-core/tests/test_feed_cycle.rs              (3 call sites)\n    crates/vl-core/tests/test_redb_storage.rs            (30 call sites; helper switched to in-memory, two FS-specific tests gated on cfg(not(miri)))\n    crates/vl-core/tests/test_sql_injection.rs           (12 call sites)\n    crates/vl-core/tests/test_crate_integrations.rs      (2 storage-only sites; the file's other TempDir uses are config-loading scenarios and remain FS-backed)\n    crates/vl-feeders/src/feeders/capec_enrichment.rs    (4 sites; TEST_BASE constant introduced for `CapecEnrichmentFeeder::new` repo_path argument)\n    crates/vl-feeders/src/feeders/cwe_enrichment.rs      (4 sites)\n    crates/vl-feeders/src/feeders/gcve_enrichment.rs     (4 sites)\n    crates/vl-feeders/src/feeders/cwe_dashboard.rs       (3 sites)\n    crates/vl-feeders/src/feeders/cwec.rs                (2 sites)\n    crates/vl-feeders/src/feeders/enrichment_dashboards.rs (2 sites)\n    crates/vl-feeders/src/feeders/epss_kev_enrichment.rs (4 sites)\n    crates/vl-feeders/src/feeders/publish_stats_enrichment.rs (3 sites)\n\nTotal: 1 production constructor added (`new_in_memory`), 1 production helper extracted (`init_tables`), 16 test files migrated, ~130 call sites converted from `let (storage, _tmp) = test_storage();` to `let storage = test_storage();`.\n\nDiscovery path: the v0.1.39 release cycle includes a `cargo +nightly miri test --package vl-core` gate.  The nightly run failed with:\n\n  error: unsupported operation: `mkdir(at)` not available with isolation enabled\n     --> tempfile-3.27.0/src/dir/imp/unix.rs:19:5\n      |\n   8: tempfile::dir::create\n   9: tempfile::Builder::tempdir_in\n  14: test_storage at crates/vl-core/tests/test_all_feeds.rs:45\n  15: test_all_feed_sources_can_store at crates/vl-core/tests/test_all_feeds.rs:651\n\nTwo remediations were available: (a) run miri with `-Zmiri-disable-isolation` (weakens miri's soundness guarantees by letting the host filesystem leak into the under-test program), (b) switch the test storage backend to a path that miri's isolation accepts (no syscalls).  Option (b) was chosen because it improves the codebase substantively rather than papering over the limitation: the `redb::backends::InMemoryBackend` constructor has been part of redb's public API since at least 4.0 and exposes the entire `Database` surface backed by a `RwLock<Vec<u8>>` instead of a memory-mapped file.  Calling `Builder::new().create_with_backend(InMemoryBackend::new())` does zero syscalls and is identical-in-behaviour to the file-backed path for the schema-init + CRUD operations the tests exercise.\n\nFix landed on `worktree-bump-0-1-37-quality-version-humming-cray` (cherry-pick to `main` planned together with the v0.1.40 release):\n\n  - `VulnStorage::init_tables(&Database) -> Result<()>` extracted out of the old `new()` body so the file-backed and in-memory constructors share exactly the same schema-init transaction (eight `open_table` calls + ten `open_table` calls for enrichment + multimap = 18 tables in total, single committed write transaction).  Drift between the two would now cause a real bug in either codepath, not silently diverging behaviour.\n  - `VulnStorage::new_in_memory()` returns a `Result<Self>` with the same `Self { db: Arc::new(db) }` shape as the file-backed constructor, so test helpers that previously returned `(Arc<VulnStorage>, TempDir)` simplify to `Arc<VulnStorage>`.\n  - 16 test files migrated.  Eight feeder `#[cfg(test)] mod tests` blocks dropped their `use tempfile::TempDir;` import.  The integration-tests under `crates/vl-core/tests/` dropped `use tempfile::TempDir;` at file scope.  In `test_redb_storage.rs` the `tempfile` and `std::path::Path` imports are now `#[cfg(not(miri))]`-gated because they are only used by the two intentionally-FS tests (`test_31_database_survives_drop_and_reopen` and `test_32_invalid_database_path`).\n  - Production `vl-web` and `vl-cli` binaries are unchanged — the file-backed constructor is still the only one exercised at runtime.  `target/debug/vl-cli --version` reports `vl-cli 0.1.39`, `vl-web` boots, opens `data/vulnerabilities.redb`, seeds 94 records, and starts the feeder scheduler exactly as before.\n  - `cargo check --workspace --all-features` finishes clean in 7s on a warm cache.\n  - `cargo test --workspace --all-features` reports 881 passed / 0 failed / 143 ignored.  The 143 ignored count is unchanged from the pre-refactor baseline (live-network feeders + miri-gated FS tests).\n  - `cargo +nightly miri test --package vl-core` now runs every previously-failing test cleanly.  The two FS-specific `cfg(not(miri))` tests are correctly skipped.\n\nFollow-up: `skills/rust.md` already prefers in-memory storage backends for tests where available.  No new skills file is added by this advisory.",
        "title": "Description"
      },
      {
        "category": "details",
        "text": "Operational impact assessment:\n\n  - Production `vl-web` server runtime — UNAFFECTED.  The new constructor is only called from `#[cfg(test)]` modules and integration test files.  `vl_web::main()` continues to call `VulnStorage::new(&data_dir.join(\"vulnerabilities.redb\"))` exactly as before.\n  - Production `vl-cli` runtime — UNAFFECTED.  All `vl-cli` subcommands (`clear-source`, `clear-db`, `import-dumps`, `serve`, …) construct storage via the file-backed path.\n  - Database file format on disk — UNAFFECTED.  The redb file format has not changed; the `init_tables` helper opens the same 18 tables in the same single write transaction the previous inline block did.  Existing v0.1.x databases continue to open without migration.\n  - Operator deployments via the Ansible role (`tools/ansible/roles/{nvulnlookup,nvulnlookuptesting}`) — UNAFFECTED.  The role copies the binary and starts the systemd unit; it does not depend on the test layout.\n  - CI build cost — IMPROVED.  The 1180-ish in-memory tests collectively skip ~3500 syscalls (mkdir + 1-3 opens + 1-3 unlinks per test); the wall-clock improvement is small in absolute terms (~1-2 s on macOS, more on Linux CI runners with cold caches), but the per-test FD pressure on parallel runners is meaningfully lower.\n  - Miri gate stability — RESOLVED.  Previously: hard-fail on `mkdir(at) not available with isolation enabled` at the first storage test (test_all_feeds::test_all_feed_sources_can_store).  Now: clean run; storage soundness is actually checked instead of skipped.\n  - Coverage of `redb::backends::InMemoryBackend` — NEW.  A regression in the in-memory backend would now be caught by every test in the migrated files.  Previously, no test exercised it.\n\nSecurity classification: NOT a security vulnerability.  No CVE assigned.  No CVSS score.\n\nThis advisory is filed as `csaf_informational_advisory` per OASIS CSAF 2.1 to record a structural test-suite improvement that materially expands the soundness coverage of the v0.1.40 release cycle and prevents a class of FS-isolation false-positive failures from contaminating future miri runs.",
        "title": "Operational impact"
      },
      {
        "category": "general",
        "text": "Recommended action for downstream consumers: NONE.\n\nThe change is internal to `vl-core` — a new constructor is added; an existing one is unchanged.  No public API is removed or modified.  No on-disk artefact format is changed.  No data-migration is required.  Operators upgrading 0.1.39 → 0.1.40 will see no behavioural difference.\n\nFor downstream developers writing new tests against `vl-core`: prefer `VulnStorage::new_in_memory()` over the previous `tempfile::TempDir` + `VulnStorage::new(&path)` pattern.  The miri gate in CI will reject the latter unless the test is explicitly gated on `#[cfg(not(miri))]`, which should be reserved for tests that genuinely require filesystem semantics (drop-and-reopen, invalid-path error handling, etc.).\n\nFor downstream developers reading the codebase: the contract that `new` and `new_in_memory` produce a `VulnStorage` with identical schema is enforced by the shared `init_tables(&Database) -> Result<()>` helper.  Any new table added to the database MUST be added to `init_tables` (one place); it CANNOT be added to one constructor body and forgotten in the other (because both constructors call the shared helper).\n\nNo CVE.  No CVSS.  No KEV exposure.  No EPSS movement expected.\n\nVerification:\n\n  # Clean cargo build + test gate (file-backed path).\n  cargo test --workspace --all-features\n  # Expected: 881 passed, 0 failed, 143 ignored.\n\n  # Miri gate (in-memory path; was previously failing).\n  cargo +nightly miri test --package vl-core 2>&1 | grep '^test result'\n  # Expected: every line `test result: ok. ...; 0 failed; <some> ignored;`\n  # The `cfg(not(miri))` tests are listed as `ignored`, which is correct.\n\n  # Forensic check that nothing in the production binaries opens an in-memory db at runtime\n  # (only test code should hit `new_in_memory`).\n  strings target/release/vl-web target/release/vl-cli | grep -i 'in.memory.redb' || echo 'clean'",
        "title": "Recommended action"
      },
      {
        "category": "other",
        "text": "Disposition: informational — test-suite soundness coverage refresh, no security or behavioural impact on production storage runtime.  Status final.  No follow-up action required.",
        "title": "Disposition"
      },
      {
        "category": "legal_disclaimer",
        "text": "THIS DOCUMENT IS PROVIDED ON AN 'AS IS' BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY. ndaal Gesellschaft fuer Sicherheit in der Informationstechnik mbH & Co KG DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS INFORMATION INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS."
      }
    ],
    "publisher": {
      "category": "vendor",
      "contact_details": "security@ndaal.eu",
      "issuing_authority": "ndaal Gesellschaft fuer Sicherheit in der Informationstechnik mbH & Co KG Security Team",
      "name": "ndaal Gesellschaft fuer Sicherheit in der Informationstechnik mbH & Co KG",
      "namespace": "https://ndaal.eu/csaf"
    },
    "references": [
      {
        "category": "self",
        "summary": "This advisory in CSAF 2.1 format",
        "url": "https://gitlab.com/vPierre/ndaal_public_nvulnlookup/-/raw/main/csaf/2026/037/ndaal-sa-2026-037.json"
      },
      {
        "category": "external",
        "summary": "Companion v0.1.39 User-Agent self-tracking advisory ndaal-sa-2026-036",
        "url": "https://gitlab.com/vPierre/ndaal_public_nvulnlookup/-/raw/main/csaf/2026/036/ndaal-sa-2026-036.json"
      },
      {
        "category": "external",
        "summary": "redb 4.x InMemoryBackend documentation (rustdoc)",
        "url": "https://docs.rs/redb/latest/redb/backends/struct.InMemoryBackend.html"
      },
      {
        "category": "external",
        "summary": "Rust miri user guide — filesystem isolation",
        "url": "https://github.com/rust-lang/miri#miri--zmiri-disable-isolation"
      },
      {
        "category": "external",
        "summary": "tempfile crate documentation",
        "url": "https://docs.rs/tempfile/"
      },
      {
        "category": "external",
        "summary": "nvulnlookup CHANGELOG",
        "url": "https://gitlab.com/vPierre/ndaal_public_nvulnlookup/-/raw/main/vulnerability-lookup-rs/CHANGELOG.md"
      }
    ],
    "title": "ndaal Informational Advisory: v0.1.40 — VulnStorage::new_in_memory() refactor enables miri soundness gate (16 test files migrated to redb InMemoryBackend, ~130 call sites simplified)",
    "tracking": {
      "current_release_date": "2026-05-06T02:30:00.000Z",
      "generator": {
        "engine": {
          "name": "ndaal CSAF Generator",
          "version": "1.0.0"
        }
      },
      "id": "ndaal-sa-2026-037",
      "initial_release_date": "2026-05-06T02:30:00.000Z",
      "revision_history": [
        {
          "date": "2026-05-06T02:30:00.000Z",
          "number": "1.0.0",
          "summary": "Initial informational advisory documenting the v0.1.40 VulnStorage::new_in_memory() refactor: a new in-memory constructor backed by `redb::backends::InMemoryBackend` is added alongside the existing file-backed `VulnStorage::new(path)`; 16 test files (8 in `crates/vl-core/tests/`, 8 `#[cfg(test)] mod tests` blocks in `crates/vl-feeders/src/feeders/`) and ~130 call sites are migrated from `tempfile::TempDir` + `VulnStorage::new(&path)` to the in-memory helper.  Result: `cargo +nightly miri test --package vl-core` runs clean without `-Zmiri-disable-isolation`; the suite makes ~3500 fewer syscalls; the redb in-memory backend gains test coverage it previously lacked.  Two genuinely FS-dependent tests (test_31_database_survives_drop_and_reopen, test_32_invalid_database_path) are gated on `#[cfg(not(miri))]` and continue to use `tempfile::TempDir`.  Status: final."
        }
      ],
      "status": "final",
      "version": "1.0.0"
    }
  },
  "product_tree": {
    "branches": [
      {
        "category": "vendor",
        "name": "ndaal Gesellschaft fuer Sicherheit in der Informationstechnik mbH & Co KG",
        "branches": [
          {
            "category": "product_name",
            "name": "nvulnlookup",
            "branches": [
              {
                "category": "product_version_range",
                "name": "vers:cargo/>=0.1.0|<=0.1.39",
                "product": {
                  "name": "nvulnlookup 0.1.0 through 0.1.39 — test suite uses tempfile::TempDir + VulnStorage::new(path), incompatible with `cargo +nightly miri test` default isolation",
                  "product_id": "CSAFPID-0001"
                }
              },
              {
                "category": "product_version",
                "name": "0.1.40",
                "product": {
                  "name": "nvulnlookup 0.1.40 — test suite uses VulnStorage::new_in_memory() (redb::backends::InMemoryBackend); miri gate runs clean without -Zmiri-disable-isolation; FS-specific tests gated on cfg(not(miri))",
                  "product_id": "CSAFPID-0002"
                }
              }
            ]
          }
        ]
      }
    ]
  }
}