ndaal Informational Advisory: v0.1.40 — EPSS/KEV enrichment feeder switches from community KEV-restricted GitHub CSV to FIRST.org bulk dataset; KEV flag now accurately cross-referenced against the CISA KEV catalog (table grows from ~1,200 to ~300,000+ rows; kev: true no longer hard-coded)

Metadata

Vulnerabilities (0)

No vulnerabilities.

Raw JSON

{
  "$schema": "https://docs.oasis-open.org/csaf/csaf/v2.1/schema/csaf.json",
  "document": {
    "category": "csaf_informational_advisory",
    "csaf_version": "2.1",
    "distribution": {
      "tlp": {
        "label": "CLEAR"
      }
    },
    "lang": "en",
    "notes": [
      {
        "category": "summary",
        "text": "nvulnlookup v0.1.40 corrects a data-source defect in the EPSS/KEV enrichment feeder (`crates/vl-feeders/src/feeders/epss_kev_enrichment.rs`).  Releases v0.1.0 through v0.1.39 fetched a community-maintained GitHub CSV (`https://raw.githubusercontent.com/jgamblin/KEV_EPSS/main/epss_kev_nvd.csv`) that pre-joins the CISA Known Exploited Vulnerabilities catalog with the EPSS scores for THAT specific subset — i.e. roughly 1,200 CVE rows out of the ~300,000+ CVEs that have an EPSS score in the official FIRST.org bulk dataset.  Net effect on a v0.1.39 production deployment: the `ENRICHMENT_EPSS_KEV` redb table was populated with EPSS data for only the ~1,200 KEV-listed CVEs, and every imported row was hard-coded with `kev: true` regardless of whether the CVE was actually on the CISA KEV catalog.  v0.1.40 switches to the official FIRST.org bulk dataset (`https://epss.cyentia.com/epss_scores-current.csv.gz`, gzipped, daily refresh, ~300,000+ entries) and cross-references each row against a freshly fetched CISA KEV catalog (`https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json`) so `kev: true` is set if and only if the CVE is on the KEV list.  Net effect on a v0.1.40 fresh run: the enrichment table grows from ~1,200 rows to ~300,000+ rows, and the `kev` flag accurately reflects CISA KEV membership.",
        "title": "Summary"
      },
      {
        "category": "description",
        "text": "## Defect 1 — wrong data source\n\nThe v0.1.0..v0.1.39 EPSS/KEV feeder fetched:\n\n  https://raw.githubusercontent.com/jgamblin/KEV_EPSS/main/epss_kev_nvd.csv\n\nThis is a community-maintained GitHub CSV that joins:\n\n  * CISA KEV — the ~1,200-entry CISA Known Exploited Vulnerabilities catalog\n  * EPSS    — but only for those same ~1,200 KEV CVEs\n\nIn other words: the upstream CSV is intentionally KEV-restricted.  The feeder consumed the CSV verbatim, so our `ENRICHMENT_EPSS_KEV` redb table contained EPSS data for ~1,200 CVEs out of the ~300,000+ CVEs that have an EPSS score in the official FIRST.org dataset.\n\nObservable downstream symptom: a CVE such as CVE-2026-25866 (MobaXterm unquoted-path) has an EPSS score of 0.00017 visible on CIRCL (which fetches from FIRST.org) but no EPSS entry in our API, because the CVE is not on the CISA KEV list.\n\n## Defect 2 — hard-coded `kev: true`\n\nFor every row imported from the GitHub CSV, the feeder built the storage payload with `kev` literal-true:\n\n  // BEFORE (epss_kev_enrichment.rs, v0.1.39)\n  let data = serde_json::json!({\n      \"epss\": epss,\n      \"percentile\": percentile,\n      \"cvss3\": cvss3,\n      \"kev\": true,\n  });\n\nThis was correct in the input data — every row in the GitHub CSV IS a KEV CVE — but the storage layer cannot tell that the upstream filter was the source of truth.  Once we switched to the FIRST.org bulk dataset (which is NOT KEV-restricted), keeping the `kev: true` literal would have flagged every CVE in the database as KEV, which would be a strictly worse error than the v0.1.39 baseline.\n\n## Fix landed in v0.1.40\n\n  * URL constant changed:\n      EPSS_BULK_URL = \"https://epss.cyentia.com/epss_scores-current.csv.gz\"\n      CISA_KEV_URL  = \"https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json\"\n\n  * Gzip decompression added:\n      use flate2::read::GzDecoder;\n      let cursor  = std::io::Cursor::new(gz_bytes);\n      let decoder = GzDecoder::new(cursor);\n      let reader  = BufReader::new(decoder);\n\n  * KEV cross-reference: a HashSet<String> of lower-cased CVE IDs\n    is built from the CISA KEV JSON.  For each EPSS row the storage\n    payload is now:\n\n      let kev = kev_set.contains(&cve_id);\n      let data = serde_json::json!({\n          \"epss\": epss,\n          \"percentile\": percentile,\n          \"kev\": kev,\n      });\n\n  * KEV-fetch failure degrades safely: if the CISA KEV download\n    or JSON parse fails, the feeder logs a warning and proceeds\n    with `kev_set = HashSet::new()`, which means every imported\n    row gets `kev: false`.  This is strictly safer than the\n    v0.1.39 hard-coded `kev: true` behaviour.\n\n  * The `cvss3` field is no longer carried in the enrichment\n    payload — the FIRST.org bulk dataset only ships `(cve, epss,\n    percentile)`.  Consumers that want CVSS should join against\n    the dedicated CVSS enrichment table.\n\n  * Dependency: `flate2 = { workspace = true }` added to\n    `crates/vl-feeders/Cargo.toml` (the workspace root already\n    pinned `flate2 = \"1\"`).\n\n## Test coverage\n\n  * `cargo check -p vl-feeders` — clean.\n  * `cargo test -p vl-feeders --lib epss_kev` — 5 / 5 pass:\n      test_feeder_construction\n      test_epss_kev_store_and_retrieve\n      test_epss_kev_clear_and_reimport\n      test_epss_kev_distinct_kev_flag\n      test_cisa_kev_json_parses     (NEW — guards CISA KEV schema drift)\n\n  * `test_epss_kev_distinct_kev_flag` is a regression test for\n    Defect 2: it stores five CVEs with alternating kev:true/false\n    flags and asserts the storage layer round-trips both values.\n    A future regression to `kev: true` literal would fail this\n    test before reaching production.",
        "title": "Description"
      },
      {
        "category": "details",
        "text": "## Operational impact assessment\n\n  * Production `vl-web` server runtime — IMPACTED.\n    Operators upgrading 0.1.39 → 0.1.40 with auditd / KEV / EPSS\n    consumers wired to the `/api/v1/enrichment/epss_kev/<cve>`\n    endpoint will see substantially more entries (table grows from\n    ~1,200 to ~300,000+) and substantially fewer false-positive\n    KEV flags (every row was previously `kev: true`; now only\n    rows for CVEs on the CISA KEV catalog are flagged).\n\n  * Production `vl-cli` runtime — IMPACTED in the same way.\n    `vl-cli enrichment epss-kev --cve <id>` returns data for\n    300,000+ CVE IDs that previously returned 404.  Entries that\n    return data now show the correct `kev` boolean.\n\n  * Database file format on disk — UNCHANGED.\n    The redb table schema (`ENRICHMENT_EPSS_KEV`) is unchanged.\n    The serde_json payload shape lost the `cvss3` field but kept\n    `epss` (f64), `percentile` (f64), and `kev` (bool) — the\n    fields downstream consumers actually use.  The first run of\n    the v0.1.40 feeder calls `clear_epss_kev()` before\n    re-population, so old `cvss3`-bearing entries are removed in\n    one redb write transaction; no on-disk migration is required.\n\n  * Storage size — INCREASED.\n    The on-disk redb file gains roughly 250x more EPSS rows.\n    Each row stores ~80 bytes of JSON; 300,000 × 80 ≈ 24 MB of\n    additional storage in the `ENRICHMENT_EPSS_KEV` table.  This\n    is well under any reasonable disk budget.\n\n  * Bandwidth at fetch time — DOUBLED ROUGHLY.\n    The feeder now downloads the FIRST.org bulk gz (~30 MB\n    gzipped, ~120 MB decompressed) plus the CISA KEV JSON\n    (~1 MB).  The previous community CSV was ~150 KB.\n    Daily refresh remains the documented cadence, so the\n    incremental traffic is single-digit MB/day for the marginal\n    daily delta after the first fetch.\n\n  * Dependency surface — `flate2` v1 added to vl-feeders.\n    The crate was already a transitive dependency of\n    `reqwest`, `git2`, and `cargo` machinery — it is not new\n    to the workspace.\n\n  * Operator deployments via the Ansible role\n    (`tools/ansible/roles/{nvulnlookup,nvulnlookuptesting}`) —\n    UNAFFECTED at the deploy layer.  The role copies the binary\n    and starts the systemd unit; the EPSS feeder runs at the\n    documented cadence inside the daemon and does not require\n    any new role variables.\n\nSecurity classification: NOT a security vulnerability under the\n  CVSS framework.  The defect did not expose data, did not allow\n  privilege escalation, did not crash the daemon, and did not\n  create an unauthenticated attack surface.\n\n  However, it WAS a security-relevant correctness bug: the\n  hard-coded `kev: true` flag could have led downstream\n  prioritisation tooling (SOAR / vulnerability-management\n  pipelines that key on `kev`) to treat ~1,200 CVEs as\n  CISA-confirmed actively-exploited when in fact every CVE that\n  ever appeared in our enrichment table was being flagged this\n  way.  In practice, those rows WERE on the CISA KEV catalog\n  (because the upstream CSV was KEV-restricted), so the bug was\n  silent.  v0.1.40 closes the latent risk before it can manifest\n  on a future upstream-data-source change.\n\nThis advisory is filed as `csaf_informational_advisory` per\nOASIS CSAF 2.1 to record a data-quality and correctness fix in\nthe v0.1.40 release cycle.  No CVE is assigned because the\ndefect did not introduce an exploitable vulnerability in the\nproduct itself.",
        "title": "Operational impact"
      },
      {
        "category": "general",
        "text": "## Recommended action for downstream consumers\n\nUPGRADE: 0.1.39 → 0.1.40.  Either:\n\n  * Pull the v0.1.40 release tarball from\n    `release/v0.1.40/` (per-triple) once published.\n  * Update the Ansible role pin from 0.1.39 → 0.1.40 (already\n    done in this commit) and re-run\n    `ansible-playbook -i inventory site.yml -e\n    nvulnlookup_version=0.1.40`.\n  * Or `cargo install --path vulnerability-lookup-rs/crates/vl-web`\n    from the v0.1.40 git tag.\n\nAfter upgrading, the EPSS/KEV feeder's next scheduled run\n(daily) will:\n\n  1. `clear_epss_kev()` — remove the ~1,200 stale rows.\n  2. Download the FIRST.org bulk dataset (~30 MB gzipped) and\n     the CISA KEV catalog (~1 MB JSON).\n  3. Populate ~300,000+ rows with accurate `kev` flags.\n\nNo manual intervention is required.  The `last_update` key in\nthe redb metadata table is set to the new run's timestamp so\nmonitoring dashboards that key on `last_update` will see the\nrefresh land.\n\n## Verification\n\n  # Confirm the feeder code is on v0.1.40:\n  grep -n 'EPSS_BULK_URL\\|CISA_KEV_URL' \\\n      vulnerability-lookup-rs/crates/vl-feeders/src/feeders/epss_kev_enrichment.rs\n  # Expected: both URL constants present, both pointing at the\n  # canonical sources (epss.cyentia.com + www.cisa.gov).\n\n  # Confirm the test that guards Defect 2 passes:\n  cargo test -p vl-feeders --lib epss_kev_enrichment::tests::test_epss_kev_distinct_kev_flag\n\n  # On a running v0.1.40 server, count the rows:\n  vl-cli enrichment epss-kev --count\n  # Expected: between 280,000 and 320,000 (the FIRST.org dataset\n  # size; varies daily as new CVEs land and EPSS scores update).\n\n  # On a running v0.1.40 server, sample a non-KEV CVE that has an\n  # EPSS score:\n  curl -sk 'https://localhost:8080/api/v1/enrichment/epss_kev/cve-2026-25866' \\\n      | jq '{epss, percentile, kev}'\n  # Expected: kev: false (the CVE is not on the CISA KEV list as\n  # of the catalog version current at the time of the fetch).\n\n  # On a running v0.1.40 server, sample a CVE on the KEV list:\n  curl -sk 'https://localhost:8080/api/v1/enrichment/epss_kev/cve-2021-44228' \\\n      | jq '.kev'\n  # Expected: true (Log4Shell is a textbook KEV entry).\n\n## Rollback\n\nNot recommended.  v0.1.39 had the data-quality bug described\nabove.  If you must roll back for an unrelated reason, restore\nthe v0.1.39 binary; the next feeder run will repopulate the\ntable with the v0.1.39 (KEV-restricted, hard-coded-true)\ndata.  No on-disk schema migration is required to roll back.\nv0.1.40 introduced no new redb tables.",
        "title": "Recommended action"
      },
      {
        "category": "other",
        "text": "Disposition: informational — data-source correction + KEV-flag accuracy fix in the EPSS/KEV enrichment feeder, no exploitable vulnerability in the product itself.  Status final.  Followed up by the v0.1.40 release cycle.",
        "title": "Disposition"
      },
      {
        "category": "legal_disclaimer",
        "text": "THIS DOCUMENT IS PROVIDED ON AN 'AS IS' BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY. ndaal Gesellschaft fuer Sicherheit in der Informationstechnik mbH & Co KG DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS INFORMATION INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS."
      }
    ],
    "publisher": {
      "category": "vendor",
      "contact_details": "security@ndaal.eu",
      "issuing_authority": "ndaal Gesellschaft fuer Sicherheit in der Informationstechnik mbH & Co KG Security Team",
      "name": "ndaal Gesellschaft fuer Sicherheit in der Informationstechnik mbH & Co KG",
      "namespace": "https://ndaal.eu/csaf"
    },
    "references": [
      {
        "category": "self",
        "summary": "This advisory in CSAF 2.1 format",
        "url": "https://gitlab.com/vPierre/ndaal_public_nvulnlookup/-/raw/main/csaf/2026/038/ndaal-sa-2026-038.json"
      },
      {
        "category": "external",
        "summary": "Companion v0.1.40 in-memory storage advisory ndaal-sa-2026-037",
        "url": "https://gitlab.com/vPierre/ndaal_public_nvulnlookup/-/raw/main/csaf/2026/037/ndaal-sa-2026-037.json"
      },
      {
        "category": "external",
        "summary": "FIRST.org EPSS — Exploit Prediction Scoring System",
        "url": "https://www.first.org/epss/"
      },
      {
        "category": "external",
        "summary": "FIRST.org EPSS bulk dataset (canonical CSV download, gzip-compressed, daily refresh)",
        "url": "https://epss.cyentia.com/epss_scores-current.csv.gz"
      },
      {
        "category": "external",
        "summary": "CISA Known Exploited Vulnerabilities catalog (canonical JSON feed)",
        "url": "https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json"
      },
      {
        "category": "external",
        "summary": "CISA KEV catalog homepage",
        "url": "https://www.cisa.gov/known-exploited-vulnerabilities-catalog"
      },
      {
        "category": "external",
        "summary": "jgamblin/KEV_EPSS GitHub repository (the previous, KEV-restricted upstream)",
        "url": "https://github.com/jgamblin/KEV_EPSS"
      },
      {
        "category": "external",
        "summary": "flate2 crate documentation (gzip / deflate / zlib for Rust)",
        "url": "https://docs.rs/flate2/"
      },
      {
        "category": "external",
        "summary": "nvulnlookup CHANGELOG (root)",
        "url": "https://gitlab.com/vPierre/ndaal_public_nvulnlookup/-/raw/main/CHANGELOG.md"
      },
      {
        "category": "external",
        "summary": "nvulnlookup CHANGELOG (workspace mirror)",
        "url": "https://gitlab.com/vPierre/ndaal_public_nvulnlookup/-/raw/main/vulnerability-lookup-rs/CHANGELOG.md"
      }
    ],
    "title": "ndaal Informational Advisory: v0.1.40 — EPSS/KEV enrichment feeder switches from community KEV-restricted GitHub CSV to FIRST.org bulk dataset; KEV flag now accurately cross-referenced against the CISA KEV catalog (table grows from ~1,200 to ~300,000+ rows; kev: true no longer hard-coded)",
    "tracking": {
      "current_release_date": "2026-05-07T13:15:00.000Z",
      "generator": {
        "engine": {
          "name": "ndaal CSAF Generator",
          "version": "1.0.0"
        }
      },
      "id": "ndaal-sa-2026-038",
      "initial_release_date": "2026-05-07T13:15:00.000Z",
      "revision_history": [
        {
          "date": "2026-05-07T13:15:00.000Z",
          "number": "1.0.0",
          "summary": "Initial informational advisory documenting the v0.1.40 EPSS/KEV enrichment feeder data-source correction in `crates/vl-feeders/src/feeders/epss_kev_enrichment.rs`.  Two defects fixed in the same rewrite: (Defect 1) the v0.1.0..v0.1.39 feeder fetched a community-maintained GitHub CSV (`jgamblin/KEV_EPSS/main/epss_kev_nvd.csv`) that pre-joins CISA KEV ∩ EPSS — i.e. only ~1,200 of the 300,000+ CVEs that have an EPSS score; (Defect 2) every imported row was hard-coded with `kev: true` regardless of actual KEV catalog membership.  v0.1.40 switches to the FIRST.org official bulk dataset (`https://epss.cyentia.com/epss_scores-current.csv.gz`, gzipped, daily refresh) and cross-references each row against a freshly fetched CISA KEV catalog (`https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json`).  KEV-fetch failure degrades safely to `kev: false` everywhere — strictly safer than the previous hard-coded-true behaviour.  Dependency: `flate2 = { workspace = true }` added to `crates/vl-feeders/Cargo.toml`.  The `cvss3` field is removed from the enrichment payload (the FIRST.org bulk dataset doesn't carry it).  Net effect on a fresh production run: the `ENRICHMENT_EPSS_KEV` table grows from ~1,200 rows to ~300,000+ rows; the `kev` flag accurately reflects CISA KEV membership.  Test coverage: 5 unit tests in `epss_kev_enrichment::tests` pass, including the new `test_epss_kev_distinct_kev_flag` regression test that guards Defect 2 and `test_cisa_kev_json_parses` that guards CISA KEV schema drift.  Status: final."
        }
      ],
      "status": "final",
      "version": "1.0.0"
    }
  },
  "product_tree": {
    "branches": [
      {
        "category": "vendor",
        "name": "ndaal Gesellschaft fuer Sicherheit in der Informationstechnik mbH & Co KG",
        "branches": [
          {
            "category": "product_name",
            "name": "nvulnlookup",
            "branches": [
              {
                "category": "product_version_range",
                "name": "vers:cargo/>=0.1.0|<=0.1.39",
                "product": {
                  "name": "nvulnlookup 0.1.0 through 0.1.39 — EPSS/KEV feeder fetches `jgamblin/KEV_EPSS/main/epss_kev_nvd.csv` (KEV-restricted, ~1,200 rows) and hard-codes `kev: true` on every imported row",
                  "product_id": "CSAFPID-0001"
                }
              },
              {
                "category": "product_version",
                "name": "0.1.40",
                "product": {
                  "name": "nvulnlookup 0.1.40 — EPSS/KEV feeder fetches `https://epss.cyentia.com/epss_scores-current.csv.gz` (FIRST.org bulk, ~300,000+ rows, gzipped) and cross-references each row against the CISA KEV catalog at `https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json` for accurate `kev` flagging",
                  "product_id": "CSAFPID-0002"
                }
              }
            ]
          }
        ]
      }
    ]
  }
}