ratefeed

methodology · last revised 2026-07-09

Methodology

Where every number comes from.

This page names every source, walks through the extraction pipeline, and states the guarantees on every record. If a number fails a spot check, the corrections policy at the bottom states what happens next.

§1Sources

Every record starts at a regulator.

We collect from state departments of insurance and federal publications only. Nothing is licensed from data vendors, nothing is inferred from private feeds, and nothing enters the database without a source URL.

SourceWhat we takeScope
SERFF Filing Access (NAIC)
filingaccess.serff.com
Rate and rule filing metadata: carrier, SERFF tracking number, filing type, status, submission and disposition dates, document lists. 49 jurisdictions (48 states + DC)
Florida OIR IRFS
irfssearch.floir.gov
Florida filing metadata and detail. Florida does not participate in SERFF Filing Access, so it gets its own client. FL
Louisiana DOI Rate Filing Search
ldi.la.gov roadmap
Louisiana P&C filings. Louisiana's SERFF instance exposes health filings only, so homeowners metadata requires this custom path. Until that build ships, Louisiana is excluded from the sweep: the one gap in the 51-jurisdiction coverage. LA
California DOI weekly public notices
insurance.ca.gov
The CDI's weekly published list of rate applications: requested rate change percentages and prior approvals, straight from the regulator's own spreadsheet. CA
California FAIR Plan
cfpnet.com
Insurer-of-last-resort enrollment statistics, refreshed as published. CA
US Senate Budget Committee County-level homeowners non-renewal dataset from the committee's 2024 climate-insurance investigation. National, 2018–2023
US Treasury, Federal Insurance Office ZIP-level homeowners premium and claims metrics from the FIO's national climate data call. National, 2018–2022

Filing documents themselves (actuarial memoranda, rate applications) are fetched from the same portals, per filing, when the extraction pipeline needs them.

§2Extraction

Deterministic first. A language model only where templates end. Evidence always.

Rate filings mix structured forms with free-form actuarial prose, so the pipeline runs in ordered passes:

1. Deterministic parsers run first

Where a state uses a standardized form, we parse it directly with no model in the loop. California's rate-application template, for example, yields requested rate changes, policy counts, and effective dates as plain cell reads. Deterministic values are the highest-trust tier in the merge.

2. The LLM pass is evidence-bound

Fields the templates cannot reach (indicated rates buried in actuarial memoranda, territory detail, count bases) go to a language model under a strict contract: one PDF per call, structured output enforced by the schema, and for every non-null value the model must cite the 1-indexed PDF page number and a verbatim snippet supporting it. A value without evidence does not enter the database.

3. Confidence grading and merge rules

Every extracted field carries a confidence grade: high, medium, or low. When multiple passes disagree, the higher-confidence non-null value wins, and nulls never overwrite. Requested, indicated, and approved rate changes are kept as distinct fields because conflating them is the most common error in this domain.

4. Golden-set testing in CI

We maintain a golden set of hand-verified filings per state, checked value by value against the source PDFs. The extraction suite runs against it in continuous integration; changes to the pipeline do not ship unless the golden set passes. Human spot-check sampling of fresh extractions continues on top of that.

§3Provenance

The provenance guarantee.

Every record served by the API carries a provenance object: the source_url pointing at the regulator's own system, and retrieved_at, the timestamp we fetched it. Not at the response level. On each record.

"provenance": {
  "source_url": "https://filingaccess.serff.com/sfa/search/filingSummary.xhtml?filingId=...",
  "retrieved_at": "2026-07-08T04:08:36Z"
}

Follow the URL, open the filing, and check any value against the regulator's copy.

§4Access posture

How we treat public infrastructure.

State filing portals are a public good run on public budgets. Collection runs inside fixed constraints:

  • Rate-limited. Requests are spaced seconds apart, enforced in code, not in configuration someone can forget.
  • Incremental. Daily sweeps use a rolling lookback window with idempotent upserts. We fetch what changed; we do not re-download archives.
  • Browser-equivalent access. Pages are fetched the way a person at a browser would fetch them, through the portals' public search interfaces, with no circumvention of any access control.
  • Stop-first. If a portal signals distress or a regulator objects, the sweep for that state stops before any discussion starts.
§5Corrections

Corrections policy.

A wrong rate number is worse than a missing one. If you find an error in any ratefeed record:

  • Email hello@ratefeed.io with the record and what you believe is wrong.
  • We verify against the source document and fix confirmed errors within 24 hours.
  • Every confirmed correction is logged below, permanently. We do not silently rewrite history.

Corrections log

No corrections filed to date (log opened 2026-07-08). This line changes the day the first one lands.