Audit Property Data Coverage Before Using It in Your Product

A schema tells you what can appear, not how often

Every property API documents fields. Far fewer tell you how often those fields are populated for the kind of properties your product handles, and a vendor-wide number would not answer that anyway. Coverage depends on the mix: property types, listing statuses, regions, units, and how clean your input addresses are. The only coverage figure that matters to your product is the one measured on inputs that look like your product's inputs.

This guide gives you the method and a script to measure two things: how many inputs resolve to the intended property, and how often the fields your product requires are supplied for those that do. It does not publish a coverage figure for PropertiesAPI. We have not run a documented live sample for this article, and the hand-built fixtures used to test the script say nothing about real-world availability.

Run it before you design screens or models that assume a field is always there. A field that arrives for most properties still needs an empty state for the rest.

Build a sample that represents your inputs

Start from the real population your product will see: last quarter's user submissions, the CRM list you plan to enrich, or the addresses a partner will send. Draw a random sample from it with a recorded seed so someone else can redraw the same rows. Hand-picking examples that you know work produces a demo, not an audit.

If some segments matter more, such as condos or rentals, sample within each segment and report each one separately rather than blending them. Fix the sample and the exclusion rules before you look at any results. Changing either after seeing the numbers is the most common way an audit stops being an audit.

Size matters for what you can say. With a small sample, report counts and an interval, not a bare percentage. The script reports a Wilson score interval for each rate, which behaves sensibly when a rate is close to the extremes or the sample is small.

• Write down the population the sample was drawn from and the date.
• Record the random seed and the selection code.
• List exclusions in advance, such as non-US addresses or PO boxes.
• Add an expected_zpid when you already know the right property, so wrong matches are caught.

Define outcomes before you call anything

Every input ends in exactly one outcome. Resolution is about identity; field availability is about the properties that resolved. Keep them separate: a correct property with a missing estimate is a different problem from a wrong property with every field filled.

Transient failures, such as rate limits, temporary upstream errors or timeouts, say nothing about coverage. The script keeps them in the per-row file but leaves them out of the denominator, and reports how many there were. If they are numerous, rerun those rows later instead of reporting a rate built on what happened to succeed.

Run the audit script

The script reads sample.csv, sends the addresses through the batch endpoint in chunks no larger than the documented batch limit, classifies every item, and writes two files. audit.csv has one row per input with the outcome, returned identity, cache flag, collection time and a present or absent marker for each required field. summary.json has the counts, the denominators, the intervals and a sentence describing the method.

Edit REQUIRED to match your product. A listing card might need price, homeStatus and photos; an investment screen might need zestimate, rentZestimate and livingArea. For a field that is a list, such as photos or priceHistory, an empty list counts as not supplied. Response metadata also includes meta.completeness, which reports supplied, not_returned or empty_array for a fixed set of fields; you can store it alongside your own check.

Cost and caching affect the run. Each successful live lookup consumes a credit, while failures do not. Repeating the audit inside the cache window returns cached results with cached set to true and the original fetched_at, so a rerun the same day does not measure anything new. Record the cached column and say so in your write-up.

• Batch request limits and item results
• Current plans and credit accounting

"""coverage_audit.py: identity resolution and required-field availability on YOUR sample.

Input: sample.csv with columns id, address, expected_zpid (expected_zpid may be blank).
Output: audit.csv (one row per input) and summary.json (counts, intervals, method).
"""
import csv
import json
import math
import os
import sys
from datetime import datetime, timezone

import requests

API = "https://api.propertiesapi.com/v1/zillow/batch"
REQUIRED = ["price", "zestimate", "livingArea", "yearBuilt", "photos", "priceHistory"]
CHUNK = 50  # the documented batch maximum
EXCLUDED = {"not_requested"}  # outcomes kept out of the resolution denominator


def wilson(k, n, z=1.96):
    """Wilson score interval for k successes in n trials, or None when n is 0."""
    if n == 0:
        return None
    p = k / n
    d = 1 + z * z / n
    centre = (p + z * z / (2 * n)) / d
    half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
    return [round(centre - half, 4), round(centre + half, 4)]


def classify(row, item):
    if item.get("ok"):
        data, meta = item["data"], item["meta"]
        if meta.get("warning"):
            return "identity_warning", data
        if row["expected_zpid"] and data.get("zpid") != row["expected_zpid"]:
            return "wrong_property", data
        return "resolved", data
    code = item["error"]["code"]
    if code in ("address_ambiguous", "zpid_mismatch"):
        return "ambiguous", None
    if code == "not_found":
        return "not_found", None
    if code in ("bad_param", "unprocessable_input"):
        return "rejected_input", None
    return "transient:" + code, None


def main(sample_path):
    rows = list(csv.DictReader(open(sample_path, newline="", encoding="utf-8")))
    session = requests.Session()
    session.headers["x-api-key"] = os.environ["PROPERTIESAPI_KEY"]
    results, stopped = [], None
    for start in range(0, len(rows), CHUNK):
        chunk = rows[start:start + CHUNK]
        if stopped:
            results += [(row, "not_requested", None, None) for row in chunk]
            continue
        response = session.post(API, json={"items": [row["address"] for row in chunk]},
                                timeout=(10, 330), allow_redirects=False)
        if response.status_code != 200:
            stopped = response.status_code  # auth, credits or rate limit: stop and report
            results += [(row, "not_requested", None, None) for row in chunk]
            continue
        payload = response.json()
        for item in payload["data"]:
            row = chunk[item["index"]]
            outcome, data = classify(row, item)
            results.append((row, outcome, data, item.get("meta")))

    with open("audit.csv", "w", newline="", encoding="utf-8") as out:
        writer = csv.writer(out)
        writer.writerow(["id", "address", "outcome", "zpid", "returned_address", "cached", "fetched_at", *REQUIRED])
        for row, outcome, data, meta in results:
            present = [("" if data is None else data.get(f) not in (None, [])) for f in REQUIRED]
            writer.writerow([row["id"], row["address"], outcome,
                             data and data.get("zpid"), data and data.get("address"),
                             meta and meta.get("cached"), meta and meta.get("fetched_at"), *present])

    counts = {}
    for _, outcome, _, _ in results:
        counts[outcome] = counts.get(outcome, 0) + 1
    attempted = sum(n for o, n in counts.items() if o not in EXCLUDED and not o.startswith("transient:"))
    resolved = [data for _, outcome, data, _ in results if outcome == "resolved"]
    fields = {}
    for f in REQUIRED:
        k = sum(1 for data in resolved if data.get(f) not in (None, []))
        fields[f] = {"supplied": k, "of_resolved": len(resolved), "interval": wilson(k, len(resolved))}
    summary = {
        "run_at": datetime.now(timezone.utc).isoformat(),
        "sample_file": sample_path,
        "inputs": len(rows),
        "outcomes": counts,
        "stopped_on_http_status": stopped,
        "resolution": {"resolved": len(resolved), "attempted": attempted,
                       "interval": wilson(len(resolved), attempted)},
        "required_fields": fields,
        "method": "Outcomes per audit.csv; transient errors and unrequested rows excluded from attempted; "
                  "field counts among resolved rows only; Wilson interval at the stated z.",
    }
    json.dump(summary, open("summary.json", "w", encoding="utf-8"), indent=2)
    print(json.dumps(summary["resolution"]), json.dumps(counts))


if __name__ == "__main__":
    main(sys.argv[1])

Check identity by hand on a subset

The script can only compare zpids when you supplied an expected one. For the rest, a resolved outcome means the API was confident, not that a person agreed. Draw a random subset of resolved rows and compare the returned address and unit with the input. Count disagreements and report them with the subset size.

Pay attention to units, directional prefixes and neighbouring house numbers. A building-level answer for a unit address, or the house next door, is exactly the kind of error that looks fine in aggregate and causes real problems for the one user it affects.

• Address identity and ambiguous matches

Report the method with the numbers

A coverage number without its method is not evidence. Publish, even internally, the population, the sampling procedure and seed, the run date, the exclusions, the outcome definitions, the denominators, the interval, and the share of cached results. Attach audit.csv so others can recount.

Scope the claim to the sample. Resolved this share of our sampled CRM addresses on this date is a defensible sentence. Covers the US housing market is not, and nothing in this audit supports it. Field availability can also change as source pages change, so treat the result as dated.

Repeat the audit when things change

Rerun the audit when your input source changes, when you add a segment, or on a regular schedule if your product depends on a field. Keep each run's files with their date instead of overwriting them. Comparing two dated audits on the same method is far more informative than one impressive number.

Validation scope: the script was exercised against a local fixture server returning successes, warnings, each error class and a stopped run. Field and error names follow the published response contract. Those fixtures are test inputs, not coverage evidence, and this article reports no coverage figure.

• Plan the rest of your integration tests