Method · Python tool · 2020

Fishbone analysis: seeing address errors on the map

A QA method that draws a line from every address point to where it geocodes on the road centerline. Clean data draws evenly spaced lines that never cross; range, parity, side-of-street and street-name errors show up as lines that do.

Organization
Yavapai County
Presented
AGIC Symposium 2020
Year
2020
Files
2 downloads

Results

  • Address and centerline errors visible at a glance.
  • The production version charted match rates and line lengths every run.
  • Script and sample data published.

Stack

  • Python
  • ArcPy
  • ArcGIS Pro
  • Dual-range address locator
  • File geodatabase
Visualizing attribute discrepancies between address and centerline datasets (AGIC 2020)

What it shows

Each line runs from an address point to where that address geocodes along the street centerline. The errors it surfaces include:

  • Addresses or road ranges out of numerical order
  • Addresses or road ranges on the wrong side of the street
  • Addresses or road ranges on the wrong block
  • Mismatched pre-directionals, post-directionals and street names
Clean data: evenly spaced lines that never cross.
Clean data: evenly spaced lines that never cross.

The lines should (almost) never cross. When they do, the road attributes or the address attributes are wrong.

Crossing lines flag an attribution problem.
Crossing lines flag an attribution problem.
Zoomed in: right-side addresses geocode to the left because the FROMLEFT and TOLEFT ranges hold right-side numbers, and the parity fields are wrong too.
Zoomed in: right-side addresses geocode to the left because the FROMLEFT and TOLEFT ranges hold right-side numbers, and the parity fields are wrong too.

What the errors look like

Diagram with two examples on made-up streets. Left: address points on both sides of a street, each with a short line to where its address geocodes; a second point carrying address 107 sits across the street and its line runs diagonally back to 107, crossing its neighbors. Right: an address point in the yard near the driveway, away from its building, and the fixed point on the building.
Two common finds, redrawn with made-up streets: a duplicate address point and a point that isn’t on its building.

How it’s built

Five steps, the same ones from the AGIC talk:

  1. Build a dual-range U.S. address locator from the road centerlines.
  2. Geocode the address points with it.
  3. Create an empty polyline feature class and draw one line per address, from its original position to its geocoded position.
  4. Find the places where lines intersect.
  5. Investigate each one and fix the address point or the road record.
# Truncate old connection lines feature class records
arcpy.management.TruncateTable('Connection_Lines')

connection_line_list = []

# Read the original (POINT_X/Y) and geocoded (DisplayX/Y) position of each address
with arcpy.da.SearchCursor('GeocodedPoint', ["Match_addr", "POINT_X", "POINT_Y", "DisplayX", "DisplayY"]) as in_table:
    for row in in_table:
        start_point = arcpy.Point(row[1], row[2])
        end_point = arcpy.Point(row[3], row[4])
        line = arcpy.Polyline(arcpy.Array([start_point, end_point]))
        connection_line_list.append([line, row[0]])

# Insert the new connection lines
with arcpy.da.InsertCursor('Connection_Lines', ["SHAPE@", "Match_addr"]) as insert_cursor:
    for line in connection_line_list:
        insert_cursor.insertRow(line)

In production

The version used at Yavapai County went further: it rebuilt the address locators, managed table views and relationship classes, and recorded each run’s totals, match/tie/unmatched percentages, average connection-line length and longest line. Charting those over time showed the cleanup’s progress.

Where it runs matters. A full run took about 40 minutes against the enterprise database and 10 to 15 minutes on a laptop file geodatabase, so the checks ran on a local copy and only the fixes went back to the server.

Run statistics logged to the database on every run.
Run statistics logged to the database on every run.

Downloads