Measuring Address Quality

The quality control tests follow a simple recipe:

1. Compare address data to domains and specifications tailored for local use
2. Identify anomalies

Tests are designed to provide data quality element information as described in ISO 19115. The test specification includes:

• Scope: the elements, attributes or classifications to be tested
• Measure: a description of what the test measures.
• Procedure: a description of the test
• Script or Function: an example of the test in SQL code, or pseudocode where the exact parameters are more difficult to anticipate. The scripts and functions were written (except where noted otherwise) using standard ISO/IEC 9075-1:2008 SQL. Exact coding will vary from system to system. Spatial predicates used in the measures are described in OpenGIS Simple Features Specification for SQL. Where the code or pseudocode uses predicates beyond the SFSQL standard, they are also noted.
• Parameters for calculating anomalies as a percentage of the data set

About Anomalies

Measures are described with the understanding that records with known anomalies are excluded from related tests. New anomalies discovered should be corrected or described with Address Anomaly Status attributes.

Calculating Conforming Records as a Percentage of the Data Set

The Perc Conforming function measures the results of test for anomalies and describes the percentage of data elements that conform. Calculating the percentage of conformance requires inverting the query: the number of anomalies found is subtracted from the total number of records before calculating the percentage.

Function: Calculating Conforming Records as a Percent of the Data Set

Description

The function receives information directly from SQL statements including the standard COUNT aggregator and calcuates percentages.

Function

CREATE OR REPLACE FUNCTION perc_conforming( integer, integer )
RETURNS numeric as $$

DECLARE
   nonconforming alias for $1;
   total_recs alias for $2;
   calc_perc numeric;

BEGIN

SELECT INTO calc_perc
ROUND( ( ( total_recs - nonconforming )::numeric / total_recs::numeric )  * 100, 2 );

RETURN calc_perc;

END; 

$$ language 'plpgsql';

Pseudocode Query

SELECT
   perc_conforming
   (
      ( SELECT
           COUNT(*) as nonconforming
        FROM
            Address Collection
        WHERE
            condition is not met
      )::integer,
      ( SELECT
           COUNT(*) as total_recs
        FROM
           Address Collection
      )::integer
  )
;

Successful Result: 100% Conforming

 perc_conforming 
-----------------
          100.00
(1 row)

Unsuccessful Result: 30% Conforming

 perc_conforming 
-----------------
           30.00
(1 row)

Notation

The tests are described using SQL constructs and operators.

Operators used include:

Operator	Description	SQL Example Statement
||	concatenation	SELECT 'a' || 'b';	ab
%	modulo	SELECT 5 % 2;	1
::	type casting	SELECT '0005'::integer;	5
~	pattern matching	SELECT Complete Street Name where Complete Street Name ~ 'Main';	Main Street

Domains of values are an important tool for controlling values for address components. Measures used to test data conformance depend on the type of domain. The Content Standard for Digital Geospatial Metadata (CSDGM) classifies domains as enumerated,range, codeset and unrepresentable. The following table lists the CSDGM definition for each type of domain, as listed in Section 5: Entity and Attribute Information, along with the measures associated with each.

Domain Type	CSDGM Definition	Quality Measures	Quality Notes
codeset	"reference to a standard or list which contains the members of an established set of valid values."	Tabular Domain Measure Spatial Domain Measure	The U.S. Postal Service list of "Primary Street Suffix Names" is a familiar example of a codeset domain. In cases where specific street suffixes are associated with a given area in the Address Reference System, the association should also be checked with the Spatial Domain Measure.
enumerated	"the members of an established set of valid values."	Tabular Domain Measure Spatial Domain Measure	A local, validated street name list is an example of an enumerated domain. In cases where specific types of values are associated with a given area in the Address Reference System, the association should also be checked with the Spatial Domain Measure.
range	"the minimum and maximum values of a continuum of valid values."	Range Domain Measure Spatial Domain Measure Address Number Fishbones Measure	Range domain examples include such things as minimum and maximum Address Number values set by some jurisdictions, or a range of address values assigned to a given grid cell in the Address Reference System. In the latter case, the Spatial Domain Measure would be required to validate the location of the grid cell. Many Address Number values are associated with a Two Number Address Range or a Four Number Address Range. In the latter case conformance can be checked with the Address Number Fishbones Measure.
unrepresentable	"description of the values and reasons why they cannot be represented."