Skip to contents

Reviewing and QA-ing links

link-review.Rmd

Automated linkage should never be the last word on a longitudinal panel. This article covers the quality-control layer: how to surface the links most worth a human’s attention, and how to route upstream name-parser defects back to peopleparser instead of silently absorbing them.

There are two distinct jobs, and two functions:

Input contract

Both operate on a linked panel (the output of link_panel()), and both need the raw name to compare against the parsed components. The relevant columns:

role default column
person id EMP_ID
organization ein, org.name
tax year taxyr
raw name name.raw
parsed parts first_name, last_name

name.raw is the literal source name; carrying it through the pipeline (see the Getting started article) is what makes this review layer possible.

flag_links() scans each linked cluster for the failure modes that survive automated scoring and tags them with a category:

linked <- link_panel(panel)
flags  <- flag_links(linked)
table(flags$category)

The categories name why a link (or non-link) is suspect — for example:

category what it catches
surname_compound compound surnames matched on one shared token
surname_credential a credential (MD, CPA) fused into the surname
parser_honorific_glue an honorific glued onto the name by the parser
nickname_or_spelling first names differing by a nickname or spelling

Turn the flagged set into a ready-to-work queue (sorted, deduplicated, one row per item to review) and export it:

q <- link_review_queue(flags)
utils::write.csv(q, "link_review_queue.csv", row.names = FALSE)

Routing parser defects upstream

Some problems are not linkage errors at all — they are defects in the parsed name that synthid merely detects. parse_fail_log() isolates these so they can be fixed at the source (peopleparser) rather than patched here:

log <- parse_fail_log(linked, path = "peopleparser_parse_fails.csv")
table(log$defect_type)

The defect classes correspond to concrete parser failure modes:

defect_type meaning
honorific_glue an honorific (REV, HON) stuck to a given name
credential_in_surname a post-nominal credential absorbed into the surname
surname_letter_glue a stray initial glued to the surname

The token vocabularies

The flaggers are driven by small, inspectable token sets — the honorifics and credentials they look for. These are exported so you can see exactly what is matched (and extend them):

head(honorific_tokens(), 20)
#>  [1] "MR"     "MRS"    "MS"     "MISS"   "DR"     "REV"    "HON"    "PROF"  
#>  [9] "FR"     "SR"     "SISTER" "RABBI"  "PASTOR" "FATHER" "SIR"    "DAME"
head(credential_tokens(), 20)
#>  [1] "RT"     "RN"     "MD"     "DDS"    "DMD"    "PHD"    "PHARMD" "CAE"   
#>  [9] "RDMS"   "RVT"    "RM"     "RMSBS"  "MSM"    "MSRS"   "CRA"    "CNMT"  
#> [17] "MBA"    "ESQ"    "CPA"    "JD"

Pass extended vectors via the hon = / cred = arguments of flag_links() to tune sensitivity.

A workflow

  1. link_panel() to produce the linked panel.
  2. flag_links() → link_review_queue() → hand the CSV to a reviewer.
  3. parse_fail_log() → hand its CSV to the peopleparser maintainer.
  4. Re-run once upstream fixes land; the deterministic ids mean unaffected records keep their EMP_ID / PERSON_YEAR_ID.

See also