Record-level parser-defect log for peopleparser

Scans a panel for records whose parsed name components bear a syntactic defect that upstream name parsing should fix, and emits one row per offending record with the raw input, the wrong output, and – where it can be inferred – the expected value. These input/wrong/expected triples are directly usable as peopleparser regression fixtures.

Usage

parse_fail_log(
  df,
  emp_id = "EMP_ID",
  org_id = "ein",
  org_name = "org.name",
  year = "taxyr",
  raw_name = "name.raw",
  first = "first_name",
  last = "last_name",
  middle = "middle_name",
  suffix = "suffix",
  salutation = "salutation",
  object_id = "OBJECTID",
  table_id = "TABLE_ID",
  hon = honorific_tokens(),
  cred = credential_tokens(),
  path = NULL
)

Arguments

df: A linked panel: the output of link_panel() (or any frame carrying the person id, org, year, raw name, and parsed given/surname columns).
emp_id, org_id, org_name, year: Column names for the cross-year person id, organization id, organization name, and tax year.
raw_name, first, last: Column names for the raw (pre-parse) name string and the parsed given and surname components.
middle, suffix, salutation: Optional parsed-component columns, echoed into the log when present so a fix can be verified against the full parse.
object_id, table_id: Optional source-key columns (see person_year_id()); echoed as the record key when present.
hon, cred: Token sets for honorific-glue and credential detection; see honorific_tokens() and credential_tokens().
path: Optional file path; when given, the log is written there with utils::write.csv() and returned invisibly.

Value

A data frame with one row per defective record: the record key, the raw_name input, the parsed components, defect_type, evidence, expected_first/expected_last (NA where not inferable), and source ("self-evident" or "consensus"). Rows are unique per record even when a record trips more than one detector (the defects are concatenated).

Details

Three defect types are detected:

honorific_glue: a honorific fused onto the given name ("MRWILLIAM" -> expected "WILLIAM"). Self-evident per record.
credential_in_surname: post-nominal credential tokens inside the surname ("RM RDMS RVT CRA MCNEIL" -> expected "MCNEIL"). Self-evident per record; the expected surname is the non-credential remainder.
surname_letter_glue: a single stray letter fused onto the surname ("WSMITH"). Detected only by consensus – another record of the same person carries the clean surname ("SMITH"), which becomes the expected value – so this type requires a linked, multi-record person.

Unlike flag_links(), which only inspects clusters whose members disagree, the self-evident defects are scanned across every record, so a defect that is consistent across a person's years (and therefore never trips a link flag) is still reported.

Examples

if (FALSE) { # \dontrun{
log <- parse_fail_log(linked, path = "peopleparser_parse_fails.csv")
table(log$defect_type)
} # }