Generate cross-organization candidate pairs by hash join

Stage 1 of cross-org linkage: instead of a materialized within-block cross product (the reclin2 pair_blocking approach used within an organization), this groups profiles by hash bucket (person_blocking_keys()) and emits the distinct person pairs that co-occur in a bucket. Only these candidates are handed to the (expensive) stage-2 scorer.

Usage

candidate_pairs(keys, profiles, max_block_size = 5000L)

Arguments

keys: Long key table from person_blocking_keys().
profiles: The profiles the keys came from; supplies each person's organization (from orgs) for the same-organization filter.
max_block_size: Skip (and report) any bucket with more than this many distinct people (default 5000).

Value

A data frame, one row per unordered candidate pair: emp_a, emp_b (string-sorted), pass and pass_rank of the tightest bucket that produced the pair, and n_shared_buckets (how many buckets they co-occur in – a cheap first-pass strength signal). Attributes: "dropped_blocks" (over- size buckets skipped) and "n_comparisons" (pairs generated, i.e. the stage-2 workload).

Details

Since each within-organization EMP_ID denotes one person in one organization, a genuine interlock links two EMP_IDs in different organizations, so pairs sharing an organization are dropped. Buckets larger than max_block_size (typically a very common surname with geography/industry relaxed away) would emit O(n^2) mostly-spurious pairs; they are skipped and reported in the "dropped_blocks" attribute rather than silently truncated.

Examples

if (FALSE) { # \dontrun{
keys  <- person_blocking_keys(profiles)
cand  <- candidate_pairs(keys, profiles)
attr(cand, "n_comparisons")            # vs choose(nrow(profiles), 2)
} # }