Dictionaries and crosswalks
dictionaries.RmdThe pipeline is driven by two kinds of reference data: crosswalks (curated lookup tables, maintained in a Google Sheet and bundled with the package as pinned CSV snapshots) and word lists (character vectors shipped as package data used by the regex-based cleaning steps).
Crosswalks
The three crosswalks live in one Google Sheet and are cached in the package under inst/extdata/crosswalks/. The loader functions read the bundled snapshot by default and take refresh = TRUE to re-pull the live sheet.
Title standardization (get_googlesheets_title_xwalk())
Maps every observed title variant to a canonical title.standard, plus a strata / strata.label for board sub-roles. Used in step 07, standardize_titles().
xwalk <- get_googlesheets_title_xwalk()
dim(xwalk)
#> [1] 5374 4
head(xwalk, 8)
#> title.variant title.standard strata strata.label
#> 1 ACCOUNTANT ACCOUNTANT
#> 2 ACCOUNTING DIRECTOR ACCOUNTING DIRECTOR
#> 3 ACTOR ACTOR
#> 4 ADMINISTRATION OFFICER ADMINISTRATION OFFICER
#> 5 ADMINISTRATION ASSISTANT ADMINISTRATIVE ASSISTANT
#> 6 ADMINISTRATIVE ASSISTANT ADMINISTRATIVE ASSISTANT
#> 7 ADMINISTRATION ADMINISTRATOR
#> 8 ADMINISTRATOR ADMINISTRATORTitle taxonomy (get_googlesheets_title_taxonomy())
Maps each canonical title.standard to its taxonomy dimensions: functional domain.category / domain.label, the BLS SOC occupation codes, and the binary role/hierarchy flags (ceo, c.level, dir.vp, mgr, spec, board, pres, vp, sec, treas, mem). Used in step 08, categorize_titles().
tax <- get_googlesheets_title_taxonomy()
dim(tax)
#> [1] 474 20
head(tax[, c("title.standard", "domain.category", "soc.label",
"ceo", "c.level", "board")], 8)
#> title.standard domain.category
#> 1 ACCOUNTANT operations
#> 2 ACCOUNTING DIRECTOR operations
#> 3 ACTOR industry-specific
#> 4 ADMINISTRATION DIRECTOR operations
#> 5 ADMINISTRATION OFFICER operations
#> 6 ADMINISTRATIVE ASSISTANT operations
#> 7 ADMINISTRATOR operations
#> 8 ADOPTION COUNSELOR industry-specific
#> soc.label ceo c.level board
#> 1 Accountants and Auditors
#> 2
#> 3 Actors, producers, and directors
#> 4
#> 5
#> 6 Secretaries and administrative assistants
#> 7 Administrative services and facilities managers
#> 8 CounselorsStatus codes (get_googlesheets_status_codes())
Maps status-qualifier variants (e.g. ACTING, EMERITUS, INTERIM) to a standardized status.qualifier (e.g. INTERIM, FORMER). Used in step 06, gen_status_codes().
sc <- get_googlesheets_status_codes()
dim(sc)
#> [1] 294 2
head(sc, 8)
#> status.variant status.qualifier
#> 1 AS NEEDED AT LARGE
#> 2 AT LARGE AT LARGE
#> 3 AT-LARGE AT LARGE
#> 4 PROXY AT LARGE
#> 5 AT LG AT LARGE
#> 6 CURRENT CURRENT
#> 7 EX OFFICIO EXOFFICIO
#> 8 EX-OFFICIO EXOFFICIOWord lists (package data)
These character vectors are lazy-loaded package datasets used by the regex cleaning steps. Load any of them with data().
| Dataset | Used in | Purpose |
|---|---|---|
date.words |
remove_dates() (02) |
month/date tokens to detect and strip |
number.words |
gen_status_codes() (06) |
number words removed from titles |
likely.titles |
standardize_conj() (03) |
recognizable titles for split detection |
likely.subjects |
fix_of() (05) |
subjects that follow “of” in a title |
possible.titles |
fix_of() (05) |
candidate title tokens |
former.words |
gen_status_codes() (06) |
FORMER status variants |
future.words |
gen_status_codes() (06) |
FUTURE status variants |
interim.words |
gen_status_codes() (06) |
INTERIM status variants |
regional.words |
gen_status_codes() (06) |
REGIONAL status variants |
possible.regional.list |
gen_status_codes() (06) |
candidate regional tokens |
data(number.words)
head(number.words, 12)
#> [1] "ONE" "TWO" "THREE" "FOUR" "FIVE" "SIX" "SEVEN" "EIGHT"
#> [9] "NINE" "TEN" "ELEVEN" "TWELVE"
data(likely.titles)
head(likely.titles, 12)
#> [1] "PRESIDENT" "CEO" "DIRECTOR\\b" "CHAIR" "TRUSTEE\\b"
#> [6] "TREASURER" "SECRETARY" "OFFICER" "COUNSEL" "FOUNDER"
#> [11] "PUBLISHER" "EDITOR\\b"Demo data
| Dataset | Description |
|---|---|
tinypartvii |
~81k raw Part VII rows (10,000 orgs) in the modern schema, ready for standardize_df(). |
raw.titles |
A small vector of raw title strings for quick experiments. |
data(tinypartvii)
str(tinypartvii[, 1:8], vec.len = 1)
#> 'data.frame': 81309 obs. of 8 variables:
#> $ EIN2 : chr "EIN-72-1424183" ...
#> $ OBJECTID : chr "OID-202400189349300715" ...
#> $ ORG_EIN : chr "721424183" ...
#> $ ORG_NAME_L1 : chr "MU TAU INC" ...
#> $ ORG_NAME_L2 : chr "" ...
#> $ RETURN_AMENDED_X: chr "FALSE" ...
#> $ RETURN_GROUP_X : chr "FALSE" ...
#> $ RETURN_PARTIAL_X: chr "FALSE" ...