End-to-end workflow

titleclassifier turns the free-text officer and key-employee titles reported on IRS Form 990 Part VII into a structured, analysis-ready taxonomy. The work is done by an eight-step pipeline; each step is a single verb you can run on its own, and each has its own article in this site.

Step	Function	What it does
01	standardize_df()	Standardize the raw Part VII data frame.
02	remove_dates()	Strip and flag date fragments.
03	standardize_conj()	Normalize conjunctions, prepositions, separators.
04	split_titles()	Split multi-title strings into one row per title.
05	standardize_spelling()	Harmonize spelling, abbreviations, short forms.
06	gen_status_codes()	Detect, flag, and clean status qualifiers.
07	standardize_titles()	Map titles to canonical forms via a crosswalk.
08	categorize_titles()	Assign taxonomy categories and engineer features.

Getting the data

Part VII compensation tables are published in the public NCCS efile store. You can pull one directly by tax year (each file is large, so it is cached on disk):

library(titleclassifier)

# fetch + read the 2023 Part VII table in one call
d <- get_partvii(2023)

# or split the two steps (download once, re-read from cache)
paths <- fetch_partvii(2023, dest = "efdata")
d     <- read_partvii(paths)

For examples the package ships a demo slice, tinypartvii, already in the expected schema.

library(titleclassifier)
library(dplyr)

data(tinypartvii)
set.seed(1234)
d <- dplyr::sample_n(tinypartvii, 300)
dim(d)
#> [1] 300  33

Running the pipeline

The steps compose with the base pipe. The title-standardization, status-code, and taxonomy crosswalks are bundled with the package (a pinned snapshot of the source Google Sheet), so a run needs no network access.

df <- d |>
  standardize_df()    |>   # 01
  remove_dates()      |>   # 02
  standardize_conj()  |>   # 03
  split_titles()      |>   # 04
  standardize_spelling() |> # 05
  gen_status_codes()  |>   # 06
  standardize_titles() |>  # 07
  categorize_titles()      # 08
#> [OK] standardize df step complete
#> [OK] remove dates step complete
#> [OK] standardize conjunctions step complete
#> [OK] split titles step complete
#> [OK] standardize spelling step complete
#> [OK] generate status codes step complete
#> [OK] standardize titles step complete
#> [OK] categorize titles step complete

dim(df)
#> [1] 317 101

The result

Each row is one standardized title for one person, carrying the raw text, the successive cleaned versions (title.v2 … title.v7), the canonical title.standard, taxonomy fields, status flags, and organization-level features (pay/hours ranks, leadership counts).

df |>
  select(dtk.name, title.raw, title.standard, domain.category, tot.comp, tot.hours) |>
  arrange(desc(tot.comp)) |>
  head(10)
#>               dtk.name                  title.raw     title.standard
#> 1     CHARLES KAHN III                  PRESIDENT    BOARD PRESIDENT
#> 2          KENT RAY DO                  SECRETARY    BOARD SECRETARY
#> 3   MICHAEL D FREDDINO                 CONTROLLER        COMPTROLLER
#> 4   MICHAEL D FREDDINO                 CONTROLLER        COMPTROLLER
#> 5         ANDRE GROCOX                   DIRECTOR       BOARD MEMBER
#> 6  CAROLINE V SEGELKEN           STAFF PHARMACIST               <NA>
#> 7       TAMMY YAMANOHA           SR ADMINISTRATOR               <NA>
#> 8        WHITNEY VASEY         DIRECTOR OF EVENTS           DIRECTOR
#> 9      ELIZABETH KEANE                   EMPLOYEE           EMPLOYEE
#> 10    BERNEITHA MCNAIR INTERIM EXECUTIVE DIRECTOR EXECUTIVE DIRECTOR
#>    domain.category tot.comp tot.hours
#> 1            board  2389606      40.0
#> 2            board   444028      53.0
#> 3       operations   235319      37.5
#> 4       operations   235319      37.5
#> 5            board   217880      55.0
#> 6             <NA>   203361      42.0
#> 7             <NA>   190745      40.0
#> 8       operations   188401      40.0
#> 9       operations   132955      40.0
#> 10      operations   115500      40.0

The successive title versions make the cleaning transparent — you can see exactly what each step changed:

df |>
  select(title.raw, title.v3, title.v5, title.v6, title.standard) |>
  distinct() |>
  head(12)
#>             title.raw              title.v3              title.v5
#> 1         H&W TRUSTEE       H AND W TRUSTEE       H AND W TRUSTEE
#> 2              MEMBER                MEMBER                MEMBER
#> 3            DIRECTOR              DIRECTOR              DIRECTOR
#> 4  2ND VICE PRESIDENT SECOND VICE PRESIDENT SECOND VICE PRESIDENT
#> 5             Officer               OFFICER               OFFICER
#> 6                 CEO                   CEO                   CEO
#> 7           President             PRESIDENT             PRESIDENT
#> 8           Secretary             SECRETARY             SECRETARY
#> 9           treasurer             TREASURER             TREASURER
#> 10          Treasurer             TREASURER             TREASURER
#> 11           Director              DIRECTOR              DIRECTOR
#> 12     Vice President        VICE PRESIDENT        VICE PRESIDENT
#>           title.v6  title.standard
#> 1  H AND W TRUSTEE            <NA>
#> 2           MEMBER    BOARD MEMBER
#> 3         DIRECTOR    BOARD MEMBER
#> 4   VICE PRESIDENT  VICE PRESIDENT
#> 5          OFFICER    BOARD MEMBER
#> 6              CEO             CEO
#> 7        PRESIDENT BOARD PRESIDENT
#> 8        SECRETARY BOARD SECRETARY
#> 9        TREASURER BOARD TREASURER
#> 10       TREASURER BOARD TREASURER
#> 11        DIRECTOR    BOARD MEMBER
#> 12  VICE PRESIDENT  VICE PRESIDENT

Refreshing the crosswalks

By default the crosswalk loaders read the bundled snapshot. To pull the current Google-Sheet version instead (and refresh the snapshot in a source checkout), pass refresh = TRUE:

xwalk <- get_googlesheets_title_xwalk(refresh = TRUE)

See the Dictionaries and crosswalks article for what each crosswalk contains.