Step 08 - Categorize titles

categorize_titles() joins the standardized titles to the taxonomy, engineers organization-level features, and produces the finished, analysis-ready data frame.

categorize_titles()
|
+- merge with the taxonomy on title.standard  (domain, SOC codes, role flags)
+- add_features()        pay/hours ranks, leadership counts, per-EIN summaries
+- simplify_varnames()   rename to the tidy output schema; zero-fill ratio NAs

Engineered features

add_features() computes, per organization (OBJECTID): counts of employees, board members, and leaders (num.ceos, num.clevel, num.dirvp, num.mgr, num.spec); officer counts (num.pres, num.vp, num.treas, num.sec); pay and hours ranks and shares; and full/part-time headcounts — weighting people who hold multiple titles so they are not double-counted.

data(tinypartvii)
set.seed(8)
d <- dplyr::sample_n(tinypartvii, 250)

df <- d |>
  standardize_df() |> remove_dates() |> standardize_conj() |>
  split_titles() |> standardize_spelling() |> gen_status_codes() |>
  standardize_titles() |> categorize_titles()
#> [OK] standardize df step complete
#> [OK] remove dates step complete
#> [OK] standardize conjunctions step complete
#> [OK] split titles step complete
#> [OK] standardize spelling step complete
#> [OK] generate status codes step complete
#> [OK] standardize titles step complete
#> [OK] categorize titles step complete

df |>
  dplyr::select(org.name, dtk.name, title.standard, domain.category,
                pay.rank, tot.comp, num.ceos, num.board) |>
  dplyr::arrange(org.name, pay.rank) |>
  head(10)
#>                                                                       org.name
#> 1                                                           A Place-2-Live Inc
#> 2                                                                  ADDGENE INC
#> 3                                                ADVANCED BIOFUELS ASSOCIATION
#> 4                                          AIDS VACCINE ADVOCACY COALITION INC
#> 5                                          ALABAMA HOME INSPECTOR ORGANIZATION
#> 6                                           ALBANY AREA CHAMBER FOUNDATION INC
#> 7                                               ALLIANCE FRANCAISE DE PORTLAND
#> 8                          ALLIANCE TO END HOMELESSNESS INSUBURBAN COOK COUNTY
#> 9  AMERICAN FEDERATION OF TEACHERS Henry Ford Community College Adjunct Faccul
#> 10                                        AMERICAN LEGION 1172 EVANSVILLE POST
#>              dtk.name  title.standard domain.category pay.rank tot.comp
#> 1        Deneen Jones    BOARD MEMBER           board        1        0
#> 2  MATTHEW E GHANTOUS            <NA>            <NA>        1   173699
#> 3   NEVILLE FERNANDES    BOARD MEMBER           board        1        0
#> 4       WANDA BUCKNER            <NA>            <NA>        1   224598
#> 5     ANDREW GRIFFITH       PRINCIPAL             xxx        1        0
#> 6        SMITH WILSON    BOARD MEMBER           board        1        0
#> 7         HOKE HARDEN    BOARD MEMBER           board        1        0
#> 8      YVETTE HOLCOMB    BOARD MEMBER           board        1        0
#> 9         Edgar Johns  VICE PRESIDENT      operations        1    14228
#> 10        MELVIN JANY FINANCE OFFICER      operations        1        0
#>    num.ceos num.board
#> 1         0         1
#> 2         0         0
#> 3         0         1
#> 4         0         0
#> 5         1         0
#> 6         0         1
#> 7         0         1
#> 8         0         1
#> 9         0         0
#> 10        0         0

Where this leaves us

Every row is now a single standardized, categorized title enriched with organization context. The remaining ambiguity (which “PRESIDENT” is the CEO?) is the target of the forthcoming machine-learning relabeling step, which will consume exactly these engineered features.

Input: title.standard. Output: the final analysis-ready data frame.