Skip to contents

Step 08 - Categorize titles

Source: vignettes/step-08-categorize-titles.Rmd
step-08-categorize-titles.Rmd

categorize_titles() attaches the taxonomy to each standardized title, adds features that describe each person’s place in their filing, and renames the columns to the tidy output schema.

Before and after

before (title.standard) domain.category domain.label SOC soc.label emp.level board.role
EXECUTIVE DIRECTOR executive general management 11-1011 Chief Executives CEO
CHIEF OPERATING OFFICER executive general management 11-1011 Chief Executives OFFICER
VICE PRESIDENT OF FINANCE AND ADMINISTRATION operations finance 11-3031 Financial Managers OFFICER
GENERAL COUNSEL operations legal 23-1011 Lawyers OFFICER
DIRECTOR OF DEVELOPMENT operations fundraising and membership 11-2033 Fundraising Managers MANAGER
DIRECTOR OF NURSING industry-specific medical 11-9111 Medical and Health Services Managers MANAGER
MANAGER operations administration 11-9199 Managers, All Other MANAGER
COACH industry-specific recreation and sports 27-2022 Coaches and Scouts PROFESSIONAL
DOCTOR industry-specific medical Physicians PROFESSIONAL
PASTOR industry-specific religion 21-2011 Clergy CEO
BOOKKEEPER operations finance 43-3031 Bookkeeping, Accounting, and Auditing Clerks STAFF
EXECUTIVE ASSISTANT operations administration 43-6011 Executive Secretaries and Executive Administrative Assistants STAFF
BOARD PRESIDENT governance board CHAIR
BOARD VICE PRESIDENT governance board VICE CHAIR
BOARD SECRETARY governance board SECRETARY
BOARD TREASURER governance board TREASURER
BOARD MEMBER governance board MEMBER
NA NA NA NA NA NA NA

Each title is either a job (an emp.level and, usually, a SOC occupation) or a board seat (a board.role), never both. An uncoded title (SENIOR RABBI) gets no taxonomy fields. The Taxonomy article explains every category.

What it does 

categorize_titles()
|
+- merge with the taxonomy on title.standard   domain, SOC codes, emp.level, board.role
+- add_features()        pay and hours ranks and shares, per-filing counts
+- simplify_varnames()   rename to the output schema (title.v1 ... title.v7, tot.comp, ...)

Taxonomy fields

Column Meaning
domain.category, domain.label the functional area (executive / general management, operations / finance, industry-specific / medical, …)
major.group … detailed.occupation, soc.label the 2018 BLS Standard Occupational Classification code, at four levels
emp.level for a job: CEO, OFFICER, MANAGER, PROFESSIONAL or STAFF
board.role for a board seat: CHAIR, VICE CHAIR, SECRETARY, TREASURER or MEMBER
emp, ceo, c.level, mgr, spec, board, pres, vp, sec, treas, mem legacy 0/1 flags, derived from emp.level and board.role

Engineered features

add_features() computes, within each filing (object.id):

  • each person’s pay and hours: tot.comp, tot.hours (and versions that include related organizations), pay.rank, hours.rank, pay.pct.of.max, pay.pct.of.tot, hours.pct.of.max;
  • the filing’s make-up: num.dtk (people), num.titles, num.paid, num.fte (40+ hours), num.fte.30h, num.pte;
  • counts by taxonomy: num.emp, num.board, num.ceos, num.clevel, num.mgr, num.spec, num.pres, num.vp, num.treas, num.sec, num.mem;
  • a person’s titles: title.count and title.order (from step 04), title.mult.x.

People with several titles are weighted so that they are not counted twice.

data(tinypartvii)
big <- tinypartvii |> group_by(OBJECTID) |>
  filter(RETURN_TYPE == "990", n() >= 12, any(as.numeric(F9_07_COMP_DTK_COMP_ORG) > 150000)) |> ungroup()
one <- tinypartvii[tinypartvii$OBJECTID == big$OBJECTID[1], ]
df <- one |>
  standardize_df() |> remove_dates() |> standardize_conj() |> split_titles() |>
  standardize_spelling() |> gen_status_codes() |> standardize_titles() |> categorize_titles()
#> [OK] standardize df step complete
#> [OK] remove dates step complete
#> [OK] standardize conjunctions step complete
#> [OK] split titles step complete
#> [OK] standardize spelling step complete
#> [OK] generate status codes step complete
#> [OK] standardize titles step complete
#> [OK] categorize titles step complete
df |>
  select(title.standard, emp.level, board.role, tot.comp, tot.hours, pay.rank, num.dtk, num.paid, num.board) |>
  arrange(pay.rank) |>
  head(10)
#>              title.standard emp.level board.role tot.comp tot.hours pay.rank
#> 1  EXECUTIVE VICE PRESIDENT   OFFICER              194586        40        1
#> 2       CORPORATE SECRETARY   OFFICER              169504        40        2
#> 3                      <NA>      <NA>       <NA>   156659        40        3
#> 4   CHIEF FINANCIAL OFFICER   OFFICER              155577        40        4
#> 5                      <NA>      <NA>       <NA>   137163        40        5
#> 6          PROGRAMS MANAGER   MANAGER              113144        40        6
#> 7                   MANAGER   MANAGER              112991        40        7
#> 8                FACILITIES     STAFF              108137        40        8
#> 9              BOARD MEMBER               MEMBER        0         1        9
#> 10     BOARD VICE PRESIDENT           VICE CHAIR        0         1        9
#>    num.dtk num.paid num.board
#> 1       18        8        10
#> 2       18        8        10
#> 3       18        8        10
#> 4       18        8        10
#> 5       18        8        10
#> 6       18        8        10
#> 7       18        8        10
#> 8       18        8        10
#> 9       18        8        10
#> 10      18        8        10

Assets

Asset Type Used for Load it with
title.taxonomy crosswalk (package data) for each title.standard: domain, SOC codes, emp.level, board.role, legacy flags get_title_taxonomy() or data(title.taxonomy)

The source is data-raw/crosswalks/title-taxonomy.csv in the source repository. The domain vocabulary, with a description of each domain, is data-raw/crosswalks/domains.csv.

tax <- get_title_taxonomy()
dim(tax)
#> [1] 450  22
table(tax$emp.level, useNA = "ifany")
#> 
#>                       CEO      MANAGER      OFFICER PROFESSIONAL        STAFF 
#>           15           11          129          139           40          116
table(tax$board.role[tax$board.role != ""])
#> 
#>      CHAIR     MEMBER  SECRETARY  TREASURER VICE CHAIR 
#>          1          1          1          1          1
tax[tax$title.standard %in% c("CEO", "DIRECTOR OF DEVELOPMENT", "BOARD TREASURER"),
    c("title.standard", "domain.label", "detailed.occupation", "soc.label", "emp.level", "board.role")]
#>              title.standard               domain.label detailed.occupation
#> 45          BOARD TREASURER                      board                    
#> 56                      CEO         general management             11-1011
#> 152 DIRECTOR OF DEVELOPMENT fundraising and membership             11-2033
#>                soc.label emp.level board.role
#> 45                                  TREASURER
#> 56      Chief Executives       CEO           
#> 152 Fundraising Managers   MANAGER

Input: the output of step 07. Output: one row per person and title with the taxonomy fields, the status flags, the engineered features, and the title at each step (title.raw, title.v1 … title.v7). Continue with step 09.