Download, filter, merge, append BMF, and stack a panel

The workhorse acquisition recipe: resolve tables and years, download (or virtually scan) the CSVs, project columns and push the sample frame's entity restriction down to read-time, merge the tables within each year, stack the years, optionally append BMF organization traits, then apply the frame's rules. Returns a panel bundling the data with the frame and a provenance log.

Usage

panelize(
  sfw = NULL,
  tables,
  years,
  source = data_source(),
  bmf = "auto",
  backend = "memory",
  cache = c("retain", "temporary", "none"),
  path = "efdata",
  filters = NULL,
  columns = NULL,
  include_many = FALSE,
  collision = c("error", "prefix"),
  overwrite = FALSE,
  retry_max = 3L,
  timeout = 300,
  verbose = TRUE
)

Arguments

sfw: Optional create_sfw() sample frame governing the build. Filters prefilter reads and merges; select rules project columns.
tables: Aliases or literal table names.
years: Tax years.
source: Efile source configuration.
bmf: Append BMF fields: "auto" (default – attach when the frame references a BMF field), FALSE, TRUE (the NCCS master), or a BMF source (data frame, path, or URL).
backend: "memory" (default) or "duckdb" ("db" is accepted).
cache: "retain", "temporary", or "none" (DuckDB-only virtual scan).
path: Retained-cache directory.
filters: Optional named value filters (merged with the frame's).
columns: Optional source fields to retain (union with the frame's).
include_many: Join one-to-many and supplemental tables.
collision: Non-key collision policy.
overwrite: Replace cached files.
retry_max: Download attempts.
timeout: Download timeout.
verbose: Print progress.

Value

A panel (see as_panel()) whose data is the stacked frame, whose sfw carries the rules and provenance log (manifest()), plus download/table/join manifests and BMF diagnostics.

Details

Keys are set automatically from the efile schema (entity EIN2, time TAX_YEAR, record OBJECTID).