Combine and Process Multi-Year XPATH Reports

The process_xpaths() function automates the full workflow of generating, combining, and summarizing XPATH usage reports across multiple DuckDB databases. It can process multiple tax years, merge the resulting reports, summarize schema versions, and add derived metadata such as first/last year and current version indicators.

Usage

process_xpaths(years, base_path, concordance = NULL, get_type = NULL)

Arguments

years: Numeric vector of tax years to process.
base_path: Character. Base path containing yearly DuckDB folders and the xpath_reports directory.
concordance: Optional data frame containing a variable xpath for merging. If not supplied, the merge step is skipped.
get_type: Optional function that accepts an XPATH string and returns a type label. Used to classify paths into "HEADER" or "DATA".

Value

Invisibly returns a data frame containing all combined and processed XPATHs.

Details

This function:

Iterates over a range of tax years and calls generate_xpath_report() for each.
Reads all CSV reports from the specified xpath_reports directory.
Combines and aggregates XPATH occurrences across years.
Cleans and consolidates schema version lists using combine_lists().
Extracts first and last schema years and merges results with a concordance table if provided.

The final combined report is saved to "xpath_reports/ALL-XPATHS.csv".

Examples

if (FALSE) { # \dontrun{
process_xpaths(
  years = 2009:2024,
  base_path = "C:/Users/jdlec/DATA/DUCKDB_2025",
  concordance = concordance,
  get_type = get_type
)
} # }