Skip to content

Add tracing regression coverage for automatic chooser-column pruning #1105

Description

@jpn--

Problem

PR #1094 makes automatic unused-column pruning run while tracing is enabled. This closes the memory gap left by #833, where pruning was intentionally bypassed for traced runs, but it also means tracing now depends on preserving the correct identity columns.

The fix in 0f5234e adds util.traceable_id_columns() and protects:

  • household_id
  • person_id
  • proto_household_id
  • proto_person_id

across the affected simple- and interaction-simulation paths.

The tests added in #1094 only partially cover this behavior. They inspect the pruned column list with tracing mocked; the interaction_sample test also mocks interaction_trace_rows(). Consequently, they do not verify that real trace targets are selected after pruning, that proto-table tracing still works, or that all affected simulation paths preserve the required identifiers.

The integration test added by #1021 verifies consistent trace filename hashes across multiprocessing, but does not validate the rows or identifiers contained in trace output.

This follows up on:

Proposed coverage

Add focused regression tests that exercise tracing without mocking the trace-row selection logic.

  1. Add table-driven tests for Tracing.interaction_trace_rows() covering:

    • person_id and household_id as an index or chooser column
    • proto_person_id and proto_household_id, including proto-table precedence
    • fixed-size interaction alternatives (sample_size set)
    • sparse or variable-size alternatives (sample_size=None)
    • the expected trace-row mask and returned trace IDs
  2. Extend the pruning tests for the paths changed by Deprecate SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS settings #1094:

    • simple simulation and logsum calculation
    • interaction_sample
    • interaction_simulate
    • interaction_sample_simulate
  3. Verify that:

    • spec-used columns remain available
    • unrelated chooser columns are pruned before constructing/evaluating the interaction dataframe
    • every identifier required for trace slicing survives pruning
    • the configured household/person is actually selected in the resulting trace rows
    • the full, unpruned chooser and alternative inputs are still written before pruning, as intended by Deprecate SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS settings #1094
  4. Strengthen the lightweight integration coverage, either by extending test/trace_id or adding a small equivalent test, to inspect trace contents rather than only filename hashes.

Acceptance criteria

  • Tests exercise real trace-row selection rather than substituting a mocked result.
  • Regular and proto household/person identifiers are covered.
  • Removing trace-ID protection from any affected simulation path causes a regression test to fail.
  • At least one test validates emitted trace data or captured trace_df input, not only the intermediate column list.
  • The tests use small synthetic fixtures and remain suitable for the normal core test suite.

Metadata

Metadata

Assignees

No one assigned

    Labels

    FeatureNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions