Map VBA macros and finish the coverage audit (priorities 1–3)

CHANGELOG.md

@@ -13,6 +13,60 @@ All notable changes to this project are documented here. The format follows
 - `install.sh`, a one-line installer for macOS and Linux that downloads the latest release binary.
 - Agent skill (`skills/sheetgraph/SKILL.md`) and a Claude Code plugin marketplace (`.claude-plugin/`), so agents run sheetgraph on their own when asked about a spreadsheet.
 - README: before/after demo image (`assets/demo.png`) and the script that generates its sample workbook.
+- Macros: each VBA procedure's trigger (workbook and sheet events, `Auto_Open`), the sheets and ranges it writes and
+  uses, the procedures it calls, and the formula cells that call it as a function, in REPORT.md and as `macro` nodes
+  with `writes` / `uses` / `calls` edges in graph.json. Apps Script functions are listed with their triggers.
+- `sheetgraph macro <dir> [name]` lists macros or prints one module's or procedure's code; `find` also searches
+  macro code.
+- Sheets' VBA code names are read from xlsx/xlsm (`Sheet.code_name`).
+- File-level settings: document and custom properties, calculation mode and iterative calculation (Excel and
+  Google), and whether the sheet structure is locked, at the top of REPORT.md.
+- Data connections (ODBC, OLE DB, web, text, Power Query) with passwords removed, and Power Query (M) code as
+  `power_query` scripts that `macro` and `find` read.
+- Pictures, shapes, text boxes and form controls over the grid (`Sheet.shapes`), with their text, alt text, the macro
+  a button runs and a control's linked cell; graph.json gets `shape` nodes with `runs` and `writes` edges.
+- Pictures placed in cells and Excel data types (Stocks, Geography) with their fields (`Cell.obj`); Google people and
+  file chips the same way.
+- Text with mixed formatting inside a cell (`Cell.runs`), from Excel and Google (`textFormatRuns`).
+- Locked/unlocked cells and hidden formulas (`Style.unlocked`, `Style.hidden_formula`), listed per sheet as the
+  inputs left editable under protection.
+- Numbers in xlsx/xlsm show as their format displays them (`25%`, `$12.50`, `(1,234)`), not only dates.
+- `tests/fixtures/audit.xlsx` and `tests/audit.rs`: one workbook with every item on the coverage checklist.
+- Very hidden sheets (only macro code can unhide them) are marked as such (`Sheet.very_hidden`).
+- Names local to one sheet keep their scope (`NamedRange.scope`), and a sheet's own definition wins over a
+  workbook-wide name of the same name. Names defined by a formula, LAMBDA functions included, link to what the
+  formula reads, through other names too.
+- Named cell styles ("Input", "Calculation", custom ones) as `Style.cell_style`, and which sides of a cell have a
+  border and its line (`Style.border`: `bottom double`, `box thin`), from Excel and Google.
+- Dropdowns, validation formulas and conditional formats are graph nodes with `choices` / `reads` edges to the ranges
+  they read; REPORT.md lists those that read another sheet, and `trace` shows them both ways.
+- `trace` follows links into other workbooks extracted to the same folder (`'[Other.xlsx]Data'!A1`, IMPORTRANGE),
+  in both directions.
+- Merged cells: `cells` and `trace` say a cell is merged or hidden under a merge and show what the merge displays;
+  REPORT.md folds repeating merges (`12× A:C, 1 row each, rows 8–30`); formulas reading a cell hidden under a merge
+  (always blank) are flagged.
+- Google Sheets slicers (as filters) and Tables.
+- Column widths and row heights in pixels where they differ from the default (`Sheet.col_widths`,
+  `Sheet.row_heights`); a column of width 0 or a row of height 0 counts as hidden. Gridlines turned off
+  (`Sheet.no_gridlines`) show in the sheet's REPORT.md header.
+- Grouped rows and columns with their level and whether they're collapsed (`Sheet.outlines`), from Excel and Google,
+  listed per sheet in REPORT.md.
+- Notes and comment threads carry their authors (`Analyst: Check with finance`, replies after `↳`, resolved threads
+  marked). Google Drive comments, which the Sheets API doesn't return, are read too (`Workbook.comments`) and tied to
+  a cell when exactly one cell shows the text they quote; REPORT.md lists them, `cells` shows them on their cell and
+  `find` searches them.
+
+### Fixed
+
+- A reference across sheets (`=SUM(Jan:Dec!B2)`) linked only the last sheet; it now links every sheet in the span.
+- A reference to another workbook (`'[Other.xlsx]Data'!A1`) also produced a broken `reads` edge in graph.json.
+- An xlsm whose VBA uses compression calamine doesn't expect no longer aborts the extraction; the macros are reported
+  missing and everything else is kept.
+
+### Changed
+
+- `Style.borders` (has any border) is replaced by `Style.border`, which says which sides and their line.
+- REPORT.md counts modules with no code (the empty module Excel gives every sheet) instead of listing each one.
 
 ## [0.1.0]
 

README.md

@@ -102,8 +102,9 @@ sheetgraph extract <id|url|file.xlsx>... -o sheetgraph-out --creds service-accou
 
 # drill down (all read workbook.json.gz; no network)
 sheetgraph cells  sheetgraph-out/<wb> "'Revenue Input'!A1:F20"
-sheetgraph trace  sheetgraph-out/<wb> "Monthly!E10" --depth 3   # precedents + dependents
-sheetgraph find   sheetgraph-out/<wb> "VAT"
+sheetgraph trace  sheetgraph-out/<wb> "Monthly!E10" --depth 3   # precedents + dependents, into other extracted workbooks too
+sheetgraph find   sheetgraph-out/<wb> "VAT"                      # values, formulas, notes, comments, macro code
+sheetgraph macro  sheetgraph-out/<wb> [Module1 | RollUp]         # macros and what they touch; code with a name
 sheetgraph sheet  sheetgraph-out/<wb> "Ledger" --full           # untruncated section
 sheetgraph report sheetgraph-out/<wb> [--full]                 # re-render without refetching
 ```
@@ -126,14 +127,18 @@ let graph = sheetgraph::graph::build(&wb, &a); // serde_json::Value
 sheetgraph-out/
   INDEX.md                    every workbook, sizes, links between workbooks (IMPORTRANGE, Excel external links)
   <workbook>/REPORT.md        the map: read this first
-  <workbook>/graph.json       nodes (sheets, blocks, formula groups, charts, pivots, externals) + edges
-  <workbook>/workbook.json.gz every cell (see Known gaps for very large files): value, formula, display text, style, validation, note, link
+  <workbook>/graph.json       nodes (sheets, blocks, formula groups, charts, pivots, dropdowns and rules, names, externals, macros) + edges
+  <workbook>/workbook.json.gz every cell (see Known gaps for very large files): value, formula, display text, style, validation, note, link; plus column widths, row heights and file comments
   <workbook>/meta.json
 ```
 
-REPORT.md contains, per workbook: sheet list with used ranges and sheet→sheet data flow; external links (IMPORTRANGE / IMPORTDATA / … with targets resolved from cells, Excel links to other workbooks, refs to deleted sheets, Connected-Sheets data sources); named ranges; function census incl. **non-builtin functions** (Apps Script custom functions / Named Functions / VBA UDFs), dynamic refs (INDIRECT/OFFSET) and volatile functions; scripts (VBA modules from xlsm; Apps Script on request); developer metadata (left by add-ons/integrations).
+REPORT.md contains, per workbook: sheet list with used ranges and sheet→sheet data flow; external links (IMPORTRANGE / IMPORTDATA / … with targets resolved from cells, Excel links to other workbooks, refs to deleted sheets, Connected-Sheets data sources); dropdowns, validation and conditional formats that read another sheet; named ranges, with the sheet a local name belongs to and which names are LAMBDA functions; function census incl. **non-builtin functions** (Apps Script custom functions / Named Functions / VBA UDFs), dynamic refs (INDIRECT/OFFSET) and volatile functions; scripts (VBA modules from xlsm; Apps Script on request), with each procedure's trigger (`Workbook_Open`, `Worksheet_Change`, `onEdit`), the sheets and ranges it writes and uses, the procedures it calls, and the formula cells that call it as a function; modules with no code (the empty one Excel gives every sheet) are counted, not listed; developer metadata (left by add-ons/integrations).
 
-Per sheet: blocks (tables with column profiles + samples, or small blocks verbatim), formula copy-groups and families with flags (`[array/spills]`, `[external]`, `[dynamic ref]`, `[#REF!]`, error counts), data validation (dropdowns, checkboxes, range-sourced lists), conditional formats, visual formatting regions, merges, filters/filter views, protected ranges, hidden rows/cols, banding, Excel tables, charts and pivots (with source ranges), notes and comments, hyperlinks.
+REPORT.md also starts with what changes how the whole file behaves: document properties, calculation mode (manual calculation means stored values may be stale; iterative means circular references are intended), a locked sheet structure, and data connections and Power Query code.
+
+graph.json edges: `reads` (formula group → the blocks and groups its references land on), `plots` (chart → source), `summarizes` (pivot → source), `imports` (formula → external workbook or URL), `writes` / `uses` (macro → what it changes or reads; a sheet the macro creates has `"to": null` and `missing_sheet`), `calls` (macro → macro, and formula group → the VBA function it calls), `runs` (button or shape → the macro it runs), `writes` from a form control to its linked cell, `choices` (dropdown → the range its list comes from), and `reads` from validation and conditional-format rules and from names defined by a formula (LAMBDA included) to what they read. A formula reading a name defined by a formula gets `reads` edges to what that formula reads, with `via` naming it; a name local to one sheet is the node `name:Sheet!Name`.
+
+Per sheet: blocks (tables with column profiles + samples, or small blocks verbatim), formula copy-groups and families with flags (`[array/spills]`, `[external]`, `[dynamic ref]`, `[#REF!]`, error counts, reads of a cell hidden under a merge), data validation (dropdowns, checkboxes, range-sourced lists), conditional formats, visual formatting regions (with named cell styles such as "Input" and which sides have a border), merges (repeats folded into patterns), filters/filter views and slicers, protected ranges, hidden rows/cols, banding, Excel tables, charts and pivots (with source ranges), pictures, shapes, text boxes and buttons over the grid (with their text and the macro they run), pictures in cells and data types (Excel Stocks/Geography, Google people and file chips), text with mixed formatting, unlocked input cells and hidden formulas, notes and comments with their authors, hyperlinks, grouped rows and columns (collapsed or not), and whether gridlines are off. Column widths and row heights are kept in workbook.json.gz only; a zero width or height counts as hidden. Google Drive comments (with replies and whether they're resolved) get their own section.
 
 ## Giving it to an AI agent
 
@@ -144,30 +149,37 @@ Point the agent at `INDEX.md`, then the `REPORT.md` it needs, and let it run the
 - "typed arithmetic" is hand-entered sums like `=3820+3770`: data entry, not logic.
 - `▸ range table …` is a data block, one line per column: type and range, `enum` for small value sets, `fmt` for the number format, `ƒN` when the column is computed.
 - "same layout as X" means a sheet is a structural clone of an earlier one with different data; `sheetgraph sheet <dir> "<name>" --full` shows it in full.
+- `[reads B8 inside merge A8:C8: blank]` flags a formula pointing at a cell a merge hides: it always reads blank, usually a mistake for the merge's top-left cell.
 - `⋯ rows A–B summarized` means a very large sheet's middle rows were summarized per column rather than kept: the column profiles still count every row, but sample rows and `cells` don't show those rows.
 - Flags: `[external]` another file, `[dynamic ref]` INDIRECT/OFFSET (graph edges incomplete), `[array/spills]`, `[#REF!]`, `[N errors]`.
+- Under Scripts, `RollUp: writes Summary!A1:B1,B2:B5 · uses Data!E10,… · calls Totals` is one procedure. `,…` means it also reaches cells whose address is built at run time (`Cells(i, 2)`); `(not in file)` is a sheet the macro creates; `ActiveSheet` is whichever sheet is active when it runs. `sheetgraph macro <dir> RollUp` prints its code.
 
 ## Layout
 
 | module | role |
 |---|---|
 | `model` | source-agnostic full-fidelity workbook (sparse cells, deduped styles/validations) |
-| `gsheets` | auth (service account JWT / ADC refresh / token), chunked parallel grid fetch with retry, conversion |
+| `gsheets` | auth (service account JWT / ADC refresh / token), chunked parallel grid fetch with retry, conversion, Drive comments |
 | `xlsx` | calamine: values, formulas, defined names, merges, VBA; csv |
-| `ooxml` | xlsx/xlsm package parts calamine skips: styles, validation, conditional formats, comments, hyperlinks, frozen/hidden rows and cols, filters, protection, tables, charts, pivots, external workbook links |
+| `ooxml` | xlsx/xlsm package parts calamine skips: styles, validation, conditional formats, comments and their authors, hyperlinks, frozen/hidden rows and cols, column widths, row heights, outline groups, gridlines, filters, protection, tables, charts, pivots, external workbook links, rich text, pictures and data types in cells, shapes and form controls, document properties, calculation settings, connections, Power Query |
+| `numfmt` | Excel number formats applied to numbers (`25%`, `$12.50`) |
 | `formula` | reference lexer (quoted and non-English sheet names, open-ended `A2:A`, whole rows/cols, `'[Book.xlsx]Sheet'`), R1C1 keys, function census |
-| `analyze` | copy-groups, families, blocks, cross-sheet edges, externals |
+| `analyze` | copy-groups, families, blocks, cross-sheet edges, externals, names, dropdown and rule refs |
+| `macros` | VBA / Apps Script procedures: triggers, sheets and ranges each writes and uses, calls |
 | `report` | REPORT.md |
 | `graph` | graph.json |
-| `query` | cells / trace / find |
+| `query` | cells / trace / find / macro |
 | `store` | on-disk layout, INDEX.md |
 
 ## Known gaps
 
 - **Apps Script / macros / add-ons on Google Sheets**: the Sheets API doesn't expose bound scripts, installed add-ons or triggers, and the Apps Script API doesn't work with service accounts. `--apps-script <scriptId>` fetches a project with user OAuth (scope `script.projects.readonly`). Custom functions are still detected from formulas.
-- **Local files**: xlsx/xlsm get the same coverage as Google Sheets, plus Excel tables and links to other workbooks (`'[Budget.xlsx]Data'!B2`). Images, shapes, sparklines and slicers aren't read (the report says when a file has them). xls, xlsb and ods get values, formulas, names and merges only.
-- Number-format display text (e.g. `$1,234.50`) is only rendered for dates in local files; the format itself is on every cell.
+- **Local files**: xlsx/xlsm get the same coverage as Google Sheets, plus Excel tables, links to other workbooks (`'[Budget.xlsx]Data'!B2`), shapes and pictures, connections and Power Query. Sparklines, Excel slicers and ActiveX controls aren't read (the report says when a file has them). xls, xlsb and ods get values, formulas, names and merges only.
+- **Comments on Google Sheets**: the Drive API doesn't say which cell a comment is on, only the text that cell showed. A comment is placed on a cell only when exactly one cell shows that text; otherwise REPORT.md lists it with the text it quotes.
+- **Pictures on Google Sheets**: the Sheets API doesn't return pictures placed over the grid or inserted into cells (an `=IMAGE()` formula is kept).
+- Number-format display text covers the common formats (percent, currency, thousands, decimals, scientific, sections); fractions, conditional sections and formats that mix digits with literals (`000-00-0000`) show the plain value. The format itself is on every cell.
 - Spill ranges of array formulas are flagged on the formula but the spilled cells look like plain values.
+- **What a macro touches** is read from its code, not by running it: literal addresses, sheets held in variables, `With` blocks, sheets passed to other procedures, code names, string constants and named ranges are followed; an address built at run time counts as the whole sheet, and a sheet picked at run time (from a cell, a loop over all sheets) isn't known. Apps Script code is listed by function without the ranges it touches.
 - INDIRECT/OFFSET targets are dynamic, so those edges are missing (flagged in the report).
 - **Very large local files**: up to 10 million cells are kept whole (about 0.2–0.4 GB of memory per million). Past that, a big sheet keeps its last 1,000 rows and its formulas, and summarizes the rows in between per column (counts, min and max, most common values; no sums), so memory stays near 2–3 GB however big the file is (a 160 MB xlsx with 44 million cells: 3 GB, 84 s). REPORT.md's column profiles still count every row and say which rows were summarized; `cells` only lists kept rows, and `find` reports text matches in summarized rows by column. The budget is `xlsx::CELL_BUDGET`; `xlsx::read_limited` takes another.
 - **Very large Google Sheets** are read whole, without that budget. Google allows up to 20 million cells, so a sheet near the limit can take several GB of memory.
@@ -178,7 +190,7 @@ Point the agent at `INDEX.md`, then the `REPORT.md` it needs, and let it run the
 cargo test
 ```
 
-`tests/fixtures/features.xlsx` exercises every xlsx feature the reader handles. It is generated by `tests/fixtures/make_features.py` (openpyxl 3.1.5); rerun the script after changing it.
+`tests/fixtures/features.xlsx` exercises every xlsx feature the reader handles. It is generated by `tests/fixtures/make_features.py` (openpyxl 3.1.5); rerun the script after changing it. `tests/fixtures/macros.xlsm` is the VBA fixture, generated the same way by `tests/fixtures/make_macros.py`, which also writes its VBA project from scratch.
 
 See [CONTRIBUTING.md](CONTRIBUTING.md) before opening an issue or pull request, and [SECURITY.md](SECURITY.md) to report a vulnerability.