create simple autogenerated table builds for the data dictionary sections in the guides

[danabauer]

yes, that's the plan. We just don't have simple autogenerated table builds yet

Originally posted by Dana Bauer (@danabauer) in #281 (comment)

[lowlydba]

Dana Bauer (@danabauer) Can you expand on this a bit? I can probably take a go at it once I understand the goal better

[danabauer]

John McCall (@lowlydba) Thanks for taking a look. I'd like to create simple data dictionary tables from our Pydantic models. Right now we create these tables by hand in our data guides. See here. Ideally we could have a semi-automated process for generating "flat" column definitions and dropping them into the MDX files in Docusaurus as a polished element. Check out the the data dictionaries in the DuckLake docs for a clean example the schema team really liked.

As you pointed out in our conversation earlier today, the first step is sorting out the tooling we're using for docs generation from Pydantic. Right now we're using a custom workflow for the nested, interactive reference docs. I'm wondering we could do something simpler for the table generation.

I said "semi-automated" because it seems like overkill to have table generation triggered with every release.

[lowlydba]

As you pointed out in our conversation earlier today, the first step is sorting out the tooling we're using for docs generation from Pydantic. Right now we're using a custom workflow for the nested, interactive reference docs. I'm wondering we could do something simpler for the table generation.

I poked around the schema codegen CLI, and I think extending it to do the first-level "simple" attributes into a data dict would be fairly trivial.

From where we are now, doing fully automated (at least until the point of a PR) should be pretty low-lift