Restructure docs lifecycle

[aleksandarkovacevic30]

Description

Work Item ID: AB#

Why

The documentation was organised by feature area. To find the page explaining a feature you had to already know what CluedIn calls it, and the same subject was covered in several places at once - a concept in Key terms, the same concept again in a training module, and a third time in a Knowledge Base article, each drifting from the others.

This restructures it around the path data takes through the platform, so a reader who knows what they are trying to do can find the page without knowing the product's vocabulary first.

The new shape

Ingest → Model → Master → Govern & improve → Publish & consume

with Administer, Deploy & operate and Develop & APIs alongside, and Learning paths, Playbooks, Solutions and Troubleshooting as supporting material.

Section	URL	Answers
Get started	/get-started	What is CluedIn and how do I begin?
Ingest	/ingest	How do I get data into CluedIn?
Model	/model	How do I describe my data so CluedIn understands it?
Master	/master	How do I turn records into trusted golden records?
Govern & improve	/govern	How do I improve and govern the data?
Publish & consume	/publish	How do I get trusted data out?
Administer	/administer	How do I manage the product?
Deploy & operate	/operate	How do I run the platform?
Develop & APIs	/develop	How do I automate or extend CluedIn?
Solutions & integrations	/solutions	How does CluedIn fit with the rest of my stack?
Learning paths	/learning-paths	How do I learn CluedIn for my role?
Playbooks	/playbooks	How do I run a CluedIn project?
Troubleshooting	/troubleshooting	Something is not working — what do I check?
Release notes	/release-notes	What changed, and what is supported?
Archive	/archive	Where did the old material go?

Two rules hold it together:

1. One canonical home per subject. Learning paths, playbooks, solution guides and troubleshooting articles link to that home instead of restating it — so there is only ever one explanation of a feature to keep current.
2. Every article declares a content type — concept, how-to, tutorial, reference or troubleshooting — so the four kinds of documentation stay distinguishable. The badge is rendered at the top of each article and in every section index.

The move is generated, not hand-arranged

This is the part worth reviewing. docs/ was produced by a script from a routing table, so every placement decision is stated in one reviewable place rather than implied by 674 file moves:

File	What it is
_migration/ia.py	The information architecture — the sections, their URLs, their audiences and intros
_migration/targets_a.py, _b.py, _c.py	The routing table: for every source article, its one section, subsection, content type and order
_migration/migrate.py	Performs the move, rewrites front matter and internal links, writes the section indexes, records redirect_from
_migration/validate.py	Checks the generated tree before Jekyll builds it
_migration/report.md	The generated record: every routing decision, every retitle, every repaired link
_migration/redirects.csv	Old URL → new URL for all 565 pre-restructure URLs

Nothing was dropped, and no URL broke

• All 568 source articles are accounted for. 548 have a new home; the other 20 were landing pages containing nothing but a card grid pointing at their children, which the generated section index now produces
  automatically.
• Every one of the 565 pre-restructure URLs still resolves. Each page carries redirect_from, and jekyll-redirect-from builds a stub at the old address.
• Section indexes are no longer hand-maintained. _includes/children.html lists a section's children from their front matter, so a page can never be added to a section and then be missing from its index.

Two link problems were fixed on the way across:

• 65 relative links (./aks, ../assets/images/x.png) are now resolved against the page's old URL before being rewritten, so they still resolve after a page changed depth.
• 17 images under docs/180-microsoft-integration/purview/media/ moved to assets/images/.... Jekyll published them at their source path, so those pages' relative links never resolved on the live site. They do now.

Contributor guidelines

The restructure only holds if the next article is placed the same way, so the second commit adds CONTRIBUTING.md — a placement procedure rather than a style guide:

1. Product documentation, or supporting material? Supporting material links to the canonical page instead of restating it. The test: delete every sentence in the draft that explains how a feature works, replace each with a link, and see whether a useful article remains.
2. Which section? A table per lifecycle stage, plus explicit rulings on the four boundaries that are genuinely ambiguous — Administer vs Deploy & operate ("add a user" vs "add a node"), Model vs Master, Model vs Govern, Publish vs Develop — and where symptom-first articles go.
3. Which subsection? Three conditions for creating a new one, and the hard three-level navigation ceiling.
4. Which content type? The test for each, and the how-to vs tutorial split.
5. File name and front matter, with templates for an article and a subsection index.

It also records what is easy to get wrong: the numeric filename prefix must equal nav_order; the directory name is not the URL (05-govern-and-improve serves /govern); parent and grand_parent are page titles spelled
exactly; and section indexes are generated and must never be hand-maintained.

Every fact in it was checked against the tree, including the six subsections whose URL differs from their directory name and the nav_order-equals-prefix invariant, which holds for all 517 articles.

Navigation reachability

validate.py checked that a parent resolves, but not that the chain climbs all the way to a top-level section. A page whose chain stops short is still built and still served — it is simply absent from the sidebar, so the only way
to find it is to already know the URL.

Verified against a real build rather than assumed: dropping grand_parent from one page took the Rules sidebar from 8 links to 7 while the page kept being served. validate.py now walks the chain for every published page and fails if
one cannot be reached, and reports how many pages are deliberately held out of the sidebar with nav_exclude.

Audit of the current tree: 490 published pages, 486 reachable from the sidebar. The remaining four carry nav_exclude: true — docs/tag.md and terms-of-service.md are utility pages, and the two under Deploy & operate
(sync-aks-to-akv, azure-policy-exemption) carried nav_exclude before the restructure too, so they are pre-existing choices rather than regressions.

One stale key removed: 170-2026-01-00-support.md still declared great_grand_parent: "PaaS operations", a section that no longer exists. A/B builds confirm just-the-docs 0.7 ignores it, so the page was reachable either way — this is cleanup, not a fix for broken navigation.

Supporting changes

File	Change
_config.yml	jekyll-redirect-from, the content-type vocabulary, nav grouping
Gemfile	jekyll-redirect-from
_layouts/cluedin.html	Content-type badge, generated section index
_includes/children.html	New — lists a section's children
assets/js/nav-groups.js	New — separates the lifecycle sections from the supporting material in the sidebar
_sass/color_schemes/clue.scss	Badges, section index, audience strip
index.markdown → index.md	Home page follows the lifecycle
README.md	Points at the guidelines; no longer cites the pre-restructure path docs/250-rest-api/

How to review this

The diff is 674 files, but almost all of it is mechanical. Suggested order:

1. _migration/ia.py — the sections, their URLs, and the audience/intro text that appears on each section index. This is where the structural decisions live.
2. _migration/targets_*.py — the routing table. If an article is in the wrong place, this is the line to comment on; the file move is the consequence, not the decision.
3. _migration/report.md — every routing decision, every retitle, every repaired link, in one generated document.
4. CONTRIBUTING.md — the go-forward rules.
5. The theme changes — _layouts/cluedin.html, _includes/children.html, assets/js/nav-groups.js, _sass/color_schemes/clue.scss.

The docs/ diff itself is mostly renames with 90–100% similarity and does not need reading page by page.

To move an article, edit the routing table and re-run — see _migration/README.md. Note that the routing table describes the pre-restructure tree; once this merges, docs/ is authored directly and a new article is a new file, not a new routing entry.

Verification

python _migration/validate.py     595 pages, 0 errors, 5 warnings
bundle exec jekyll build          943 pages and redirect stubs

The 5 warnings are links that were already broken before this change and are left alone; all 45 pre-existing broken links are listed in _migration/report.md.

Known, and deliberately not addressed here

• 42 articles are not linked from their own section index page. 18 subsection indexes set list_children: false and carry a hand-written list inherited from the old page body, and 11 of those lists are incomplete — /govern/enrichment lists 4 of 20 enrichers, /govern/rules 4 of 9. Every one of those articles is in the sidebar, so this is not a reachability failure, but it is exactly what the generated index exists to prevent. Fixing it means deleting the stale lists and, to lose nothing, moving their descriptions into each child's summary. That is an editorial change to live content and belongs in its own PR.
• CI does not run validate.py. Adding it to .github/workflows/main.yml would make these checks gate a deploy rather than rely on a contributor remembering to run them.