Improve documentation website

CLAUDE.md

@@ -81,6 +81,7 @@ Check that the bug reproduces before the fix and is gone after it.
 - After any code change, check whether docstrings, inline docs, or `docs/` pages need updating and update them as part of the same task.
 - After updating `docs/`, verify it builds without errors: `python -m mkdocs build --strict`. In strict mode warnings are also errors, matching CI.
 - Write docstrings and documentation concisely and directly. No em-dashes or en-dashes; use commas, colons, or plain sentences instead. Avoid AI-sounding phrasing such as bold lead-ins ("**Why this matters:**"), filler constructions ("it is worth noting", "this allows", "this ensures"), and verbose qualifiers.
+- Avoid contrast constructions used for emphasis, such as "X, not Y", "not X but Y", "X rather than Y" or "X instead of Y". State what is true directly; keep a comparison only when the reader needs both sides.
 
 ### Key configuration options (`model_config`)
 

README.md

@@ -1,12 +1,13 @@
-# Loading XML files into a relational database
+# Load XML files into a SQL database with Python
 
 `xml2db` is a Python package that parses and loads XML files into a relational database. It handles complex 
 XML files which cannot be denormalized to flat tables, and works out of the box, without any custom mapping rules.
 
 It fits naturally into an [Extract, Load, Transform](https://docs.getdbt.com/terms/elt) pipeline: it 
-loads XML files into a relational data model that stays close to the source data while remaining easy to query as 
-flat database tables. The raw data can then be transformed using [DBT](https://www.getdbt.com/), SQL views, or stored procedures to produce
-more user-friendly tables.
+loads all the content of XML files into a relational data model that stays close to the source data while remaining
+easy to query as flat database tables. Parsing XML is slow compared to SQL, so you do not need to decide upfront what
+to extract: explore the loaded data, then build more user-friendly tables with [dbt](https://www.getdbt.com/), SQL
+views, or stored procedures, and change them later without parsing the files again.
 
 Starting from an XSD schema which represents a given XML structure, `xml2db` builds a data model, i.e. a set of database 
 tables linked to each other by foreign keys relationships. Then, it allows parsing and loading XML files into the 
@@ -15,10 +16,31 @@ database, and getting them back from the database into XML format if needed.
 This package uses `sqlalchemy` to interact with the database, so it should work with different database backends. 
 Automated integration tests run against PostgreSQL, MySQL, MS SQL Server and DuckDB. You may have to install additional 
 packages to connect to your database (e.g. `psycopg2` or `psycopg` for PostgreSQL, `pymysql` or `mysqlclient` for
-MySQL, `pyodbc` for MS SQL Server, or `duckdb-engine` for DuckDB).
+MySQL, `pyodbc` for MS SQL Server, or `duckdb-engine` for DuckDB). Through DuckDB, the loaded tables can also be
+transformed and exported to Parquet files, Delta Lake or Iceberg.
 
 **Please read the [package documentation website](https://cre-dev.github.io/xml2db) for all the details!**
 
+## When to use xml2db
+
+| Situation | Usually enough | What xml2db adds |
+|---|---|---|
+| A few fields from flat, record-like XML | `pandas.read_xml`, a short `lxml` script, or the XML functions of your database | Little: these are simpler |
+| One repeated record element with shallow nesting | XML readers that turn each record into a row, such as Spark's XML data source or DuckDB XML extensions | Normalized tables, with no nested columns to explode |
+| Deep nesting, repeated elements at several levels, large XSD | Hand-written mappings, one per element | Tables, keys and loading code generated from the XSD, no mapping to maintain |
+| Recurring feed of files following the same schema | A custom script per feed | The same load for every file, with redundant content stored once |
+| Needs not settled yet: which fields, which aggregations | Extracting the fields you expect to need, and parsing the files again when needs change | Everything loaded once: new questions are answered with SQL |
+| All the data must be kept, or written back to XML | Hand-written mappings, with the risk of dropping fields | Every element and attribute kept, with a round trip back to XML |
+
+`xml2db` needs an XSD schema. See the documentation for an [example](https://cre-dev.github.io/xml2db/#from-xml-to-tables)
+and for database-specific guides: [PostgreSQL](https://cre-dev.github.io/xml2db/databases/postgresql/),
+[SQL Server](https://cre-dev.github.io/xml2db/databases/sql-server/),
+[MySQL and MariaDB](https://cre-dev.github.io/xml2db/databases/mysql/),
+[DuckDB](https://cre-dev.github.io/xml2db/databases/duckdb/),
+[export to files, lakehouses and other databases](https://cre-dev.github.io/xml2db/databases/export/). To transform the loaded
+data, see [Transform with dbt](https://cre-dev.github.io/xml2db/dbt/). To only generate SQL tables from an XSD, see
+[XSD to SQL tables](https://cre-dev.github.io/xml2db/xsd-to-sql/).
+
 ## Installation
 
 The package can be installed, preferably in a virtual environment, using `pip`:
@@ -79,24 +101,35 @@ document.insert_into_target_tables()
 
 ## Testing
 
-Running the tests requires installing additional development dependencies, after cloning the repo, with:
+After cloning the repo, install the package with its development dependencies, plus DuckDB and `pytz` to run the
+database tests locally:
 
 ```bash
-pip install -e .[tests,docs]
+pip install -e .[tests,docs] duckdb_engine pytz
 ```
 
-Run all tests with the following command:
+Run all tests, including database integration tests, against an in-memory DuckDB database:
 
 ```bash
-python -m pytest
+TZ="Europe/Paris" DB_STRING="duckdb:///:memory:" python -m pytest
 ```
 
-Integration tests require write access to a PostgreSQL or MS SQL Server database; the connection string is provided as an
-environment variable `DB_STRING`. If you want to run only conversion tests that do not require a database you can run:
+Round-trip tests compare datetimes read back from the database in the `Europe/Paris` time zone, so `TZ` must be set to
+it.
+
+Integration tests use the database given by the `DB_STRING` connection string. To run them against another database,
+for instance PostgreSQL:
 
 ```bash
-pytest -m "not dbtest"
-`````
+TZ="Europe/Paris" DB_STRING="postgresql+psycopg2://user:pw@localhost:5432/testdb" python -m pytest
+```
+
+Continuous integration runs them against DuckDB, PostgreSQL, MySQL and MS SQL Server. To run only the tests that do
+not need a database:
+
+```bash
+TZ="Europe/Paris" python -m pytest -m "not dbtest"
+```
 
 ## Contributing
 

docs/api/data_model.md

@@ -1,6 +1,6 @@
 ---
 title: "DataModel"
-description: "API reference for the xml2db DataModel class: constructor parameters, methods for parsing XML files, visualizing the schema as ERD or tree, and generating DDL statements."
+description: "API reference for the xml2db DataModel class: constructor parameters, parsing XML files, ERD and tree views of the schema, and DDL generation."
 ---
 
 # DataModel

docs/api/overview.md

@@ -1,6 +1,6 @@
 ---
 title: "API Overview"
-description: "Overview of the xml2db Python API: building DataModel objects, parsing and loading XML files into a database, and extracting data back to XML, including multiprocessing patterns."
+description: "Overview of the xml2db Python API: building a DataModel, loading XML files into a database, extracting data back to XML, and multiprocessing."
 ---
 
 # API Overview

docs/cli.md

@@ -1,6 +1,6 @@
 ---
-title: "CLI usage"
-description: "Reference for the xml2db command-line interface: import XML files, render ERDs and DDL, and launch the interactive browser explorer."
+title: "Command line: import XML into a database, render DDL"
+description: "Reference for the xml2db command line: import XML files into a database, render diagrams and DDL from an XSD, and explore a schema in the browser."
 ---
 
 # CLI usage
@@ -81,6 +81,8 @@ xml2db render schema.xsd --format ddl --db-type postgresql
 xml2db render schema.xsd --format erd --output diagram.md
 ```
 
+[XSD to SQL](xsd-to-sql.md) shows the generated DDL and how to adjust it.
+
 ## xml2db serve
 
 Launch an interactive schema explorer in the browser.

docs/configuring.md

@@ -1,6 +1,6 @@
 ---
 title: "Configuring your data model"
-description: "Configure xml2db's data model via model_config: override column types, control field elevation, adjust deduplication, simplify XSD choice groups, and add custom indexes."
+description: "Configure the tables xml2db derives from an XSD: column types and names, flattening of child elements, deduplication, choice groups and indexes."
 ---
 
 # Configuring your data model
@@ -391,6 +391,9 @@ records.
 
 Configuration: `"reuse":` `True` (default) or `False` (disable)
 
+The root table is always deduplicated, so that loading the same file twice stores it once: setting `"reuse": False` on
+it raises an error.
+
 !!! example
     Disabling deduplication for a given table:
     ``` python

docs/databases/duckdb.md

@@ -0,0 +1,75 @@
+---
+title: "Load XML into DuckDB"
+description: "Load complex XML files into DuckDB tables generated from an XSD with xml2db, query them without a server, or read simple XML with DuckDB extensions."
+---
+
+# Load XML into DuckDB
+
+DuckDB runs inside the Python process and stores a whole database in a single file. With `xml2db`, it gives a
+relational copy of XML files without any server to set up, which suits local analysis, trying a data model before
+deploying it, and tests.
+
+## Reading XML in DuckDB
+
+DuckDB community extensions, such as [webbed](https://duckdb.org/community_extensions/extensions/webbed), add a
+`read_xml` function that reads XML files directly in SQL. It turns a repeated element into rows, with nested elements
+as `STRUCT` and `LIST` columns, and needs no XSD. It fits quick queries on record-like files.
+
+`xml2db` fits files described by an XSD, with deep nesting or repeated elements at several levels: it loads them into
+normalized tables with keys, stores identical records once, and gives the same tables for every file of a feed. Both
+work in the same DuckDB database, so each can be used where it fits.
+
+## Connecting
+
+``` bash
+pip install xml2db duckdb-engine
+
+xml2db import invoices.xml invoices.xsd --connection-string "duckdb:///invoices.duckdb"
+```
+
+``` py title="Python API" linenums="1"
+from xml2db import DataModel
+
+data_model = DataModel(
+    xsd_file="invoices.xsd",
+    connection_string="duckdb:///invoices.duckdb",
+)
+for xml_file in ["invoices_1.xml", "invoices_2.xml"]:
+    data_model.parse_xml(xml_file).insert_into_target_tables()
+```
+
+Use `duckdb:///:memory:` for a database that only lives during the Python process.
+
+Then query the file with the DuckDB command line or from Python:
+
+``` py
+import duckdb
+
+con = duckdb.connect("invoices.duckdb", read_only=True)
+con.sql("SELECT id, customer_name, CAST(date AS DATE) AS date FROM invoice").show()
+```
+
+XSD `date` and `time` values are stored as text, since they may carry a time zone: cast them in queries or views.
+
+## Bulk loading
+
+Batches of 100 rows or more are written to a temporary CSV file and loaded with DuckDB's `read_csv`, every column being
+read as text and then cast to its target type. Smaller batches use `executemany`.
+
+## One writer at a time
+
+A DuckDB file accepts a single writing process. When loading files from several processes, parse in parallel and
+load one file at a time, as shown in [Multiprocessing](../how_it_works.md#multiprocessing). Readers using
+`read_only=True` cannot open the file while a process writes to it either.
+
+## Keys and names
+
+Primary keys are filled from DuckDB sequences, named `pk_sequ_<table>`. Table and column names longer than 63
+characters are shortened with a hash suffix, as for PostgreSQL. This limit comes from the SQLAlchemy integration: DuckDB
+itself accepts longer names.
+
+## Transforming and exporting
+
+Keep the DuckDB file as the raw copy of your XML data, and build the tables you need with SQL views or
+[dbt](../dbt.md). DuckDB then exports them to Parquet files, Delta Lake, Iceberg and other targets that `xml2db` does
+not support directly: see [Export to files, lakehouses and other databases](export.md).