Skip to content

Commit 9e16877

Browse files
hossain-rayhanhossain-rayhan
authored
Add Beanie + Document DB Playground (#2)
Add Beanie + DocumentDB Playground Signed-off-by: Rayhan Hossain <hossain.rayhan@outlook.com>
1 parent 284acfb commit 9e16877

14 files changed

Lines changed: 1051 additions & 3 deletions

File tree

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,11 @@ yarn-debug.log*
55
yarn-error.log*
66
.pnpm-debug.log*
77

8+
# Python
9+
.venv/
10+
__pycache__/
11+
*.pyc
12+
813
# Runtime / logs
914
*.log
1015
logs/

‎README.md‎

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,9 +16,10 @@ playground's README states what it needs.
1616
| Playground | Language / Stack | What it shows |
1717
| --------------------------------- | ---------------------- | ------------------------------------------------------------------------- |
1818
| [mongoose](playgrounds/mongoose/) | Node.js — Mongoose ODM | Express REST API + a CRUD/compatibility test suite using the Mongoose ODM. |
19+
| [beanie](playgrounds/beanie/) | Python — Beanie ODM | FastAPI REST API + a CRUD/compatibility test suite using the Beanie ODM. |
1920

20-
More playgrounds are planned (for example **PyMongo**, **Beanie**, and other
21-
MongoDB drivers). Contributions are welcome.
21+
More playgrounds are planned (for example **PyMongo** and other MongoDB
22+
drivers). Contributions are welcome.
2223

2324
## Getting Started
2425

@@ -36,14 +37,23 @@ cd playgrounds/mongoose
3637
./scripts/run-app.sh # or run the demo REST API
3738
```
3839

40+
To try the Beanie playground:
41+
42+
```bash
43+
cd playgrounds/beanie
44+
./scripts/run-test.sh # start DocumentDB locally and run the compatibility suite
45+
./scripts/run-app.sh # or run the demo REST API
46+
```
47+
3948
## Repository Layout
4049

4150
```
4251
documentdb-playground/
4352
├── README.md
4453
├── LICENSE
4554
└── playgrounds/
46-
└── mongoose/ # Node.js + Mongoose ODM
55+
├── mongoose/ # Node.js + Mongoose ODM
56+
└── beanie/ # Python + Beanie ODM
4757
```
4858

4959
## License

‎playgrounds/beanie/README.md‎

Lines changed: 302 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,302 @@
1+
# Beanie with DocumentDB (local)
2+
3+
This playground shows how to use [Beanie](https://beanie-odm.dev/), a popular
4+
**asynchronous Python ODM** built on [Motor](https://motor.readthedocs.io/)
5+
(async PyMongo) and [Pydantic](https://docs.pydantic.dev/), against DocumentDB —
6+
running **entirely on your machine**. It includes:
7+
8+
- a small **FastAPI + Beanie REST API** (`app/`), and
9+
- a standalone **Beanie CRUD/compatibility test suite**
10+
(`app/beanie_crud_test.py`) that exercises connect, index creation, insert,
11+
query, update, aggregation, unique-index enforcement, and delete.
12+
13+
There is **no Kubernetes and no cloud**. DocumentDB runs as the
14+
[`documentdb-local`](https://github.com/documentdb/documentdb) emulator in a
15+
single Docker container, and the app/test run as local Python processes that
16+
connect straight to it.
17+
18+
> **What is Beanie?** Beanie is a **Python library** (an async ODM, Object
19+
> Document Mapper), not a CLI tool or a server. Your application imports it to
20+
> define `Document` models (Pydantic classes) and talk to a MongoDB-compatible
21+
> database over Motor. Here it is used by the demo **app**
22+
> ([`app/main.py`](app/main.py)) and the standalone **test script**
23+
> ([`app/beanie_crud_test.py`](app/beanie_crud_test.py)).
24+
25+
## Architecture
26+
27+
Everything is local. The emulator container exposes the MongoDB wire protocol on
28+
`localhost:10260`; the Python processes connect to it directly.
29+
30+
```
31+
Your machine (WSL / Linux / macOS)
32+
┌──────────────────────────────────────────────────────────────────┐
33+
│ ┌────────────────────┐ ┌──────────────────────────────┐ │
34+
│ │ beanie app / │ TLS, │ documentdb-local (Docker) │ │
35+
│ │ test script │ wire │ ┌────────────┐ ┌─────────┐ │ │
36+
│ │ (Python + Beanie) │────────▶│ │ Gateway │▶│Postgres │ │ │
37+
│ │ │ :10260 │ │ (10260) │ │ (engine)│ │ │
38+
│ └────────────────────┘ │ └────────────┘ └─────────┘ │ │
39+
│ └──────────────────────────────┘ │
40+
└──────────────────────────────────────────────────────────────────┘
41+
```
42+
43+
Beanie talks to the emulator through Motor exactly as it would to a standalone
44+
`mongod`, with a few required options (see [Connecting Beanie to
45+
DocumentDB](#connecting-beanie-to-documentdb)).
46+
47+
## Prerequisites
48+
49+
- **Docker** (to run the `documentdb-local` emulator)
50+
- **Python 3.10+** with `venv` (to run the app and test suite)
51+
52+
The scripts create a Python virtualenv and install dependencies for you. On
53+
Windows, run these from a **WSL** shell.
54+
55+
## Quick Start
56+
57+
From this directory (`playgrounds/beanie/`). `run-test.sh` and `run-app.sh`
58+
are **two independent operations** — each starts DocumentDB on its own if it
59+
isn't already running.
60+
61+
### Option A — run the test suite
62+
63+
```bash
64+
# Run the full CRUD/compatibility suite end-to-end.
65+
# Starts DocumentDB in Docker (first run pulls the image), then runs the tests.
66+
./scripts/run-test.sh
67+
```
68+
69+
### Option B — run the demo REST API
70+
71+
```bash
72+
# Starts DocumentDB (if not already running) and serves the API on :3000.
73+
# This stays in the foreground until you press Ctrl-C.
74+
./scripts/run-app.sh
75+
```
76+
77+
Stop the database when you are done:
78+
79+
```bash
80+
./scripts/stop-documentdb.sh
81+
```
82+
83+
The suite should end with `Passed: 13 Failed: 0`.
84+
85+
## Trying the API
86+
87+
With `./scripts/run-app.sh` running, the API is on `http://localhost:3000`
88+
(interactive docs at `http://localhost:3000/docs`):
89+
90+
```bash
91+
# Health
92+
curl -s http://localhost:3000/health
93+
# {"status":"healthy","db":"connected"}
94+
95+
# Create a book
96+
curl -s -X POST http://localhost:3000/books \
97+
-H 'Content-Type: application/json' \
98+
-d '{"title":"Dune","author":"Herbert","genres":["sci-fi"],"pages":412,"rating":5}'
99+
100+
# List books
101+
curl -s http://localhost:3000/books | jq .
102+
103+
# Count books per genre (aggregation)
104+
curl -s http://localhost:3000/stats/genres | jq .
105+
```
106+
107+
## Connecting Beanie to DocumentDB
108+
109+
The DocumentDB gateway speaks the MongoDB wire protocol but advertises itself as
110+
a **standalone** server over **TLS** (with a self-signed cert). Beanie talks to
111+
it through Motor, which therefore needs these options (see [`app/db.py`](app/db.py)):
112+
113+
```python
114+
from motor.motor_asyncio import AsyncIOMotorClient
115+
from beanie import init_beanie
116+
117+
client = AsyncIOMotorClient(
118+
uri,
119+
directConnection=True, # gateway is standalone, not a replica set
120+
tls=True, # gateway only accepts TLS
121+
tlsAllowInvalidCertificates=True, # emulator uses a self-signed cert
122+
)
123+
await init_beanie(database=client[db_name], document_models=[Book])
124+
```
125+
126+
The connection string built by the scripts is:
127+
128+
```
129+
mongodb://<user>:<pass>@localhost:10260/?tls=true&tlsAllowInvalidCertificates=true&directConnection=true
130+
```
131+
132+
For production against a real (non-emulator) deployment, set `TLS_INSECURE=false`
133+
and pass a CA bundle via `tlsCAFile` instead of `tlsAllowInvalidCertificates`.
134+
135+
## Configuration Reference
136+
137+
All settings are passed via environment variables; there is no config file.
138+
139+
### Emulator + scripts (`scripts/`)
140+
141+
Read by [`lib.sh`](scripts/lib.sh) and the `start`/`stop`/`run` scripts.
142+
143+
| Variable | Default | Description |
144+
| ---------------------- | -------------------------------------------------------- | -------------------------------------------------------- |
145+
| `DOCUMENTDB_IMAGE` | `ghcr.io/documentdb/documentdb/documentdb-local:latest` | Emulator image to pull/run. |
146+
| `DOCUMENTDB_CONTAINER` | `documentdb-local` | Docker container name. |
147+
| `DOCUMENTDB_HOST` | `localhost` | Host the app/test connect to. |
148+
| `DOCUMENTDB_PORT` | `10260` | Host port mapped to the gateway. |
149+
| `DOCUMENTDB_USERNAME` | `docdbadmin` | Emulator admin username. **Do not use `documentdb`** (reserved — the gateway rejects it as "Username is invalid"). |
150+
| `DOCUMENTDB_PASSWORD` | `Documentdb!Local1` | Emulator admin password. If you use special characters, URL-encode them in the connection string. |
151+
| `PORT` | `3000` | Local port the FastAPI app listens on (`run-app.sh`). |
152+
153+
### App + test script (`app/`)
154+
155+
Read by [`app/db.py`](app/db.py), [`app/main.py`](app/main.py), and
156+
[`app/beanie_crud_test.py`](app/beanie_crud_test.py). The scripts set `MONGO_URI`
157+
for you from the variables above.
158+
159+
| Variable | Default | Description |
160+
| ----------------------------- | ---------------- | -------------------------------------------------------------------------------------------- |
161+
| `MONGO_URI` | _(set by scripts)_ | DocumentDB connection string. The test script also accepts it as the first CLI argument. |
162+
| `MONGO_DB` | `beanie_demo` (app), `beanie_test` (test) | Database name Beanie connects to. |
163+
| `TLS_INSECURE` | `true` | When `true`, accepts the self-signed cert. Set `false` for CA-verified TLS. |
164+
| `SERVER_SELECTION_TIMEOUT_MS` | `10000` | How long Motor waits to select a server before erroring. |
165+
| `PORT` | `3000` | Port the FastAPI/Uvicorn API listens on. |
166+
167+
## DocumentDB Compatibility Notes
168+
169+
Verified against `documentdb-local:latest` (release `0.114`):
170+
171+
| Beanie feature | Status | Notes |
172+
| --------------------------------------- | ------------- | --------------------------------------------------------------------- |
173+
| CRUD (`insert`/`find`/`save`/`delete`) | ✅ Supported | Standard document operations work as expected. |
174+
| `get(id)` / `_id` point lookups | ✅ Supported | Works on `0.114`. (Older gateway `0.109.0` failed these with "trying to open a pruned relation".) |
175+
| Index creation via `Settings.indexes` | ✅ Supported | Built asynchronously by the engine; `createIndexes` returns in ~2s. Avoid `collation`. |
176+
| Unique indexes | ✅ Supported | Duplicate keys raise `DuplicateKeyError` (code `11000`). |
177+
| Aggregation pipelines | ✅ Common stages | `$match`, `$group`, `$unwind`, `$sort`, etc. Atlas-only stages differ. |
178+
| `$vectorSearch` | ❌ Not supported | Atlas-only operator. |
179+
| Index `collation` | ❌ Not supported | `createIndex.collation is not implemented yet`; omit it. |
180+
| Change streams / transactions | ⚠️ Check version | Verify against your DocumentDB version before relying on them. |
181+
182+
The CRUD test suite ([`app/beanie_crud_test.py`](app/beanie_crud_test.py))
183+
covers the supported rows above and prints a pass/fail summary.
184+
185+
## Running the Test Suite Manually
186+
187+
`scripts/run-test.sh` sets `MONGO_URI` and runs the suite for you. To run it
188+
directly against any reachable connection string:
189+
190+
```bash
191+
cd app
192+
python3 -m venv .venv
193+
.venv/bin/pip install -r requirements.txt
194+
MONGO_URI="mongodb://docdbadmin:Documentdb!Local1@localhost:10260/?tls=true&tlsAllowInvalidCertificates=true&directConnection=true" \
195+
.venv/bin/python beanie_crud_test.py
196+
```
197+
198+
Expected output:
199+
200+
```
201+
Beanie DocumentDB compatibility test
202+
====================================
203+
✅ connect
204+
✅ create indexes
205+
✅ insert_one (Document.insert)
206+
✅ insert_many
207+
✅ get by _id (Document.get)
208+
✅ find with filter + sort + limit
209+
✅ count_documents
210+
✅ update_one ($set)
211+
✅ find_one_and_update (returns new)
212+
✅ aggregation ($unwind/$group)
213+
✅ unique index enforcement (duplicate sku rejected)
214+
✅ delete_one
215+
✅ cleanup (drop collection)
216+
====================================
217+
Passed: 13 Failed: 0
218+
```
219+
220+
## What the Scripts Do
221+
222+
| Script | Purpose |
223+
| ----------------------------- | ---------------------------------------------------------------------------------------- |
224+
| `scripts/start-documentdb.sh` | Start the local emulator container and wait until the gateway is ready. |
225+
| `scripts/run-test.sh` | Start DocumentDB (if needed), set up the venv, and run the Beanie CRUD/compatibility suite. |
226+
| `scripts/run-app.sh` | Start DocumentDB (if needed), set up the venv, and run the FastAPI + Beanie demo app. |
227+
| `scripts/stop-documentdb.sh` | Stop and remove the emulator container (full reset of its data). |
228+
| `scripts/lib.sh` | Shared helpers: container lifecycle, readiness wait, connection-string builder, and venv setup. |
229+
230+
## Verification
231+
232+
- `./scripts/run-test.sh` ends with `Passed: 13 Failed: 0`.
233+
- With `./scripts/run-app.sh` running, `curl http://localhost:3000/health`
234+
returns `{"status":"healthy","db":"connected"}`, `POST /books` returns `201`
235+
with the created document, and `GET /stats/genres` returns per-genre counts.
236+
237+
## Cleanup
238+
239+
- **App / test:** press `Ctrl-C` to stop the app; the test exits on its own.
240+
Optionally remove the virtualenv: `rm -rf app/.venv`.
241+
- **Emulator:** `./scripts/stop-documentdb.sh` removes the container and all its
242+
data.
243+
244+
## Troubleshooting
245+
246+
### `AuthenticationFailed: Username is invalid.`
247+
248+
The emulator rejects certain reserved usernames — notably `documentdb`. Use a
249+
different admin username (the default here is `docdbadmin`). If you changed
250+
`DOCUMENTDB_USERNAME`, recreate the container so the new credentials take effect:
251+
252+
```bash
253+
./scripts/stop-documentdb.sh && ./scripts/start-documentdb.sh
254+
```
255+
256+
### `ServerSelectionTimeoutError` / TLS handshake failures
257+
258+
The gateway requires TLS. Confirm the connection string includes `tls=true` and
259+
`tlsAllowInvalidCertificates=true` (the scripts add these). Make sure the
260+
emulator is running: `docker ps` should list `documentdb-local`, and
261+
`docker logs documentdb-local` should show the gateway accepting connections.
262+
263+
### Port `10260` already in use
264+
265+
Another process (or a previous emulator) holds the port. Stop it, or run on a
266+
different port:
267+
268+
```bash
269+
DOCUMENTDB_PORT=10261 ./scripts/start-documentdb.sh
270+
DOCUMENTDB_PORT=10261 ./scripts/run-test.sh
271+
```
272+
273+
### Credentials changed but auth still fails
274+
275+
The username/password are baked into the container at creation time. Changing
276+
`DOCUMENTDB_USERNAME`/`DOCUMENTDB_PASSWORD` only takes effect after you recreate
277+
the container (`stop-documentdb.sh` then `start-documentdb.sh`).
278+
279+
### `createIndex.collation is not implemented yet`
280+
281+
A model index uses `collation`. Remove it; DocumentDB does not support collation
282+
indexes. The models in this playground intentionally avoid it.
283+
284+
## Directory Layout
285+
286+
```
287+
beanie/
288+
├── README.md
289+
├── app/
290+
│ ├── requirements.txt
291+
│ ├── db.py # Motor connection + init_beanie (DocumentDB options)
292+
│ ├── main.py # FastAPI REST API (/books, /health, /stats)
293+
│ ├── models/
294+
│ │ └── book.py # Example Beanie Document model
295+
│ └── beanie_crud_test.py # Standalone CRUD/compatibility test suite
296+
└── scripts/
297+
├── lib.sh # Connection-string builder + readiness wait + venv setup
298+
├── start-documentdb.sh # Start the local emulator (Docker)
299+
├── run-app.sh # Run the demo app locally
300+
├── run-test.sh # Run the test suite locally
301+
└── stop-documentdb.sh # Stop + remove the emulator
302+
```

0 commit comments

Comments
 (0)