diff --git a/README.md b/README.md index a88caa0..8406551 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ So the trade is really: | | traditional RDB | no-RDB DBs | |---|---|---| -| a *selective, indexed* query | **wins** (often by a lot) | scans instead | +| a *selective, indexed* query | **wins** (often by a lot) | scans instead *(unless `usememattr` is on)* | | everything else (most scans, aggregates) | ~the same per query | ~the same per query | | concurrent query **throughput** | single-threaded | **N DBs in parallel, sharing the OS page cache** | | operational complexity | RDB + HDB + gateway | one DB process | @@ -54,13 +54,17 @@ A few deliberate changes make the single-database model work: - **mapped** (the default) — maps the whole database once (via `.Q.MAP`) and keeps it mapped, refreshing only the live partition on each flush, so reads skip the per-query mapping entirely. Fastest reads; query latency stays flat as history grows. - **deferred** — the stock kdb+ behaviour above: re-map per query. Slower, and increasingly so as history grows, but simple and always fresh. -The mode is a per-process flag, `-.idb.usemapping` (`1` = mapped, the default; `0` = deferred), set in the `extras` column of `process.csv` — see `bench/process.csv`, which runs one DB each way. +The mode is a per-process flag, `-.idb.usemapping` (`1` = mapped, the default; `0` = deferred), set in the `extras` column of `process.csv` — see `bench/process.csv`, which runs one DB each way. A second per-process flag, `-.idb.usememattr`, adds an in-memory index for the live partition on top of mapped mode — see [Attributes](#key-design-choices) below. Both serve identical results; mapped is just faster. On the [benchmark](#benchmarks)'s single-day, 50M-row database, deferred costs **5–50% more per query** depending on the query; least on compute-heavy aggregates, where the mapping cost is lost in the work, and most on cheap scans, where it dominates. That gap widens with the number of partitions and columns each query touches, so on a database with real history it will be larger than these one-partition figures suggest. **Page cache (where the data actually lives.)** Today's partition is warm as a *side effect of writing it*: the WDB has just written those pages, so they're already resident before any DB reads them, and because the cache belongs to the kernel rather than to any one process, every DB shares it instead of each holding its own copy. That's what makes the freshest data (which is also the most-queried data) free to serve. If you want to manually control exactly what stays in the page cache you can: tools like [`vmtouch`](https://hoytech.com/vmtouch/) will pull a directory into cache (`vmtouch -t`) or pin it there (`vmtouch -l`). The pack deliberately doesn't, leaving it to the kernel, which is the right default for most workloads. -**Attributes (the one place the RDB still wins.)** kdb+ attributes like `g#` (grouped) make selective lookups on a column near-instant, but they **can't be maintained on a partition that's being continuously appended to**. So *today's* partition has no index: an intraday `where sym=AAPL` scans it. An in-memory RDB keeps `g#` on `sym`, so it wins those selective intraday lookups, sometimes by 100×+. At end of day this pack sorts the day's partition and applies a `p#` (parted) attribute, so every *previous* day gets fast lookups too; only the live day is un-indexed. And if a handful of selective live queries genuinely matter, the no-RDB model doesn't stop you adding a small, *targeted* in-memory cache or CEP process for just those, which is much less resource intensive than full general-purpose RDB. +**Attributes (the one place the RDB still wins — mostly.)** kdb+ attributes like `g#` (grouped) make selective lookups on a column near-instant, but an index-carrying attribute **can't be maintained on an on-disk column that's being appended to**: the index lives in a footer after the data, so kdb+ drops it on every append rather than shift it under any memory-mapped reader. So *today's* partition has no index: an intraday `where sym=AAPL` scans it. An in-memory RDB keeps `g#` on `sym`, so it wins those selective intraday lookups, sometimes by 100×+. At end of day this pack sorts the day's partition and applies a `p#` (parted) attribute, so every *previous* day gets fast lookups too; only the live day is un-indexed. And if a handful of selective live queries genuinely matter, the no-RDB model doesn't stop you adding a small, *targeted* in-memory cache or CEP process for just those, which is much less resource intensive than full general-purpose RDB. + +That limitation is specific to *disk* — in memory, `g#` survives an append fine. So there is now an **experimental, opt-in** overlay (`-.idb.usememattr 1`, default off, mapped mode only) that keeps a heap-resident `g#` copy of the indexed column(s) for the live partition and splices it into that partition's map in place of the on-disk column, leaving every other column mapped. The engine sees a genuine attribute, so it's fully transparent: plain q-sql, `by sym`, `aj`, and queries with no date filter all benefit with no change at the call site — `p#` serves history, in-memory `g#` serves today. The copy is derived from the disk file on each flush, appending only the new rows, so there's no second source of truth and no writer changes. Which columns get it comes from `sort.csv`, the same file the EOD sort uses to place `p#`. + +It isn't free: roughly 24 bytes per row per process (~1.1 GB for a 50M-row day), and unlike the mapped columns that memory is *private to each DB* rather than shared page cache, so N DBs pay it N times. Enable it on readers serving selective lookups and `by sym` aggregation, which both get faster; leave it off on ones dominated by large scans of *un-indexed* columns, which get ~10–25% slower (see [Benchmarks](#benchmarks)). Being per-process, it's easy to run a mixed fleet over the same directory — some DBs indexed, some lean. It leans on `.Q.pm`, which is undocumented, and has a known gap around the end-of-day partition swap — hence off by default. See [`code/idb/mapping.q`](code/idb/mapping.q). **End-of-day rollover.** Sorting the live partition is a little tricky: there's no gateway to hold queries and no separate HDB to move to as the DBs are serving the very partition being sorted. So the pack copies today's partition to a hidden staging directory, sorts the *copy*, then swaps it in with two atomic renames. DBs never observe a half-sorted state, and mapped DBs keep serving right across the swap. Zero downtime, no gateway required. @@ -68,16 +72,17 @@ Both serve identical results; mapped is just faster. On the [benchmark](#benchma The `bench/` directory compares this design to a traditional RDB on the same data. Representative figures on a **50M-row synthetic trading day** (server-side ms per query; reproduce with `bash bench/run.sh`): -| query | 1 RDB | 1 mapped DB | 4 DBs in parallel | -|---|---:|---:|---:| -| selective lookup, **indexed** col (`sym`, ~5k rows) | **0.7** | 370 | 96 | -| selective lookup, **un-indexed** col (`tradetime`, ~5k rows) | 115 | 132 | 33 | -| aggregate (`avg`/`sum` by `sym`) | 540 | 610 | 170 | -| filtered aggregate | 1220 | 1030 | 300 | +| query | 1 RDB | 1 mapped DB | 1 mapped DB + `usememattr` | 4 DBs in parallel | +|---|---:|---:|---:|---:| +| selective lookup, **indexed** col (`sym`, ~5k rows) | **0.5** | 392 | **0.6** | 101 | +| selective lookup, **un-indexed** col (`tradetime`, ~5k rows) | 124 | 118 | 136 | 33 | +| aggregate (`avg`/`sum` by `sym`) | 590 | 660 | 576 | 181 | +| filtered aggregate | 1247 | 1153 | 1184 | 296 | Important notes: -- **The RDB's edge is the *index*, not memory.** That makes sense since, of course, the DBs are *also* ideally serving from memory, just the OS page cache the memory-mapped files live in, instead of the RDB's own heap. Both are reading RAM; the only real difference is the index. Same 5k-row result, in memory both times: *with* an index it's 0.7 ms, *without* one it's 115 ms — a full scan, right in the DBs' ballpark. Memory alone buys almost nothing. +- **The RDB's edge is the *index*, not memory.** That makes sense since, of course, the DBs are *also* ideally serving from memory, just the OS page cache the memory-mapped files live in, instead of the RDB's own heap. Both are reading RAM; the only real difference is the index. Same 5k-row result, in memory both times: *with* an index it's 0.5 ms, *without* one it's 124 ms — a full scan, right in the DBs' ballpark. Memory alone buys almost nothing. The third column is the proof: give the DB the same index and it lands at 0.6 ms, on the same on-disk data. +- **`usememattr` closes that gap, at a price.** The in-memory attribute (column 3, off by default — see [Attributes](#key-design-choices)) takes the selective indexed lookup from 392 ms to 0.6 ms, a large indexed lookup (2.5M rows on `sym`) from 588 ms to 235 ms, and `by sym` aggregation from 660 ms to 576 ms — all to RDB parity or better, since the grouping reads straight off the index. It is not free everywhere: a large scan of an *un-indexed* column that also returns the indexed one runs ~10–25% slower, and the RDB shows the same pattern, so it looks like the ~1.1 GB of private heap displacing page cache rather than anything about the attribute itself. - **On anything that scans, one DB ≈ one RDB** (within ~0–20%, sometimes even slightly faster), and **four DBs beat the single RDB 3–4×**. The RDB is single-threaded, the DBs aren't. - **The numbers above compare the average time for one query over multiple runs.** This is particularly important to note for the column with 4 DBs. These numbers cleanly scale to 3-4x faster **if you need to run 4 queries**. For a single query, having 4 processes obviously does not help you. Query **throughput** goes up: four DBs serve a fixed batch of queries **~3.6× faster** than one RDB. @@ -145,4 +150,4 @@ It's a demo source, not a load generator: a few hundred rows a second, enough to The KDB-X [community edition](https://code.kx.com/kdb-x/releases/release-notes-latest.html#2-qlim-resource-limits) caps resources such as concurrent connections and memory. The default setup in this pack is deliberately an absolute minimum — a tickerplant, a writer, a sort process and a single DB, plus the discovery service and the feed — so it starts comfortably within those limits. You can add processes back (extra DBs, a gateway, monitoring, and so on) as your license allows. -Because this is a no-RDB architecture, live data is served straight from disk rather than held in an in-memory RDB, so overall memory use _by kdb+_ is lower than an equivalent RDB-based setup — making it easier to stay within the community edition's limits. +Because this is a no-RDB architecture, live data is served straight from disk rather than held in an in-memory RDB, so overall memory use _by kdb+_ is lower than an equivalent RDB-based setup — making it easier to stay within the community edition's limits. (The optional in-memory attribute overlay trades some of that back — see [Attributes](#key-design-choices).) diff --git a/appconfig/settings/idb.q b/appconfig/settings/idb.q index c5cf0a0..9c94424 100644 --- a/appconfig/settings/idb.q +++ b/appconfig/settings/idb.q @@ -4,6 +4,11 @@ // Override per-process, e.g. -.idb.usemapping 0 on the command line. usemapping:@[value;`usemapping;1b]; +// in-memory attribute flag: 1b = keep a heap-resident `g# copy of the indexed +// column(s) for the live partition, spliced into its .Q.pm slot alongside the mapped +// columns, so selective intraday lookups stop scanning; 0b = leave it un-indexed. +usememattr:@[value;`usememattr;0b]; + // Re-enable proctype-directory code loading so .proc.reloadcode picks up our // read-mode behaviour from $KDBAPPCODE/idb/ (i.e. code/idb/mapping.q). Core // config/settings/idb.q sets this 0b because stock TorQ ships no code/idb dir; diff --git a/bench/bench-env.sh b/bench/bench-env.sh index e687464..03b56f4 100644 --- a/bench/bench-env.sh +++ b/bench/bench-env.sh @@ -17,11 +17,12 @@ export BENCH_SORT=$((KDBBASEPORT + 6)) # sort1 export BENCH_DEFERRED=$((KDBBASEPORT + 2)) # db1 - setup A, deferred reader export BENCH_RDB=$((KDBBASEPORT + 4)) # rdb1 - setup D, in-memory control export BENCH_MAPPED=$((KDBBASEPORT + 8)) # db2 - setup B, mapped reader +export BENCH_MEMATTR=$((KDBBASEPORT + 13)) # db7 - setup C, mapped + in-memory attribute export BENCH_PAR="$((KDBBASEPORT + 9)) $((KDBBASEPORT + 10)) $((KDBBASEPORT + 11)) $((KDBBASEPORT + 12))" - # db3-6 - setup C, parallel mapped readers -export BENCH_NPAR=$(set -- $BENCH_PAR; echo $#) # how many readers setup C uses + # db3-6 - setup D, parallel mapped readers +export BENCH_NPAR=$(set -- $BENCH_PAR; echo $#) # how many readers setup D uses # every port the bench topology must have listening before it can run # (discovery is deliberately excluded - nothing here queries it directly) -export BENCH_PORTS="$BENCH_STP $BENCH_WDB $BENCH_SORT $BENCH_DEFERRED $BENCH_RDB $BENCH_MAPPED $BENCH_PAR" +export BENCH_PORTS="$BENCH_STP $BENCH_WDB $BENCH_SORT $BENCH_DEFERRED $BENCH_RDB $BENCH_MAPPED $BENCH_MEMATTR $BENCH_PAR" export BENCH_NPORTS=$(set -- $BENCH_PORTS; echo $#) diff --git a/bench/matrix.sh b/bench/matrix.sh index 1de2784..d69c8f6 100644 --- a/bench/matrix.sh +++ b/bench/matrix.sh @@ -1,13 +1,17 @@ #!/bin/bash # ============================================================================= -# bench/matrix.sh - latency matrix across FOUR setups, all server-side timed (\t): +# bench/matrix.sh - latency matrix across FIVE setups, all server-side timed (\t): # A = db1 deferred (1 reader) $BENCH_DEFERRED # B = db2 mapped (1 reader) $BENCH_MAPPED -# C = db3-6 mapped, N readers IN PARALLEL $BENCH_PAR -# D = rdb1 in-memory (1 reader) $BENCH_RDB -# A/B/D are per-query ms (one reader). C is the EFFECTIVE ms/query when N readers +# C = db7 mapped + in-memory attribute (1 reader) $BENCH_MEMATTR +# D = db3-6 mapped, N readers IN PARALLEL $BENCH_PAR +# E = rdb1 in-memory (1 reader) $BENCH_RDB +# A/B/C/E are per-query ms (one reader). D is the EFFECTIVE ms/query when N readers # serve the load in parallel = aggregate throughput as latency = max(server-side -# time over the N) / (N x reps). C ~= B/N if it scales; compare C against D. +# time over the N) / (N x reps). D ~= B/N if it scales; compare D against E. +# C is B plus -.idb.usememattr 1: the live partition carries a real `g# on sym, so +# the filter_sym* rows should drop to RDB (column E) latency. Everything else should +# track B closely - that is the point of the column. # Ports come from bench-env.sh (derived from KDBBASEPORT), sourced below so this # script works standalone as well as via run.sh. # Usage: REPS=10 bash bench/matrix.sh @@ -43,20 +47,37 @@ ORDER=(filter_sym filter_sym_sel filter_time_lg filter_time_sm agg_by_sym filter one(){ $Q "$CC" -q -st -port "$1" -reps "$REPS" -query "$2" 2>&1 | awk '/^RESULT/{print $3}'; } -printf "ms/query, server-side, REPS=%s (C = effective ms/q across %s parallel readers)\n\n" "$REPS" "$BENCH_NPAR" -printf "%-14s %12s %12s %14s %12s\n" "query" "A_deferred" "B_mapped" "C_${BENCH_NPAR}par(eff)" "D_rdb" -printf -- "------------------------------------------------------------------------\n" +# Preflight: confirm db7 really has the in-memory attribute applied. Without this a +# silently-ignored flag (bad extras string, deferred mode, sort.csv not found) would +# just look like "the overlay does not help" instead of "the overlay is not on". +MEMCHK=$($Q -q 2>/dev/null <<'EOF' +p:"J"$getenv`BENCH_MEMATTR; h:@[hopen;`$":localhost:",string[p],":admin:admin";0Ni]; +-1 $[null h;"unreachable";h"\"usememattr=\",string[.idb.usememattr],\" attr=\",(string attr .idb.memcols[`trade;`sym]),\" rows=\",string count .idb.memcols[`trade;`sym]"]; exit 0 +EOF +) +echo "db7 (column C) state: ${MEMCHK:-}" +case "$MEMCHK" in + *"usememattr=1 attr=g"*) : ;; + *) echo "WARNING: db7 has no \`g# in-memory attribute - column C is NOT measuring what it claims" ;; +esac +echo "" + +printf "ms/query, server-side, REPS=%s (D = effective ms/q across %s parallel readers)\n\n" "$REPS" "$BENCH_NPAR" +printf "%-14s %12s %12s %12s %14s %12s\n" "query" "A_deferred" "B_mapped" "C_memattr" "D_${BENCH_NPAR}par(eff)" "E_rdb" +printf -- "-------------------------------------------------------------------------------------\n" TMP=$(mktemp -d) for k in "${ORDER[@]}"; do q="${LBL[$k]}" A=$(one "$BENCH_DEFERRED" "$q") B=$(one "$BENCH_MAPPED" "$q") - D=$(one "$BENCH_RDB" "$q") + C=$(one "$BENCH_MEMATTR" "$q") + E=$(one "$BENCH_RDB" "$q") + rm -f "$TMP"/* for p in $BENCH_PAR; do $Q "$CC" -q -st -port "$p" -reps "$REPS" -query "$q" > "$TMP/$p" 2>&1 & done wait - Cmax=$(cat "$TMP"/* | awk '/^RESULT/{print $3}' | sort -n | tail -1) - awk -v k="$k" -v a="$A" -v b="$B" -v c="$Cmax" -v d="$D" -v r="$REPS" -v n="$BENCH_NPAR" 'BEGIN{ - printf "%-14s %12.2f %12.2f %14.2f %12.2f\n", k, a/r, b/r, c/(r*n), d/r }' + Dmax=$(cat "$TMP"/* | awk '/^RESULT/{print $3}' | sort -n | tail -1) + awk -v k="$k" -v a="$A" -v b="$B" -v c="$C" -v d="$Dmax" -v e="$E" -v r="$REPS" -v n="$BENCH_NPAR" 'BEGIN{ + printf "%-14s %12.2f %12.2f %12.2f %14.2f %12.2f\n", k, a/r, b/r, c/r, d/(r*n), e/r }' done rm -rf "$TMP" diff --git a/bench/process.csv b/bench/process.csv index 327b6ec..0bd6845 100644 --- a/bench/process.csv +++ b/bench/process.csv @@ -9,4 +9,5 @@ localhost,{KDBBASEPORT}+9,idb,db3,${TORQAPPHOME}/appconfig/passwords/accesslist. localhost,{KDBBASEPORT}+10,idb,db4,${TORQAPPHOME}/appconfig/passwords/accesslist.txt,1,1,,,${KDBCODE}/processes/idb.q,1,-.idb.usemapping 1,q localhost,{KDBBASEPORT}+11,idb,db5,${TORQAPPHOME}/appconfig/passwords/accesslist.txt,1,1,,,${KDBCODE}/processes/idb.q,1,-.idb.usemapping 1,q localhost,{KDBBASEPORT}+12,idb,db6,${TORQAPPHOME}/appconfig/passwords/accesslist.txt,1,1,,,${KDBCODE}/processes/idb.q,1,-.idb.usemapping 1,q +localhost,{KDBBASEPORT}+13,idb,db7,${TORQAPPHOME}/appconfig/passwords/accesslist.txt,1,1,,,${KDBCODE}/processes/idb.q,1,-.idb.usemapping 1 -.idb.usememattr 1,q localhost,{KDBBASEPORT}+4,rdb,rdb1,${TORQAPPHOME}/appconfig/passwords/accesslist.txt,1,1,180,,${KDBCODE}/processes/rdb.q,1,,q diff --git a/code/idb/mapping.q b/code/idb/mapping.q index 331a255..b15528c 100644 --- a/code/idb/mapping.q +++ b/code/idb/mapping.q @@ -13,25 +13,86 @@ // Both modes reload the sym file identically (stock intradayreload does it on each // WDB flush); usemapping changes only what happens to the partition maps. // +// On top of Mode B, -.idb.usememattr (default 0b, opt-in) adds the IN-MEMORY +// ATTRIBUTE overlay. In kdb+ you cannot maintain an attribute on an on-disk column +// while appending to it. The attribute flag sits in the file header, but the index it +// points at is written as a FOOTER, after the data - so every append would shift it, +// pulling it out from under any memory-mapped reader. kdb+ avoids that by silently +// DROPPING the attribute on append (the file truncates back to header+data). So in +// the two modes above the live partition is un-indexed and selective lookups scan it. +// +// In-memory columns on the heap have no such problem: `g# is maintained across an +// append. And .Q.MAP[] leaves behind .Q.pm, a fixed dictionary of the mapped tables +// and their columns. .Q.pm is not mentioned at all in KX's documentation, so this +// should be considered somewhat experimental - but its existence means we can swap +// one column of the live slot for an in-memory copy carrying a real `g#, leaving the +// rest mapped: +// +// .Q.pm[t]: , (enlist lk)!enlist @[T;`sym;:;memsym] +// +// The engine simply sees a column with a genuine attribute; it does not care whether +// that column is in-memory or mapped from disk. So this is transparent to everything +// - plain q-sql, `by sym`, aj - and composes with history: `p# serves the older +// slots, heap `g# the live one. The copy is derived from the disk file itself, +// appending only each flush's delta, so there is no second source of truth and no WDB +// changes. Indexed columns come from sort.csv, the same file EOD uses to place `p#. +// // Load order: this file loads before the stock idb.q, so the wiring is deferred to // .proc.initlist (runs last) to wrap intradayreload/rollover rather than be // clobbered by them. \d .idb +memattrpart:`; // partition the cache below is built for +memcols:()!(); // table ! (column ! heap `g# vector); registry AND cache + +// Indexed columns for table t, mirroring .sort.sorttab's precedence: a table's own +// sort.csv rows win if it has ANY, else the `default rows +memattrcols:{[t] + if[0=count .sort.params;@[.sort.getsortcsv;.sort.defaultfile;{.lg.e[`idb.memattr;"could not read sort.csv (",x,"); no in-memory attributes"]}]]; + t:$[t in .sort.params`tabname;t;`default]; + exec distinct column from .sort.params where tabname=t, att in `p`g + }; + +// Rebuild the registry with one EMPTY `g#-attributed placeholder per indexed column. +resetmemattr:{[cp] + memcols::.Q.pt!{[t] c:memattrcols t; c!count[c]#enlist `g#()} each .Q.pt; + memattrpart::cp; + .Q.gc[]; // or yesterday's in-memory data stays resident + .lg.o[`idb.memattr;"live partition now ",string[cp],"; indexing ","; " sv {[t] string[t],": ",", " sv string key memcols t} each key memcols]; + }; + +// Bring each indexed column level with disk and splice it into the mapped slot. +buildmemattr:{[t;T] + if[not t in key memcols; :T]; // if table appeared since the last reset return early + {[t;T;c] + d:T c; // snapshot mapping: count was fixed at `get pdir` i.e. any additional writes to the end of the file are invisible + if[(count memcols[t;c])>count d; // column on disk shrank i.e. the partition was rebuilt under us + .lg.w[`idb.memattr;"disk shrank for ",string[t],".",string[c],"; rebuilding"]; + .[`.idb.memcols;(t;c);:;`g#()]]; // wipe out the in-memory columns so we can start over + n:count memcols[t;c]; // current in-memory count + if[n`; we splice a fresh map of just the live partition // back in with a dict-merge (nested-index assign .Q.pm[t][k]:v is `nyi). refreshliveslot:{ cp:currentpartition; + if[usememattr and not cp~memattrpart; resetmemattr cp]; // new day -> start the cache over {[cp;t] - ks:key m:.Q.pm t; + ks:key .Q.pm t; idx:where cp=last each ks; // the live (dir;partition) key if[0=count idx;:()]; // no live-partition slot for this table yet lk:ks first idx; pdir:.Q.dd[.Q.dd[lk 0;`$string lk 1];t]; // // splay - .Q.pm[t]:m,(enlist lk)!enlist get pdir; + // NOTE this is unguarded: `get` on a ragged splay (during a WDB write) throws 'length. It will still correctly reload on the next try, but worth fixing at some point. + .Q.pm[t]:(.Q.pm t),(enlist lk)!enlist T:get pdir; // Refresh the live partition slot. Must be its own assignment: it drops the old slot's reference to memcols, keeping the append below in-place + if[not usememattr;:()]; + .Q.pm[t]:(.Q.pm t),(enlist lk)!enlist buildmemattr[t;T]; // Splice in the in-memory columns with attrs }[cp] each .Q.pt; }; @@ -44,16 +105,19 @@ refreshliveslot:{ // the re-sorted partition leaves the whole map stale and \l does not refresh a // mapped session. ensuremapped:{[force] - if[0=count .Q.pv;:()]; // no partitions on disk yet + if[0=count @[value;`.Q.pv;()];:()]; // no partitions on disk yet - .Q.pv is UNDEFINED (not empty) on a partitionless db if[force or (0=count .Q.pm) or any (count .Q.pv) <> count each .Q.pm each .Q.pt; - :.Q.MAP[]]; // forced / first map / coverage gap -> full remap - refreshliveslot[]; // else: cheap live-slot refresh - }; + .Q.MAP[]]; // forced / first map / coverage gap -> full remap + refreshliveslot[]; // runs after .Q.MAP[] too: it rebuilds every slot + }; // from disk, dropping any overlay // Applied post-load via .proc.initlist, once idb.q has defined intradayreload. applymapmode:{[] if[not usemapping; .lg.o[`idb.mode;"IDB read mode A (deferred): no .Q.MAP, selects re-map per query"]; + if[usememattr; + .lg.w[`idb.memattr;"usememattr ignored: deferred mode has no .Q.pm to overlay"]; + usememattr::0b]; :()]; .lg.o[`idb.mode;"IDB read mode B (mapped): .Q.MAP[] + per-reload live-slot refresh"]; ensuremapped[0b]; // map what already exists at startup