Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
260 changes: 6 additions & 254 deletions BUGS.md

Large diffs are not rendered by default.

39 changes: 39 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,44 @@
# Changelog

---

## 2026-11-03

- Query result headers now also show the names of enums, domains and extension types (looked up once per connection) (2026-11-03)
- The connection dialog's Read-only switch now proposes a default and says why: on when the name says prod/live/main or the server is remote, off for a database on this machine; it stops following once you flip it (2026-11-03)
- Re-importing a connection link (or auto-connecting) for an open profile now reconnects when the host, user, password or other connection settings changed, instead of keeping the old connection (2026-11-03)
- Fix Postgres query result headers showing Type(oid:N) instead of the type name (2026-11-03)
- Fix job tabs showing a fast job as running forever when it finished before the tab subscribed (2026-11-03)
- Fix one window closing a stream (job progress, profile or metadata updates) belonging to another window (2026-11-03)
- Fix soft delete on SQLite, which wrote now(): it now writes CURRENT_TIMESTAMP (2026-11-03)
- Fix keyset paging stopping at the first NULL when sorting by a nullable column (2026-11-03)
- Fix table tabs ignoring a connection's read-only setting: the grid is no longer editable on read-only connections (2026-11-03)
- Fix CSV export leaving a bare carriage return unquoted, which split the row in two (2026-11-03)
- Hide the .db export button on the web build, where it did nothing (2026-11-03)
- Fix copy jobs blocking every other query on the target profile while they run: the target now gets its own connection (2026-11-03)
- Fix .db export destroying an existing file when the export fails, and failing on duplicate column names (they become id_2, …) (2026-11-03)
- Fix cell edits on phones with SQLite older than 3.39 (Android 13 and earlier) failing on IS NOT DISTINCT FROM (2026-11-03)
- Fix two windows saving to the project file or query history at once losing one of the changes (2026-11-03)
- Fix an interrupted project-file save wiping saved connections and queries: writes are atomic and an unreadable file is kept aside as .corrupt-<time> (2026-11-03)
- Fix the Navigator's row filter on SQLite, which used Postgres-only ::text ~* syntax (2026-11-03)
- Fix Export schema as DBML on SQLite: it no longer queries pg_type and labels the file as SQLite (2026-11-03)
- Fix the date/time picker dropping the seconds (and zone) of a timestamp or time cell (2026-11-03)
- Fix metadata SQL with typed placeholders (@id:int8, $1::int8) failing on SQLite (2026-11-03)
- Fix @me_principal binding NULL when no identity is known: actions now say they need it, and on Postgres it falls back to the login (claimed emails are lowercased) (2026-11-03)
- Fix pressing Cmd/Ctrl+Enter twice in a Query tab running two queries at once and losing the Stop button (2026-11-03)
- Fix the row inspector saving an open edit into a different row after you select another row: switching rows now closes the editor (2026-11-03)
- Fix SQLite metadata queries that start with a comment, or a RETURNING after a newline, silently returning no rows (2026-11-03)
- Fix importing a hostile .dbscopeconn file hanging in the key derivation or failing with a raw error: iteration count, nonce and tag sizes are now checked (2026-11-03)
- Fix the timestamp renderer showing a date-only value one day early for viewers east of UTC (2026-11-03)
- Fix metadata charts and queries on SQLite ignoring their timeout: a runaway statement is now stopped at the deadline (2026-11-03)
- Fix the AI assistant's read-only check letting through data-modifying CTEs, EXPLAIN ANALYZE DELETE and SELECT INTO, which could write on the pg proxy and on phones (2026-11-03)
- Fix profiles named like Liverpool, Olive or Maintenance defaulting to read-only: the prod/live/main guess now needs a whole word (2026-11-03)
- Fix connections leaking: a profile deleted in another window now closes this window's connection, connecting an already-open profile reuses it, and two server-side opens of one profile share one connection (2026-11-03)
- Fix the connection dialog's password: you can now tick 'Forget the saved password' to delete it from the keychain (leaving the field empty still keeps it) (2026-11-03)
- Fix scripts/add_changelog_line.sh refusing every line when CHANGELOG.md has no --- header rule: it now adds the rule itself (2026-11-03)
- Fix inline cell editing in table grids: the editor now takes keyboard focus on double-click / Enter, stays open while you type (it used to save and close after one character), lets arrow keys and Cmd-A work inside the field, and saves on Enter, Tab or clicking another cell (Esc discards) (2026-11-03)
- Landing page: canonical URL, fuller social tags, SoftwareApplication and FAQPage structured data, a visible FAQ, sitemap.xml and llms.txt, checked on every Pages deploy by scripts/check_site_seo.sh (2026-11-03)

## 2026-10-19

- Launching with `DBSCOPE_AUTO_HOST` (plus `DBSCOPE_AUTO_DATABASE`/`DBSCOPE_AUTO_USERNAME`/etc., or `DBSCOPE_AUTO_CLOUDSQL_INSTANCE` for Cloud SQL) set auto-connects to that database on startup instead of restoring the last GUI connection — a fully command-line launch, no "Import connection link" step. See `lib/connection/cli_auto_connect.dart`; the compledo repo's `backend/scripts/dbscope/start_dbscope.sh local|dev|prod` drives it. (2026-10-19)
Expand Down
6 changes: 3 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ The grid is a hand-rolled `ListView.builder` widget (`lib/grid/data_grid.dart` a
- The grid keeps **its own copy** of the row list (the row lists inside are shared). `TableTabView` mutates its tab buffer *and* forwards the same delta to the controller, which is only correct because the two outer lists are separate. Pinned by `test/grid_row_ownership_test.dart`. Selection and in-progress edits follow their row across `removeRowAt` / `removeFirstRows` / `prependRow` (`GridModel.rowsRemoved` / `rowsInserted`), and are dropped when their row is removed.
- The app-side wrapper (`lib/grid/data_grid.dart`) converts `widget.data` once per object. A fresh `GridData` on every build looks like new data to a controller-less grid, which then resets column widths and selection.
- `TableTabView` enforces a 50 000-row in-memory window with 5 000-row trim chunks (`_maxBufferedRows`, `_trimChunk`) to make billion-row tables work. Trade-off: scrollback past the window loses rows (use Refresh); changing these numbers shifts the memory-vs-scrollback balance. `removeFirstRows` compensates the scroll offset (`jumpTo(prev - removed * _kRowHeight)`) so the viewport stays put when the top is trimmed.
- Pagination is keyset, not OFFSET. `KeysetPager` (`lib/query/pager.dart`) builds `(sort_col, pk_cols...) > ($lastSort, $lastPk...)` cursors. The optional `baseWhere`/`baseWhereParams` slot is how the Navigator threads inbound-FK and regex filters through the same pager.
- Pagination is keyset, not OFFSET. `KeysetPager` (`lib/query/pager.dart`) builds `(sort_col, pk_cols...) > ($lastSort, $lastPk...)` cursors. A nullable sort column splits the cursor into its NULL and non-NULL runs (Postgres sorts NULLs last in ASC, SQLite first), because a tuple compare against NULL matches nothing. The ORDER BY stays plain so an index still serves it. The optional `baseWhere`/`baseWhereParams` slot is how the Navigator threads inbound-FK and regex filters through the same pager.
- Query tabs cap at 10 000 rows (no pagination). This is deliberate.
- Cancellation: `CancelToken` (`lib/query/cancel_token.dart`) lives on the `TabModel`. The fallback cancel mechanism is closing the underlying connection (`ConnectionRegistry.killConnection`); the user-facing Stop button cancels all currently-running tabs.

Expand Down Expand Up @@ -268,7 +268,7 @@ User-facing reference: README "Schema metadata". The shape of the code:
- **Sources** (`meta_sources.dart`): `NativeCommentSource` (COMMENT ON on schemas, relations, columns, types), `DbTableSource` (three outcomes — absent / loaded / **unreadable, which fails closed**), `FileSource` (YAML/JSON with includes), `ProfileOverrideSource`. `native_facts.dart` reads constraints/indexes/triggers/policies live for the About dialogs.
- **`MetadataCache`** (per window, per profile, like `DbmlCache`): loaded by `ConnectionRegistry._connect` *before* the connection is marked connected (so hidden tables never flash), reloaded via `reloadMetadata` and broadcast to other windows on the bus (`bus.publish` → `meta.changed:<id>`; BusService wildcards only match `.*`, so windows subscribe to `meta.*`). It maps executor → profile with an Expando so code holding only a `DbExecutor` finds the policy.
- **The choke points** — keep them the only ones: `SchemaCatalog` (default constructor filters through the policy, adds virtual FKs; `SchemaCatalog.unfiltered` for DML/internals), `KeysetPager` (masked → `NULL AS col`, PK fetched then masked; `explicitColumns` when access control is active), `buildEditPolicies` (`lib/db/edit_policies.dart` — every EditPolicy list), `loadRowByPk`, `applyColumnAccess` for query results. A new `SELECT *`, a new `EditPolicy(` construction or a new AI egress path bypasses access control — route it through these.
- **Metadata SQL is untrusted code**: `MetaTrust` (`ui/action_trust.dart`) pins a SHA-256 per scope (read / exec) in SharedPreferences (machine-local on purpose); `executeReadOnly` fences reads in `BEGIN TRANSACTION READ ONLY` (Postgres) / `PRAGMA query_only` (local SQLite). Backends without transactions (pg-proxy, on-device sqflite) can't be fenced.
- **Metadata SQL is untrusted code**: `MetaTrust` (`ui/action_trust.dart`) pins a SHA-256 per scope (read / exec) in SharedPreferences (machine-local on purpose); `executeReadOnly` fences reads in `BEGIN TRANSACTION READ ONLY` (Postgres) / `PRAGMA query_only` (local SQLite). Its `timeout` is `SET LOCAL statement_timeout` on Postgres and, on SQLite (local and on-device), a `sqlite3_progress_handler` deadline (`lib/db/sqlite_progress.dart`, via `StatementTimeoutCapable`), since package:sqlite3 runs statements synchronously and has no interrupt. Backends without transactions (pg-proxy, on-device sqflite) can't be fenced.
- **UI**: `meta_plugin.dart` registers the named-query and dashboard tab kinds, the Metadata menu (fixed labels — menus are built once, everything decides at click time) and a sidebar action provider. `ui/meta_menu.dart` builds every right-click's metadata entries from one `MetaClickContext`.
- `tool/meta_to_sql.dart` reuses `MetaWriter` through a recording executor, so the CLI and the app write identical SQL. `test/meta_live_pg_test.dart` runs every query/check/chart/action of a real database's metadata (skipped unless `DBSCOPE_LIVE_PG` is set).
- Gotcha that cost a hang: `future.whenComplete(() => map.remove(key))` returns the removed future, and `whenComplete` waits on it — a load that waits on itself. Use a block body.
Expand All @@ -280,7 +280,7 @@ These are non-negotiable; the plan covers them and the codebase enforces them:
- **Identifier vs value quoting.** Schema/table/column names go through `quoteIdent` / `quoteQualified` (`lib/util/sql_quote.dart`). Values are positional params (`$1`, `$2`). Never concatenate user values into SQL.
- **Optimistic concurrency on UPDATE.** All cell edits issue `... AND <col> IS NOT DISTINCT FROM $oldValue`. 0 rows affected = conflict toast + refresh that row. See `lib/db/dml.dart`.
- **Statement splitting** (`containsMultipleStatements` / `stripCommentsAndLeading` in `lib/util/sql_parse.dart`) takes a required `SqlDialect`. Derive it with `sqlDialectOf(executor)` (`lib/meta/meta_sources.dart`). Postgres syntax (`E''`, `$$`, nested comments) on SQLite would hide a real second statement.
- **AI-generated SQL** auto-runs through `runSql(readOnly: true)`, which fences it into a read-only transaction (`executeReadOnly`). `enforceReadOnly` only checks the first keyword. Backends without transactions (pg-proxy, on-device sqflite) can't be fenced.
- **AI-generated SQL** auto-runs through `runSql(readOnly: true)`, which fences it into a read-only transaction (`executeReadOnly`). `enforceReadOnly` checks the first keyword and refuses `INSERT`/`UPDATE`/`DELETE`/`MERGE`/`CREATE`/`EXECUTE`, `REPLACE INTO` and `SELECT … INTO` anywhere outside literals and comments (so a data-modifying CTE or `EXPLAIN ANALYZE DELETE` is caught). Backends without transactions (pg-proxy, on-device sqflite) can't be fenced, so that scan is their only guard.
- **Read-only mode** is enforced by `SET default_transaction_read_only = ON` on the connection itself, not by parsing SQL. Toggled per-profile; `ConnectionService` and `ConnectionRegistry` both apply it on open and on `setReadOnly`.

## Postgres version floor
Expand Down
33 changes: 33 additions & 0 deletions lib/ai/sql_assistant.dart
Original file line number Diff line number Diff line change
Expand Up @@ -266,8 +266,41 @@ void enforceReadOnly(String sql, {required SqlDialect dialect}) {
'AI returned non-read-only SQL ($firstWord); refusing.',
);
}
// An allowed first word is not enough: `WITH d AS (DELETE … RETURNING *)
// SELECT …` and `EXPLAIN ANALYZE DELETE …` both write, and on backends
// that can't fence a read-only transaction (pg proxy, on-device SQLite)
// nothing else stops them. Refuse any write keyword anywhere outside
// literals and comments. This also refuses a column that happens to be
// named `delete`; the user can still run such SQL by hand.
final words = bareWords(stripped, dialect: dialect);
for (var i = 0; i < words.length; i++) {
final word = words[i];
final writes =
_writeKeywords.contains(word) ||
// SQLite's `REPLACE INTO` (bare REPLACE is also a string function).
(word == 'REPLACE' && i + 1 < words.length && words[i + 1] == 'INTO') ||
// `SELECT … INTO new_table` creates a table on Postgres.
(word == 'INTO' && _selectInto(words, i));
if (writes) {
throw AssistantException(
'AI returned SQL that can write ($word); refusing.',
);
}
}
}

// The statements Postgres accepts inside a CTE or under EXPLAIN that
// change data (`EXECUTE` runs a prepared statement that may). Statements
// that can only come first (DROP, TRUNCATE, …) are already refused by the
// first-word check, and listing them here would refuse ordinary columns
// named `comment` or `lock`.
const _writeKeywords = {'INSERT', 'UPDATE', 'DELETE', 'MERGE', 'CREATE', 'EXECUTE'};

/// `INTO` at [i] belongs to a `SELECT … INTO` (not an `INSERT INTO`, which
/// is refused anyway).
bool _selectInto(List<String> words, int i) =>
i > 0 && !{'INSERT', 'REPLACE', 'MERGE'}.contains(words[i - 1]);

String _firstWord(String sql) {
final match = RegExp('^[A-Za-z_]+').firstMatch(sql);
return match?.group(0) ?? '';
Expand Down
24 changes: 19 additions & 5 deletions lib/catalog/dbml_export.dart
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import '../db/executor.dart';
import '../db/sqlite_kind.dart';
import '../meta/meta_cache.dart';
import '../meta/meta_index.dart';
import 'schema_catalog.dart';
Expand Down Expand Up @@ -45,15 +46,16 @@ class DbmlExporter {
allTables.addAll(tables.where((table) => table.kind == 'BASE TABLE'));
}

final enums = await _loadEnums(
schemas.map((schema) => schema.name).toList(),
);
// SQLite has no enum types (and no pg_type to ask).
final enums = _isSqlite
? const <DbmlEnum>[]
: await _loadEnums(schemas.map((schema) => schema.name).toList());

final foreignKeys = await catalog.listForeignKeys();

final buffer = StringBuffer();
buffer.writeln('Project $databaseName {');
buffer.writeln(" database_type: 'PostgreSQL'");
buffer.writeln(" database_type: '${_isSqlite ? 'SQLite' : 'PostgreSQL'}'");
buffer.writeln('}');
buffer.writeln();

Expand Down Expand Up @@ -188,14 +190,26 @@ class DbmlExporter {
}
}

bool get _isSqlite => conn.kind == kSqliteKind;

String _dbmlType(ColumnInfo column) {
final name = column.typeName;
final name = column.typeName.trim();
if (_isSqlite) {
// A SQLite declared type is free text (`VARCHAR(20)`, `UNSIGNED BIG
// INT`) and may be empty; DBML needs one token, quoted if it isn't.
if (name.isEmpty) return 'any';
return _simpleType.hasMatch(name) ? name : '"${name.replaceAll('"', "'")}"';
}
if (name.startsWith('_')) {
return '${name.substring(1)}[]';
}
return name;
}

static final _simpleType = RegExp(
r'^[A-Za-z_][A-Za-z0-9_]*(\(\d+(,\s*\d+)?\))?$',
);

String _qualifiedRef(String schema, String name) => '"$schema"."$name"';

String _columnName(String name) {
Expand Down
4 changes: 4 additions & 0 deletions lib/catalog/pg_type.dart
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,10 @@ PgType pgTypeFromSqlite(String declared) {
PgType pgTypeFromTypeName(String? typeName) {
if (typeName == null) return PgType.unknown;
final lowerName = typeName.toLowerCase();
// Arrays first: `int4[]` / `_int4` would otherwise match their element.
if (lowerName.endsWith('[]') || lowerName.startsWith('_')) {
return PgType.array;
}
if (lowerName.contains('bigint') || lowerName.contains('int8')) {
return PgType.int8;
}
Expand Down
Loading
Loading