Fix six more bugs (phone DB push on reads, JSON export, json/xml/geometric edits, env bools, SQLite row counts)

BUGS.md

@@ -26,14 +26,6 @@ rg -n --max-count 5 '<file or symbol>' BUGS.md
 
 <!-- New entries go here, newest first. -->
 
-### Every SELECT that returns rows on a phone's database pushes the file back to the device
-
-- **Where.** `packages/dbscope_sqflite_plugin/lib/src/executor.dart:209-210` (`_writesToFile` returns true when `result.affectedRows > 0`), fed by `lib/db/sqlite_executor.dart` (`_resultFromSelect` reports `affectedRows: rows.length` for every result set, so a read of 3 rows says 3).
-- **Failure.** Open a table (or run any query that returns rows) on a USB-connected phone's app database: the read comes back with `affectedRows > 0`, so `DeviceSqliteExecutor.execute` calls `_pushBack()` and copies the whole `.db` back onto the device and deletes its `-wal`/`-shm` after every page of every grid. Each read costs an adb/devicectl round trip, and each push is the exact operation the dialog warns can corrupt the database or lose the app's own writes if the app is running, though the user only read data.
-- **Noticed.** 2026-11-03, while wiring the statement timeout through `DeviceSqliteExecutor`.
-- **Confidence.** likely (both lines are plain in the code; not run against a phone).
-- **Why not fixed on the spot.** Outside the task (the BUGS.md sweep). The fix is to decide "wrote" from the statement (`!PreparedStatement.isReadOnly`, or `_db.totalChanges` before and after) instead of the row count, which needs the inner executor to expose it.
-
 ## Accepted limitations
 
 Bugs we decided not to fix. Each one says why, and what would change the decision.

CHANGELOG.md

@@ -4,6 +4,12 @@
 
 ## 2026-11-03
 
+- Cells in point, line, lseg, box, path, polygon and circle columns can be edited on Postgres; the save used to fail every time, and the editor now starts from Postgres syntax like (1,2) (2026-11-03)
+- SQLite reports 0 rows affected for DDL instead of the count from the previous write (2026-11-03)
+- DBSCOPE_AUTO_READ_ONLY (and DBSCOPE_AUTO_ASSIGN_IP) accept 1/0, yes/no, on/off in any case; =1 used to open the connection read-write (2026-11-03)
+- Cells in json and xml columns can be edited on Postgres; the save used to fail every time with "operator does not exist" (2026-11-03)
+- JSON export of a query result no longer fails on date and timestamp columns, and no longer drops a column whose name repeats (2026-11-03)
+- On a phone's app database, reads no longer push the whole file back to the device; only statements that change the file do, and writes inside BEGIN … COMMIT push once at the COMMIT (2026-11-03)
 - Query result headers now also show the names of enums, domains and extension types (looked up once per connection) (2026-11-03)
 - The connection dialog's Read-only switch now proposes a default and says why: on when the name says prod/live/main or the server is remote, off for a database on this machine; it stops following once you flip it (2026-11-03)
 - Re-importing a connection link (or auto-connecting) for an open profile now reconnects when the host, user, password or other connection settings changed, instead of keeping the old connection (2026-11-03)

docs/architecture.md

@@ -137,7 +137,7 @@ dbscope supports more than one kind of database backend. The core never talks to
 - Profile-kind id on disk stays `'sqflite_remote'` (unchanged from the prior VM-service implementation) so `ConnectionKindRegistry` lookups still match; profile *extras* are different (`deviceKind`, `deviceId`, `appId`, `dbPath`) so old VM-service profiles must be re-saved.
 - Add-Connection dialog (`profile_dialog.dart`) is a three-step picker: device → app → on-device .db file. Single-device / single-DB cases auto-select. Apps are filtered to "has reachable data" via `androidAppHasData` / `iosAppHasData`. Files are filtered to `.db`/`.sqlite`/`.sqlite3`/`.db3`, excluding `-wal`/`-shm`/`-journal` sidecars (they're paired automatically at open time).
 - Open flow (`DeviceSqliteExecutor.open` in `executor.dart`): pull `<name>.db` plus any `-wal` and `-shm` sidecar into a temp dir, run `PRAGMA wal_checkpoint(TRUNCATE)` to fold WAL into the main file, switch to `journal_mode = DELETE` so subsequent local writes don't recreate WAL sidecars, then wrap the resulting `sqlite3.Database` in a regular `SqliteExecutor`.
-- Write path: every `execute` is delegated to the inner `SqliteExecutor`, then if the statement was a write (affectedRows > 0, or a non-SELECT keyword), the main `.db` is pushed back to the device and stale `-wal`/`-shm` on device are deleted. Push uses `adb push /data/local/tmp/ + run-as cp` on Android (because `adb push` can't write directly into `/data/data/<pkg>/`) and `devicectl device copy to` on iOS — see `Backend.push` in `packages/app_data_explorer/lib/app_data_explorer.dart`.
+- Write path: every `execute` is delegated to the inner `SqliteExecutor`, then if the statement changed the file (SQLite's `total_changes()` or `schema_version` moved, or a write keyword with no counter such as VACUUM or `PRAGMA x = y`), the main `.db` is pushed back once no transaction is open (a `BEGIN … COMMIT` pushes once, at the COMMIT). The result's `affectedRows` is not used: for a SELECT it is the row count to the device and stale `-wal`/`-shm` on device are deleted. Push uses `adb push /data/local/tmp/ + run-as cp` on Android (because `adb push` can't write directly into `/data/data/<pkg>/`) and `devicectl device copy to` on iOS — see `Backend.push` in `packages/app_data_explorer/lib/app_data_explorer.dart`.
 - Concurrency caveat shown in the dialog as a banner: the file is a snapshot. If the device app writes between pull and push, those writes are lost; pushing while the app holds the DB open can corrupt it. Users are told to force-stop the app before editing.
 - The executor reports `kind == kSqliteKind` so `SchemaCatalog` reuses the sqlite branch. Capabilities: `sessionReadOnly: true`, `dml: true`, `transactions: false` (each push is an independent network round-trip, can't span a multi-statement TX), `cancelByClose: false`.
 - macOS sandbox is **disabled** in `macos/Runner/{DebugProfile,Release}.entitlements` because `Process.run('adb', ...)` / `Process.run('xcrun', ...)` doesn't work inside the sandbox. Do not ship this build to the App Store.

lib/connection/cli_auto_connect.dart

@@ -18,7 +18,7 @@ import 'profile.dart';
 /// Required: `DBSCOPE_AUTO_HOST`, `DBSCOPE_AUTO_DATABASE`,
 /// `DBSCOPE_AUTO_USERNAME`. Optional: `DBSCOPE_AUTO_PORT` (default 5432),
 /// `DBSCOPE_AUTO_PASSWORD`, `DBSCOPE_AUTO_NAME` (the connection's label),
-/// `DBSCOPE_AUTO_READ_ONLY` (`true`/`false`). Set
+/// `DBSCOPE_AUTO_READ_ONLY` (`true`/`false`, also `1`/`0`, `yes`/`no`). Set
 /// `DBSCOPE_AUTO_CLOUDSQL_INSTANCE` (`project:region:instance`) to open a
 /// Cloud SQL profile instead of a plain Postgres one — dbscope then
 /// launches its own Cloud SQL Auth Proxy (see
@@ -31,17 +31,18 @@ import 'profile.dart';
 /// own identity (host/port/database/username), so re-running `start_dbscope.sh`
 /// updates the same saved profile instead of piling up a fresh one in the
 /// picker every launch.
-ConnectionProfile? autoConnectProfileFromEnv() {
-  final env = processEnvironment();
+///
+/// [environment] defaults to the process environment (tests pass their own).
+ConnectionProfile? autoConnectProfileFromEnv([Map<String, String>? environment]) {
+  final env = environment ?? processEnvironment();
   final host = env['DBSCOPE_AUTO_HOST'];
   if (host == null || host.isEmpty) return null;
   final database = env['DBSCOPE_AUTO_DATABASE'] ?? 'postgres';
   final username = env['DBSCOPE_AUTO_USERNAME'] ?? 'postgres';
   final port = int.tryParse(env['DBSCOPE_AUTO_PORT'] ?? '') ?? 5432;
   final password = env['DBSCOPE_AUTO_PASSWORD'] ?? '';
   final name = env['DBSCOPE_AUTO_NAME'] ?? '$username@$host/$database';
-  final readOnlyRaw = env['DBSCOPE_AUTO_READ_ONLY'];
-  final readOnly = readOnlyRaw == null ? null : readOnlyRaw == 'true';
+  final readOnly = parseEnvBool(env['DBSCOPE_AUTO_READ_ONLY']);
   final id = const Uuid().v5(
     Namespace.url.value,
     'dbscope-cli:$host:$port:$database:$username',
@@ -67,7 +68,7 @@ ConnectionProfile? autoConnectProfileFromEnv() {
     gcpProject: env['DBSCOPE_AUTO_GCP_PROJECT'] ?? segments.first,
     instanceName: segments.last,
     localPort: port,
-    assignPublicIp: (env['DBSCOPE_AUTO_ASSIGN_IP'] ?? 'true') != 'false',
+    assignPublicIp: parseEnvBool(env['DBSCOPE_AUTO_ASSIGN_IP']) ?? true,
   );
   return ConnectionProfile(
     id: id,
@@ -82,3 +83,18 @@ ConnectionProfile? autoConnectProfileFromEnv() {
     extra: cloudSql.toExtra(),
   );
 }
+
+/// `true`/`1`/`yes`/`on` and `false`/`0`/`no`/`off`, in any case; null for
+/// anything else (unset, empty, a typo), so the profile's read-only
+/// heuristic decides. A strict `== 'true'` read `DBSCOPE_AUTO_READ_ONLY=1`
+/// or `=TRUE` as read-write, on the variable people set for production.
+bool? parseEnvBool(String? raw) {
+  switch (raw?.trim().toLowerCase()) {
+    case 'true' || '1' || 'yes' || 'on':
+      return true;
+    case 'false' || '0' || 'no' || 'off':
+      return false;
+    default:
+      return null;
+  }
+}

lib/db/dml.dart

@@ -4,6 +4,7 @@ import '../catalog/pg_type.dart';
 import '../db/executor.dart';
 import '../util/sql_quote.dart';
 import 'edit_policy.dart';
+import 'pg_geometry.dart';
 import 'sqlite_kind.dart';
 
 class UpdateResult {
@@ -29,29 +30,58 @@ class CellEditor {
       params.add(pkValues[primaryKey]);
       pkPlaceholders.add('${quoteIdent(primaryKey)} = \$${params.length}');
     }
-    params.add(oldValue);
+    params.add(
+      conn.kind == kSqliteKind || !pgGeometricTypes.contains(policy.typeName.toLowerCase())
+          ? oldValue
+          : pgGeometryText(oldValue),
+    );
     final oldPlaceholder = '\$${params.length}';
 
     final qualified = quoteQualified(policy.schema, policy.table);
-    // SQLite before 3.39 (Android 13 and older phones) has no
-    // `IS NOT DISTINCT FROM`; its `IS` is the same NULL-safe comparison and
-    // works on every version.
-    final nullSafeEq = conn.kind == kSqliteKind ? 'IS' : 'IS NOT DISTINCT FROM';
     // CURRENT_TIMESTAMP rather than now(): it works on Postgres and SQLite.
     final touches = [
       for (final touchColumn in policy.touchColumns)
         if (touchColumn != policy.column) ', ${quoteIdent(touchColumn)} = CURRENT_TIMESTAMP',
     ].join();
+    // SQLite before 3.39 (Android 13 and older phones) has no
+    // `IS NOT DISTINCT FROM`; its `IS` is the same NULL-safe comparison and
+    // works on every version.
+    final guard = conn.kind == kSqliteKind
+        ? '${quoteIdent(policy.column)} IS $oldPlaceholder'
+        : _postgresGuard(policy, oldPlaceholder);
     final sql =
         'UPDATE $qualified '
         'SET ${quoteIdent(policy.column)} = \$1$touches '
         'WHERE ${pkPlaceholders.join(' AND ')} '
-        'AND ${quoteIdent(policy.column)} $nullSafeEq $oldPlaceholder';
+        'AND $guard';
 
     final result = await conn.execute(sql, positional: params);
     return UpdateResult(affectedRows: result.affectedRows, newValue: parsed);
   }
 
+  /// The optimistic-concurrency guard on Postgres: the column still holds
+  /// the value the user saw. `json`, `xml` and the geometric types have no
+  /// usable `=` operator, so a plain `IS NOT DISTINCT FROM` fails every edit
+  /// of such a column ("operator does not exist: json = unknown").
+  /// - json: compared as `jsonb` (key order and whitespace don't count).
+  /// - xml: compared as text.
+  /// - geometric: the old value is sent as Postgres text ([pgGeometryText]),
+  ///   parsed by the server into the column's type and printed back, so
+  ///   both sides are the server's own text form of the value.
+  static String _postgresGuard(EditPolicy policy, String oldPlaceholder) {
+    final column = quoteIdent(policy.column);
+    final typeName = policy.typeName.toLowerCase();
+    if (pgGeometricTypes.contains(typeName)) {
+      return '$column::text IS NOT DISTINCT FROM ($oldPlaceholder::$typeName)::text';
+    }
+    final cast = switch (typeName) {
+      'json' => '::jsonb',
+      'xml' => '::text',
+      _ => '',
+    };
+    return '$column$cast IS NOT DISTINCT FROM $oldPlaceholder$cast';
+  }
+
   Future<int> deleteRow({
     required String schema,
     required String table,

lib/db/pg_geometry.dart

@@ -0,0 +1,27 @@
+import 'package:postgres/postgres.dart' as pg;
+
+/// Postgres's geometric types. None of them has a usable `=` (point, path
+/// and polygon have none at all; box and circle compare areas), and the
+/// driver can't send most of them back as a bind parameter, so the edit
+/// guard compares them as Postgres text (see `CellEditor`).
+const pgGeometricTypes = {'point', 'line', 'lseg', 'box', 'path', 'polygon', 'circle'};
+
+/// [value] in Postgres's input syntax when it is one of the driver's
+/// geometric objects (`Point(1.0,2.0)` → `(1.0,2.0)`), else [value]
+/// unchanged. Dart's shortest round-trip doubles parse back to the same
+/// float8, so the server reads exactly the value it sent.
+Object? pgGeometryText(Object? value) => switch (value) {
+  pg.Point() => _point(value),
+  pg.Line() => '{${value.a},${value.b},${value.c}}',
+  pg.LineSegment() => '[${_point(value.p1)},${_point(value.p2)}]',
+  pg.Box() => '${_point(value.p1)},${_point(value.p2)}',
+  // Path extends Polygon, so it has to come first.
+  pg.Path() => value.open ? '[${_points(value)}]' : '(${_points(value)})',
+  pg.Polygon() => '(${_points(value)})',
+  pg.Circle() => '<${_point(value.center)},${value.radius}>',
+  _ => value,
+};
+
+String _point(pg.Point point) => '(${point.latitude},${point.longitude})';
+
+String _points(pg.Polygon polygon) => polygon.points.map(_point).join(',');

lib/db/sqlite_executor.dart

@@ -117,6 +117,7 @@ class SqliteExecutor implements DbExecutor, StatementTimeoutCapable {
     // SELECT) through here when the SQL doesn't carry placeholders. For
     // DML we need `_db.updatedRows` to surface the real affected count;
     // for SELECT we fall back to the row count from the result set.
+    final changesBefore = _totalChanges();
     final result = _db.select(sql);
     if (cancelToken?.isCancelled ?? false) {
       return const DbResult(columns: [], rows: []);
@@ -126,7 +127,7 @@ class SqliteExecutor implements DbExecutor, StatementTimeoutCapable {
       return DbResult(
         columns: base.columns,
         rows: base.rows,
-        affectedRows: _db.updatedRows,
+        affectedRows: _changedRows(changesBefore),
       );
     }
     return base;
@@ -139,6 +140,7 @@ class SqliteExecutor implements DbExecutor, StatementTimeoutCapable {
     CancelToken? cancelToken,
   }) {
     final statement = _db.prepare(sql);
+    final changesBefore = _totalChanges();
     try {
       // Always step through `select`: it is safe for statements that return
       // no rows, and it never throws a result set away. Guessing from the
@@ -156,7 +158,7 @@ class SqliteExecutor implements DbExecutor, StatementTimeoutCapable {
         return DbResult(
           columns: const [],
           rows: const [],
-          affectedRows: _db.updatedRows,
+          affectedRows: _changedRows(changesBefore),
         );
       }
       return _resultFromSelect(resultSet);
@@ -165,6 +167,16 @@ class SqliteExecutor implements DbExecutor, StatementTimeoutCapable {
     }
   }
 
+  int _totalChanges() =>
+      (_db.select('SELECT total_changes()').first.columnAt(0) as int?) ?? 0;
+
+  /// Rows the statement just run inserted, updated or deleted.
+  /// `sqlite3_changes()` alone keeps the count of the last INSERT / UPDATE /
+  /// DELETE, so a `CREATE TABLE` after an UPDATE of 5 rows reported 5; it
+  /// only counts when this statement changed something.
+  int _changedRows(int totalChangesBefore) =>
+      _totalChanges() == totalChangesBefore ? 0 : _db.updatedRows;
+
   DbResult _resultFromSelect(sq.ResultSet resultSet) {
     final columns = <DbColumn>[];
     for (var i = 0; i < resultSet.columnNames.length; i++) {

lib/grid/export_io.dart

@@ -1,4 +1,3 @@
-import 'dart:convert';
 import 'dart:io';
 
 import 'package:file_selector/file_selector.dart';
@@ -7,6 +6,7 @@ import 'package:sqlite3/sqlite3.dart' as sq;
 import '../catalog/pg_type.dart';
 import '../grid/data_grid.dart';
 import '../util/sql_quote.dart';
+import 'export_json.dart';
 import 'export_names.dart';
 
 class ResultExporter {
@@ -56,17 +56,10 @@ class ResultExporter {
       ],
     );
     if (location == null) return null;
-    final colNames = data.columns.map((column) => column.name).toList();
-    final out = <Map<String, Object?>>[];
-    for (final row in data.rows) {
-      final entry = <String, Object?>{};
-      for (var i = 0; i < colNames.length; i++) {
-        entry[colNames[i]] = row[i];
-      }
-      out.add(entry);
-    }
     final file = File(location.path);
-    await file.writeAsString(const JsonEncoder.withIndent(' ').convert(out));
+    await file.writeAsString(
+      encodeRowsAsJson([for (final c in data.columns) c.name], data.rows),
+    );
     return location.path;
   }
 

lib/grid/export_json.dart

@@ -0,0 +1,24 @@
+import 'dart:convert';
+
+import 'export_names.dart';
+
+/// The JSON export of a result set: an array with one object per row.
+///
+/// Values the Postgres driver hands back that `jsonEncode` can't take on
+/// its own (a `DateTime` for every date and timestamp column, `BigInt`,
+/// intervals, geometric types) are written as text, so exporting any
+/// result with a timestamp column doesn't fail. Repeated column names
+/// (`SELECT a.id, b.id …`) get suffixes, so no column overwrites another.
+String encodeRowsAsJson(List<String> columnNames, List<List<Object?>> rows) {
+  final keys = uniqueColumnNames(columnNames);
+  final out = [
+    for (final row in rows)
+      {for (var i = 0; i < keys.length; i++) keys[i]: i < row.length ? row[i] : null},
+  ];
+  return const JsonEncoder.withIndent(' ', _toEncodable).convert(out);
+}
+
+Object? _toEncodable(Object? value) {
+  if (value is DateTime) return value.toUtc().toIso8601String();
+  return value.toString();
+}

lib/grid/export_web.dart

@@ -4,11 +4,11 @@
 // disk. There is no sqlite export: [ResultExporter.supportsSqliteExport] is
 // false and the UI hides the `.db` button.
 
-import 'dart:convert';
 
 import 'package:flutter/services.dart';
 
 import '../grid/data_grid.dart';
+import 'export_json.dart';
 
 class ResultExporter {
   /// Whether [exportSqlite] can write a file on this platform.
@@ -35,18 +35,9 @@ class ResultExporter {
   }
 
   static Future<String?> exportJson(GridData data) async {
-    final colNames = data.columns.map((column) => column.name).toList();
-    final out = <Map<String, Object?>>[];
-    for (final row in data.rows) {
-      final entry = <String, Object?>{};
-      for (var i = 0; i < colNames.length; i++) {
-        entry[colNames[i]] = row[i];
-      }
-      out.add(entry);
-    }
     await Clipboard.setData(
       ClipboardData(
-        text: const JsonEncoder.withIndent(' ').convert(out),
+        text: encodeRowsAsJson([for (final c in data.columns) c.name], data.rows),
       ),
     );
     return 'clipboard';

lib/grid/inputs/column_input.dart

@@ -9,6 +9,7 @@ import 'package:widgets/widgets.dart';
 
 import '../../catalog/pg_type.dart';
 import '../../db/edit_policy.dart';
+import '../../db/pg_geometry.dart';
 
 part 'parts/bool_input.dart';
 part 'parts/date_input.dart';
@@ -188,7 +189,10 @@ class _ColumnInputField extends StatelessWidget {
   @override
   Widget build(BuildContext context) {
     final raw = value;
-    final asString = raw == null ? null : (raw is String ? raw : raw.toString());
+    // Geometric values start out in Postgres's own syntax, so saving an
+    // untouched `(1,2)` works (the driver's `Point(1.0,2.0)` is not input
+    // Postgres accepts).
+    final asString = raw == null ? null : (raw is String ? raw : '${pgGeometryText(raw)}');
     if (policy.enumValues != null && policy.enumValues!.isNotEmpty) {
       return _EnumInput(
         values: policy.enumValues!,

packages/dbscope_sqflite_plugin/lib/dbscope_sqflite_plugin.dart

@@ -17,4 +17,5 @@
 // `lib/main.dart`
 // 4. delete any saved profiles of this kind from the connection picker.
 
+export 'src/executor.dart' show DeviceSqliteExecutor;
 export 'src/plugin.dart' show SqfliteRemoteKindPlugin, SqfliteRemotePlugin, kSqfliteRemoteKind;