fix(sql): coerceToDate accepts a Number as epoch milliseconds (story 20.5) - #266
Merged
Conversation
…20.5)
An Elasticsearch date field whose _source stores the value numerically
(legal under ES's default strict_date_optional_time||epoch_millis) reaches
the coercion layer as a java.lang.Long and fell to coerceToDate's catch-all:
'Cannot convert java.lang.Long to DATE'. That took down every cross-index
JOIN leg projecting such a column (softclient4es-arrow#168's Superset
'schema probe' failure) and JDBC ResultSet.getDate with it.
- ValueCoercion.coerceToDate: new 'case n: Number => new java.sql.Date(n.longValue())'
after the String arm, symmetric with coerceToTimestamp's long-standing arm;
scaladoc records the deliberate NON-extension to coerceToTime (a bare number
in a TIME position is ambiguous, and SQLTypes.Time is unreachable from an
ES mapping).
- New ValueCoercionSpec: RED-first on the exact pre-fix message; Number shapes
(Long/Integer/BigDecimal/Double), pre-existing conversions, the coerceToTime
negative control, and the UTC-calendar-date invariant.
- documentation/client/{arrow_flight_sql,adbc_driver}.md: ES_* env spellings
(read by nothing, softclient4es-arrow#169) corrected to the canonical
ELASTIC_* family, plus 'user' -> 'username' and the real credential env
names (ELASTIC_CREDENTIALS_*).
Both cross legs green (sql/test 556+556), + core/compile green, scalafmtAll +
headerCheck clean.
Issue #265 (companion of softclient4es-arrow#168/#169)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Sep 2, 2026
Closed
fupelaqu
marked this pull request as ready for review
September 2, 2026 15:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Story 20.5 — elasticsql half:
coerceToDateaccepts epoch-millisNumber(+ELASTIC_*doc mirrors)Closes #265
Companion of softclient4es-arrow#168 (date columns) and softclient4es-arrow#169 (env vars). The arrow-side halves (Arrow Date64 UTC-midnight normalisation, the
date32[day]readValuearm, theES_*→ELASTIC_*startup alias, and the baked image-ENVremoval) ride the arrow repo'sfeature/20.5branch, whose PR opens once elasticsql0.22.0is published (it will carryCloses #168+Closes #169).What changed
sql/.../type/ValueCoercion.scala—coerceToDategainscase n: Number => new java.sql.Date(n.longValue())(epoch milliseconds), placed after theStringarm exactly like its long-standing twin incoerceToTimestamp(:203). The scaladoc records why, and whycoerceToTimedeliberately does not get the arm (a bare number in a TIME position is ambiguous — epoch millis vs millis-of-day — andSQLTypes.Timeis unreachable from an ES mapping).sql/src/test/.../ValueCoercionSpec.scala(new) — 8 tests: the Number arm (all shapes incl.BigDecimal, pinned deliberately), every pre-existing conversion, the rejection catch-all, thecoerceToTimenegative control,SQLTypes("date")→SQLTypes.Date, and the UTC-calendar-date invariant (zone-independent assertion).documentation/client/arrow_flight_sql.md+adbc_driver.md— theES_HOST/ES_PORT/ES_USER/ES_PASSWORD/ES_API_KEY/ES_BEARERspellings (read by nothing — arrow#169) become the canonicalELASTIC_*family; theuser =config key (binds to nothing) becomesusername =; credential env names are the realELASTIC_CREDENTIALS_*(never the REPL-onlyELASTIC_USERNAME/ELASTIC_PASSWORD). The.mdxtwins are updated in the siblingsoftclient4es-webPR.Red-first evidence (spec AC 14)
Pre-fix run of the new spec fails with the issue's exact message on exactly the 3 Number-arm tests:
Post-fix: 8/8 green.
Verification
sbt "+ sql/test"— 556/556 green on BOTH cross legs (2.12.20 + 2.13.16)sbt "+ core/compile"— green (downstream consumer ofsql)sbt scalafmtAll headerCheck— cleandocumentation/: zeroES_*survivors — the only pattern hits areSOFTCLIENT4ES_API_KEY/SOFTCLIENT4ES_LICENSE_KEYinfederation_operator_guide.md(a different, correct env family; substring false-positives of theES_API_KEYpattern)Blast radius (spec D-3, re-verified)
Three live
coerceToDateconsumers (arrowArrowTypeMapping.writeValue, jdbcElasticResultSet.getDate, jdbcTypeMapping.coerceValue); nothing inside elasticsql calls it; no test anywhere pinned the oldNumberthrow.ElasticStreamingResultSetis dead code (Story 20.1 finding).Release-note lines (for the 0.22.0 notes)
ValueCoercion.coerceToDate(modulesoftclient4es-sql) now accepts aNumberas epoch milliseconds, symmetrically withcoerceToTimestamp;coerceToTimedeliberately unchanged.ResultSet.getDateon an epoch-millis-stored date column now returns the date instead of throwing.🤖 Generated with Claude Code