Skip to content

Incorrect filter statistics for overflowing integer multiplication #25794

Description

@haohuaijin

Describe the bug

Integer multiplication overflow can produce false-empty filter statistics. The following reproduction uses TINYINT to demonstrate the issue with small values.

For columns containing (0, -3) and (20, 7), a * b < -100 correctly returns (20, 7, -116), but the filter statistics report Rows=Inexact(0), exact NULL bounds, and Distinct=Exact(0) for both input columns.

To Reproduce

Use a Parquet destination that does not already exist.

SET datafusion.execution.target_partitions = 1;
SET datafusion.execution.collect_statistics = true;
SET datafusion.explain.show_statistics = true;

COPY (
    SELECT a::TINYINT AS a, b::TINYINT AS b
    FROM (VALUES (0, -3), (20, 7)) AS t(a, b)
) TO '/tmp/df_mul_wrap.parquet' STORED AS PARQUET;

CREATE EXTERNAL TABLE mul_wrap STORED AS PARQUET
LOCATION '/tmp/df_mul_wrap.parquet';

SELECT a, b, a * b AS x FROM mul_wrap WHERE a * b < -100;
EXPLAIN SELECT a, b FROM mul_wrap WHERE a * b < -100;

Query result:

a   b   x
20  7   -116

Filter statistics include:

Rows=Inexact(0), Bytes=Inexact(0)

For both input columns:
Min=Exact(Int8(NULL))
Max=Exact(Int8(NULL))
Distinct=Exact(0)

Expected behavior

Statistics should remain conservative rather than infer an empty result. The returned rows are correct; this reproduction demonstrates a statistics error, including exact-empty column statistics, rather than just an inaccurate row-count estimate.

Additional context

Reproduced with DataFusion CLI 55.1.0 at e2ca7f38051744b2010cf09b80db8e9dfa4b5d38.

Multiplication inference discards an overflow-generated unbounded endpoint when combining the ranges [0, 20] and [-3, 7], producing [-60, 0] and excluding the actual wrapped value -116. The issue is in the shared multiplication interval logic, rather than a TINYINT-specific implementation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions