Set ops give wrong answers on mixed int/float vectors: hashset hashes i64 and f64 cells differently

[singaraiona]

Summary

The shared row hashset in src/ops/collection.c hashes integer cells with ray_hash_i64 and float cells with ray_hash_f64 (hs_hash_row, ~line 74). When the build and probe sides are different numeric types, equal values like 3 and 3.0 land in different buckets, so the probe misses. hs_eq_rows would compare them correctly (it falls back to atom_eq, which coerces through f64), but it never gets called because the bucket is wrong. The occasional correct match (1 vs 1.0) is a hash collision.

The intended meaning is numeric equality: (== [1 2 3] [1.0 2.0 3.0]) is [true true true], and hs_hash_row already hashes boxed RAY_LIST numerics through f64 so that 1 and 1.0 collide. The typed-vector cases just don't do the same.

Reproducer (current dev)

(find [1.0 2.0 3.0] [3 1])     ;; [0Nl 0]        expected [2 0]
(find [1 2 3] [3.0 1.0])       ;; [0Nl 0Nl]      expected [2 0]
(find (as 'F64 (til 100)) [3 1]) ;; [0Nl 1]      expected [3 1]
(except [1 2 3] [1.0 3.0])     ;; [1 2 3]        expected [2]
(except [1.0 2.0 3.0] [1 3])   ;; [2.0 3.0]      expected [2.0]
(union [1 2] [2.0 3.0])        ;; [1 2 2 3]      expected 2 once
(in [1 0Nl 3] [1.0 0Nf])       ;; [false true false]  expected [true true false]
(in [1.0 0Nf 3.0] [0Nl 3])     ;; [false true false]  expected [false true true]

Same-type inputs are unaffected ((find [1 2 3] [3 1]) gives [2 0], (except [1 2 3] [1 3]) gives [2]).

Scope

find, except and union go through the hashset for any mixed int/float input, nulls or not. in reaches it for mixed types only when both sides carry a null: other shapes take the typed kernel, which promotes to double correctly, and #644 narrows the gate so one-sided nulls also avoid the hashset. Other hashset_init sites in collection.c (~lines 395, 1115, 1571, 1640, 1702, 2612) should be checked for the same exposure.

Fix direction

When build and probe types differ in numeric class (one int-family, one float-family), hash both sides through f64, as the RAY_LIST branch already does. Hashing everything through f64 would also work, but it would slow down the same-type int path and make distinct i64 values above 2^53 collide. Those collisions are harmless for correctness, since hs_eq_rows is exact, but they cost probe time. The hashset already knows src_type, and the probe knows probe_type, so the mixed case can be detected once per call rather than per row.

Found while fixing #593 (PR #644); the both-null in rows above were deliberately left unpinned in test/rfl/collection/in.rfl for this issue.