Skip to content

perf: use new PyLong* API for num-bigint feature - #6144

Merged
davidhewitt merged 42 commits into
PyO3:mainfrom
chirizxc:PyLongForBigInt
Sep 6, 2026
Merged

perf: use new PyLong* API for num-bigint feature#6144
davidhewitt merged 42 commits into
PyO3:mainfrom
chirizxc:PyLongForBigInt

Conversation

@chirizxc

@chirizxc chirizxc commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Related: #6040

See codspeed comment for benchmark improvements.

@codspeed-hq

codspeed-hq Bot commented Jun 20, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by ×2.3

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 26 improved benchmarks
✅ 115 untouched benchmarks
⏩ 6 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
into_biguint_small 6.5 µs 1.2 µs ×5.5
into_biguint_zero 5.4 µs 1.1 µs ×4.9
into_bigint_huge_negative 43.4 µs 10.4 µs ×4.2
into_u128_zero 3,616.9 ns 925.7 ns ×3.9
into_u128_small 3,447 ns 930.2 ns ×3.7
into_i128_zero 3,563.1 ns 982.4 ns ×3.6
extract_biguint_small 4.7 µs 1.4 µs ×3.5
into_i128_small_pos 3,393.2 ns 986.9 ns ×3.4
extract_biguint_zero 4.7 µs 1.4 µs ×3.4
into_bigint_huge_positive 34.9 µs 10.4 µs ×3.4
into_biguint_huge 34.5 µs 10.4 µs ×3.3
extract_biguint_negative_fail 13.4 µs 4.1 µs ×3.3
extract_bigint_small 4.9 µs 1.5 µs ×3.2
into_bigint_big_negative 9.8 µs 4.2 µs ×2.4
into_bigint_small 6 µs 2.6 µs ×2.3
into_bigint_big_positive 8.9 µs 4.1 µs ×2.1
into_biguint_big 8.5 µs 4.1 µs ×2.1
extract_bigint_huge_negative 21.4 µs 15.4 µs +38.37%
extract_bigint_huge_positive 19.7 µs 15.4 µs +27.94%
extract_biguint_huge 19.8 µs 15.7 µs +26.28%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing chirizxc:PyLongForBigInt (033bda2) with main (ac9b689)

Open in CodSpeed

Footnotes

  1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@davidhewitt davidhewitt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, generally looks good on the from-python implementation; I think possible gains yet to be had in the to-python implementation.

Comment thread src/conversions/num_bigint.rs Outdated
Comment thread src/conversions/num_bigint.rs Outdated
Comment thread src/conversions/num_bigint.rs Outdated
@davidhewitt

Copy link
Copy Markdown
Member

Also, I think worth giving this a changed newsfragment to note to users we've optimized the bigint conversions on 3.14+.

@chirizxc chirizxc changed the title internal: try use new PyLong API for num-bigint feature perf: use new PyLong* API for num-bigint feature Jun 21, 2026
@chirizxc

This comment was marked as outdated.

@chirizxc
chirizxc requested a review from davidhewitt August 3, 2026 10:24
Comment thread src/conversions/std/num.rs Outdated
Comment on lines +480 to +495

let digits_len = if abs >> 30 == 0 {
1
} else if abs >> 60 == 0 {
2
} else if abs >> 90 == 0 {
3
} else if abs >> 120 == 0 {
4
} else {
5
};

let digits = (0..digits_len)
.map(|i| (abs >> (i * PYLONG_BITS_IN_DIGIT)) as u32 & DIGIT_MASK);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compiler Explorer GodBolt Difff

How do I view the diff after clicking the link? 🤔 Image Image

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On rustc beta I get the following as the fewest instructions:

use std::num::NonZero;

const PYLONG_BITS_IN_DIGIT: u32 = 30;
const BITS: NonZero<usize> = NonZero::new(128).unwrap();

#[unsafe(no_mangle)]
pub fn digits_len(abs: NonZero<u128>) -> u32 {
    let bits: u32 = u128::BITS - abs.leading_zeros();
    bits.div_ceil(PYLONG_BITS_IN_DIGIT)
}

... this requires first checking whether abs is zero with a conversion to NonZero, which I think makes sense to make a fast path for zero anyway?

Additionally this reminds me of this article I recently read, which essentially concluded that branchless code (i.e. div_ceil) is better for consistent performance than branches when you don't know the input distribution. So I think given inputs here are entirely controlled by users and it looks like next stable Rust will optimize the div_ceil form better, div_ceil seems better than a chain of ifs?

Comment thread src/conversions/std/num.rs
@davidhewitt

Copy link
Copy Markdown
Member

@chirizxc sorry for the long delay in review here. I ended up playing around locally with the implementation and ended up pushing a commit which optimizes for small integers a little more. Beyond that, this looks great, thanks!

If you're happy with the new commit, let's merge?

@chirizxc

Copy link
Copy Markdown
Contributor Author

@chirizxc sorry for the long delay in review here. I ended up playing around locally with the implementation and ended up pushing a commit which optimizes for small integers a little more. Beyond that, this looks great, thanks!

If you're happy with the new commit, let's merge?

let's do it, but it looks like CI red

@davidhewitt
davidhewitt added this pull request to the merge queue Sep 6, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 6, 2026
@davidhewitt
davidhewitt added this pull request to the merge queue Sep 6, 2026
Merged via the queue into PyO3:main with commit af3ea98 Sep 6, 2026
49 of 53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants