Implementing Kolmogorov Smirnov Distribution and Test

[AshrafIbrahim03]

I was looking for distribution similarity testing in this crate, but noticed there really isn't any. I found issues #288 and #39 which both mention KS testing. It seems like there hasn't been any movement with that in a few years. #39 mentions that the implementation of KS was too slow. I found an implementation in this paper that I could translate into Rust and it seems reasonably fast. I'm pretty sure I tracked down this implementation from what they use in scipy.

I really want to use a KS test in a project of mine and was hoping to contribute to this in some way!

[YeungOnion]

I'd definitely accept a contribution for something like this, if you can implement the two sided test statistics with two iterators over univariate float data, that would go a long way as a standalone function. The only sharp edge to consider is licensing, please don't study copyleft material.

[AshrafIbrahim03]

What do you mean by the last sentence? Would there be something wrong with me translating the algorithm to Rust?

[YeungOnion]

It was more just a general notice because licensing can sometimes be a concern as you look for reference implementations.

Scipy is BSD licensed which is very permissive, so you can reference and modify as needed without issue, and that would include a rust translation. The paper you linked would also be fair game.

[AshrafIbrahim03]

Sounds good. The code itself has a GPL license, so I don't think there's any problem there!

I want to test for parity between my translation and the original version. Given that there's two original files (one in C and one in Java). Is there a preferred method of testing for parity between my translation and the original code? I know that Rust can natively interface with C(which would make for easy test writing), but I'm translating from the Java code because it's more readable for me.

Is there a place to add tests, not to verify statistical test outputs(like it seems in the tests directory), but to verify parity between statrs and original implementations?

Also, when going through the stats_tests directory, I noticed that there's already a ks_test file that references the same paper that I'm translating from AND there's already implementation for one sided and two sided tests. For clarification, in your initial message in this thread, did you mean that contributing a stand alone function would be the only contribution right now because I was going into this thinking I was going to implement the KS distribution and tests for statrs.

[YeungOnion]

I forgot that was there. Maybe I was recalling how it wasn't yet in use for gof testing...? But I mentioned the piece around what to build as a first step to compose nicer API. The distribution itself would be net new, but I also think we're lacking discoverability on the tests. Contributions on either are welcome.

Regarding comparisons to reference implementations, we've stuck to values instead of direct to reference implementations. And its good to add comments in how why one chose or where one found those values as some come from reference implementations and some come from their respective test suites.

While I'd generally like to avoid directly coupling to a reference implementation for tests, if you wanted to create a way to support more dynamic testing where statrs was extern crate and wasn't going to be a fight with CI/build, I'd probably merge it.

However, I'm not sure about how best to include it, but for simple builds one could consider,

• cephes (much of scipy is cephes backed) is accessible in the spec_math crate
• Fortran's CFDLIB was recently ported, I've not looked much at this - looks like they wanted all rust but wanted something narrower or more tested than statrs. I will mention their reference values approach is nicer than ours, a more standard dataset file instead of tuples in the source code is nice.

[AshrafIbrahim03]

I think I'll start with implementing the KS distribution, then see what I can do about the stat test API later. I submitted a small PR to change the two sample ks test function to accept slices(like the mann whitney tests) instead of moving ownership of a vector to the function.