[Question] Swapping out html5ever

[amirshukayev]

I've been profiling our project and I'm under the impression that html5ever is causing a significant performance bottleneck. There are other html parsers/tokenizers that are more performance focused, such as https://docs.rs/tl/, and https://docs.rs/lol_html/

I was curious on the viability of this. Of course this is a larger project, and we'd be happy to work on this internally and upstream any progress, but would appreciate some advice here. Is this something this project could accept in the long run?

[adamreichold]

I think lol_html is a completely different paradigm (you basically have to specify your selectors upfront instead of operating on a fully realised DOM), so I don't think it can be realistically replace html5ever as used in this crate.

I don't know about tl specifically, but generally speaking, this crate is really just a thin glue code layer over html5ever, selectors and ego-tree. So replacing any of the three will probably end up with different crate even if it starts out as a fork of the current scraper code.