refactor registry-api, switch to using sparse index for fetching release data

[syphar]

We currently have three use-cases where the crates.io API is used:

1. fetch release-data (yanked, release-date)
2. fetch owners
3. run crate search

1) and 2) run when we build the crate, 3) when the user requests it in the UI.

Since a couple of weeks ago, we're running into crates.io rate limits every couple of days, for a couple of yours each time. This leads to the release-data (yanked, release-time) being outdated or filled with dummy values, and owners not being up-to-date after the build.

Likely this is because we now run more builders (6), and the rate limiting for crates.io moved to the fastly NgWaf instead of the heroku servers.

Thing is: since we removed the downloads field, which wasn't used any more, and the sparse & git index got pubtime as new field, we can replace one of the requests with a request to the sparse index. The sparse index is mostly cached (in the CDN & locally), and doesn't have a rate limit (apart from some DDOS protection).

Generally I believe we probably will do more and more with the sparse index and start live-fetching stuff instead of syncing things.

So I invested some time into a bigger refactor of the registry_api crate.

So:

• For its SparseIndex support I added the crates_index crate to our registry api.
• I added pubtime support to crates-index
• we now initially fetch config.json from the index to figure out the API & download server, as cargo does too.
• import-release-command: can how use the registry api crate to fetch the sparse index entry, and also to generate the tarball download url.
• for get_release_data, we use the sparse index. So it's quite unlikely that we'll run into a rate limit on these while we're building.
• the TestEnvironment now got a working TestRegistry with which other crates in the workspace can test interaction with the registry api & sparse index, without having to mock HTTP calls directly. We could even think about making FakeRelease also fill the sparse index at some point, or the new fake I'll build.
• the global test environment doesn't need a registry-config overwrite any more. All other crates just use the mock. The tests for the registry-api crate itself directly use the TestEnvironment::builder.
• the search API got a bigger refactor, with some test improvements. More types, more builders.
• the consistency check will also update releases.release_time when we have pubtime in the index

What I didn't do yet is changing the dummy data thing, I only made it explicit. In the future I would like to only insert data we're getting from the index, and otherwise leave the fields empty, even in error cases. Or perhaps have additional fields with the original data, and then generated fields with fake or real data. But that I'll do in another PR. Another thought could be to even fail the build when the sparse index fails, our re-attempts will retry the build soon.

What is also not covered: we're still fetching owners via API, and also doing the search API calls. But IMO both are much less important than the yank-state or release-time.

What I'll also add in the future (depenending on the design discussions) is:

• fetching & extracting the crate tarball
• fetching single files from the crates.io source zip (we could use these for showing readmes, when we don't have the source archive any more at some point)

[syphar]

r? @GuillaumeGomez

[syphar]