Skip to content
This repository was archived by the owner on Nov 13, 2026. It is now read-only.

Download the Lite and Asn files from the Distributor - #43

Draft
jwrosewell wants to merge 4 commits into
rcfrom
fix/distributor-downloads
Draft

jwrosewell wants to merge 4 commits into
rcfrom
fix/distributor-downloads

Conversation

@jwrosewell

Copy link
Copy Markdown
Contributor

get-lite-file-from-azure.ps1 and get-lite-file-from-azure.sh downloaded the Lite and Asn files from Azure blob storage (https://51ddatafiles.blob.core.windows.net/enterpriseipi/). This repository is being replaced by https://github.com/51Degrees/data, whose links point at the Distributor's free downloads, and this pull request moves these two scripts there too before the repository is archived, so anyone still using them gets the files from the same place.

What changes

  • Both scripts download https://distributor.51degrees.com/api/v2/free/ipi-lite-v41 and ipi-asn-v41, which need no License Key, with source=ip-intelligence-data, so downloads can be counted by where they come from. Their options (-Force, -Lite, -Asn and -f, -lite, -asn) and the files they write are unchanged.
  • The Distributor sends each file as two gzip members, the first holding the file's first 32 bytes. gunzip and PowerShell 7 unpack both, but the GZipStream of Windows PowerShell stops after the first, which leaves a 32 byte file. The PowerShell script now unpacks each member in turn there, using the same code as the data repository's get-data-files.ps1, which its Download scripts workflow tests under Windows PowerShell.
  • The shell script passes -f to curl, so an error answer such as a 429 is no longer saved as the archive.
  • The README and the test file's description say the Distributor instead of Azure storage.

How it was checked

  • bash -n, and the PowerShell parser in PowerShell 7 and Windows PowerShell.
  • The unit test workflow on this pull request runs the Pester tests, which download both files for real and check that each unpacks to a larger file.

Merge it after today's release pull request, #42, so that release is unchanged. It is part of the run book for moving every SDK to the data repository, 51Degrees/data#1.

get-lite-file-from-azure.ps1 and get-lite-file-from-azure.sh downloaded
both files from Azure blob storage. They now use the Distributor's free
downloads, ipi-lite-v41 and ipi-asn-v41, which need no License Key, and
send source=ip-intelligence-data so downloads can be counted. Their
options and output files are unchanged.

The Distributor sends each file as more than one gzip member. The
PowerShell script now unpacks every member under Windows PowerShell,
whose GZipStream stops after the first, and PowerShell 7 and gunzip
already read them all. The shell script passes -f to curl, so an error
answer is no longer saved as the archive.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant