[jwrosewell]

Problem

The IP Intelligence cloud engine keeps every value as the cloud service sends it. CloudEngine stores each one unchanged, result[key].value = value; (pipeline/fiftyone.pipeline.cloudrequestengine/cloudEngine.js lines 152 to 159 at 6d5d4fef), and IpIntelligenceCloud adds nothing (ip-intelligence/fiftyone.ipintelligence.cloud/ipIntelligenceCloud.js lines 24 to 42). So for the same address:

1. A weighted value reads from the cloud engine as { rawweighting, value }, where the on-premise engine gives { value, weight, rawWeight } (ip-intelligence/fiftyone.ipintelligence.onpremise/swigHelpers.js lines 41 to 61). The class comment says weighted values come with value and weight, which they did not, and the shared dataExtension.js reads no weight from a cloud value (lines 36 to 40).
2. A WKT value, such as an area, reads as the object { value: 'POLYGON(...)' }, where the on-premise engine gives the WKT text (swigData.js lines 123 to 126).

What the specification says

From ip-intelligence-specification/pipeline-elements/ip-intelligence-cloud.md lines 55 to 61 at 706de1f0:

Note that IP Intelligence properties may include weighted values. The cloud response returns these as JSON arrays of objects with a rawweighting member and a value member (see weighted values), and the Engine MUST parse them into the appropriate weighted value types used by the Element Data.

From pipeline-specification/features/value-types.md lines 123 to 130 at 706de1f0:

Where a WKT value is written to JSON, for example by the JSON Builder Element or in a response from the remote server, it MUST be written as an object with one member, value, holding the WKT text, for example {"value": "POINT(2 4)"}. This is the form the .NET and Java implementations write, and their IP Intelligence Cloud Engines read. A reader MUST accept this form. A reader MAY also accept the WKT text alone as a string.

weighted-values.md lines 60 to 62 says the raw weighting is a 16-bit unsigned integer, with 65535 standing for a weighting of 1.0.

.NET, Java and Python each parse the cloud's weighted values and test it, for example .NET's APVValueConversionTests.cs lines 285 to 319 at f0ba9653 and Python's weighted_cloud_engine.py at 6ad5ae2d.

What changed

IpIntelligenceCloud.processInternal reads the response as CloudEngine does, then gives each weighted value in the form the on-premise engine gives it.

1. A list whose every item has a rawweighting member and a value member becomes a list of { value, weight, rawWeight, rawweighting }, with weight the raw weighting divided by 65535 and rawWeight the raw weighting. Each item keeps rawweighting, the member every earlier release gave, deprecated in favour of rawWeight, because a patch release removes no name a caller reads.
2. Every other value, a property with no value and its reason, and a WKT value are as the service gave them. A WKT value stays an object with one member, value, holding the WKT text, where the on-premise engine gives the text itself. Giving the text would change the type a caller reads, so that change, and removing rawweighting, wait for a release that may change public names.

The class comment says what the values are, and that rawweighting is deprecated, in the code and in types/ipIntelligenceCloud.d.ts.

Before you upgrade

Nothing to do. A weighted IP Intelligence value from the cloud engine now also carries weight, a number from 0 to 1, and rawWeight, as the on-premise engine's values do. rawweighting is still there, deprecated, so read rawWeight in new code.

Tests

ip-intelligence/fiftyone.ipintelligence.cloud/tests/cloudValues.test.js builds a Pipeline with the real CloudRequestEngine and a request client that answers as the cloud service does, so it needs no key or network. It checks that weighted values come out with value, weight, rawWeight and rawweighting, that an area given as a weighted {"value": "POLYGON(...)"} and a location given as {"value": "POINT(2 4)"} keep that form, and that a plain value, a number and a property with no value are unchanged.

Red, run 43637335532 on 593c7ee, the first test commit on rc's code, with only a throwaway workflow added. On Ubuntu and on Windows exactly the two tests of that first design failed, the test of weighted values, as rc gives no weight or rawWeight, and the test of WKT values, which then expected the text.

The first fix, c542429, also dropped rawweighting and gave a WKT value as its text, which a patch release may not do, so the tests were changed at 4c6b1ed to expect rawweighting kept and the WKT form unchanged.

Red, run 43654002716 on 4c6b1ed, those tests on c542429's code, with only a throwaway workflow added. On Ubuntu and on Windows the two tests fail, as c542429 drops rawweighting and changes the WKT form, and the test of other values passes.

Green, run 43654009988 on 7a1811a, the fix that keeps both, with only the same throwaway workflow added. All 3 pass on Ubuntu and on Windows. Its lint step failed on the indentation that a test name split over two lines gave the first test, which 5a37d1f puts back on one line, and the full build below lints it.

Types, run 43654071569 ran npm run tsc on 4d9277e, the merge of rc into this draft, and found no difference in the committed types but pipeline/fiftyone.pipeline.did/types/owid-js/v1.d.ts, which #127 decides.

Full build

Run 43654584127 on tmp/pr-matrix-124-c, started by tools/pr-matrix.py without macOS. All 7 Setting 'PullRequestSha' lines give this head, 5a37d1f, and Configure printed "Left out 2 macOS entries". Each of the six Build and Test jobs, Ubuntu, Ubuntu ARM and Windows on Node.js 22 and 24, passed 1,515 unit tests with 2 skipped and 96 integration tests with 4 skipped.

macOS was left out of this draft's full build under the 9 November decision. The rc head is built with macOS before it merges into main.

What remains

Giving a WKT value as its text, as the on-premise engine does, and removing rawweighting wait for a release that may change public names, #138. The cloud engines for Device Detection and for location are unchanged, as their values are not weighted.