Skip to content

Release candidate - #152

Draft
jwrosewell wants to merge 51 commits into
mainfrom
rc
Draft

jwrosewell wants to merge 51 commits into
mainfrom
rc

Conversation

@jwrosewell

Copy link
Copy Markdown
Contributor

Releases what has collected on rc since the last release, #83, which merged into main on 13 November 2026 at 10:49 UTC. Its Publish run 43412143722 failed in one example test before it published anything, so npm still has 4.5.102 of every package, and this release takes #83's changes to npm too. Everything below "Before you upgrade" that #83 listed is listed here again, because no one has had it yet.

This is the first release in which each package has its own version (#37). A package is released when its own code changed since its last release, or when a package it depends on is released, and only the patch number moves. Since 4.5.102 every package has changed or depends on one that has, so all 16 are released, each at 4.5.103.

This release comes three days after #83 merged, sooner than the week common-ci's CONTRIBUTING.md asks between releases, by James Rosewell's decision of 13 November to release every 51Degrees repository on Monday 16 November. npm last had a release of these packages on 4 November, so the Publish workflow's weekly check, which counts from each package's last release, does not refuse it.

This release moves Node.js toward one behavior across every 51Degrees language. Each element it changes now follows the published 51Degrees specifications, which in turn follow the RFCs they cite, so that the same input gives the same answer in .NET, Java, Node.js, Python, PHP, Go, Rust and C. The native modules take the C library release 51degrees-cxx 4.5.106, whose C code is that of 4.5.105.

Before you upgrade

  1. On-premise Device Detection defaults change. performanceProfile now defaults to 'Balanced', where it was 'LowMemory', and updateMatchedUserAgent is now on. Set performanceProfile: 'LowMemory' and updateMatchedUserAgent: false to keep the old behavior (#42).
  2. The JSON Builder needs a Sequence Element. It fails with "The JSON Builder needs a sequence number. Place a Sequence Element before the JSON Builder in the Pipeline." when there is none. PipelineBuilder now adds the Sequence Element first, so only a Pipeline built with addSequenceElement: false, or one that adds JSONBundlerElement by hand, is affected (#53, #74).
  3. The JSON sent to the browser gains a sequence entry, which holds session-id and sequence. A request with a session id and no sequence now gets sequence 1, where it got 2 (#59).
  4. The Cloud Request Engine sends more Evidence. It sends every Evidence value apart from the reserved names resource, license, licence and values, which it writes itself. Before, it sent only the Evidence that a Flow Element in the Pipeline accepted, so do not add Evidence that must not reach the cloud service (#72).
  5. Usage Sharing shares no cookie but 51D_ ones. It shared every cookie. It now shares a cookie only when its name starts 51D_, whatever the settings, shareAll: true included, and never the Cookie header. With trackSession on, the session cookie named by cookie is read for the repeat check and is not sent unless its name starts 51D_. Every query string and form value is still shared by default. To share only those whose names start 51D_ or are in queryWhitelist, set the new option shareAllQueryStringParameters: false (#35, #45, #135).
  6. A Flow Element of your own must name its Evidence. An element with no evidenceKeyFilter accepted every key and now accepts none, and flowData.evidence.get gives undefined for a key that no element accepts. Give the element evidenceKeyFilter: new BasicListEvidenceKeyFilter([...]) with the keys it reads (#45).
  7. Data update log entries change. Each error entry starts Error during check for data update, so a log search or alert on the old text needs to change (#69).
  8. Latitude and Longitude have 3 decimal places, as do the points of WKT values such as Areas, so 18.542741172521133 is now 18.543 (#62).
  9. A client IP value is read only when all of it is one address. Values such as 1.2.3.4/24, 1.2.3.4, 5.6.7.8 and 01.2.3.4 were read and are now skipped. Pass one address in each value (#67).
  10. An Engine built with no properties keeps the first Data File's Properties across every data update. A Property that only a later Data File holds has no value until the Engine is created again (#55).
  11. Settings that were accepted now throw when the element is built. These are a JavaScript Builder protocol other than http or https, an objName such as window or document (#31, #60), and a Translation Engine behavior that names none of Original, EmptyString or FlowError (#49).
  12. A 51Did with white space inside it, or with characters from both base64 alphabets, is refused with the status INVALID_BASE64. FodId.toStandardBase64 returns null for such a text, so a TypeScript caller needs to handle null (#40, #56).
  13. Where one name is under more than one Evidence prefix, the Cloud Request Engine sends it once. It sends the value of the highest prefix in the order query, header, cookie, server, fiftyone, location, and compares prefixes and names ignoring letter case, so header.User-Agent and query.user-agent are sent as one field (#75).
  14. _51DEGREES_CLOUD_ENDPOINT replaces FOD_CLOUD_API_URL, which still works in this release. The Cloud Request Engine and the 51Did client read _51DEGREES_CLOUD_ENDPOINT, and read FOD_CLOUD_API_URL only when the new name is unset or empty. An engine whose endpoint came from the old name logs one warning to its Pipeline, and the first 51Did client that uses it raises one Node DeprecationWarning for the process. The baseURL and endpoint options still come first. Set _51DEGREES_CLOUD_ENDPOINT to the same value as the old name (#77, #88).
  15. Each package now has its own version. All 16 are 4.5.103 in this release, and from the next release on only the packages that changed, and those that depend on them, get a new number, so the numbers will differ between packages. Each package requires the others here with ^, so a caller that pins every fiftyone.* package to one number should pin each to its own (#37).
  16. The shared packages keep their code in lib. fiftyone.devicedetection.shared and fiftyone.ipintelligence.shared moved the modules their index.js exports out of examples and tests. A require of an old path, such as fiftyone.devicedetection.shared/tests/keyUtils, still works, as each old file now re-exports the module in lib. The generated types are only under types/lib (#37).
  17. The JSON sent to the browser marks delayexecution only on a Property with a value, as the JSON Builder specification asks, so a Property with no value no longer carries it (#66).
  18. The client-side endpoints should answer a matching If-None-Match before processing. Call Helpers.writeClientSideNotModified(request, response, flowData, options) once the evidence is added and stop when it returns true, as every web example now does. Without it a revalidation is still answered correctly but processed first. A 304 sent while processing records errors now carries Cache-Control: private, max-age=<max-age>, the Vary header and the ETag of a good response, where it carried private, no-cache and no ETag (#120).
  19. A Resource Key that includes no product a cloud engine reads is logged at warning level, once for each engine, where it was an error, and warnings the cloud service sends are now logged (#123).
  20. FodId's raw flags byte _flags is deprecated. Read type, usage and usageIsIndirect, which name its bits. A later release removes it (#115, #128).
  21. The TypeScript types are written again from the code. HardwareProfileCloudEngine from the Device Detection cloud package now has its real type, where a caller that skips checking type files got any, so that caller's use of it is now checked (#113).
  22. The JSON sent to the browser writes each item of a list of Element Data, such as the cloud hardware lookup's profiles, as plain lower-case values, with null and a nullreason for a Property with no value, where it wrote each Property's _value, hasValue and noValueMessage. With no Sequence Element in the Pipeline, the script carries an empty session id and a sequence of 1, where it carried the evidence's (#118).
  23. A weighted IP Intelligence value from the cloud engine also carries weight and rawWeight, as the on-premise engine's values do. rawweighting is still there and deprecated, so read rawWeight in new code (#124, #138).
  24. A query.sequence added as a JavaScript number now counts as absent. The Sequence Element counts a given sequence only when it is a BigInt or text made only of digits, from 0 up to 2,147,483,646, so an application that adds query.sequence itself as a number now gets a sequence of 1. Add the text of the number, as a query string gives it, or a BigInt. A web integration that hands over the request sees no change (#134).
  25. The match metric Properties' category is now "Device Metrics", where it was "Device metrics", and they are listed in the order of the specification's table. Code that compares the category or relies on their order needs to change (#137).
  26. A built Pipeline cannot be changed. It keeps a frozen copy of the list of elements it was built from, and of each list of parallel elements in it, and freezes pipeline.flowElements. Code that changed a list given to addParallel or new Pipeline, flowElementsChain or pipeline.flowElements after the build needs to build a new Pipeline instead. A method that changes flowElementsChain now throws a TypeError, and so does a change to pipeline.flowElements in strict mode code (#142).
  27. weight and rawWeight on each weighted item in the JSON sent to the browser are deprecated. Each item now also carries rawweighting and value, as the specification writes them, so read those. A later release removes weight and rawWeight (#139, #147).
  28. A cloud answer whose errors array has entries now fails the request, whatever its HTTP status, with an error for each entry, and the error messages name the URL the request went to. A log search or alert on "No data in response from cloud service" or "Cloud service returned status code" needs to change, and a failed answer that is not JSON gives the status code message, not "Error parsing response" (#140).
  29. Usage Sharing's warnings are emitted at 'warn', as every other element's are, and at 'warning' as well, which is deprecated. Listen with pipeline.on('warn', ...). A later release stops emitting 'warning' (#141, #149).
  30. minify: true now minifies the client script. It never did, as the JavaScript Builder threw the minified script away. Minifying takes time on every request for the script, so measure it before keeping the setting (#150, #132).

What changes

Packages and versions

  • Each package has its own version, tagged <package>/<version> in this repository with one GitHub release for each, and the Publish workflow releases only the packages that changed since their last tag, with every package that depends on one of them. A change only to documentation, tests, examples, or a manifest's description, keywords or links releases nothing (#37).
  • fiftyone.pipeline.core packs only the template its JavaScript Builder reads, JavaScriptResource.mustache, and not the rest of the JavaScript templates repository, its tests, workflows, README and licence (#106).
  • fiftyone.pipeline.did carries the OWID library as a submodule and packs only its v1.js and its licence, where it installed OWID from GitHub as a bundled dependency, so installing the package needs no access to GitHub (#107).
  • The six main Device Detection and IP Intelligence packages describe themselves in the words people search npm for, and name the products they are an alternative to, being UAParser.js, DeviceAtlas and WURFL, and MaxMind and IPinfo (#15).
  • Every package's TypeScript types are written again with npm run tsc, so they match the code, and each package's types compile for a TypeScript caller (#113).

Web integration and client-side caching

  • Helpers.writeClientSideResponse writes the response of a client-side JavaScript or JSON endpoint with Cache-Control, Vary and an ETag. Node.js has no web integration package, so call it from your own endpoints, as the web examples do (#44).

  • The ETag is a weak entity tag, W/"...", and If-None-Match is compared weakly, as RFC 9110 section 8.8.3 and section 13.1.2 ask. A match gets 304 for GET and HEAD, and 412 for any other method. A response to a request whose processing failed has Cache-Control: private, no-cache and no ETag. The same Evidence gives the same ETag in every process (#44).

  • Evidence.addFromRequest adds the web request as a read-only view under server.http-request (#45).

  • Helpers.writeClientSideNotModified answers a request whose If-None-Match matches before the Pipeline processes it, with 304 for GET and HEAD and 412 for any other method, as the web integration specification asks. A 304 carries a good response's Cache-Control, Vary and ETag, during an outage too, and writeClientSideResponse gives a matching request the same answer (#120).

  • A Set Header Property's identifier may hold digits and other characters, and the header name starts at the next upper-case letter, so SetHeaderBrowser2Accept-CH sets Accept-CH (#120).

  • PipelineBuilder adds the Sequence Element, the JSON Builder, the JavaScript Builder and the Set Headers Element only where the Pipeline has none with that key, compared ignoring case, so one the caller added is the one used, and building twice adds none of them twice (#105).

JavaScript Builder

  • The JavaScript Builder checks the callback URL before it writes it into the client script. The protocol must be http or https, and the host must be ASCII labels or an IPv6 address in brackets, with no user information, query, fragment or zone identifier, following RFC 3986. A URL from the request that fails is left out of the script and logged (#31, #51).

  • The script always carries the page request's query parameters, whether or not the builder has a callback URL, as step 7 of the JavaScript Builder specification asks (#97).

  • The script takes its session id and sequence from the Sequence Element's data, with sequence 1 and no session id where there is no Sequence Element, and the JSON writes each item of a list of Element Data by the same rules as the Element Data's own Properties (#118).

  • The JSON sent to the browser writes each item of a weighted value with rawweighting and value, keeping weight and rawWeight beside them for this release (#139).

Evidence

  • Evidence keys and Property names are compared ignoring letter case, so header.User-Agent and header.user-agent are one key, and flowData.device.get('IsMobile') works in any letter case (#71, #68).
  • An Engine's cache key is its accepted Evidence sorted by key, with each key in lower case and each value as given, made once for each request before processing, so the same Evidence added in another order or with keys in other letters is served from the cache (#125).
  • Element Data is found by its key in any letter case, so flowData.get('Device') and flowData.Device work as flowData.device does (#122).
  • The Sequence Element counts a given query.sequence only when it is a BigInt or text made only of digits, from 0 up to 2,147,483,646. Any other value gives a sequence of 1 (#134).

Cloud

  • The Cloud Request Engine can authenticate with a License Key, alone or with a Resource Key, through licenseKey. With a License Key alone the Properties wanted must be named with requestedProperties. Both keys now travel in the request body (#64).
  • A License Key appears in log entries and error messages only as its last 4 characters, with the rest as *. The packages mask a key where one can enter a message, so the mask no longer runs on each detection, and Pipeline.log passes the message it is given to its listeners unchanged (#58, #85).
  • A cloud response that holds a 51Did is never stored in or served from the cache that cacheSize sets (#57).
  • The Cloud Request Engine fetches its accessible Properties and accepted Evidence keys before its first request, whatever else is in the Pipeline, so a Pipeline with a cache and no Cloud Engine serves a repeated request from the cache (#73).
  • Each entry of a cloud response's warnings array, from discovery or processing, is logged as a warning with any License Key masked, and a product the Resource Key does not include is logged once for each engine at warning level (#123).
  • The IP Intelligence cloud engine gives each weighted value as value, weight and rawWeight, as the on-premise engine does, keeping the service's rawweighting beside them for this release (#124).
  • An empty answer, or one whose errors array has entries, fails the request with the specification's messages, which name the URL (#140).

Usage Sharing

  • Usage Sharing accepts no Evidence, so a Pipeline that holds it no longer accepts every header, and a header no other Flow Element reads takes no part in a cache key, the ETag or the Vary header. The Flow Data still keeps what Usage Sharing shares. evidence.add keeps a value when a Flow Element accepts its key or when Usage Sharing would share it. An application that adds its own Evidence therefore has nothing to change, and the values it adds are shared under the same rules as the values of a request. evidence.addFromRequest also adds the whole request as server.http-request, which is Constants.evidenceHttpRequest in fiftyone.pipeline.core, a frozen object that holds each value of the request under the key it has as Evidence. Read it with evidence.get('server.http-request'). evidence.getAll() leaves it out, and it is never shared whole (#45).
  • After 10 failed sends in 100 seconds, sending pauses, and the pause doubles up to 300 seconds while sends keep failing. A new trackSession option tells sessions apart (#36, #35).
  • The record names the version of the Pipeline that wrote it, read from fiftyone.pipeline.core, where every release wrote 4.4.7, so the spread of a release can be measured. The session id and sequence are recorded as SessionId and Sequence whatever the letters of their keys (#109).
  • The record marks a value with replaced characters escaped="true", where it was replaced="true", and cuts a value at 10,000 code points, where it was 512 characters (#54).
  • A new shareAllQueryStringParameters option, true unless set to false, shares every query string and form value. Set to false, only the names that start 51D_ and those in queryWhitelist are shared. No cookie but 51D_ ones is shared, shareAll included (#135).

Data updates

  • Data update log entries and errors use the text and levels of the specification's error table, which every language now shares. Every check reports a status, and an updateComplete listener that throws is logged (#69).

  • With updateOnStart and a License Key or an update URL, an on-premise Engine downloads a missing Data File, where the build threw. DataFileUpdateService.updateFromMemory is new (#52).

  • The file system watcher runs on its own fileSystemWatcher setting, whether automatic updates are on or not, and needs no License Key. This reverses a change on rc that made the watcher wait for automatic updates, and so for a License Key (#52), which was a mistake. That change never reached npm, so the watcher behaves as in 4.5.102 (#90).

  • A data file has one polling timer. An update started by hand with updateDataFile replaces the waiting timer rather than adding another, so the file is checked once in each polling interval however many updates are started (#63).

Device Detection and IP Intelligence

  • Node.js reads each on-premise Property by name, because a read by name measured faster than a read by position, at 711 ns against 2,797 ns for one read of IsMobile on Ubuntu ARM with Node.js 22 (run 42995016804, #68). The IP Intelligence engine was timed on its own as well, and a read by name took about a third of the time of a read by position on Ubuntu x64 and ARM (run 43487633019, #81).
  • Both on-premise engines read their accepted Evidence keys again after each data refresh, as the specification asks, so the keys a new data file accepts are used from then on (#130).
  • Each match metric Property of the on-premise Device Detection engine has the category "Device Metrics", the data files "Lite", "Premium", "Enterprise" and "TAC" and a default value, and they are listed in the order of the specification's table, after the data file's Properties (#137).

51Did

  • A 51Did payload too short for its header whose flags byte names version 1, 2 or 3 is refused as an unsupported version naming that version, as one of normal length is (#115).

Translation

  • TranslationEngine accepts the specification's option names, sourceElementDataKey and missingTranslationBehavior. sourceDataKey and behavior still work and are deprecated (#49).

C library

  • The C library moves from 51degrees-cxx 4.5.103 to 4.5.106, whose C code is that of 4.5.105. The HighPerformance profile holds in memory only what the named Properties read, and for 51D_GetHighEntropyValues and 51D_StructuredUserAgent the query value is used before the cookie value (#67, #55, #92).

Configuration

  • A list setting may be given as one comma separated string, as the pipeline configuration specification says a configuration file gives a list. The engines' restrictedProperties and usage sharing's queryWhitelist and headerBlacklist read it as its items, through the new listSetting of fiftyone.pipeline.core, and an array works as before (#129).

Examples and documentation

  • Every on-premise example runs with no settings on the free Data Files, which it downloads the first time. A Property that only the Enterprise Data File holds is shown as needing that file, with a link to ask for a copy (#32).
  • Every example, console and web, ends with a Find out more section giving the repositories and tagged 51degrees.com pages to follow up, including where it stops early for want of a key or a data file (#103).
  • Each example's 51d.json names its engines by package, so the same file works in this repository and with the installed packages, where eleven named them by a path into the source (#110).
  • The IP Intelligence mixed API example runs with no settings on the free Device Detection and IP Intelligence Lite files, where it searched the disk for paid files first. It names each Property only the Enterprise data files hold as needing them, with a link to ask for a copy (#111).
  • The data update examples check for a License Key before they look for a data file, the IP Intelligence ones use the small ASN file, every automatic update example ends with status 0 when it succeeds, and both update data file examples read _51DEGREES_LICENSE_KEY first. The cloud metadata example says a resource key is needed when none is set, and the match metrics example prints a documentation link a person can follow (#101).
  • fiftyone.pipeline.cloudrequestengine has the star sign Cloud Engine example the pipeline specification asks for, using the real Cloud Request Engine against a star sign service it starts on the same machine, so it needs no key or network. The simple star sign example takes a date of birth as its argument (#121).
  • The IP Intelligence on-premise web example shares usage when its configuration names the usage sharing element, as its own 51d.json does and as the Device Detection one does, with a comment saying production should leave usage sharing on. The console examples still turn it off (#131).
  • Every cloud example run with no Resource Key ends without an error and with status 0, names _51DEGREES_RESOURCE_KEY, gives a tagged link to where a key is created and ends with its Find out more section, and a test runs each of the 20 with no key (#133).
  • Links to the configuration page, in the READMEs, code comments, example output and the messages that say where to create a Resource Key, point at https://51degrees.com/configure, where the page has moved from configure.51degrees.com (#87).
  • The repository has one LICENSE file at its root, the EUPL 1.2 text, and every JavaScript source file carries the standard 51Degrees header. The fiftyone.geolocation package no longer carries a LICENSE file of its own (#79).

Building and testing

  • The Selenium contract tests run for real on Linux. The CI removed .NET to free disk space, so dotnet was missing and the step reported a pass without running a test. The step now fails when dotnet is missing or when no test passed, and a pull request into rc is built on the configurations its changes need, with macOS and the performance tests left to builds of rc and the release (#27).
  • The pipeline examples' tests run each example in its own process and check what it prints or serves, where they built a function and never called it, and the 51Did offline reader and usage sharing examples have tests of their own (#104).
  • The umbrella getting started tests run each example in its own process and check its answers, and a new test reloads a Device Detection engine from the Lite data file onto the Enterprise data file and checks that it keeps its first Properties, as the C library does (#108).
  • The web integration elements test runs as a unit test, so the Publish tests, which run the integration tests against the packed packages, no longer look for a test file the core package does not pack (#112).
  • The native read timing test compares paired rounds and counts the case work, so it no longer fails at random on macOS (#96).
  • The Publish workflow takes allow-early-release, for a release agreed to go out less than a week after the last, and common-ci-ref, to run common-ci's steps from a branch other than main (#99).
  • The cloud 51Did tests run on pull requests into rc as well as main, so a change to the 51Did package is tested against the live cloud service before it reaches a release (#100).
  • AGENTS.md says that a test failing on timing is fixed rather than run again, and points at common-ci's one list of environment variable names (#93, #89).
  • Tests of the cloud Property metadata before and after discovery and of the Evidence a Cloud Aspect Engine accepts, and of a session cookie found only in the web request (#144, #145).

Environment variables

Each name below has a new name that starts _51DEGREES_, which every shell can set. The Cloud Request Engine and the 51Did client read the cloud endpoint, and the examples and tests read the rest. Every old name is still read in this release, but only when its new name is unset or empty, and its first use is reported as deprecated, through the Pipeline's logger or a DeprecationWarning in the packages, and in a printed message in the examples and tests. Each constant that held an old name holds it again and is marked @deprecated (#70, #77, #88).

Old name, deprecated New name
FOD_CLOUD_API_URL _51DEGREES_CLOUD_ENDPOINT
51DEGREES_DD_PATH _51DEGREES_DD_PATH
51DEGREES_IPI_PATH _51DEGREES_IPI_PATH
51DEGREES_RESOURCE_KEY, RESOURCE_KEY _51DEGREES_RESOURCE_KEY
LICENSE_KEY _51DEGREES_LICENSE_KEY
LicenseKey, LICENSEKEY (the update data file examples) _51DEGREES_LICENSE_KEY
TEST_SUPER_RESOURCE_KEY, TEST_PLATFORM_RESOURCE_KEY, TEST_HARDWARE_RESOURCE_KEY, TEST_BROWSER_RESOURCE_KEY, TEST_NO_SETHEADER_RESOURCE_KEY _51DEGREES_RESOURCE_KEY_SUPER, _51DEGREES_RESOURCE_KEY_PLATFORM, _51DEGREES_RESOURCE_KEY_HARDWARE, _51DEGREES_RESOURCE_KEY_BROWSER, _51DEGREES_RESOURCE_KEY_NO_SETHEADER

Fixed

  • From the second request on, the JSON sent to the browser lost delayexecution and evidenceproperties for every element but one, because the JSON Builder kept each element's delayed Properties under one shared key (#66).
  • An update started by hand with updateDataFile started another polling timer for the same data file each time, so the file was checked once for each of them in every polling interval (#63).
  • With no callback URL, the client script's renderedParameters returned nothing, and the script then threw a TypeError on a page whose tab held a record of an earlier request and values saved from it (#97).
  • A data update download cut short part way was logged as "No response from data update service", or on Windows as the access error, depending on timing. It is now always the access error, "Error accessing data update service", with its cause, and the part file is removed before the failure is reported (#94).
  • The mixed web example named its IP Intelligence engine by a path into the source folder, so it failed against the published packages, as it did in the 13 November publish. It now names the package. The location examples printed ReferenceError: resourceKey is not defined, and now read the key as the other examples do (#95).
  • The Cloud Request Engine asked the cloud service for its Evidence keys and accessible Properties before every request, so each request made three calls to the cloud service in place of one. Release 4.5.102 and earlier have the fault (#73).
  • In 4.5.102 a Cloud Engine threw TypeError: Cannot convert undefined or null to object when the Resource Key did not include its product. One error is now logged for each Engine and the request carries on (#41).
  • An updateComplete listener that threw ended the Node.js process (#65).
  • The wait before a data update check used only the millisecond part of the Data File's expected update time, so polling did not wait for that time (#52).
  • Usage Sharing put a comma between the <Device> elements of a message, logged a failed send as Usage data sent, and wrote <ClientIP>undefined</ClientIP> for a request with no client IP (#35, #36, #54).
  • The on-premise web examples never showed their warning about an old data file, because each was given the current time as the file's published date and compared days with milliseconds. They now warn from the file's own published date (#102).
  • The client-side evidence and on-premise flow element web examples ended their own server on the first page, because each page read a star sign or hemisphere that the element left out of its data until it was known. Both are now null until known (#104).
  • The IP Intelligence compare, metrics, suspicious and mixed examples stopped with RangeError: Maximum call stack size exceeded while they searched for a Data File (#32).
  • flowData.get('Device') and flowData.Device threw where flowData.device worked (#122).
  • The Device Detection cloud package's types did not compile for a TypeScript caller (TS2307), because its index.js required its own files through .. (#113).
  • A list setting given as one comma separated string stopped the build with a TypeError for restrictedProperties and headerBlacklist, and was split into single characters for queryWhitelist (#129).
  • The simple and client-side star sign examples gave no star sign for 21 June (#121).
  • With no Resource Key set, three cloud web examples stopped with an uncaught error and the 51Did creator context demo exited with status 1 and no link to where a key is created (#133).
  • A list of parallel elements given to addParallel stayed the caller's list, so an element added to it after the build was processed by the built Pipeline, and one taken out was not (#142).
  • The same Evidence added in another order, or with a key in other letters, missed an Engine's cache and was processed again, and the key stored was made after processing (#125).
  • A failed cloud answer with an empty body read as "Error parsing response - [object Object]", and one whose JSON had no errors member threw a TypeError (#140).
  • One JSON Builder added to a second Pipeline failed on any element the first Pipeline lacked, and used the first Pipeline's delayed execution for an element with the same data key. An element whose Properties were replaced after its first request, as a cloud engine's are when discovery succeeds late, kept the delayed execution of its old Properties (#143).
  • Usage Sharing's warnings of a full queue, a pause and each failed send were emitted only as the event warning, which no example or document listens for (#141).
  • Helpers.setResponseHeaders threw when processing stopped before the Set Headers element, which ended the process where it was called in a promise with no catch, as the getting started web examples do. It now sets no headers and logs a warning (#146).
  • With minify: true the JavaScript Builder gave its script unminified, because it used the minified script only when uglify-js reported error as null, where uglify-js leaves it undefined. A failure to minify is now logged as a warning (#150).

Issues this closes

Build

The release build, the timing against main, the Publish dry run and the Nightly Pipeline dry run on this head are running, and this section is filled in from them when they end.

Only the patch number moves.

AGENTS.md gains a section linking common-ci's list of environment variable
names (environment-variables.md, made from environment-variables.json) at
commit 18f9a570, and saying the list decides. README.md points at the same page
where it names the variables the tests or examples read.

Documentation only.
…file before reporting it (#94)

* Test that a download cut short reports the access error

The data update specification's Errors table gives the messages of the
.NET data update service. .NET reads the whole response before its
request returns, so a download cut short fails that request, and .NET
reports it as "Error accessing data update service at '<url>' for engine
'<name>'" with the cause attached.

The test of a download cut short closed the connections 10 milliseconds
after the check started, whether or not the response had arrived, so it
passed or failed by chance. On a slow runner the connection closed before
the response, and the message was the one for a failed connection. The
test now cuts the connection only once the service's request has its
response, from a server that never sends the whole file, and a
connection closed before any response is tested on its own. The test of
a reset part way through the download expects the same error, which a
reset gives whether it comes before or after the response.

* Report a download cut short as the access error, with its cause

A download that ended before the whole file arrived was logged as "No
response from data update service", a message .NET writes only when its
request returns no response at all. .NET reads the whole response before
its request returns, so it reports a download cut short as "Error
accessing data update service at '<url>' for engine '<name>'" with the
cause attached, which is what Node already wrote for a connection that
failed before any response. The service now writes that message for a
download cut short too, with the first error the request or the response
raised after the response arrived as the cause. The status is unchanged.

* Test that a download cut short leaves no part file, with no timer

The test of a reset part way through the download reset the connection 50
milliseconds after the server wrote, and waited 500 milliseconds before it
looked, so its check that the part file had gone held only because of
those two times. It now resets once the service's request has its
response, waits for the update to be reported, and then checks that there
is one report, one scheduled check and no part file. The test of a
download cut short checks the folder too. In both tests the cut comes
before the service has opened its part file.

* Remove a cut short download's part file before reporting the update

When a download was cut short the service destroyed the write stream for
the part file, asked for the file to be removed straight away, and
reported the update. A stream still opening its file has not created it
yet, so the removal could find nothing and the file the stream then
created would stay in the temporary folder. In every order the file could
still be there when the update was reported. The service now removes the
part file once the stream has closed, which is after the file is open,
and reports the update once the file has gone.

* Test that the part file is removed before the update is reported

The folder checks in the two cut short tests passed or failed by chance on
the service before the fix, because fs.unlink removes the part file on
another thread while the test goes on. A spy on fs.unlink now records each
removal as it finishes, and both tests check that the part file's removal
had finished when the update was reported. Before the fix the service
reports the update first, so the check fails every time.
…or (#15)

* Package text: say what each package does in the words people search for

Developers search npm with words such as user agent parser, device
detection, and IP geolocation. The descriptions of the six Device
Detection and IP Intelligence packages now say what each package does in
those words, and the keywords add the matching terms. Existing keywords
that are still true are kept, and keywords naming other companies'
products are removed.

Both package.json and remote_package.json change, because
ci/build-package.ps1 publishes remote_package.json and the two files
should match. Only the description and keywords fields change, and no
version number moves.

* Package text: name the products these packages are an alternative to

People searching npm for WURFL, DeviceAtlas, MaxMind, or ipinfo should
find these packages too. The keywords naming those products are restored
exactly as they were, and each description now ends with the products
the package is an alternative to. Only the description and keywords
lines change, and no version number moves.

* Package text: name UAParser.js in full

The Device Detection descriptions now say "An alternative to UAParser.js,
DeviceAtlas, and WURFL." The 51Degrees fork is called 51Degrees UAParser,
so the bare name could read as our own package. Only the description line
changes, and no version number moves.
…se work (#96)

Reviewed and approved at head 1ef641c. The test times both arms side by side, takes the middle of 200 rounds and holds them to 1.05, set from 293 measurements on every runner (red 43443859591, green 43473620570, full build 43473656997 with macOS).
…ntries (#27)

On Ubuntu the Selenium contract step reports a pass without running a test, so the browser contract has had no Linux coverage. And a pull request into `rc` runs all eight configurations, two of them on macOS, of which only five run at once across the whole organisation.
…ue (#66)

The JSON Builder wrote `delayexecution` for every property whose metadata sets the delay execution flag, before it read the value ([`jsonbundler.js` line 168 at 24592ac](https://github.com/51Degrees/51degrees-node/blob/24592acbee7fd2881a96ac661296535daed6f157/pipeline/fiftyone.pipeline.core/jsonbundler.js#L168)). So a property with no value, and one that could not be read at all, were marked too. The cross-language audit found this, and PHP ([51degrees-php#61](51Degrees/51degrees-php#61)), Go ([51degrees-go#91](51Degrees/51degrees-go#91)) and Python ([51degrees-python#92](51Degrees/51degrees-python#92)) have already merged the same change.
…heck what the location examples print (#95)

The Publish run for release 4.5.103, [run 43412143722](https://github.com/51Degrees/51degrees-node/actions/runs/43412143722) on `main` at 4d5832f (the merge of #83), failed, so nothing was published and npm still has 4.5.102. In each of the eight "Publish / Test" jobs (Ubuntu, Ubuntu ARM, Windows and macOS, on Node.js 22 and 24) exactly one test failed, "Examples without settings › the mixed web page marks the Enterprise properties" in [`examplesWithoutSettings.test.js`](https://github.com/51Degrees/51degrees-node/blob/24592acbee7fd2881a96ac661296535daed6f157/ip-intelligence/fiftyone.ipintelligence.onpremise/examples/onpremise/examplesWithoutSettings.test.js#L234-L244).
[#8](#8). The JavaScript builder sets the template's `_parameters` only when it has a callback URL ([`javascriptbuilder.js` lines 340 to 365 at 24592ac](https://github.com/51Degrees/51degrees-node/blob/24592acbee7fd2881a96ac661296535daed6f157/pipeline/fiftyone.pipeline.core/javascriptbuilder.js#L340-L365)). With no endpoint, the template's line `var renderedParameters = function() { return {{&_parameters}}; };` renders as `return ;`.
… a named branch (#99)

common-ci refuses a release that comes less than a week after the last one unless `allow-early-release` is set, and the refusal tells the person to "run this repository's Publish workflow with allow-early-release set." ([common-ci `.github/workflows/nightly-publish.yml` lines 28 to 31, 202 and 210 at 1c5c0e0b](https://github.com/51Degrees/common-ci/blob/1c5c0e0b09a1f77722e20ba2d10cbd1031caca9e/.github/workflows/nightly-publish.yml#L28-L31)). This repository's Publish workflow has only the `dryrun` input ([`.github/workflows/publish.yml` lines 3 to 8 at 758fea8](https://github.com/51Degrees/51degrees-node/blob/758fea819c13fe0df4275a0a0478cdd03f57d5ed/.github/workflows/publish.yml#L3-L8)), so a release agreed to go out early cannot be started, whatever the message says.
)

The Cloud 51Did workflow runs the live 51Did tests on a pull request only when it targets `main` ([`.github/workflows/cloud-51did.yml` lines 32 and 33 at 758fea8](https://github.com/51Degrees/51degrees-node/blob/758fea819c13fe0df4275a0a0478cdd03f57d5ed/.github/workflows/cloud-51did.yml#L32-L33)). Changes reach `main` through `rc`, so a change to the 51Did package is first tested against the live cloud service when the release pull request from `rc` opens, after it has been reviewed and merged into `rc`.
…es when a value is not yet known (#104)

Five pipeline examples had tests that never ran them, and two examples had no test at all (gaps DX-node-34 and DX-node-35).
…kage (#106)

`fiftyone.pipeline.core` has no `files` list ([`pipeline/fiftyone.pipeline.core/package.json` at 758fea8](https://github.com/51Degrees/51degrees-node/blob/758fea819c13fe0df4275a0a0478cdd03f57d5ed/pipeline/fiftyone.pipeline.core/package.json)), so npm packs the whole `javascript-templates` submodule, its tests, README, LICENSE and GitHub workflows, where the JavaScript Builder reads only `JavaScriptResource.mustache` ([`javascriptbuilder.js` lines 27 to 30](https://github.com/51Degrees/51degrees-node/blob/758fea819c13fe0df4275a0a0478cdd03f57d5ed/pipeline/fiftyone.pipeline.core/javascriptbuilder.js#L27-L30)). Anything the templates repository adds later would be published in this package too, so the submodule could not move on without changing what the package holds. The Device Detection and IP Intelligence packages already have a `files` list, for example [`device-detection/fiftyone.devicedetection.cloud/package.json`](https://github.com/51Degrees/51degrees-node/blob/758fea819c13fe0df4275a0a0478cdd03f57d5ed/device-detection/fiftyone.devicedetection.cloud/package.json). This is gap DX-node-15, item 6 of run book 9.
Step 3.1 of [common-ci#306](51Degrees/common-ci#306). [`ci/build-package.ps1` line 35 at 21fa119](https://github.com/51Degrees/51degrees-node/blob/21fa11996a1e584e855ef548871cc006d23e5873/ci/build-package.ps1#L35) runs `npm version` with the run's one version for every package, so all 16 packages published 4.5.102 on 4 November 2026 ([Publish run 41918901313](https://github.com/51Degrees/51degrees-node/actions/runs/41918901313)) whatever had changed. That spends every package's allowance under the weekly release rule, and the [release of 13 November](#83) would have done the same.
)

`PipelineBuilder.addRequiredElements` adds the Sequence Element, the JSON Builder, the JavaScript Builder and the Set Headers Element whatever the caller already added ([`pipeline/fiftyone.pipeline.core/pipelineBuilder.js` lines 200 to 205 at 758fea8](https://github.com/51Degrees/51degrees-node/blob/758fea819c13fe0df4275a0a0478cdd03f57d5ed/pipeline/fiftyone.pipeline.core/pipelineBuilder.js#L200-L205)). This has three effects.
…he Device Id row (#102)

Two gaps from the examples audit, both recorded under "Not changed" in [#32](#32).
This draft closes two gaps from the audit of the deferred work and the examples, DX-node-26 and DX-node-27. On `rc` at 758fea8 the examples did not end with a "Find out more" section, which is the part of an example that gives the person running it somewhere to go next.
Eleven of the examples' `51d.json` files name an engine by a path into this repository's source, such as `"../../../deviceDetectionCloud"` in [`device-detection/fiftyone.devicedetection.cloud/examples/cloud/gettingstarted-console/51d.json` at fe81e01](https://github.com/51Degrees/51degrees-node/blob/fe81e01866b5aae0e2c6ec11a25a4f0a5d1ccd31/device-detection/fiftyone.devicedetection.cloud/examples/cloud/gettingstarted-console/51d.json). Such a file only works beside the source. [`ci/install-package.ps1` lines 43 to 55](https://github.com/51Degrees/51degrees-node/blob/fe81e01866b5aae0e2c6ec11a25a4f0a5d1ccd31/ci/install-package.ps1#L43-L55) rewrites two of them so the Publish tests run them against the packed packages, and [`device-detection/run_examples.md` lines 72 to 92](https://github.com/51Degrees/51degrees-node/blob/fe81e01866b5aae0e2c6ec11a25a4f0a5d1ccd31/device-detection/run_examples.md#L72-L92) and [`ip-intelligence/run_examples.md` lines 61 to 81](https://github.com/51Degrees/51degrees-node/blob/fe81e01866b5aae0e2c6ec11a25a4f0a5d1ccd31/ip-intelligence/run_examples.md#L61-L81) tell a person running an example against the published packages to edit the rest by hand. This is gap DX-node-20, left under "What remains" in [#95](#95), which named the mixed web example's engine by package.
This draft was to wait until https://51degrees.com/configure answers. At 17:35 UTC on 13 November it does. `/configure`, `/configure/`, `/configure/gGc3T5pT` and `/configure?utm_source=code` each answer 200 with the page title "Cloud Configuration | 51Degrees", the same title `configure.51degrees.com` serves, and the page at the new address loads the configuration app with `<base href="/configure/">`. On 13 November at 15:01 the same addresses answered 404.
…ata file reload (#108)

Two test gaps, with no package code to change.
…id and sequence in any letters (#109)

Two faults in the usage sharing record, and two cases no test covered, all at [fe81e01](https://github.com/51Degrees/51degrees-node/tree/fe81e01866b5aae0e2c6ec11a25a4f0a5d1ccd31).
… name what needs the Enterprise files (#111)

The IP Intelligence mixed API example, which runs a local copy of the cloud service's API on both on-premise engines, does not run from a fresh clone as the other examples do (gap DX-node-29, left under "Not changed" in [#32](#32) and "What remains" in [#95](#95)). At [7539a0b](https://github.com/51Degrees/51degrees-node/tree/7539a0be284aec3e0aae12cf50cf727ead68fb4c):
…n the packed packages (#112)

The Nightly Pipeline dry run [43578169805](https://github.com/51Degrees/51degrees-node/actions/runs/43578169805), on a throwaway branch holding `rc` at 7539a0b with the drafts then approved merged in, fails one test in all eight Test jobs of its Publish stage, being Ubuntu, Ubuntu ARM, Windows and macOS on Node.js 22 and 24.
fiftyone.pipeline.did took the [OWID library](https://github.com/SWAN-community/owid-js) as the npm dependency `"owid": "github:SWAN-community/owid-js#6c5ff4e7"`, bundled into its tarball, where every other 51Degrees language takes OWID as a git submodule. This takes owid-js as the submodule `pipeline/fiftyone.pipeline.did/owid-js`, pinned at the same commit, and packs only the library's `v1.js` and its `LICENSE` from it.
…le for a TypeScript caller (#113)

Each package publishes the `types/*.d.ts` files committed in this repository, and nothing writes them again before a package is packed. The readmes say that "Calling the generation step is manual for now" ([`device-detection/readme.md` line 236 at a9a3bd2](https://github.com/51Degrees/51degrees-node/blob/a9a3bd2ec192f0006296a39366a710c5ec090f18/device-detection/readme.md?plain=1#L236)). So the published types have fallen behind the code. With TypeScript 5.9.3, `npm run tsc` on `rc` at a9a3bd2 changes 23 of the committed files and writes one that was never committed, `pipeline/fiftyone.pipeline.core/types/callbackUrl.d.ts` ([run 43580575276](https://github.com/51Degrees/51degrees-node/actions/runs/43580575276)).
…and mark the raw flags byte deprecated (#115)

1. **A short payload that names another version is refused as too short, and the version it names is not reported.** `unpack` checks the length against version 0's header before it reads the version ([`pipeline/fiftyone.pipeline.did/internal/payload.js` lines 88 to 113 at 6d5d4fe](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.did/internal/payload.js#L88-L113)), so a payload of 1 to 4 bytes whose flags byte names version 1, 2 or 3 gets `PayloadTooShort`, measured against a header that a later version need not have. A payload of normal length is refused naming its version, and tested ([`tests/fodId.test.js` lines 611 to 632](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.did/tests/fodId.test.js#L611-L632)).
2. **The raw flags byte is part of the typed surface.** `FodId` declares `_flags: number` ([`types/fodId.d.ts` line 189](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.did/types/fodId.d.ts#L189)), set at [`fodId.js` line 132](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.did/fodId.js#L132) and [line 480](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.did/fodId.js#L480). The published fiftyone.pipeline.did 4.5.102 declares it too ([its `types/fodId.d.ts` line 177](https://unpkg.com/fiftyone.pipeline.did@4.5.102/types/fodId.d.ts)), where the offsets and the Terms index are kept in modules that are not exported.
…e's headers, and read Set Header identifiers that hold digits (#120)

1. **The client-side endpoints process a request before they compare its If-None-Match with the ETag.** `Helpers.writeClientSideResponse` takes "the processed flow data for the request" ([`pipeline/fiftyone.pipeline.core/helpers.js` lines 148 to 168 at 6d5d4fe](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/helpers.js#L148-L168)), so every example that serves the JSON or JavaScript endpoint runs the Pipeline first, for example the Device Detection on-premise web example ([`gettingStarted.js` lines 233 to 250](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/device-detection/fiftyone.devicedetection.onpremise/examples/onpremise/gettingstarted-web/gettingStarted.js#L233-L250)). A browser that revalidates a script it already holds still costs a full detection.
2. **A 304 sent while processing records errors loses the ETag and max-age.** The response then carries `Cache-Control: private, no-cache` and no ETag, 304 included ([`helpers.js` lines 185 to 202](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/helpers.js#L185-L202)), and a test asserts it ([`tests/clientSideCaching.test.js` lines 226 to 230](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/tests/clientSideCaching.test.js#L226-L230)). So a browser that holds a good response keeps it through an outage but its cache drops the ETag and max-age.
3. **A Set Header identifier that holds a digit ends early.** `getHeaderName` skips only `a` to `z` after the identifier's first character ([`setHeadersElement.js` lines 61 to 72](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/setHeadersElement.js#L61-L72)), so `SetHeaderBrowser2Accept-CH` gives the header `2Accept-CH`. No property in use today has a digit in its identifier, so no response is wrong today.
4. **No test passes the max-age option**, which `writeClientSideResponse` takes ([line 167](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/helpers.js#L167)).
Element Data is found only by its key as written. `FlowData.get` looks the key up as given, `const elementData = this.data[flowElementDataKey];` ([`pipeline/fiftyone.pipeline.core/flowData.js` line 191 at 6d5d4fe](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/flowData.js#L191)), so `flowData.get('Device')` and `flowData.Device` throw where `flowData.device` works. Property names already ignore letter case ([`elementData.js` lines 100 to 123](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/elementData.js#L100-L123)).
…warning (#123)

1. **The warnings a cloud response carries are never logged.** The cloud service can answer with a `warnings` array beside its results, and no code in `fiftyone.pipeline.cloudrequestengine` reads it. The request engine parses the discovery response for its products ([`cloudRequestEngine.js` lines 404 to 415 at 6d5d4fe](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.cloudrequestengine/cloudRequestEngine.js#L404-L415)) and keeps the processing response as it is ([lines 504 to 515](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.cloudrequestengine/cloudRequestEngine.js#L504-L515)), so a warning from the service reaches nobody.
2. **A product the key does not include is logged as an error.** The cloud engine logs it once, at error level ([`cloudEngine.js` lines 131 to 141](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.cloudrequestengine/cloudEngine.js#L131-L141)), though nothing is wrong with the request and the other products in the answer are used.
…, read _51DEGREES_LICENSE_KEY and end with status 0 (#101)

Five faults in the examples that update a data file, and in two example messages, all at [758fea8](https://github.com/51Degrees/51degrees-node/tree/758fea819c13fe0df4275a0a0478cdd03f57d5ed).
…uires (#121)

The pipeline specification asks for four star sign examples in the core, and Node.js had three. The simple Flow Element and the client-side evidence Flow Element are in [`pipeline/fiftyone.pipeline.core/examples/customFlowElements`](https://github.com/51Degrees/51degrees-node/tree/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/examples/customFlowElements) and the on-premise Engine is [`pipeline/fiftyone.pipeline.engines/examples/onPremiseFlowElement.js`](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.engines/examples/onPremiseFlowElement.js), but no example makes a Cloud Engine, and `fiftyone.pipeline.cloudrequestengine` had no examples folder.
…tion files give it (#129)

A list setting can be given only as an array, though a configuration file gives a list as one comma separated string. So a `51d.json` that gives a list the way the specification says fails or is misread in three places, at `rc` 6d5d4fe.
…a is refreshed (#130)

Both on-premise engines read the Evidence keys they accept from the native engine once, when the native engine is created, and never again. A refresh of the data file calls `refreshData()` and keeps the filter made from the data file the engine started with.
…e on-premise form (#124)

The IP Intelligence cloud engine keeps every value as the cloud service sends it. `CloudEngine` stores each one unchanged, `result[key].value = value;` ([`pipeline/fiftyone.pipeline.cloudrequestengine/cloudEngine.js` lines 152 to 159 at 6d5d4fe](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.cloudrequestengine/cloudEngine.js#L152-L159)), and `IpIntelligenceCloud` adds nothing ([`ip-intelligence/fiftyone.ipintelligence.cloud/ipIntelligenceCloud.js` lines 24 to 42](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/ip-intelligence/fiftyone.ipintelligence.cloud/ipIntelligenceCloud.js#L24-L42)). So for the same address:
…idence keys (#125)

An Engine's cache key is the JSON of the Evidence it accepts, in the order the Evidence was added and with each key as it was first spelled ([`pipeline/fiftyone.pipeline.engines/engine.js` lines 99 to 104 and 136 to 141 at 6d5d4fe](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.engines/engine.js#L99-L141)). So the same Evidence added in another order, or with a key in other letters, such as `header.User-Agent` where an earlier request sent `header.user-agent`, misses the cache and is processed again. The key stored was also made after processing, from Evidence processing may have added to, where the lookup is made before.
…and write lists of Element Data by the JSON rules (#118)

1. **The script takes its session id and sequence from the evidence, not from the Sequence Element.** The JavaScript Builder reads `query.session-id` and `query.sequence` from the evidence ([`pipeline/fiftyone.pipeline.core/javascriptbuilder.js` lines 383 to 386 at 6d5d4fe](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/javascriptbuilder.js#L383-L386)). With a Sequence Element the two agree, because the element writes the same values to the evidence and to its data. With no Sequence Element, which the JSON Builder allows when a caller adds a whole number as `query.sequence`, the script carries whatever the evidence held, and a test asserted that ([`tests/sessionIdAndSequence.test.js` lines 157 to 167](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/tests/sessionIdAndSequence.test.js#L157-L167)).
2. **A list of Element Data is written with each item's internal fields.** The JSON Builder writes a Property whose value is a list as it is ([`jsonbundler.js` lines 186 to 190](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/jsonbundler.js#L186-L190)). The cloud hardware profile lookup gives `profiles` as a list of objects of Aspect Property Values ([`hardwareProfileCloudEngine.js` lines 103 to 126](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/device-detection/fiftyone.devicedetection.cloud/hardwareProfileCloudEngine.js#L103-L126)), so each profile reaches the JSON as `noValueMessage`, `_value` and `hasValue` for every Property, rather than the value, or null with its reason.
3. **Two JSON rules the code meets had no test**, that a plain null has no entry and that every key is in lower case.
The Sequence Element counted any JavaScript number with no fractional part as a sequence ([`pipeline/fiftyone.pipeline.core/sequenceElement.js` lines 89 to 95 at 6d5d4fe](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/pipeline/fiftyone.pipeline.core/sequenceElement.js#L89-L95)), so a floating-point 2.0 counted as 2, and a `BigInt` did not count at all. Every ordinary JavaScript number is a floating-point value, so 2.0 and 2 are the same number and cannot be told apart.
…iles, defaults and order (#137)

The on-premise Device Detection engine adds the seven match metric Properties to its Property metadata itself, as the data file does not hold them ([`device-detection/fiftyone.devicedetection.onpremise/deviceDetectionOnPremise.js` lines 93 to 172 at cdbbee8](https://github.com/51Degrees/51degrees-node/blob/cdbbee828ec2e2af1b48209a7deb95a660c64570/device-detection/fiftyone.devicedetection.onpremise/deviceDetectionOnPremise.js#L93-L172)). Their metadata differed from the specification's in three ways.
…ice Detection one does (#131)

The IP Intelligence on-premise getting started web example builds its Pipeline with usage sharing turned off ([`ip-intelligence/fiftyone.ipintelligence.onpremise/examples/onpremise/gettingstarted-web/gettingStarted.js` line 122 at 6d5d4fe](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/ip-intelligence/fiftyone.ipintelligence.onpremise/examples/onpremise/gettingstarted-web/gettingStarted.js#L122)), though the example's own `51d.json` lists the usage sharing element. The Device Detection on-premise web example shares usage through its `51d.json` ([lines 13 to 23](https://github.com/51Degrees/51degrees-node/blob/6d5d4fef292afffffae3e7892d3e8c043f1b22c7/device-detection/fiftyone.devicedetection.onpremise/examples/onpremise/gettingstarted-web/51d.json#L13-L23)), and every console example turns it off.
…get one (#133)

A person who runs a cloud example from a fresh clone has no resource key set. Every cloud example should then end without an error, name the environment variable to set, give a tagged link to the page where a key is created, end with its Find out more section and exit with status 0, so that CI can run it with no settings. At `rc` 6d5d4fe, 11 of the 20 cloud examples did not.
…nes even when all Evidence is shared (#135)

At `rc` cdbbee8 usage sharing shared too little by default and too much when all Evidence was shared.
…e changed (#142)

At `rc` eb72aa1 a built Pipeline could still be changed in three ways.
… messages (#140)

At `rc` 7c99e78 the Cloud Request Engine handled the cloud service's answers in three ways the specification does not.
…e it is in (#143)

At `rc` 1ed293b the JSON Builder worked out which Properties of each element have delayed execution once, from the first Pipeline it processed, and used that for every Pipeline after ([`pipeline/fiftyone.pipeline.core/jsonbundler.js` lines 145 to 155](https://github.com/51Degrees/51degrees-node/blob/1ed293b2876f39853c93f4a83291d531b293cffe/pipeline/fiftyone.pipeline.core/jsonbundler.js#L145-L155) and [lines 207 to 213](https://github.com/51Degrees/51degrees-node/blob/1ed293b2876f39853c93f4a83291d531b293cffe/pipeline/fiftyone.pipeline.core/jsonbundler.js#L207-L213)). One JSON Builder can be added to several Pipelines, as any element can, so in a second Pipeline:
…ight and rawWeight for this release (#139)

The JSON Builder writes a Property's value as the Element Data gives it ([`pipeline/fiftyone.pipeline.core/jsonbundler.js` line 271 at 7c99e78](https://github.com/51Degrees/51degrees-node/blob/7c99e7802ea3fa564c360d13af94332511ffdda1/pipeline/fiftyone.pipeline.core/jsonbundler.js#L271)), with only a list of Element Data rewritten ([lines 242 to 246](https://github.com/51Degrees/51degrees-node/blob/7c99e7802ea3fa564c360d13af94332511ffdda1/pipeline/fiftyone.pipeline.core/jsonbundler.js#L242-L246)). So each item of a weighted value in the JSON sent to the browser has the members the Element Data uses, `value`, `weight` and `rawWeight` ([`ip-intelligence/fiftyone.ipintelligence.onpremise/swigHelpers.js` line 59](https://github.com/51Degrees/51degrees-node/blob/7c99e7802ea3fa564c360d13af94332511ffdda1/ip-intelligence/fiftyone.ipintelligence.onpremise/swigHelpers.js#L59)), where the specification writes `rawweighting` and `value`, in lower case. A cloud weighted value carries `rawweighting` as well since [#124](#124), and an on-premise one does not.
…bsent until discovery succeeds (#144)

The Cloud Request Engine and the Cloud Aspect Engines met four requirements on Property metadata and Evidence at `rc` 1ed293b, but no test held them, so a change could break any of them unseen.
…sharing's sessions apart (#145)

With session tracking on, usage sharing compares a request with those already shared using the values it would share and the session cookie ([`pipeline/fiftyone.pipeline.engines.fiftyone/shareUsage.js` lines 263 to 276 at 0e89a40](https://github.com/51Degrees/51degrees-node/blob/0e89a4009ae74a42eef36110c1f88262cf616c4b/pipeline/fiftyone.pipeline.engines.fiftyone/shareUsage.js#L263-L276)). Those values come from the Evidence and from the web request supplied as `server.http-request` ([lines 289 to 313](https://github.com/51Degrees/51degrees-node/blob/0e89a4009ae74a42eef36110c1f88262cf616c4b/pipeline/fiftyone.pipeline.engines.fiftyone/shareUsage.js#L289-L313)), as the specification asks. The only test of session tracking, in `shareUsage.test.js`, gives the session cookie as Evidence, so nothing showed that a cookie found only in the web request is read too.
…ers result (#146)

At `rc` 0e89a40 `Helpers.setResponseHeaders` read the Set Headers element's result as `flowData['set-headers']` ([`pipeline/fiftyone.pipeline.core/helpers.js` line 183](https://github.com/51Degrees/51degrees-node/blob/0e89a4009ae74a42eef36110c1f88262cf616c4b/pipeline/fiftyone.pipeline.core/helpers.js#L183)), which throws when the Flow Data has no such result ([`flowData.js` lines 188 to 208](https://github.com/51Degrees/51degrees-node/blob/0e89a4009ae74a42eef36110c1f88262cf616c4b/pipeline/fiftyone.pipeline.core/flowData.js#L188-L208)), for example when an element stopped processing before the Set Headers element ran. The Device Detection getting started web examples call it inside `flowData.process().then(...)` with no `catch` ([on-premise `gettingStarted.js` line 268](https://github.com/51Degrees/51degrees-node/blob/0e89a4009ae74a42eef36110c1f88262cf616c4b/device-detection/fiftyone.devicedetection.onpremise/examples/onpremise/gettingstarted-web/gettingStarted.js#L268), [cloud `gettingStarted.js` line 267](https://github.com/51Degrees/51degrees-node/blob/0e89a4009ae74a42eef36110c1f88262cf616c4b/device-detection/fiftyone.devicedetection.cloud/examples/cloud/gettingstarted-web/gettingStarted.js#L267)), so code written the same way meets an unhandled rejection, which ends a Node.js process by default.
…rs hear (#141)

Usage sharing logged its six warnings at the level `'warning'` ([`pipeline/fiftyone.pipeline.engines.fiftyone/shareUsage.js` lines 393, 522, 565, 586, 603 and 609 at a2aef52](https://github.com/51Degrees/51degrees-node/blob/a2aef527e72caad6f2ba0cf46c1ca3a0a35c23b9/pipeline/fiftyone.pipeline.engines.fiftyone/shareUsage.js#L393-L609)), being a full queue, a pause and its delay, and each failed send. The Pipeline emits a log entry as an event named by its level ([`pipeline.js` line 236](https://github.com/51Degrees/51degrees-node/blob/a2aef527e72caad6f2ba0cf46c1ca3a0a35c23b9/pipeline/fiftyone.pipeline.core/pipeline.js#L236)), and the levels callers listen for are `info`, `debug`, `warn` and `error`, as the examples say (for example [`nativeModelLookup.js` line 142](https://github.com/51Degrees/51degrees-node/blob/a2aef527e72caad6f2ba0cf46c1ca3a0a35c23b9/device-detection/fiftyone.devicedetection.cloud/examples/cloud/nativemodellookup-console/nativeModelLookup.js#L142)), and as every other element warns. So none of those warnings reached a caller listening for warnings. The tests of the back-off replaced the element's `_log` and read what it was given, so they never saw that no listener heard it.
…150)

At `rc` d4f2615, with `minify: true` the JavaScript Builder gave its script unminified. It called `uglifyJS.minify` and used the result only when `minified.error === null` ([`pipeline/fiftyone.pipeline.core/javascriptbuilder.js` lines 418 to 423](https://github.com/51Degrees/51degrees-node/blob/d4f26155f50aaf7b518c3efbd4fcecc76b2bedca/pipeline/fiftyone.pipeline.core/javascriptbuilder.js#L418-L423)), but uglify-js leaves `error` undefined when minifying succeeds, as [its README at 111746bb](https://github.com/mishoo/UglifyJS/blob/111746bbae5f55c88e3b82b42f14fd0f3129ea53/README.md?plain=1#L396) says (``console.log(result.error); // runtime error, or `undefined` if no error``). So the minified code was never used, and a failure to minify was not reported either. No test ran the builder with `minify` on, as `javascriptbuilder.test.js` only checks that it is off by default.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant