[jwrosewell]

Problem

uach-high-entropy-decoder.md on rc at 13642c4 says the decoder Element "is only implemented for .NET at time of writing", that "The reference implementation writes the decoded values to Evidence", and that "The Device Detection Engines MUST be capable of using these values from Element Data if available" (lines 33 to 44). None of the three holds for an on-premise Engine built on the native code in 51degrees-cxx.

• The native code already turns 51D_GetHighEntropyValues evidence into client hint headers before it matches, so an Engine built on it gets the same answer without the Element.
• .NET has no decoder Element, and neither does Java, so there is no reference implementation writing to Evidence.

The draft 51degrees-rust#350 leaves the conversion to the C library and stops adding the Rust decoder element to the on-premise pipeline. Its comparison with .NET shows device answers do not change. This pull request changes the specification to match, under the rule that the specification is followed where it is clear, and changes where it is silent or out of date and .NET agrees.

What was checked

The native code, at 51degrees-cxx cfad897, which is the cxx submodule of the .NET and Python repositories on rc.

• setGetHighEntropyValuesHeader turns a 51D_GetHighEntropyValues value into headers through TransformIterateGhevFromBase64 (src/dd/hash/hash.c lines 3902 to 3920). setSpecialHeaders stops at the first value that gives a header (lines 3949 to 3959). resultsHashFromEvidence_setSpecialHeaders applies it to every query value first and to the cookie values only when no query value gave a header (lines 4287 to 4305). Detection calls that before it reads the headers to match, whenever the evidence holds no device id (lines 4679 to 4689).
• That runs while processSpecialEvidence is true, and the default is true (src/dd/config-dd.h line 90).
• The headers made are Sec-CH-UA, Sec-CH-UA-Platform, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Platform-Version, Sec-CH-UA-Arch and Sec-CH-UA-Bitness (src/dd/transform.h lines 49 to 56), which include the six in the page's Element Data table. A structured User-Agent gives no sec-ch-ua (src/dd/transformc.c lines 1105 to 1107).
• While special evidence is processed the data set accepts 51D_gethighentropyvalues and 51D_structureduseragent under both the query. and cookie. prefixes (hash.c lines 5011 to 5019), so an Engine that takes the native key list passes the raw values on.

Java, Node, PHP and Go on rc still build 51degrees-cxx e866f6d. It converts the values in the same way, but reads query and cookie values together in the order of the evidence (hash.c lines 3922 to 3933), so those languages follow "Accepted Evidence" on query before cookie only once they take cfad897. That gap is not changed by this pull request.

.NET, at 51degrees-dotnet 4dd312b, its rc on 10 November.

• No source file holds a decoder element. No file is named for one, and a search of the tree for gethighentropyvalues finds only the JavascriptGetHighEntropyValues property, examples, tests and configuration. UachJsConversionElementBuilder appears nowhere, and the SWIG Transform class is used only by the tests.
• The examples say the engine consumes the values as query or cookie parameters, converting them to headers internally (examples/device-detection/ExampleBase/ExampleUtils.cs lines 722 to 726).

Java, at 51degrees-java ff6e4c0, its rc on 10 November. No library file is named for a decoder. UACHServiceCore sets the Accept-CH response header and decodes nothing.

Change

One file, device-detection-specification/pipeline-elements/uach-high-entropy-decoder.md. The branch has rc at 13642c4 merged in, which resolved a conflict with #116 and #119 in the Element Data paragraph.

Overview, a new paragraph at the end.

An on-premise Device Detection Engine whose native code does this
conversion itself does not need this Element. See
Engines that convert the values.

Element Data, the text on rc.

This Element is only implemented for
.NET
at time of writing.
The reference implementation writes the decoded values to Evidence, which
a Flow Element MAY do outside a
parallel group,
as Evidence is not immutable.
Alternatively, the Element can output its values using Element Data as
described in the table below.
The Device Detection Engines MUST be capable of using these values from
Element Data if available, and falling back to Evidence values if they are
not.

New text. It keeps #116's rule that a Flow Element MAY write to Evidence outside a parallel group, which agrees with evidence.md line 60, and #119's "from Element Data", and adds the exception.

The .NET and Java packages have no implementation of this Element, and the
native code their on-premise Engines use converts the values itself.
An implementation of this Element can write the decoded values to Evidence,
which a Flow Element MAY do outside a
parallel group,
as Evidence is not immutable.
Alternatively, the Element can output its values using Element Data as
described in the table below.
A Device Detection Engine MUST be capable of using these values from
Element Data if available, and falling back to Evidence values if they are
not, unless it is an on-premise Engine whose native code converts the
values itself.

A new section, "Engines that convert the values", between Process and Configuration options. It says the native code converts both keys before it matches while special evidence processing is on, which is the default, that an Engine passing both keys to that code unchanged need not read this Element's data, and that a Pipeline built around such an Engine does not need this Element. Its last paragraph agrees with "Accepted Evidence" (lines 25 to 29 on rc).

The native code converts only one such value, as
Accepted Evidence says. It tries every query value
before any cookie value, and uses the first 51d_gethighentropyvalues or
51d_structureduseragent value that gives at least one header
(hash.c lines 3949 to 3959
and lines 4287 to 4305).
So where that first value is a 51d_structureduseragent value, the high
entropy values are not converted, and the header values come from the
structured User-Agent alone, which gives no sec-ch-ua value
(transformc.c lines 1105 to 1107).

The cloud Engines are not covered by the new wording, so the MUST still applies to them as before.

Other drafts

No other open pull request in this repository changes uach-high-entropy-decoder.md, apart from #134, which brings rc into main. #116 and #119 changed it and have merged into rc.

Verification

• The page's two 51degrees.com links are unchanged, so their campaign tags are as before.
• No conflict markers remain, and the file keeps its LF line endings in the repository.
• The repository's checks on the new head are listed on this pull request.

Outstanding

• Java, Node, PHP and Go take query before cookie only once their cxx submodule moves to cfad897 or later.

No version directive is carried.

[jwrosewell]

Added a paragraph to "Engines that convert the values". The native code converts only the first 51d_gethighentropyvalues or 51d_structureduseragent value from the query or a cookie that gives at least one header (hash.c lines 3587 to 3596), so a structured User-Agent value read first means the high entropy values are not converted, and no sec-ch-ua value comes from it (transformc.c lines 1105 to 1107). The paragraph says "reads first" rather than naming an order, because a draft change to the native code makes it read query values before cookies. This came from the second look at 51Degrees/51degrees-rust#350.