proxy conversion and assignment operators use unprotected fresh SEXPs

[kevinushey]

The proxy classes share a pattern that passes freshly returned, unprotected SEXPs across allocating calls. One member of the family is reproducible today through the public API on a protect-checking build of R-devel; the rest have the same shape and were genuinely exposed on R versions Rcpp still supports (details below).

Reproducible example

Requires an R built with structure checking, e.g. --enable-strict-barrier (the configuration used for CRAN's ASan checks; tested against an ASan/UBSan build of R-devel r90339). On standard builds the failure is silent and timing-dependent.

writeLines('
#include <Rcpp.h>
using namespace Rcpp;

// [[Rcpp::export]]
void copy_field(Reference a, Reference b) {
    a.field("x") = b.field("y");
}
', "repro.cpp")
Rcpp::sourceCpp("repro.cpp")

## an active field, so b$y computes a fresh value on every read
Foo <- setRefClass("Foo", fields = list(
    x = "ANY",
    y = function(value) {
        if (missing(value)) new.env(parent = emptyenv()) else stop("read-only")
    }
))

a <- Foo$new(x = NULL)
b <- Foo$new(x = NULL)

gctorture(TRUE)
for (i in 1:200)
    copy_field(a, b)
gctorture(FALSE)

Output:

Error in copy_field(a, b) :
  unprotected object (0x6250035f0238) encountered (was ENVSXP)

R's protect checking is reporting that the value read from b$y was garbage-collected and then used.

Mechanism (the reproduced case)

FieldProxy::operator=(const FieldProxy&) (inst/include/Rcpp/api/meat/proxy.h:179) does set(rhs.get()). rhs.get() returns the result of an Rcpp_fast_eval of b$y -- referenced from nowhere the GC can see -- and set() (inst/include/Rcpp/proxy/FieldProxy.h:48-53) then runs Rf_mkString(field_name) before the value is captured by Rf_lang4:

void set(SEXP x ) {
    SEXP dollarGetsSym = Rf_install( "$<-");
    Shield<SEXP> str(Rf_mkString(field_name.c_str()));   // <- allocates; x unprotected
    Shield<SEXP> call(Rf_lang4(dollarGetsSym, parent, str, x));
    parent.set__( Rcpp_fast_eval( call, R_GlobalEnv ) );
}

A GC triggered by that allocation collects x; the subsequent $<- call assigns a freed object into the field.

The wider family

The same shape -- fresh get() result handed to something that allocates before securing it -- appears in:

• All 12 operator T() conversions in inst/include/Rcpp/api/meat/proxy.h (AttributeProxy, NamesProxy, SlotProxy, TagProxy, Binding, DottedPairProxy, FieldProxy and their const variants), plus const_SlotProxy::operator T() (proxy/SlotProxy.h:68) and the attribute proxy in vector/proxy.h:173. These do as<T>(get()); for coercing targets, r_cast reaches Rf_coerceVector, which allocates the destination. Fresh-get() producers include active bindings through Environment::get(), Rf_getAttrib() on compact row names, and field access through Rcpp_fast_eval.
• The proxy-to-proxy assignments Binding::operator=(const Binding&) (proxy/Binding.h:49, i.e. envA["x"] = envB["y"]), SlotProxy::operator=(const SlotProxy&) (proxy/SlotProxy.h:36) and NamesProxy::operator=(const NamesProxy&) (proxy/NamesProxy.h:34), whose set() implementations reach allocating calls (Rf_defineVar, R_do_slot_assign) with the fresh value unprotected.

Why standard builds rarely show this

Three unrelated internals currently mask the windows on recent R:

1. coerceVector() protects its argument internally -- but only since R 4.1.0 (PROTECT(v) before the dispatch switch; absent in 3.5.0 through 4.0.5, verified against the R sources for 3.5.0, 3.6.0, 4.0.0, 4.0.5, 4.1.0). Rcpp supports R >= 3.5.0, so on 3.5.0-4.0.5 the operator T() coercion chains (e.g. double x = env["count"] where the binding is active, or std::vector<double> rn = df.attr("row.names")) are genuinely exposed.
2. Rf_cons/Rf_lang* protect their arguments, which covers several construction sequences.
3. R_UnwindProtect() stores the evaluation result into the unwind continuation token (SETCAR(cont, result), src/main/context.c). Rcpp's token is protected for the duration of the eval, so it is old-generation by the time it is dropped, and it accidentally keeps every Rcpp_fast_eval() result reachable until the next full collection. This is undocumented behavior of R internals, not a guarantee, and it is what makes the windows above hard to hit on standard builds: they require a full GC landing inside a one-or-two-allocation window.

None of these is a documented contract; the documented convention is that the caller protects. The fix is mechanical: Shield<SEXP> the get() result in the conversion operators and in the proxy-to-proxy assignments. A PR will follow.