Compatibility with upcoming ggplot2 version

[teunbrand]

Hi there,

We've been preparing a new major release for ggplot2 and found an issue during a reverse dependency check.
I couldn't dig into the root cause of the issue myself due to bumping into #9, but I'll share what the oracle told me below.

You can test your code with the development version of ggplot2 by installing it as follows:

# install.packages("pak")
pak::pak("tidyverse/ggplot2")

We aim to release the new ggplot2 version in about 2 weeks, and hope you can submit a fix to CRAN around that time or let us know when ggplot2 should change. Hopefully this will inform you in a timely manner.

Best wishes,
Teun

## Newly broken

*   checking examples ... ERROR
    ```
    Running examples in ‘plotdap-Ex.R’ failed
    The error most likely occurred in:
    
    > ### Name: add_ggplot
    > ### Title: Add ggplot2 elements to a plotdap object
    > ### Aliases: add_ggplot
    > 
    > ### ** Examples
    > 
    > 
    ...
    +   crs = "+proj=laea +y_0=0 +lon_0=155 +lat_0=-90 +ellps=WGS84 +no_defs")
    Loading required package: maps
    > p <- add_ggplot(
    +  p,
    +  ggplot2::theme_bw()
    + )
    Error in .Primitive("@")(NULL, properties) : 
      no applicable method for `@` applied to an object of class "NULL"
    Calls: add_ggplot ...  -> validate -> validate_properties -> @ -> do.call
    Execution halted
    ```

*   checking re-building of vignette outputs ... ERROR
    ```
    Error(s) in re-building vignettes:
    --- re-building ‘using_plotdap.Rmd’ using rmarkdown
    
    Quitting from using_plotdap.Rmd:57-61 [world]
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    Error:
    ! no applicable method for `@` applied to an object of class "NULL"
    ---
    Backtrace:
    ...
    
    Error: processing vignette 'using_plotdap.Rmd' failed with diagnostics:
    no applicable method for `@` applied to an object of class "NULL"
    --- failed re-building ‘using_plotdap.Rmd’
    
    SUMMARY: processing the following file failed:
      ‘using_plotdap.Rmd’
    
    Error: Vignette re-building failed.
    Execution halted
    ```

[rmendels]

@teunbrand Thanks for the heads-up. I will work on isolating where the problem occurs, but may need help in figuring out the fix. Also I have never quite figured out submissions to CRAN in cases like this, because if I resubmit before you do and I depend on the new version my submission will fail, and I have no way of testing my fix against the new version on anything other than mu local computer - always try to test with WInbuilder etc. But let me isolate the problem and see if I can fix it and then we can work out submissions.

[teunbrand]

Thanks for the reponse! It depends a bit on the fix really. If the fix is both forward and backward compatible, you can put a new release on CRAN before or shortly after the ggplot2 release. If it only works with the new version, it might be best to wait until ggplot2 releases.
I'm happy to advise if I understand the issue, but it is currently obscured to me because I couldn't run tests or checks.

[rmendels]

@teunbrand I am already close to figuring out the problem I think, and may need help. `plotdap' has the function which the following in it

  for (i in seq_along(dots)) {
    plot$ggplot <- plot$ggplot + dots[[i]]
  }

plot$ggplot for want of a better word contains a ggplot structure, and dots[[i]] contains what is being added to the plot, The test that is failing is:

p <- add_ggplot(
 p,
 ggplot2::theme_bw()
)

When I look at what is contained in dots[] for the old version and the new version the structure is different, so that is probably what is breaking it. Let me see if I can figure out the change (I may put in a version dependent change), or if you have a suggest of a better way to do this, let me know. Also, 'plotdap' was originally written by Carson Sievert, who now works for who?

[teunbrand]

Yeah we updated all S3 classes to S7 classes, so some structures have for sure changed.
I think a problem might also be that in the following code, you'd expect the p$ggplot to be a plot instead of NULL.

library(plotdap)

p <- plotdap(crs = "+proj=laea +y_0=0 +lon_0=155 +lat_0=-90 +ellps=WGS84 +no_defs")
#> Loading required package: maps
p$ggplot
#> NULL

^{Created on 2025-06-26 with reprex v2.1.1}

[rmendels]

@teunbrand I didn't send you the full code, I am in the debugger now and 'plot$ggplot' is a ggplot2 structure, The problem is more fundamental than that, because the function 'add_ggplot()' gets called for all plots to adjust. So if I give just the simplest plotdap call:

plotdap()

in the old version I get:

str(plot)
List of 3
$ ggplot:List of 11
 ..$ data       : list()
 .. ..- attr(*, "class")= chr "waiver"
 ..$ layers     :List of 1
 .. ..$ :Classes 'LayerInstance', 'LayerSf', 'Layer', 'ggproto', 'gg' <ggproto object: Class LayerInstance, LayerSf, Layer, gg>

while in the new version I get:

str(plot)
<ggplotdap> List of 3
 $ ggplot: <ggplot2::ggplot>

so in the old version plot$ggplot does contain the ggplot structure while in the new version plot$ggplot returns NULL. It is early here and I am also old and slow so right off I am not seeing why that should fail, but what I need to do is point to the ggplot2 structure and add the new info. So basically if I can be told how to correctly refer to the ggplot2 object that would get me started, though is quite possible other parts will still be broken. But right now that is the error being seen

[teunbrand]

I think I've an idea what might go wrong. In the line below, you're appending the ggplot class to the list that contains the plot.

plotdap/R/plotdap.R

Line 73 in 4640606

return(structure(p2, class = c("ggplotdap", class(p))))

This makes R think this might be an S7 object (whereas it is S3) and it goes downhill from there. If you'd replace that line with the following, it might work a lot better:

return(structure(p2, class = "ggplotdap"))

[rmendels]

@teunbrand ok i think this is perhaps beyond my pay grade. Take the simplest command:

p <- plotdap()

The key lines here are the following (in plotdap.R):

 if (identical(method, "ggplot2")) {
    p <- ggplot() +
      geom_sf(data = bgMap, fill = mapFill, color = mapColor, ...) +
      theme_bw() + ggtitle(mapTitle)
    # keep track of some "global" properties...
    # this seems to be the only way to do things like train axis ranges
    # without a ggplot_build() hack...
    p2 <- list(
      ggplot = p,
      crs = sf::st_crs(bgMap),
      datum = datum
    )
    return(structure(p2, class = c("ggplotdap", class(p))))
  }

so if I stop after 'p2' is defined and do:

str(p2$crs)
List of 2
 $ input: chr "+proj=longlat +ellps=clrk66 +no_defs +type=crs"
 $ wkt  : chr "GEOGCRS[\"unknown\",\n    DATUM[\"Unknown based on Clarke 1866 ellipsoid\",\n        ELLIPSOID[\"Clarke 1866\","| __truncated__
 - attr(*, "class")= chr "crs"

so that list is well-defined, but if I do

str(p$crs)
 NULL

looking at this even more closely, if I pause and create the structure myself:

junk <- structure(p2, class = c("ggplotdap", class(p)))
str(junk$crs)

I get null, but if instead I do:

junk <- structure(p2, class = "ggplotdap")
str(junk$crs)

that works. If I compare class(p) in the two versions, in the new version I get:

[1] "ggplot2::ggplot" "ggplot" "ggplot2::gg" "S7_object" "gg"

while the old version just returns:

[1] "gg" "ggplot"

somehow adding that class corrupts the structure, and to be honest at this point it is beyond what I know how to fix. All we are trying to do is create an object that has the correct classes so that ggplot2 will recognize it as aa ggplot2 object.

[rmendels]

@teunbrand I believe our message crossed. The question is if I only pass

return(structure(p2, class = "ggplotdap"))

will that have enough information so that ggplot2 will see that as structure as a ggplot2 structure in its internal checks. What if I used all the classes except S7?

[teunbrand]

Does the structure(p2, class = "ggplotdap") strategy not work in other parts of the package?

[teunbrand]

will that have enough information so that ggplot2 will see that as structure as a ggplot2 structure in its internal checks.

Well it will consider the p2$ggplot part a valid ggplot2 object. I'm not sure if the ggplotdap class needs to be a subclass if it behaves as a container class.

[rmendels]

@teunbrand thanks. Since I have the problem isolated, I need to go back to my real work, will test all this out later today. Thank you for your help. Will let you know.

[rmendels]

@teunbrand two things. Mostly I am getting success now, except it appears to break my use of gganimate, I have two computers, both Mac with Mac silicon, the only difference between my two R's is the new ggplot2, my animations work with the old version, not with the new. Is there a new version of gganimate being worked on? I am not certain I would know how to fix this.

Also while I have your attention and as long as I am working on this, I am getting warnings about aes_() and aes_string() being deprecated, and I might as well fix that, though again I am not certain I know how to. My usages are:

        geom_sf(data = sg,
              mapping = ggplot2::aes_string(fill = var_name, colour = var_name), ...),
        scale_fill_gradientn(name = var_name, colors = cols),
        scale_colour_gradientn(colors = cols)

          geom_sf(data = table, mapping = aes_(colour = var),
                  size = size, pch = shape, ...),
          scale_colour_gradientn(name = lazyeval::f_text(var), colours = cols)

Any help would be greatly appreciated

[teunbrand]

Is there a new version of gganimate being worked on

Yeah there is a recent gganimate out that should be compatible with the upcoming ggplot2.

I am getting warnings about aes_() and aes_string()

The aes_string can be translated from the following:

ggplot2::aes_string(fill = var_name, colour = var_name)

to this:

ggplot2::aes(fill = .data[[var_name]], colour = .data[[var_name]])

I think aes_() can be replaced with aes() with the same arguments. The only difference should be if you use formulas like aes_(x = ~foo, y = ~bar) this should then become aes(x = foo, y = bar).

[rmendels]

@teunbrand thanks. I am taking a break from this, but I should expect (hope) by Monday to have a working version that passes all my tests, as well as tests from another package that uses plotdap.

Thanks again for all of your help

[rmendels]

@teunbrand I have gganimate working, and I think I have everything working. I need to do some more extensive testing and see if the changes work across versions, and test another package that uses plotdap, but I believe I basically have it as well as the changes to aes_()

Thanks,

-Roy