Skip to content

[Bug]: Cannot issue new certificates for nodes #192

Description

@cyberm8

Is this a critical security issue?

  • This is not a security issue.

Describe the Bug

Hello,

When I run Puppet agent on a freshly installed computer (Puppet has never run before), the following error is displayed.

Certificate revocation is disabled, skipping...
Info: Creating a new RSA SSL key for auc-pc
...
Error: failed to submit the CSR, HTTP response was 500

Meanwhile, server side:

2026-08-06 08:52:34,363 INFO  [p.p.certificate-authority] auc-pc already has a requested certificate; new certificate will overwrite it
2026-08-06 08:52:34,381 ERROR [p.r.core] Internal Server Error for PUT /puppet-ca/v1/certificate_request/auc-pc: java.nio.file.FileSystemException: /etc/puppetlabs/puppetserver/ca/serial8183282223335225560tmp: Operation not permitted
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setOwners(UnixFileAttributeViews.java:291)
        at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setGroup(UnixFileAttributeViews.java:330)
        at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setAttribute(UnixFileAttributeViews.java:218)
        at java.base/sun.nio.fs.AbstractFileSystemProvider.setAttribute(AbstractFileSystemProvider.java:81)
        at java.base/java.nio.file.Files.setAttribute(Files.java:1922)
        at puppetlabs.kitchensink.file$atomic_write.invokeStatic(file.clj:83)
        at puppetlabs.kitchensink.file$atomic_write.invoke(file.clj:47)
        at puppetlabs.kitchensink.file$atomic_write_string.invokeStatic(file.clj:95)
        at puppetlabs.kitchensink.file$atomic_write_string.invoke(file.clj:90)
        at puppetlabs.puppetserver.certificate_authority$fn__37324$next_serial_number_BANG___37329$fn__37333.invoke(certificate_authority.clj:624)
        at puppetlabs.puppetserver.certificate_authority$fn__37324$next_serial_number_BANG___37329.invoke(certificate_authority.clj:616)
        at puppetlabs.puppetserver.certificate_authority$fn__38948$autosign_certificate_request_BANG___38953$fn__38957.invoke(certificate_authority.clj:1685)
        at puppetlabs.puppetserver.certificate_authority$fn__38948$autosign_certificate_request_BANG___38953.invoke(certificate_authority.clj:1667)
        at puppetlabs.puppetserver.certificate_authority$fn__39164$process_csr_submission_BANG___39169$fn__39173.invoke(certificate_authority.clj:1815)
        at puppetlabs.puppetserver.certificate_authority$fn__39164$process_csr_submission_BANG___39169.invoke(certificate_authority.clj:1793)
        at puppetlabs.services.ca.certificate_authority_core$fn__41921$handle_put_certificate_request_BANG___41926$fn__41932.invoke(certificate_authority_core.clj:96)
        at puppetlabs.services.ca.certificate_authority_core$fn__41921$handle_put_certificate_request_BANG___41926.invoke(certificate_authority_core.clj:90)
        at puppetlabs.services.ca.certificate_authority_core$fn__42374$web_routes__42379$fn__42380$fn__42391.invoke(certificate_authority_core.clj:580)
        at bidi.ring$fn__16404.invokeStatic(ring.cljc:25)
        at bidi.ring$fn__16404.invoke(ring.cljc:21)
        at bidi.ring$fn__16389$G__16384__16398.invoke(ring.cljc:16)
        at puppetlabs.comidi$make_handler$fn__18539.invoke(comidi.clj:245)
        at puppetlabs.trapperkeeper.authorization.ring_middleware$fn__25646$wrap_authorization_check__25651$fn__25652$fn__25653.invoke(ring_middleware.clj:293)
        at puppetlabs.ring_middleware.core$fn__23025$wrap_bad_request__23034$fn__23037$fn__23043.invoke(core.clj:188)
        at puppetlabs.puppetserver.ringutils$fn__36748$wrap_with_trapperkeeper_or_client_whitelist_authorization__36753$fn__36754$fn__36758.invoke(ringutils.clj:131)
        at puppetlabs.i18n.core$locale_negotiator$fn__477.invoke(core.clj:366)
        at puppetlabs.ring_middleware.core$fn__23128$wrap_uncaught_errors__23137$fn__23140$fn__23141.invoke(core.clj:236)
        at puppetlabs.puppetserver.ringutils$wrap_with_puppet_version_header$fn__36736.invoke(ringutils.clj:90)
        at puppetlabs.ring_middleware.core$fn__22713$wrap_response_logging__22718$fn__22719$fn__22720.invoke(core.clj:57)
        at puppetlabs.trapperkeeper.services.webserver.jetty_core$ring_servlet$fn__29298.invoke(jetty_core.clj:819)
        at puppetlabs.trapperkeeper.services.webserver.jetty_core.proxy$jakarta.servlet.http.HttpServlet$ff19274a.service(Unknown Source)
        at org.eclipse.jetty.ee10.servlet.ServletHolder.handle(ServletHolder.java:754)
        at org.eclipse.jetty.ee10.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1642)
        at puppetlabs.trapperkeeper.services.webserver.normalized_uri_helpers$fn__28552$normalized_uri_filter__28557$fn$reify__28559.doFilter(normalized_uri_helpers.clj:126)
        at org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205)
        at org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
        at org.eclipse.jetty.ee10.servlet.ServletHandler$MappedServlet.handle(ServletHandler.java:1556)
        at org.eclipse.jetty.ee10.servlet.ServletChannel.dispatch(ServletChannel.java:871)
        at org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:449)
        at org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:469)
        at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1253)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:178)
        at org.eclipse.jetty.server.Handler$Sequence.handle(Handler.java:859)
        at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:571)
        at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:794)
        at com.puppetlabs.trapperkeeper.services.webserver.jetty.utils.MDCHandler.handle(MDCHandler.java:77)
        at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:794)
        at org.eclipse.jetty.server.handler.GracefulHandler.handle(GracefulHandler.java:120)
        at org.eclipse.jetty.server.Server.handle(Server.java:197)
        at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:804)
        at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:420)
        at org.eclipse.jetty.server.internal.HttpConnection$FillableCallback.succeeded(HttpConnection.java:1790)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
        at org.eclipse.jetty.io.ssl.SslConnection$SslEndPoint.onFillable(SslConnection.java:614)
        at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:415)
        at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:156)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
        at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:54)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:492)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.epcRunTask(AdaptiveExecutionStrategy.java:428)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:401)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:255)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:204)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:317)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:1009)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1240)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1194)
        at java.base/java.lang.Thread.run(Thread.java:1583)

The /etc/puppetlabs/puppetserver/ca/ is mounted on host.

-rw-r--r-- 1 64604 64604 9.3K 2026-07-06 11:21 ca_crl.pem
-rw------- 1 64604 64604 9.5K 2026-08-03 11:20 ca_crl.pem1348125108170710848tmp
-rw------- 1 64604 64604 9.4K 2026-08-03 11:20 ca_crl.pem4887723151847971648tmp
-rw------- 1 64604 64604 9.3K 2026-08-03 11:20 ca_crl.pem7216690377475044003tmp
-rw-r----- 1 64604 64604 3.9K 2021-12-10 17:04 ca_crt.pem
-rw-r----- 1 64604 64604 3.2K 2021-12-10 17:04 ca_key.pem
-rw-r----- 1 64604 64604  800 2021-12-10 17:04 ca_pub.pem
-rw-r----- 1 64604 64604 2.0K 2021-12-10 17:04 infra_crl.pem
-rw-r----- 1 64604 64604    1 2021-12-10 17:04 infra_inventory.txt
-rw-r----- 1 64604 64604    1 2021-12-10 17:04 infra_serials
-rw------- 1 64604 64604    0 2026-08-06 10:27 infra_serials2203901840119138422tmp
-rw------- 1 64604 root     0 2026-08-06 10:29 infra_serials3343141991730316101tmp
-rw------- 1 64604 64604    0 2026-08-06 10:28 infra_serials8199743333381294610tmp
-rw-r----- 1 64604 64604  30K 2026-06-30 15:45 inventory.txt
drwxr-x--- 2 64604 64604 4.0K 2026-08-06 10:52 requests/
-rw-r----- 1 64604 64604 3.2K 2021-12-10 17:04 root_key.pem
-rw-r--r-- 1 64604 64604    4 2026-06-30 15:45 serial
-rw------- 1 64604 root     4 2026-08-06 10:48 serial16107528223798870526tmp
-rw------- 1 64604 root     4 2026-08-06 10:50 serial17223632340358243869tmp
-rw------- 1 64604 root     4 2026-08-06 10:48 serial7350919704732672876tmp
-rw------- 1 64604 root     4 2026-08-06 10:46 serial7939265957234723454tmp
-rw------- 1 64604 root     4 2026-08-06 10:52 serial8183282223335225560tmp
drwxr-x--- 2 64604 64604 4.0K 2026-07-06 11:21 signed/

The request file has been created. Permissions looks good to me:

-rw-r--r-- 1 64604 root 1.6K 2026-08-06 10:52 requests/auc-pc.pem
puppet@puppetmaster:/$ id
uid=64604(puppet) gid=0(root) groups=0(root)

I tried to remove the file from the container itself, I get the same result.
I upgraded to 8.15.1, same result.

Looking at the docker logs right after startup:

│2026-08-07 05:51:28,941 ERROR [p.p.certificate-authority] Unable to synchronize crl file /etc/puppetlabs/puppetserver/ca/ca_crl.pem to /etc/puppetlabs/puppet/ssl/crl.pem: /etc/puppetlabs/puppet/ssl/crl.pem12719266150772828544tmp: Operation not permitted               │
puppet@puppetmaster:/$ ls -l /etc/puppetlabs/puppetserver/ca/ca_crl.pem
-rw-r--r-- 1 puppet puppet 9465 Jul  6 09:21 /etc/puppetlabs/puppetserver/ca/ca_crl.pem
puppet@puppetmaster:/$ ls -la /etc/puppetlabs/puppet/ssl/crl.pem12719266150772828544tmp
-rw------- 1 puppet root 9465 Aug  7 05:51 /etc/puppetlabs/puppet/ssl/crl.pem12719266150772828544tmp

Expected Behavior

The computer gets a certificate from Puppet and proceeds to gathering facts and applying configurations.

Steps to Reproduce

Install a brand new computer with openvox agent
Configure Puppet
Run puppet agent -t

Environment

Version 8.14.1 and 8.15.1 (ubuntu based)
Platform: Docker image on Debian host

Additional Context

Existing computers have no issue. Only certificate generate seems broken.

UID has changed according to the migration notes.

I made sure to apply correct permissions but anyway, it seems a script is already taking care of it (/container-entrypoint.d/87-ca-permissions.sh).

Relevant log output

# docker-compose.yml
services:
  puppet:
    restart: always
    container_name: puppetmaster
    hostname: puppetmaster
    build:
      context: puppetserver
      dockerfile: Dockerfile
    ports:
      - 8140:8140
    environment:
      OPENVOXSERVER_HOSTNAME: puppet.lan.XX.fr
      CERTNAME: puppet.lan.XX.fr
      OPENVOX_MASTERPORT: 8140
      AUTOSIGN: "true"
      CA_ENABLED: "true"
      CA_HOSTNAME: puppet
      CA_MASTERPORT: 8140
      ALLOW_DUPLICATES: "true"
      USE_OPENVOXDB: "true"
      OPENVOXDB_SERVER_URLS: https://puppetdb:8081
      OPENVOX_REPORTS: puppetdb
      OPENVOX_STORECONFIGS: "true"
      OPENVOX_STORECONFIGS_BACKEND: puppetdb
      OPENVOXSERVER_ENABLE_ENV_CACHE_DEL_API: "true"
    volumes:
      - /puppet/code/:/git/
      - /puppet/ssl/:/etc/puppetlabs/puppet/ssl/
      - /puppet/ca/:/etc/puppetlabs/puppetserver/ca/
    depends_on:
      - puppetdb



# Dockerfile
FROM voxpupuli/openvoxserver:8.15.1-main

ENV ALLOW_DUPLICATES=false

# Configure r10k
RUN true \
 && mkdir --parents /etc/puppetlabs/r10k \
 && chown puppet /etc/puppetlabs/r10k
COPY --chown=puppet conf/r10k.yaml /etc/puppetlabs/r10k/r10k.yaml

# Enable duplicates certificates
COPY --chown=puppet 81-duplicate-certs /container-entrypoint.d/81-duplicate-certs.sh

# Mute dubious ownership error with r10k
RUN git config --global --add safe.directory /git/.git



# 81-duplicate-certs.sh
#!/bin/sh

puppet config set allow_duplicate_certs "${ALLOW_DUPLICATES}" --section server

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Status
    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions