{"id":12,"date":"2026-06-27T00:00:00","date_gmt":"2026-06-27T00:00:00","guid":{"rendered":"http:\/\/\/var\/www\/html\/?p=12"},"modified":"2026-07-21T01:02:30","modified_gmt":"2026-07-21T01:02:30","slug":"caddy-reverse-proxy-tls-certificate-verification-fails-handshake-errors","status":"publish","type":"post","link":"https:\/\/butitworkedlocal.com\/caddy-reverse-proxy-tls-certificate-verification-fails-handshake-errors\/","title":{"rendered":"Caddy Reverse Proxy: Troubleshooting TLS Certificate Verification Failures & Handshake Errors"},"content":{"rendered":"\n

Resolve Caddy reverse proxy TLS certificate verification and handshake errors when connecting to upstream servers, covering common causes like self-signed certificates and untrusted CAs.<\/strong><\/p>\n\n\n

Caddy is an incredibly powerful and user-friendly web server and reverse proxy, known for its automatic HTTPS capabilities. However, when configuring Caddy to act as a reverse proxy to an upstream backend server that also uses HTTPS (e.g., an internal API, another web server, or a microservice), you might encounter TLS certificate verification failures or handshake errors. These issues prevent Caddy from establishing a secure connection to your backend, resulting in 502 Bad Gateway<\/code> errors for your users or a broken application.<\/p>\n

This guide will walk you through diagnosing and resolving these common TLS issues, focusing on scenarios where Caddy, as a client, cannot trust the backend server's presented certificate.<\/p>\n

Symptom & Error Signature<\/h3>\n

When Caddy fails to verify the TLS certificate of an upstream backend, end-users will typically see a 502 Bad Gateway<\/code> error in their browser. In your Caddy logs, accessible via journalctl -u caddy<\/code> (for systemd installations) or docker logs <caddy-container-name><\/code> (for Docker deployments), you'll find entries similar to these:<\/p>\n

{"level":"error","ts":1678886400.000000,"logger":"http.log.error","msg":"x509: certificate signed by unknown authority","request":{"remote_ip":"192.168.1.1","remote_port":"54321","proto":"HTTP\/2.0","method":"GET","host":"your-domain.com","uri":"\/app\/","headers":{"User-Agent":["Mozilla\/5.0..."],"Accept":["text\/html,..."]}},"duration":0.000000,"status":502,"err_id":"some-uuid","err_trace":"reverseproxy.go:881"}\n{"level":"error","ts":1678886400.000000,"logger":"http.reverseproxy","msg":"handshake error: x509: certificate signed by unknown authority","error":"x509: certificate signed by unknown authority"}\n{"level":"error","ts":1678886400.000000,"logger":"http.reverseproxy","msg":"tls: failed to verify certificate: x509: certificate is not valid yet","error":"tls: failed to verify certificate: x509: certificate is not valid yet"}\n{"level":"error","ts":1678886400.000000,"logger":"http.reverseproxy","msg":"tls: failed to verify certificate: x509: cannot validate certificate for 10.0.0.5 because it doesn't contain any IP SANs","error":"tls: failed to verify certificate: x509: cannot validate certificate for 10.0.0.5 because it doesn't contain any IP SANs"}\n<\/code><\/pre>\n
Common error messages include:<\/p>\n
    \n
  • x509: certificate signed by unknown authority<\/code><\/li>\n
  • tls: failed to verify certificate<\/code><\/li>\n
  • x509: certificate is not valid yet<\/code> or x509: certificate has expired<\/code><\/li>\n
  • x509: cannot validate certificate for <hostname\/IP> because it doesn't contain any IP SANs<\/code> (Subject Alternative Names)<\/li>\n<\/ul>\n
    Root Cause Analysis<\/h3>\n
    The "certificate verification fails" error means Caddy, acting as a TLS client, received a certificate from the backend server but could not trust it based on its configured trust stores and policies. Here are the most common underlying reasons:<\/p>\n
      \n
    1. Untrusted Certificate Authority (CA)<\/strong>: This is the most frequent cause.<\/p>\n
        \n
      • The backend server uses a self-signed certificate<\/strong>.<\/li>\n
      • The backend's certificate is issued by a private or internal CA<\/strong> whose root certificate is not trusted by Caddy (or the underlying operating system).<\/li>\n
      • The certificate is issued by a publicly trusted CA, but its intermediate certificates are not sent<\/strong> by the backend, preventing Caddy from building a complete trust chain.<\/li>\n<\/ul>\n<\/li>\n
      • Expired or Not Yet Valid Certificate<\/strong>: The backend server's certificate is outside its designated validity period. Caddy will correctly reject it.<\/p>\n<\/li>\n
      • Hostname Mismatch<\/strong>: The hostname or IP address Caddy uses to connect to the backend (e.g., https:\/\/backend-service.internal<\/code>) does not match any of the Subject Alternative Names (SANs) or the Common Name (CN) specified in the backend's certificate.<\/p>\n<\/li>\n
      • Time Skew<\/strong>: The system clock on the Caddy server is significantly out of sync with the actual time, leading to incorrect validation of certificate validity periods.<\/p>\n<\/li>\n
      • Backend Misconfiguration<\/strong>: The backend server itself might be configured to present an incorrect certificate, or its TLS setup is otherwise flawed.<\/p>\n<\/li>\n<\/ol>\n
        Step-by-Step Resolution<\/h3>\n
        Follow these steps to diagnose and resolve Caddy's TLS certificate verification issues with your backend.<\/p>\n
        1. Verify Backend Certificate Status<\/h4>\n
        Before modifying Caddy, inspect the backend server's certificate. This will reveal if the certificate itself is the problem.<\/p>\n
        Replace backend.example.com<\/code> with your backend's hostname or IP, and 8443<\/code> with its HTTPS port. If your backend uses a hostname for its certificate but you're connecting via IP, specify the hostname with -servername<\/code>.<\/p>\n<\/blockquote>\n
        # Connect to the backend and inspect its certificate\nopenssl s_client -connect backend.example.com:8443 -showcerts -servername backend.example.com\n<\/code><\/pre>\n
        Carefully examine the output:<\/p>\n
          \n
        • Verify return code: 0 (ok)<\/code><\/strong>: Indicates the certificate is<\/em> trusted by your current system's OpenSSL (which might differ from Caddy's trust store depending on installation). If it's not 0 (ok)<\/code>, you have a clear indication of a trust issue.<\/li>\n
        • subject=<\/code> and X509v3 Subject Alternative Name:<\/code><\/strong>: Check if the hostname\/IP you're connecting to is listed here.<\/li>\n
        • Not Before:<\/code> and Not After:<\/code><\/strong>: Ensure the current date falls within this validity window.<\/li>\n
        • Issuer:<\/code><\/strong>: Note the issuer. If it's your own domain, it's likely self-signed or from an internal CA.<\/li>\n
        • The presented certificate chain (multiple ---BEGIN CERTIFICATE---<\/code> blocks).<\/li>\n<\/ul>\n
          2. Install\/Trust Custom CA or Self-Signed Certificate<\/h4>\n
          If openssl s_client<\/code> shows Verify return code: 21 (unable to verify the first certificate)<\/code> or 18 (self signed certificate)<\/code>, you need to make Caddy trust the backend's certificate or its issuing CA.<\/p>\n
          Method A: System-wide Trust (Recommended for internal CA roots)<\/strong><\/p>\n
          If your backend's certificate is signed by an internal Certificate Authority, add that CA's root certificate to the operating system's trust store. Caddy, when installed via systemd, typically respects this.<\/p>\n
            \n
          1. Obtain the CA Root Certificate<\/strong>:<\/p>\n
              \n
            • If you have the .crt<\/code> file for your internal CA's root certificate (e.g., my-internal-ca-root.crt<\/code>), copy it to the Caddy server.<\/li>\n
            • If your backend is self-signed, you can extract its server certificate<\/em> (the first one in openssl s_client -showcerts<\/code> output) and use it as the "CA" for itself. Save the first ---BEGIN CERTIFICATE---<\/code> to backend-self-signed.crt<\/code>.<\/li>\n<\/ul>\n<\/li>\n
            • Add to System Trust Store<\/strong>:<\/p>\n
              sudo cp \/path\/to\/my-internal-ca-root.crt \/usr\/local\/share\/ca-certificates\/\nsudo update-ca-certificates\n<\/code><\/pre>\n<\/li>\n
            • Restart Caddy<\/strong>:<\/p>\n
              sudo systemctl reload caddy\n# If issues persist, try a full restart:\n# sudo systemctl restart caddy\n<\/code><\/pre>\n<\/li>\n<\/ol>\n
              Method B: Caddy-specific Trust (For specific upstream certificates\/CA)<\/strong><\/p>\n
              Caddy allows you to specify trusted CA certificates directly within your Caddyfile for specific upstream reverse proxies. This is often cleaner for isolated scenarios or when you don't want system-wide changes.<\/p>\n
                \n
              1. Obtain the CA Root Certificate<\/strong>: Same as Method A (or the self-signed server cert). Place it in a location accessible by Caddy (e.g., \/etc\/caddy\/certs\/my-internal-ca-root.crt<\/code>).<\/p>\n<\/li>\n
              2. Modify Caddyfile<\/strong>:<\/p>\n
                your-domain.com {\n    reverse_proxy https:\/\/backend.example.com:8443 {\n        transport http {\n            tls_trusted_ca_certs \/etc\/caddy\/certs\/my-internal-ca-root.crt\n        }\n    }\n}\n<\/code><\/pre>\n<\/li>\n
              3. Apply Caddyfile Changes<\/strong>:<\/p>\n
                sudo caddy fmt --overwrite \/etc\/caddy\/Caddyfile # Format your Caddyfile\nsudo systemctl reload caddy\n<\/code><\/pre>\n<\/li>\n<\/ol>\n
                3. Handle Hostname Mismatch (SAN\/CN Issues)<\/h4>\n
                If the backend's certificate doesn't list the hostname or IP Caddy uses to connect, you have a hostname mismatch.<\/p>\n
                  \n
                1. Option A: Correct Caddy's Backend Address<\/strong>:\nThe simplest fix is to ensure the hostname Caddy uses in reverse_proxy<\/code> matches a SAN or CN in the backend's certificate.<\/p>\n
                  your-domain.com {\n    # If backend.example.com is in the backend's certificate SANs\n    reverse_proxy https:\/\/backend.example.com:8443\n}\n<\/code><\/pre>\n
                  If you're connecting to an IP (e.g., 10.0.0.5<\/code>) but the certificate is for a hostname (e.g., api.internal<\/code>), you must connect via the hostname. If the certificate only<\/em> has IP SANs and you're connecting via hostname, connect via IP.<\/p>\n<\/li>\n
                2. Option B: Specify tls_server_name<\/code><\/strong>:\nIf you must connect to a specific IP or hostname (e.g., 10.0.0.5<\/code>) but the backend's certificate is issued for a different hostname (e.g., api.internal<\/code>), you can tell Caddy which Host<\/code> header to send during the TLS handshake for SNI and certificate validation.<\/p>\n
                  your-domain.com {\n    reverse_proxy https:\/\/10.0.0.5:8443 {\n        transport http {\n            tls_server_name api.internal\n        }\n    }\n}\n<\/code><\/pre>\n
                  tls_server_name<\/code> affects the Host<\/code> header sent during the TLS handshake (SNI) for certificate validation. It does not<\/em> change the Host<\/code> header sent in the HTTP request itself. If your backend relies on the HTTP Host<\/code> header, you might also need header_up Host api.internal<\/code> in the reverse_proxy<\/code> block.<\/p>\n<\/blockquote>\n<\/li>\n<\/ol>\n
                  4. Address Expired or Not Yet Valid Certificates<\/h4>\n
                  If openssl s_client<\/code> showed Not Before<\/code> \/ Not After<\/code> issues, the backend's certificate is invalid by date.<\/p>\n
                    \n
                  1. Action<\/strong>: The only proper solution is to renew the backend server's certificate<\/strong>. This is a configuration task on the backend server itself.<\/p>\n<\/li>\n
                  2. Temporary Workaround (USE WITH EXTREME CAUTION)<\/strong>:\nFor development, testing, or truly isolated internal networks where security is less critical, you can instruct Caddy to skip<\/em> TLS certificate verification.<\/p>\n
                    Do not use tls_insecure_skip_verify<\/code> in production environments<\/strong> unless you fully understand the security implications. It disables crucial security checks, making your connection vulnerable to man-in-the-middle attacks.<\/p>\n<\/blockquote>\n
                    your-domain.com {\n    reverse_proxy https:\/\/backend.example.com:8443 {\n        transport http {\n            tls_insecure_skip_verify\n        }\n    }\n}\n<\/code><\/pre>\n
                    After modifying, sudo systemctl reload caddy<\/code>.<\/p>\n<\/li>\n<\/ol>\n
                    5. Ensure Full Certificate Chain is Sent by Backend<\/h4>\n
                    If openssl s_client<\/code> shows missing intermediate certificates (only the leaf certificate is returned, and Verify return code<\/code> is not 0 (ok)<\/code> but indicates a chain issue), the backend server is misconfigured.<\/p>\n
                      \n
                    1. Action<\/strong>: Configure the backend server to send its full certificate chain, including any intermediate CA certificates, along with its server certificate.
                        \n
                      • For Nginx backend<\/strong>: Ensure your ssl_certificate<\/code> directive points to a file containing both your server certificate and<\/em> all intermediate certificates (usually a fullchain.pem<\/code> file).
                        # Example Nginx backend config\nserver {\n    listen 443 ssl;\n    server_name api.internal;\n\n    ssl_certificate \/etc\/nginx\/ssl\/fullchain.pem; # Contains server cert + intermediates\n    ssl_certificate_key \/etc\/nginx\/ssl\/private.key;\n    # ...\n}\n<\/code><\/pre>\n<\/li>\n
                      • Consult your backend server's documentation for correct certificate chain configuration.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n
                        6. Synchronize System Time<\/h4>\n
                        If time skew is suspected as the cause for certificate is not valid yet<\/code> or has expired<\/code> messages, ensure your Caddy server's time is accurate.<\/p>\n
                          \n
                        1. Install\/Verify NTP\/Chrony<\/strong>: Most modern Linux distributions use systemd-timesyncd<\/code> by default or you can install ntp<\/code> or chrony<\/code>.
                          # For systemd-timesyncd (common on Ubuntu 20.04+)\ntimedatectl status\n\n# For NTP\nsudo apt update && sudo apt install ntp\nsudo systemctl status ntp\nntpq -p # Check NTP peer synchronization\n\n# For Chrony\nsudo apt update && sudo apt install chrony\nsudo systemctl status chrony\nchronyc sources -v # Check Chrony sources\n<\/code><\/pre>\n<\/li>\n
                        2. Restart Caddy after ensuring time synchronization: sudo systemctl restart caddy<\/code>.<\/li>\n<\/ol>\n
                          7. Debugging with Caddy Logs<\/h4>\n
                          If after all these steps, you are still experiencing issues, increase Caddy's log verbosity to DEBUG<\/code> to get more insight into the TLS handshake process.<\/p>\n
                            \n
                          1. Modify Caddyfile Global Configuration<\/strong>:
                            {\n    # Other global options...\n    log {\n        output stderr\n        level DEBUG\n    }\n}\n\nyour-domain.com {\n    reverse_proxy https:\/\/backend.example.com:8443 {\n        # ...\n    }\n}\n<\/code><\/pre>\n<\/li>\n
                          2. Apply Changes and Monitor Logs<\/strong>:
                            sudo caddy fmt --overwrite \/etc\/caddy\/Caddyfile\nsudo systemctl reload caddy\njournalctl -u caddy -f # Monitor logs in real-time\n<\/code><\/pre>\nLook for more detailed TLS-related messages during the handshake attempt.<\/li>\n<\/ol>\n
                            By systematically working through these troubleshooting steps, you should be able to identify and resolve the root cause of your Caddy reverse proxy TLS certificate verification failures.<\/p>\n","protected":false},"excerpt":{"rendered":"
                            Resolve Caddy reverse proxy TLS certificate verification and handshake errors when connecting to upstream servers, covering common causes like self-signed certificates and untrusted CAs.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[36,38,40,39,37,27,34,28],"class_list":["post-12","post","type-post","status-publish","format-standard","hentry","category-web-server","tag-caddy","tag-certificate-verification","tag-devops","tag-handshake-error","tag-reverse-proxy","tag-ssl","tag-sysadmin","tag-tls"],"_links":{"self":[{"href":"https:\/\/butitworkedlocal.com\/wp-json\/wp\/v2\/posts\/12","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/butitworkedlocal.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/butitworkedlocal.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/butitworkedlocal.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/butitworkedlocal.com\/wp-json\/wp\/v2\/comments?post=12"}],"version-history":[{"count":2,"href":"https:\/\/butitworkedlocal.com\/wp-json\/wp\/v2\/posts\/12\/revisions"}],"predecessor-version":[{"id":260,"href":"https:\/\/butitworkedlocal.com\/wp-json\/wp\/v2\/posts\/12\/revisions\/260"}],"wp:attachment":[{"href":"https:\/\/butitworkedlocal.com\/wp-json\/wp\/v2\/media?parent=12"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/butitworkedlocal.com\/wp-json\/wp\/v2\/categories?post=12"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/butitworkedlocal.com\/wp-json\/wp\/v2\/tags?post=12"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}