TLS on the internal and external endpoints¶
Context¶
Kolla-Ansible terminates TLS in HAProxy, on the VIPs. Three switches in globals.yml decide what is encrypted, and
each has its own certificate files, trust settings and consequences for clients. Operators often enable the external
one at first deployment and leave the internal one for later; enabling it later is possible, but it rewrites every
endpoint in the Keystone catalogue and the configuration of every service.
This page sorts the switches, shows where the files go, and says what an operator does to enable TLS on a platform
that is already running and to renew certificates. It follows the Kolla-Ansible 2026.1 TLS guide and the code of the
loadbalancer role.
Topology¶
flowchart LR
client[API clients] -- "external VIP: kolla_enable_tls_external" --> ha[HAProxy on the loadbalancer hosts]
svc[OpenStack services] -- "internal VIP: kolla_enable_tls_internal" --> ha
ha -- "kolla_enable_tls_backend" --> api[API containers on the controllers]
| Switch | Encrypts | Certificate (default path on the deploy host) |
|---|---|---|
kolla_enable_tls_external |
clients to HAProxy on the external VIP | kolla_external_fqdn_cert: /etc/ (certificate, chain and private key in one file) |
kolla_enable_tls_internal |
services and internal clients to HAProxy on the internal VIP | kolla_internal_fqdn_cert: /etc/ |
kolla_enable_tls_backend |
HAProxy to the API containers, for Barbican, Cinder, Glance, Heat, Horizon, Ironic, Keystone, Neutron, Nova and Placement | per host and per service, see below; default backend-cert.pem and backend-key.pem |
Network design¶
- Separate VIPs. If only one of internal or external TLS is enabled, the guide requires
kolla_internal_vip_addressandkolla_external_vip_addressto differ. - A single VIP (one network for everything): the guide says TLS can then be enabled only through the internal variables.
- Names. Endpoints use
kolla_internal_fqdnandkolla_external_fqdn. They default to the VIP addresses, so without DNS names the certificates must carry the VIP addresses as IP subject alternative names. With names, the certificate must carry those names, and every host and client must resolve them.
Storage design¶
Not affected.
Inventory and configuration layout¶
/etc/kolla/
globals.yml # the switches and trust settings below
certificates/ # kolla_certificates_dir; never in git in clear text
haproxy.pem # external: certificate + chain + key
haproxy-internal.pem # internal: certificate + chain + key
ca/ # CA certificates copied into the containers (kolla_copy_ca_into_containers)
<your-ca>.crt
proxysql-cert.pem, proxysql-key.pem # database TLS through ProxySQL: needed as soon as internal TLS is on
# and ProxySQL is enabled (the default), see below
proxysql-ca.pem # optional: the intermediate that signed proxysql-cert.pem, if any
backend-cert.pem, backend-key.pem # backend TLS, used when nothing more specific exists
<host>/<service>-cert.pem, <host>/<service>-key.pem # most specific backend files win
# globals.yml
kolla_enable_tls_external: "yes"
kolla_enable_tls_internal: "yes"
kolla_enable_tls_backend: "no" # optional, see Key choices
kolla_external_fqdn: "<api.example.org>"
kolla_internal_fqdn: "<api-internal.example.org>"
kolla_copy_ca_into_containers: "yes" # needed for a private CA or self-signed certificates
openstack_cacert: "/etc/ssl/certs/ca-certificates.crt" # Ubuntu or Debian images; /etc/pki/tls/certs/ca-bundle.crt on Rocky
kolla_admin_openrc_cacert: "<path to the CA bundle on the hosts that use admin-openrc.sh>"
# kolla_haproxy_ssl_settings: "modern" # default; "intermediate" for older clients
The backend certificate is looked up in this order: <host>/<service>-cert.pem, <host>-cert.pem,
<service>-cert.pem, then kolla_tls_backend_cert; the key in the same order. A certificate used for the backend must
be valid for the address HAProxy uses to reach each host.
Internal TLS also turns on database TLS. In 2026.1, database_enable_tls_internal follows
kolla_enable_tls_internal whenever enable_proxysql is true, and enable_proxysql defaults to enable_mariadb
(ansible/group_vars/all/database.yml and proxysql.yml). So on a default deployment, enabling internal TLS also:
- makes the
loadbalancerrole copyproxysql-cert.pemandproxysql-key.pemfrom/etc/kolla/certificates/, andproxysql-ca.pemif it exists (tasks/copy-certs.yml). The prechecks test only the HAProxy files, so a missing ProxySQL file passesprechecksand fails thereconfigurein theloadbalancerrole; - needs
proxysql-ca.pemwhen the ProxySQL certificate is signed by an intermediate: a copy of that intermediate (release note of bug 2150449); - appends
?ssl_ca=<openstack_cacert>to every service's database connection: every service then connects to the database over TLS and must trust its CA, so with a private CAopenstack_cacertandkolla_copy_ca_into_containersare required, not optional.
kolla-ansible certificates generates the ProxySQL files too, which is why a lab using it does not show the gap.
Backend TLS has the same link: database_enable_tls_backend follows kolla_enable_tls_backend with ProxySQL.
Handle the private keys like passwords.yml: encrypted in the configuration repository, or kept out of it (see
laying out the configuration).
Key choices and their consequences¶
| Choice | Alternative(s) | Why this one here | Consequence in operation |
|---|---|---|---|
| Certificates from the organisation's CA or a public CA | kolla-ansible certificates (a generated test CA) |
The guide says the generated CA is not suitable for production | Renewal is the operator's job, outside Kolla-Ansible; plan it |
| External TLS from day one | Add it later | Clients see HTTPS from the start; no endpoint change later | One certificate to renew per external name |
| Internal TLS too | Plain HTTP inside | Service passwords and tokens cross the internal network on every call | Every service must trust the CA (kolla_copy_ca_into_containers, openstack_cacert); more files to renew |
| Backend TLS | Stop at HAProxy | Encrypts the last hop, for networks where HAProxy and the API containers do not share a trusted segment | A certificate per host (and possibly per service); more to renew; kolla_verify_tls_backend: false exists but the guide does not recommend it |
Let's Encrypt (enable_letsencrypt) |
Certificates managed by hand | Renewal every 12 hours by a Kolla-Ansible container, deployed to HAProxy over SSH | Needs an ACME server reachable from the controllers; internal certificates need an internal ACME server (letsencrypt_internal_cert_server); HAProxy's socket then runs with admin access, which the guide notes |
kolla_externally_managed_cert: true |
Kolla-Ansible copies the files | Certificates are put on the hosts by another tool | Kolla-Ansible no longer copies them; the other tool must put them on every host, in the place Kolla-Ansible expects |
Enabling TLS on a running platform¶
Before: read-only checks of the certificate files on the deploy host.
openssl x509 -in /etc/kolla/certificates/haproxy-internal.pem -noout -subject -enddate -ext subjectAltName
openssl verify -CAfile /etc/kolla/certificates/ca/<your-ca>.crt /etc/kolla/certificates/haproxy-internal.pem
# with an intermediate CA, pass it too: -untrusted <intermediate>.crt
openssl x509 -in /etc/kolla/certificates/proxysql-cert.pem -noout -subject -enddate # internal TLS with ProxySQL
kolla-ansible prechecks -i <inventory> # checks that the files named by kolla_*_fqdn_cert exist
⚠️ State-changing. With
kolla_enable_tls_internalorkolla_enable_tls_externalswitched on, areconfigureof the whole deployment rewrites HAProxy's configuration, the configuration of every service that calls another one, and the endpoints in the Keystone catalogue (http://becomeshttps://). With internal TLS and ProxySQL (the default), every service's database connection also moves to TLS. Every API container restarts; API calls fail or are refused during the run, and clients that do not trust the CA fail afterwards. The internal VIP serves only HTTPS as soon as theloadbalancerplay has run, before the services are reconfigured. Endpoint URLs stored outside Kolla-Ansible (tenant Heat stacks, external tooling, monitoring) must be updated too. Running instances are not affected. Do it in a maintenance window, after a configuration backup and a MariaDB backup. Undo: set the switches back and runreconfigureagain, which restores thehttp://endpoints the same way.
kolla-ansible reconfigure -i <inventory>
Do not limit this run with --tags: the endpoint URLs and the CA settings concern every service.
After: give clients the CA (OS_CACERT, or the system trust store), and regenerate admin-openrc.sh with
kolla-ansible post-deploy -i <inventory> so that it carries OS_CACERT from kolla_admin_openrc_cacert.
Renewing a certificate¶
Replace the file under /etc/kolla/certificates/ on the deploy host, check it with the same openssl commands, then
apply it to HAProxy:
⚠️ State-changing. Copies the new certificate to the load balancer hosts. Kolla-Ansible compares the configuration files of each running container with the new ones and restarts HAProxy where they differ; the VIP handling in the
loadbalancerhandlers restarts the backups first, then the host that holds the VIP, so API calls are interrupted for a few seconds. Undo: put the previous file back and run the same command.
kolla-ansible reconfigure -i <inventory> --tags loadbalancer
The ProxySQL certificate (proxysql-cert.pem, proxysql-key.pem, proxysql-ca.pem) is copied by the same
loadbalancer role, so the same command renews it, with the same short interruption of database connections.
A renewed backend certificate is copied by each service's role instead: run reconfigure with that service's tag
(--tags keystone, --tags nova, and so on), with the same short interruption for that service.
A new CA (not only a new certificate) is a larger change: the new CA must be trusted everywhere before a certificate
it signed is served. Put the new CA in /etc/kolla/certificates/ca/ next to the old one, reconfigure everything,
then switch the certificate, then remove the old CA.
What went wrong during deployment¶
Nothing observed for this page. Failures the code makes predictable:
- Prechecks fail with
External haproxy certificate file is not foundorInternal haproxy certificate file is not found: the file named bykolla_external_fqdn_certorkolla_internal_fqdn_certis missing on the deploy host. - The deploy times out at
Wait for haproxy to listen on VIP: HAProxy does not start, often because the.pemfile has no private key or a key that does not match; readdocker logs haproxy. - Services fail with
CERTIFICATE_VERIFY_FAILEDafter internal TLS is enabled: the CA is not trusted inside the containers. Checkkolla_copy_ca_into_containersandopenstack_cacert, and that the CA file is under/etc/kolla/certificates/ca/.
Version notes¶
Written against Kolla-Ansible 2026.1. The variable names and default paths are those of the 2026.1 TLS guide and
ansible/group_vars/all/; kolla_haproxy_ssl_settings defaults to modern there, which refuses older TLS clients.
Lessons learned¶
- Decide on internal TLS before the first deployment if you can: enabling it later is a full
reconfigurewith a visible API interruption. - Put the certificate expiry dates where someone looks at them. Kolla-Ansible copies certificates; it does not watch them (except Let's Encrypt, which renews its own).
- Keep the VIP addresses in the certificates even when you use names: some internal clients may still be configured with addresses.
Validity and limits¶
- Written from the Kolla-Ansible 2026.1 TLS and ACME guides and the code of
ansible/roles/loadbalancer(copy-certs.yml,precheck.yml) andansible/module_utils/kolla_docker_worker.py(the configuration comparison that restarts a container whose files changed). Not run on a lab for this page. Statusdraft. - Database TLS is covered only as far as internal and backend TLS switch it on through ProxySQL (above); its certificate rotation was not detailed.
- Not covered: RabbitMQ TLS (
rabbitmq_enable_tls), libvirt TLS, Octavia's own certificates (kolla-ansible octavia-certificates), and Podman. - The CA rotation order above follows from how trust works; it was not run.
Related¶
- Rotating the secrets in passwords.yml
- Laying out the configuration
- Kolla-Ansible 2026.1: TLS, ACME, HAProxy guide
Provenance: written by agent:ozarland-author, from the Kolla-Ansible 2026.1 documentation and code. Reviewed in its pull request (see the page history). Checked by a human operator: not yet.
Worked for you, or not? Tell us in one line: your release, your topology, and what happened.