Fact sheet D¶
Source: Kolla-Ansible tag 2024.1-eom (commit 890396f), kolla tag 2024.1-eom (commit ed7a36a),
ansible-collection-kolla 2024.1-eom (commit 82a0f5c). All paths below are relative to the repo root.
Labels: FACT (the default, not written out) = read directly from the cited source: kolla-ansible, the kolla image
repository (cited as kolla docker/...), the collection (cited as ansible-collection-kolla roles/...) or Ansible
(cited as ansible-core 2.16.20 lib/ansible/...). NOT AVAILABLE = these sources do not establish it, often because
it depends on upstream OpenStack, Swift, Ceph, libvirt, iSCSI, Redis, Pacemaker or container-engine behaviour that is
not in them.
Out of scope here, because other fact sheets cover them: site.yml mechanics, nova/nova-cell/neutron/openvswitch/common,
kolla_container semantics, service-ks-register, service-rabbitmq, service-images-pull, service-stop. These are cited
only as cross-references.
Common facts used throughout:
- Every role below that still exists in 2024.1 has
tasks/main.yml=include_tasks: "{{ kolla_action }}.yml"(for exampleansible/roles/barbican/tasks/main.yml:2). No role has a meta dependency oncommon: no role inansible/roles/has ameta/directory, andcommonis its own play taggedcommon(ansible/site.yml:88-103). The roleschrony,skydive,nova-hypervand, new in 2024.1,monascado not exist:ansible/roles/monascaandansible/monasca_cleanup.ymlare gone, and no file ofansible/mentions monasca (sections 5.10 to 5.13). - The container module is now
kolla_container(ansible/library/kolla_container.py;kolla_dockeris gone, see Repository architecture §6). Bootstrap containers are started withkolla_container action=start_container detach=False restart_policy=oneshot(for exampleansible/roles/barbican/tasks/bootstrap_service.yml:6-17); aoneshotcontainer is started directly, without a systemd unit (ansible/module_utils/kolla_docker_worker.py:379-389).remove_on_exitdefaults toTrue(ansible/library/kolla_container.py:314;ansible/module_utils/kolla_docker_worker.py:391-410), so a bootstrap container (DB sync) is created afresh and re-runs on every deploy/reconfigure for any role whosedeploy.ymlimportsbootstrap.yml. kolla-ansible upgradeandreconfigurepass-e kolla_serial=${ANSIBLE_SERIAL}, which defaults to 0 (tools/kolla-ansible:296,522,531). In every play,run_oncetasks (keystone registration, DB bootstrap) run once per serial batch: each batch is a separate run of the play, and arun_oncetask is queued for the first host left only (ansible-core 2.16.20lib/ansible/executor/playbook_executor.py:181-190,lib/ansible/plugins/strategy/linear.py:181,207-209).- Every play below now also sets
max_fail_percentage(default 100, for exampleansible/site.yml:861-864for barbican); with 100 it never stops a play early (see Repository architecture). - The transport URLs are global host lists:
ansible/group_vars/all.yml:248rpc_transport_url: "{{ om_rpc_transport }}://{% for host in groups[om_rpc_group] %}...{% endfor %}/{{ om_rpc_vhost }}"withom_rpc_group: "rabbitmq"(:245). Similarlynotify_transport_urlat:258withom_notify_group: "rabbitmq"(:255). - Changed in 2024.1, common to most roles below: RabbitMQ quorum queues are on by default, so
rabbit_quorum_queue = trueis now rendered in the[oslo_messaging_rabbit]section of the barbican, octavia, manila, ceilometer, zun, cyborg and masakari templates (for exampleansible/roles/barbican/templates/barbican.conf.j2:98-107): a cloud that kept the 2023.1 default gets a changed config, and a restart, on its first 2024.1 run (F17). The systemd unit template also changed, so the first 2024.1 check of a container whose unit 2023.1 wrote reports a difference (F16).
1. Barbican¶
1.1 Plays in site.yml¶
| Play | Lines | Hosts | Tags |
|---|---|---|---|
| loadbalancer include | ansible/ |
loadbalancer (the "Apply role loadbalancer" play, :105-114) |
barbican (include_role tasks_from: loadbalancer, when: enable_barbican). The whole block applies only when enable_haproxy and kolla_action in ['deploy','reconfigure','upgrade','config'] (:324-326) |
| Apply role barbican | ansible/ |
barbican-api, barbican-keystone-listener, barbican-worker, &enable_barbican_True |
barbican; serial: kolla_serial (:860) |
Default inventory: all three groups are children of barbican, which has control as its child
(ansible/inventory/multinode:125-126,340-347).
1.2 Services (ansible/roles/barbican/defaults/main.yml:2-42)¶
| Service | Container | Group | Enabled | Notable |
|---|---|---|---|---|
| barbican-api | barbican_api |
barbican-api | true |
haproxy barbican_api (internal) and barbican_api_external, both with tls_backend: barbican_enable_tls_backend; the external entry now has external_fqdn and barbican_api_public_port (:11-26). Named volume barbican:/ (:136) |
| barbican-keystone-listener | barbican_keystone_listener |
barbican-keystone-listener | true |
no haproxy |
| barbican-worker | barbican_worker |
barbican-worker | true |
no haproxy |
New in 2024.1: each service mounts /dev/shm when om_enable_queue_manager is set (defaults/main.yml:138,145,152;
off by default).
Keystone registration: service type key-manager (:186-192), user barbican with admin on service (:194-198),
and roles key-manager:service-admin, creator, observer, audit (:166-169,200-204, tasks/register.yml:2-8).
1.3 Task flow per kolla_action¶
- deploy (
tasks/deploy.yml:1-14): register.yml (keystone) → config.yml → check-containers.yml (compare_container, notify restart) → clone.yml (dev mode only) → bootstrap.yml → flush handlers. - reconfigure = deploy (
tasks/reconfigure.yml:2). So keystone registration and the DB bootstrap container re-run on every reconfigure. Still holds in 2024.1. - upgrade (
tasks/upgrade.yml:1-9): config → check-containers → bootstrap_service (DB sync only, no DB/user creation) → flush. No keystone re-registration. - stop / pull / precheck / deploy-containers / loadbalancer / config_validate use the standard helpers
(
kolla-ansible validate-configsetskolla_action=config_validate,tools/kolla-ansible:561-564). There is still nocheck.ymlinansible/roles/barbican/tasks/.
1.4 Config files, overrides, handlers (tasks/config.yml)¶
| Output on host | Source and override chain | Notifies |
|---|---|---|
/etc/ |
templates/ (:54-65) |
Restart <svc> |
/etc/ (uwsgi) |
merge: role template → {{node_custom_config}}/ → .../ (:67-82) |
Restart barbican-api |
/etc/ |
only if {{node_custom_config}}/ exists (:84-109) |
Restart barbican-api |
/etc/ |
merge: template → global.conf → barbican.conf → barbican/ → barbican/ (:111-129) |
Restart <svc> |
| policy file | first found in {{node_custom_config}}/ (:31-48,131-143) |
Restart <svc> |
Handlers (handlers/main.yml:2-48): recreate_or_restart_container for each of the 3 services, skipped when
kolla_action == "config".
1.5 Dependencies (from templates)¶
- MariaDB: changed in 2024.1, the connection moved from
[DEFAULT] sql_connectionto[database] connection, withconnection_recycle_time,max_pool_sizeandmax_retries = -1(templates/barbican.conf.j2:18-22). The DB is created viakolla_toolbox mysql_db/mysql_user, delegated togroups['barbican-api'][0](tasks/bootstrap.yml:2-34). The login user isbarbican_database_shard_root_user(defaults/main.yml:61), which isroot_shard_<id>whenenable_proxysqlis set. - RabbitMQ:
transport_url = {{ rpc_transport_url }}(barbican.conf.j2:16) and the notifications transport (:77-84). New in 2024.1: the notifications driver ismessagingv2with topicnotificationswhenenable_ceilometer, otherwisenoop(barbican.conf.j2:79-84;defaults/main.yml:209-213). Barbican does not use theservice-rabbitmqrole. - Memcached:
memcached_serversloops overgroups['memcached'](barbican.conf.j2:75). - Keystone: authtoken (
:60-75). Keystone notifications are coupled:[keystone_notifications] enable = True, andtopic = barbican_notificationswhenenable_keystone(barbican.conf.j2:54-58). The keystone role adds thebarbican_notificationstopic whenenable_barbicanis set (ansible/roles/keystone/defaults/main.yml:188-194). - Consumers: cinder (
ansible/roles/cinder/templates/cinder.conf.j2:264-269), glance (glance-api.conf.j2:163-168), nova and nova-cell (ansible/roles/nova-cell/templates/nova.conf.j2:259-264, so computenova.confchanges when barbican is enabled), octavia (octavia.conf.j2:21-25), magnum, tacker and skyline (templates that testenable_barbican). Changed in 2024.1: horizon and murano no longer test it (nobarbicaninansible/roles/horizon/; no murano role).
1.6 Operationally notable¶
- The crypto KEK lives in passwords.yml.
barbican_crypto_plugin: "simple_crypto"(ansible/group_vars/all.yml:1022), andkek = '{{ barbican_crypto_key }}'(barbican.conf.j2:47-51;etc/kolla/passwords.yml:56).kolla-genpwdgenerates it as a Fernet key (kolla_ansible/cmd/genpwd.py:100-101,158) and only fills empty values (:91). NOT AVAILABLE: whether losing or changingbarbican_crypto_keymakes stored secrets undecryptable (simple_crypto semantics are Barbican code, which is not in the sources). This key must be in the passwords.yml backup. - p11_crypto: the library path
/usr/lib/libCryptoki2_64.so(all.yml:1023) is used only in the conf template (barbican.conf.j2:37) and is not mounted into the containers (defaults/main.yml:132-153). No task generates the MKEK or HMAC keys: a grep formkek|gen_hmac|barbican-manageinansible/finds only the conf template (barbican.conf.j2:41,43). NOT AVAILABLE: what else an HSM setup needs (for example a custom image or extra volumes, plus manual key generation). - Running
kolla-ansible deploy -t barbicanalone does not run the keystone play, so thebarbican_notificationstopic is not enabled in keystone.conf. NOT AVAILABLE: whether barbican-keystone-listener then receives project-deletion events. - Copying the policy file uses
become: true(tasks/config.yml:136). That is fine.
2. Octavia¶
2.1 Plays¶
| Play | Lines | Hosts | Tags |
|---|---|---|---|
| loadbalancer include | ansible/ |
loadbalancer | octavia |
| Apply role octavia | ansible/ |
octavia-api, octavia-health-manager, octavia-housekeeping, octavia-worker, &enable_octavia_True |
octavia; serial kolla_serial (:989) |
Default inventory: all five groups (octavia-api, octavia-driver-agent, octavia-health-manager,
octavia-housekeeping, octavia-worker) are children of octavia, which is control
(ansible/inventory/multinode:165-166,467-480). The play still does not list octavia-driver-agent; with the default
inventory its hosts are the same control hosts.
2.2 Services (ansible/roles/octavia/defaults/main.yml:2-57)¶
octavia_api (haproxy octavia_api/octavia_api_external on port 9876, with listen_port and
tls_backend: octavia_enable_tls_backend; the external entry now has external_fqdn and octavia_api_public_port,
:11-26), octavia_driver_agent (enabled: enable_octavia_driver_agent, :27-33; default: only with
neutron_plugin_agent: ovn, ansible/group_vars/all.yml:831), octavia_health_manager, octavia_housekeeping,
octavia_worker. The other four have enabled: true, and none has privileged or special flags.
Keystone: service type load-balancer (:252-258). User octavia is given admin on service and on
{{ octavia_service_auth_project }}, default service (:241,260-271). Five load-balancer_* roles are created
(:59-64, tasks/register.yml:9-21).
2.3 Task flow¶
- deploy (
tasks/deploy.yml:1-19): register → prepare.yml whenoctavia_auto_configure(see 2.5) → config-host.yml (the health-manager interface whenoctavia_network_type: tenant,tasks/config-host.yml:2-6) → config → check-containers → clone (dev mode only) → bootstrap → flush. Changed in 2024.1: bootstrap now creates two databases and two users,octaviaand the newoctavia_persistence, before thebootstrap_octaviacontainer (tasks/bootstrap.yml:2-74;defaults/main.yml:80). - reconfigure = deploy (
tasks/reconfigure.yml:2), so the DB sync, keystone registration and, withoctavia_auto_configure, the resource creation re-run. - upgrade (
tasks/upgrade.yml:1-46): config → check-containers → new in 2024.1: create theoctavia_persistencedatabase and its user (:6-41, gated only bynot use_preconfigured_databases, not by the jobboard flag) → bootstrap_service → flush. Upgrade still does not run register.yml, prepare.yml or config-host.yml, and there is still no task that removes a role from the octavia user.- With
octavia_auto_configureon,upgrade(like every action that runs config.yml,genconfigincluded) still runsget_resources_info.ymlfirst (tasks/config.yml:2-3), which looks up the flavor, project, security groups and network by name; the repo doc says genconfig and check mode need access to the Nova and Neutron APIs for this (doc/source/reference/networking/octavia.rst:136-138).ansible/group_vars/all.yml:1321-1322still says to setoctavia_auto_configure: noand keep the previous Octavia configuration when upgrading from Ussuri. NOT AVAILABLE: what that lookup does when the resources were created manually under other names (theos_*_infomodules are not in these sources).
- With
check.ymlis empty (tasks/check.yml:1).- New in 2024.1:
kolla-ansible octavia-certificates --check-expiry <days>only checks the certificates and generates nothing (tools/kolla-ansible:550-556;ansible/roles/octavia-certificates/tasks/main.yml:10-12,51).
2.4 Config files and handlers (tasks/config.yml)¶
| Output | Source and overrides | Notifies |
|---|---|---|
| (lookup) | get_resources_info.yml when octavia_auto_configure (:2-3) |
– |
| policy file | first-found in {{node_custom_config}}/ (:18-49) |
Restart <svc> |
| config.json | templates/ (:55-66) |
Restart <svc> |
octavia-wsgi.conf (octavia-api) |
first-found: octavia/ → octavia/ → role template (:68-84) |
Restart octavia-api |
octavia.conf (all services) |
merge: template → global.conf → octavia.conf → octavia/ → octavia/ (:86-104) |
Restart <svc> |
| SSH private key for the amphora keypair | role template octavia-ssh-key.j2 into octavia-worker, only when octavia_auto_configure (:108-118) |
Restart octavia-worker |
4 PEM files: client.cert-and-key.pem, client_ca.cert.pem, server_ca.cert.pem, server_ca.key.pem |
plain copy from {{node_custom_config}}/ into worker (:120-133), housekeeping (:135-148) and health-manager (:150-163); the block runs only when 'amphora' in octavia_provider_drivers (:165-171) |
Restart that service |
Handlers (ansible/roles/octavia/handlers/main.yml:2-79) use recreate_or_restart_container for each of the five
services. Volumes are passed with reject('equalto','') (for example :12); the module would skip empty volume strings
anyway (ansible/module_utils/kolla_container_worker.py:412-414).
2.5 OpenStack resources: Kolla creates none¶
Still not true by default in 2024.1. With octavia_auto_configure (default: true when amphora is in
octavia_provider_drivers, which it is by default, ansible/group_vars/all.yml:1323,
ansible/roles/octavia/defaults/main.yml:386), deploy and reconfigure run prepare.yml, which creates, run_once
and delegated to groups['octavia-api'][0]: the amphora flavor (amphora, 1 vCPU, 1024 MB, 5 GB, not public,
defaults/main.yml:319-324), the nova keypair, the security groups and their rules, the lb-mgmt network and subnet
(octavia_amp_network_cidr: 10.1.0.0/24, :379) and, for IPv6, a router (tasks/prepare.yml:2-161).
kolla-ansible octavia-certificates generates the certificates (tools/kolla-ansible:550-556,
ansible/octavia-certificates.yml). Kolla still does not create the amphora image. The repo's own doc describes
automatic registration as the default ("Option 1: Automatic resource registration (default, recommended)",
doc/source/reference/networking/octavia.rst:123-138).
The template consumes the IDs:
- With
octavia_auto_configure,amp_image_owner_id,amp_boot_network_list,amp_secgroup_listandamp_flavor_idcome from the facts thatget_resources_info.ymlregistered (templates/octavia.conf.j2:104-108). Changed in 2024.1: the fallbacks to the olderopenstack_projects/secgroupresult keys are gone (octavia.conf.j2:105,107;tasks/get_resources_info.yml:72-75). With it off, each is written only if the operator definesoctavia_amp_image_owner_id,octavia_amp_boot_network_list,octavia_amp_secgroup_listoroctavia_amp_flavor_id(octavia.conf.j2:91-103); no group default defines them, and no precheck validates them:tasks/precheck.yml:1-73checks only the API and HM ports, the 4 PEM files (when the amphora provider is enabled), the neutron plugin fortenant, and Redis for the jobboard (below). amp_image_tag = {{ octavia_amp_image_tag }}(defaultamphora) andamp_ssh_key_name = {{ octavia_amp_ssh_key_name }}(defaultoctavia_ssh_key) are variables (octavia.conf.j2:88-89,defaults/main.yml:293-295), butheartbeat_key = insecure(octavia.conf.j2:81) is still a security oddity with no variable to change it except an override file.[certificates] ca_private_key_passphrase = {{ octavia_ca_password }}(:17-20) must match theserver_ca.key.pemin use.- Barbican: if
enable_barbican, the region and endpoint for the cert manager are set (:21-25). NOT AVAILABLE: whether TLS-terminated listeners need barbican. - New in 2024.1, the jobboard:
enable_octavia_jobboarddefaults to true whenever octavia is enabled with the amphora provider (ansible/group_vars/all.yml:832). It renders a[task_flow]section withjobboard_backend_hostslooping overgroups['redis'], the Redis password and sentinel port, andpersistence_connectionto theoctavia_persistencedatabase (octavia.conf.j2:177-189). A new precheck fails unlessenable_redisis set (tasks/precheck.yml:67-73), andenable_redisdefaults to"no"(all.yml:840): with the defaults, enabling octavia with the amphora provider failsprechecksuntilenable_redis: yes(orenable_octavia_jobboard: no) is set.prechecksis the only place that checks it:deploydoes not run the precheck. NOT AVAILABLE: what Octavia does when the jobboard Redis is unreachable. - Upgrades from Ussuri: the code comment says to set
octavia_auto_configure: noand keep the existing configuration (ansible/group_vars/all.yml:1321-1322). The doc still notes that Train and earlier registered resources inadmin(octavia.rst:257-264);octavia_service_auth_projectdefaults toservice(defaults/main.yml:241).
2.6 Health-manager network requirements¶
bind_ip = {{ octavia_network_interface_address }},bind_port = {{ octavia_health_manager_port }}(5555,all.yml:536) (octavia.conf.j2:78-80).controller_ip_port_listloops over allgroups['octavia-health-manager']using'octavia_network' | kolla_address(host)(octavia.conf.j2:82). This needs facts for every HM host.octavia_network_interfacedefaults toapi_interface, or too-hm0whenoctavia_network_type: tenant(all.yml:292;octavia_network_type: "provider"by default,:1331), and the address comes from'octavia_network' | kolla_address(all.yml:317).kolla_addressraisesFilterErrorif the interface or address family is missing on the host (kolla_ansible/kolla_address.py:84-100,132-136). The[health_manager]section is rendered into octavia.conf for all services, so every octavia host (api, worker and housekeeping too) must haveoctavia_network_interfacewith an address, or templating fails.- With the default
providertype, Kolla does not create the lb-mgmt interface, bridge or VLAN on the controllers; the doc says to bridge it to OVS manually (doc/source/reference/networking/octavia.rst:91-109). Withoctavia_network_type: tenant(andoctavia_auto_configure), on each health-manager host Kolla creates a Neutron port, adds it tobr-intthrough theopenvswitch_vswitchdcontainer, and installs and starts anoctavia-interfacesystemd unit (tasks/config-host.yml:2-6,tasks/hm-interface.yml:2-100); a precheck rejectstenantunlessneutron_plugin_agentisopenvswitch(tasks/precheck.yml:56-65). - Precheck oddity: the HM free-port check uses
api_interface_address, notoctavia_network_interface_address(tasks/precheck.yml:29-38). The two differ only whenoctavia_network_interfaceis notapi_interface(for example withtenant). Still present. - Adding or removing a health-manager host changes octavia.conf on all octavia hosts and restarts them
(
octavia.conf.j2:82;tasks/config.yml:86-104). New in 2024.1: the same holds for a host added to or removed fromredis, throughjobboard_backend_hosts(octavia.conf.j2:180). NOT AVAILABLE: whether the controller list is written into an amphora at boot, so that existing amphorae keep the old list until failover. - Security spread:
server_ca.key.pem(the CA private key) is copied to the worker, housekeeping and health-manager config dirs (tasks/config.yml:120-171).
2.7 Other dependencies¶
MariaDB (octavia.conf.j2:41-45, plus the persistence DB at :188), RabbitMQ rpc (:10) and notify (:123-130;
new: driver messagingv2 with topic notifications when enable_ceilometer, else noop,
defaults/main.yml:276-280), memcached ×2 loops (:59,76), keystone (:47-76), glance, neutron and nova via internal
endpoints (:163-176) and, new in 2024.1, Redis with the jobboard (:177-189). With neutron_plugin_agent: ovn,
the OVN NB and SB connections (:30-34). Topology is octavia_loadbalancer_topology: "SINGLE"
(roles/octavia/defaults/main.yml:298).
3. Swift¶
3.1 Plays¶
| Play | Lines | Hosts | Tags |
|---|---|---|---|
| loadbalancer include | ansible/ |
loadbalancer | swift |
| Apply role swift | ansible/ |
swift-account-server, swift-container-server, swift-object-server, swift-proxy-server, &enable_swift_True |
swift; serial kolla_serial (:519) |
Default inventory: proxy is in swift (control); account, container and object servers are in storage
(ansible/inventory/multinode:122-123,327-337).
3.2 Services: "old-style" role¶
swift_services contains only swift-proxy-server, and only for haproxy
(ansible/roles/swift/defaults/main.yml:2-17; the external entry now has external_fqdn). It has no container_name,
image, volumes or check-containers. Containers are started imperatively in tasks/start.yml with
kolla_container action: start_container:
| Container | Group | Notes (tasks/start.yml) |
|---|---|---|
swift_rsyncd |
any of account, container or object | :36-49. Rsync daemon on port swift_rsync_port 10873 (all.yml:628, templates/). It is the only swift image that fixes disk ownership at start: it chowns /srv/node and its top-level mount points to swift:swift (kolla docker/). The other swift images have no extend_start (kolla docker/swift/*/ hold only Dockerfile.j2, plus swift_sudoers in swift-base) |
swift_account_server/ (+ _replication_server if a separate replication net exists) |
swift-account-server | :51-128 |
swift_container_server/ (+ replication server) |
swift-container-server | :130-208 |
swift_object_server/ (+ replication server) |
swift-object-server | :210-303 |
swift_proxy_server |
swift-proxy-server | :305-317, only when run_start_swift_proxy_server |
Disks: find_disks runs via
{{ kolla_container_engine }} exec -t kolla_toolbox ... -m find_disks name=KOLLA_SWIFT_DATA match_mode=strict
(start.yml:2-14; defaults/main.yml:58-60). Each disk is then mounted xfs by UUID at /srv/node/<fs_label> with
noatime,nodiratime,logbufs=8 (start.yml:23-34). The find_disks module is not in this repo (ansible/library/ does
not have it); it ships in the kolla_toolbox image.
Volumes: /srv/node with :shared, /etc/kolla/swift mounted at /var/lib/kolla/swift (ring files), and the named
volume swift_recon_cache (also on the account and container servers, start.yml:63,142). No kolla_logs volume:
all swift logs go over UDP syslog to syslog_server = api_interface_address (defaults/main.yml:70,
templates/object.conf.j2:8-11).
3.3 Ring files¶
- Kolla does not build rings. Precheck fails unless
account|container|object.builderand.ring.gzexist under{{node_custom_config}}/swift/(tasks/precheck.yml:108-131). config.yml:183-197copies them (plusswift_extra_ring_files) to/etc/kolla/swift/on every host in the swift play, withbackup: yes. Timestamped backups accumulate on each host.- Containers read the rings via
config.json, for exampletemplates/swift-object-server.json.j2:4-15(source/var/lib/kolla/swift/*.ring.gz). Under COPY_ALWAYS they are copied into/etc/swiftat container start.
3.4 Task flow¶
- deploy (
tasks/deploy.yml:1-10): register (keystone service, user, and theResellerAdminrole attasks/register.yml:9-20) → config → start.yml → check.yml (empty).- No task in the swift role has
notify:(a grep fornotifyinansible/roles/swift/tasks/finds nothing). The only handler is a no-opdebug(handlers/main.yml:2-4), triggered only by service-cert-copy. start_containerrecreates a container only when container properties differ (image, volumes, ...;module_utils/kolla_docker_worker.py:365-374). Sodeploydoes not apply swift config or ring changes to running containers (verified in code):start_containercompares only container properties and never runskolla_set_configs --check, which onlycompare_containercalls (module_utils/kolla_container_worker.py:20,58-65,module_utils/kolla_docker_worker.py:146-181);kolla_set_configscopies config and rings into the container only at container start (kolladocker/base/start.sh:11,docker/base/set_configs.py:535-553; both files are byte-identical to 2023.1); every swift command reads those copies, not the bind-mounted sources (/etc/swift/*.confor/etc/rsyncd.confintemplates/swift-*.json.j2; no template setsswift_dir); and no task notifies a handler. Still holds in 2024.1. See Kolla images §2.
- No task in the swift role has
-
reconfigure (
tasks/reconfigure.yml:1-105) is a different, legacy flow:- It builds the list of all swift containers, plus replication servers if
swift_has_replication_network(:2-30). - It fails if any listed container is not running (
:32-42), so reconfigure cannot repair a crashed container. - It runs config.yml, then
{{ kolla_container_engine }} exec -u root <c> kolla_set_configs --check(:44-54). - It removes and re-creates containers under COPY_ONCE (
:70-89), or does a plainrestart_containerunder COPY_ALWAYS when the check returns rc==1 (:91-105).restart_containerrestarts the container through its systemd unit (module_utils/kolla_docker_worker.py:439-459). Changed in 2024.1: thesekolla_containercalls now passcommon_options(reconfigure.yml:35,62,73,94), so they followkolla_container_enginelike the other roles.
- Under COPY_ALWAYS (the default,
all.yml:43), all changed containers on all hosts of the batch restart in the same task (:91-105; the linear strategy queues a task for every host of the batch, ansible-core 2.16.20lib/ansible/plugins/strategy/linear.py:50-93). With defaultkolla_serial=0(tools/kolla-ansible:296) the batch is every host of the play (lib/ansible/executor/playbook_executor.py:300-302), so every changed proxy, on every proxy host, restarts in that one task. NOT AVAILABLE: whether this opens an outage window.
- It builds the list of all swift containers, plus replication servers if
-
upgrade (
tasks/upgrade.yml:1-6) usesrolling_upgrade.ymlby default (swift_enable_rolling_upgrade: "yes",defaults/main.yml:73). Despite the name, it stops all storage containers on all storage hosts at once (rolling_upgrade.yml:31-39, with a TODO admitting it at:28-30), restarts them, then stops and starts the proxies (:46-68). The proxy start here omits/etc/timezoneandkolla_logs(:64-67), which is consistent with start.yml. - stop (
tasks/stop.yml) still contains a bug: the account-auditor stop checks'swift_object_auditor' not in skip_stop_containers(:23-31).
3.5 Config files (tasks/config.yml)¶
tasks/config.yml, tasks/precheck.yml and every swift template are unchanged from 2023.1 (same text, same lines).
- Config dirs and config.json for all 18 services are created on every host in the play, with no group filter
(
:2-29,35-59). swift.conf is copied for all 17 services on every host (:61-91).swift.confcontainsswift_hash_path_suffix/prefixfrom passwords.yml (templates/swift.conf.j2:1-3,etc/kolla/passwords.yml:103-104). NOT AVAILABLE: what changing them after data is written does (Swift semantics are not in the sources). - account, container and object
*.confare written for their groups (:93-155). Override chain: template →global.conf→swift/<type>.conf→swift/<svc>.conf→swift/<host>/<svc>.conf. - The proxy chain is template → global.conf →
swift/proxy-server.conf→swift/<host>/proxy-server.conf(:157-171). rsyncd.confis a plain template (not mergeable) bound to'swift_replication' | kolla_address(templates/rsyncd.conf.j2:1).- policy.json is copied without
become(config.yml:206-230).
3.6 Networks and ports¶
- Storage services bind to
swift_storage;replicat*services bind toswift_replication(templates/object.conf.j2:1-3,account.conf.j2:1-3).swift_storage_interfacedefaults tonetwork_interfaceandswift_replication_interfacetoswift_storage_interface(all.yml:288-289). swift_has_replication_networkis true when the interfaces differ (defaults/main.yml:66). Replicators useRSYNC_CONNECT_PROG: "nc %H {{ swift_rsync_port }}"(start.yml:87,105,166etc.).- Precheck bug: the rsync free-port check probes port 873 (
precheck.yml:86-95), but rsyncd listens on 10873. Still present.
3.7 Dependencies¶
- Keystone authtoken (
proxy-server.conf.j2:36-51). - Memcached: two loops over
groups['memcached'](:25,51). - RabbitMQ only when
enable_ceilometer: the ceilometer middleware usesurl = {{ notify_transport_url }}(:58-66). - No MariaDB.
- Cinder backup can point at swift (
ansible/roles/cinder/templates/cinder.conf.j2:43-50). Ceilometer's register.yml grantsResellerAdminwhen swift is enabled (ansible/roles/ceilometer/tasks/register.yml:8-22).
3.8 Oddities¶
enable_swift_reconis tested without| bool:{% if enable_swift_recon %}(account.conf.j2:15,17,container.conf.j2:15,17,object.conf.j2:19,22). The default is the string"no"(all.yml:844), not a boolean. NOT AVAILABLE: whether the template treats that string as true and so always enables the recon middleware (the Jinja test of a string is not in the sources).- The
proxy-loggingandversioned_writesfilters are defined but not in the proxy pipeline (proxy-server.conf.j2:13,33-34,92-94).
4. Manila¶
4.1 Plays¶
| Play | Lines | Hosts | Tags |
|---|---|---|---|
| loadbalancer include | ansible/ |
loadbalancer | manila |
| Apply role neutron | ansible/ |
includes manila-share (:684) and compute (:683) |
neutron |
| Apply role manila | ansible/ |
manila-api, manila-data, manila-share, manila-scheduler, &enable_manila_True |
manila |
Default inventory: manila-share is a child of network (ansible/inventory/multinode:320-321). The other groups
are under manila (control) (:140-141,314-324). openvswitch children include manila-share (:105-108).
Why manila-share is in the neutron play: the agent host_in_groups include manila-share hosts.
- OVS agent, only with the generic backend:
(enable_manila_backend_generic | bool and inventory_hostname in groups['manila-share'])(ansible/roles/neutron/defaults/main.yml:36). openvswitch db and vswitchd have the same condition (ansible/roles/openvswitch/defaults/main.yml:11,27). - Linuxbridge agent: whenever
enable_manilais set, for any backend (neutron/defaults/main.yml:60). This inconsistency is still present. - Generic-driver share servers are plugged with
OVSInterfaceDriver(openvswitch) orBridgeInterfaceDriver(linuxbridge) (templates/manila-share.conf.j2:75-79). - Ordering: the neutron play (
:671) runs before manila (:786).
4.2 Services (ansible/roles/manila/defaults/main.yml:2-50)¶
| Service | Container | Notable |
|---|---|---|
| manila-api | manila_api |
haproxy port 8786 (all.yml:463); new: listen_port on both entries and external_fqdn/manila_api_public_port on the external one |
| manila-scheduler | manila_scheduler |
— |
| manila-share | manila_share |
privileged, /run:/run (:shared only with Docker), /lib/modules:ro, plus the engine's run_default_volumes_<engine> (empty for Docker) (:38-39, :158-166; ansible/). Its handler adds ipc_mode: host (handlers/, ipc_mode at :62), which is not in the services dict and not compared by check-containers (tasks/) |
| manila-data | manila_data |
privileged, /dev/:/dev/, /run as above (:47-48, :181-188) |
Backends: generic, hnas and glusterfsnfs in manila_backends (:221-238) and, changed in 2024.1, the two CephFS
backends moved to a new list manila_ceph_backends, each with a share_name and a cluster (default ceph)
(:240-256); manila_enabled_backends joins both lists (:257). Each is gated by enable_manila_backend_*
(all.yml:799-803, all "no").
4.3 Task flow¶
- deploy (
tasks/deploy.yml:1-14): clone (dev) → register (keystoneshareandsharev2,defaults/main.yml:282-295) → config → check-containers → bootstrap (DB, thenbootstrap_manila,tasks/bootstrap.yml:2-38) → flush. - reconfigure = deploy (
tasks/reconfigure.yml:2). upgrade: config → check-containers → bootstrap_service → flush (tasks/upgrade.yml:1-9). - There is still no
fix_cephfs_owner.yml(no such file underansible/roles/manila/tasks/, and no task of the role mounts CephFS or passessecret=). With a CephFS backend, config includesexternal_ceph.ymlon manila-share hosts (tasks/config.yml:15-18), see 4.4.
4.4 Config (tasks/config.yml)¶
| Output | Chain | Notifies |
|---|---|---|
Changed in 2024.1: a manila-share/ sub-directory (external_ceph.yml:2-14), then per enabled entry of manila_ceph_backends: ceph/ merged from {{node_custom_config}}/ and .../ (:16-29), and the keyring ceph/ templated from {{node_custom_config}}/ (:31-42; ceph_manila_keyring: "client.{{ ceph_manila_user }}.keyring", all.yml:1186); only with enable_manila_backend_cephfs_native or _cephfs_nfs on manila-share hosts (config.yml:15-18) |
For the default cluster ceph the operator file names are still ceph.conf and ceph.client.manila.keyring, but config.json now copies the whole ceph/ directory to /etc/ceph (templates/) and the backend sections read /etc/ (manila-share.conf.j2:108-143) |
Restart manila-share |
| config.json | :43-54 |
Restart <svc> |
| manila.conf (api, data, scheduler) | template → global.conf → manila.conf → manila/ → manila/ (:56-75) |
Restart <svc> |
| manila-share manila.conf | manila.conf.j2 + manila-share.conf.j2 → global.conf → manila.conf → {{node_custom_config}}/ → manila/manila-share.conf → manila/ (:77-99) |
Restart manila-share |
| policy | :101-112, no become |
Restart <svc> |
Because the destination files and config.json change, a CephFS-backed manila-share is expected to see a config
difference, and a restart, on its first 2024.1 run (kolla_set_configs --check compares the config files it would copy,
Kolla images §3).
4.5 Dependencies¶
- MariaDB (
manila.conf.j2:27-28), RabbitMQ rpc (:22) and notify (:50-51), memcached (:48). - manila-share.conf additionally loops over memcached 3× (
manila-share.conf.j2:35,52,70) and holds glance, cinder, nova and neutron service-user passwords (manila-share.conf.j2:8-70). - Ceph is external only: there is no ceph deployment role in 2024.1 (
ansible/roles/has nocephrole; onlyceph-rgw, which has "no container deployment" and only configures the load balancer,ansible/roles/ceph-rgw/defaults/main.yml:3-4). The config is supplied by the operator (external_ceph.yml).
4.6 Operationally notable¶
- Kolla does not create share types, the service image or networks (no such task under
ansible/roles/manila/tasks/).default_share_type = default_share_typeis hard-coded (manila-share.conf.j2:6) andservice_image_name = manila-service-image(:85). - The generic backend's service-instance credentials default to
manila/manila(defaults/main.yml:211-212,manila-share.conf.j2:83-84). - Changed in 2024.1: endpoints no longer use
kolla_internal_fqdnandkolla_external_fqdndirectly butmanila_internal_fqdn/manila_external_fqdnthrough thekolla_urlfilter (defaults/main.yml:199-200); those two default to the global FQDNs (all.yml:461-462).
5. Other roles that deploy containers onto compute hosts¶
Default multinode membership (compute ∈): collectd (multinode:56-57), telegraf (:71-76), hacluster-remote
(:81-82), cyborg and cyborg-agent (:149-151,425-426), iscsid (:305-308), ceilometer-compute and ceilometer-ipmi
(:402-406), multipathd (:452-454), zun-compute and zun-cni-daemon (:515-519), prometheus-node-exporter,
prometheus-cadvisor and prometheus-libvirt-exporter (:543-548,556-561,575-576), masakari-instancemonitor (:590-591),
ovn-controller-compute (:597-598). The kuryr play targets compute directly (site.yml:695-707). There is no
chrony, skydive-agent, hyperv or (new in 2024.1) monasca-agent group in the 2024.1 inventory/multinode.
Podman note (new in 2024.1): the compute-side containers that mount /run (ceilometer_compute, iscsid, tgtd,
multipathd, zun_compute, zun_cni_daemon, kuryr) now mount it :shared only when kolla_container_engine == 'docker'
and add run_default_volumes_<engine>, which is empty for Docker (for example
ansible/roles/iscsi/defaults/main.yml:10,46; ansible/group_vars/all.yml:165-172). With the default Docker engine the
volume list is unchanged.
5.0 Summary table (compute-side)¶
| Role (enable flag, default) | Compute containers | Restart trigger (handler) | Config: per-host or global? | Data-plane / VM impact |
|---|---|---|---|---|
ceilometer (enable_ceilometer no; ipmi enable_ceilometer_ipmi no, all.yml:743-744) |
ceilometer_compute (privileged, /run, nova_libvirt volume), ceilometer_ipmi |
handlers/, :51-65 |
Global: rabbit rpc and notify, memcached | VM impact NOT AVAILABLE (ceilometer_compute runs only ceilometer-polling, templates/) |
iscsi (enable_iscsid derived, all.yml:794) |
iscsid (privileged, ipc host, /dev, /run, /sys/, iscsi_info:/) |
handlers/ |
Per-host (config.json only) | Risk topic, outcome NOT AVAILABLE; see 5.2 |
multipathd (enable_multipathd no) |
multipathd (privileged, ipc host) |
handlers/ |
Per-host (role template or operator override) | Risk topic, outcome NOT AVAILABLE; see 5.3 |
| collectd (no) | collectd (privileged, /sys, /dev ro) |
handlers/ |
Per-host | none |
| telegraf (no) | telegraf (pid host, docker.sock ro) |
handlers/ |
Global: influxdb hosts | none |
| prometheus (no) | prometheus_node_exporter (pid host, /:/), prometheus_cadvisor (/var/run:rw, /var/), prometheus_libvirt_exporter (/run/libvirt:ro; enabled with prometheus when nova uses kvm or qemu, all.yml:1227) |
handlers/, :63-77, :139-153 |
Per-host on compute. The server config lists every compute | none on compute; prometheus_server restarts when computes are added |
| masakari (no) | masakari_instancemonitor (privileged) |
handlers/ |
Per-host | none directly; see 5.6 |
hacluster (enable_hacluster = enable_masakari_hostmonitor = enable_masakari, all.yml:771,807) |
hacluster_pacemaker_remote (ipc host) |
hacluster/ |
Per-host (shared Pacemaker authkey) | NOT AVAILABLE; see 5.15 |
| zun (no) | zun_compute (privileged, /var/lib/docker, /dev, /run), zun_cni_daemon (privileged) |
handlers/, :59-76, plus a loopback copy at :78-88 |
Global: rabbit rpc, memcached ×2, DB creds | container workloads, outcome NOT AVAILABLE; see 5.7 |
| kuryr (no) | kuryr (privileged, NET_ADMIN, /run, /usr/lib/docker) |
handlers/ |
Per-host | docker network ops, outcome NOT AVAILABLE; see 5.8 |
| cyborg (no) | cyborg_agent (privileged) |
handlers/ |
Global: rabbit rpc and notify, memcached, DB creds, placement and nova creds | none |
| skydive | no role in 2024.1 (see 5.10) | – | – | – |
| monasca | no role at all in 2024.1 (see 5.11) | – | – | – |
| chrony | no role in 2024.1 (see 5.12) | – | – | – |
| nova-hyperv | no role in 2024.1 (see 5.13) | – | – | – |
| (cross-ref) ovn-controller | ovn_controller |
ovn-controller role | Global: ovn-remote = SB host list |
out of scope |
5.1 ceilometer¶
- Play:
ansible/site.yml:819-834, hosts central, notification, compute, ipmi. - Services:
ansible/roles/ceilometer/defaults/main.yml:2-35. Compute volumes include/runand{{ ceilometer_libvirt_volume }}:/var/lib/libvirt(:137-146), whereceilometer_libvirt_volumeisnova_libvirtwith the libvirt container and/var/lib/libvirtotherwise (:162). ceilometer-ipmi has no/devmount and is not privileged (:28-35,147-154). NOT AVAILABLE: whether local IPMI (/dev/ipmi0) is reachable from it. - Flow: deploy = register → config → check-containers → clone → bootstrap → flush (
tasks/deploy.yml).bootstrap.ymlstill has aTODO(Jeffrey4l): fix idempotentcomment (tasks/bootstrap.yml:2) and always runs thebootstrap_ceilometercontainer (ceilometer-upgrade, with the database typegnocchiwhen gnocchi is enabled,defaults/main.yml:196), delegated togroups['ceilometer-notification'][0](tasks/bootstrap_service.yml:2-22). - Config:
ceilometer.confmerge chain attasks/config.yml:196-214. The template carriestransport_url = rpc_transport_url(templates/ceilometer.conf.j2:6),notify_transport_url(:36) andmemcache_serversovergroups['memcached'](:69). The same file is rendered for ceilometer-compute on every compute, so a rabbitmq or memcached membership change restartsceilometer_computeon every compute. - Changed in 2024.1:
event_pipeline.yamlis now amerge_yamlof the role template,{{node_custom_config}}/ceilometer/event_pipeline.yamland a per-host file (tasks/config.yml:255-270), instead of "custom file if it exists, else template"; thestaton localhost for it is gone. - Scaling oddities:
- Several
delegate_to: localhoststat/find tasks still lackrun_once: custom meters (:15-31), polling.yaml (:106-110), gnocchi_resources (:133-137), event_definitions (:216-220), pipeline (:272-276). That is roughly 6 local tasks per host (two for the custom meters), multiplied by the compute count. - "Clean default folder for dynamic pollsters definitions" has no
becomeand no host condition (:77-80). It deletes/etc/kolla/ceilometer-central/pollsters.d/on every run, then recreates and recopies it (:82-104). NOT AVAILABLE: whether ceilometer-central then restarts on every deploy/reconfigure whenpollsters.dexists, and whether the delete, having no become, hits permission errors for non-root ansible users (config_owner_user: root,all.yml:20).
- Several
- Precheck: an
assert(was afail) still requires gnocchi orenable_ceilometer_prometheus_pushgatewaywhen ceilometer is enabled (tasks/precheck.yml:8-14).
5.2 iscsi (iscsid, tgtd)¶
- Play:
ansible/site.yml:439-452, hostsiscsid,tgtd,&enable_iscsid_True. enable_iscsidisenable_cinder and enable_cinder_backend_iscsi(ansible/group_vars/all.yml:794), andenable_cinder_backend_iscsidefaults toenable_cinder_backend_lvm(:753-754).- Per-host enablement (
ansible/roles/iscsi/defaults/main.yml:62-73): iscsid runs on compute, cinder-backup or cinder-volume hosts only when cinder and the iscsi backend are enabled, or on ironic-conductor hosts with ironic. tgtd runs only with the LVM backend andcinder_target_helper == tgtadm. Since the play only targets&enable_iscsid_Truehosts, ironic without the cinder iSCSI backend gets no iscsid. - Containers (
:2-20): both privileged,ipc_mode: host, with volumes/dev,/run(:sharedwith Docker),/lib/modules:ro,/sys/kernel/config:/configfs, and, for iscsid only,iscsi_info:/etc/iscsi(:40-58). - Config: config.json only. The iscsid command is still
iscsid -d 8 -f(templates/iscsid.json.j2:2), which is debug level 8, very verbose. So the container restarts only on config.json, image, volume or dimension change, in practice upgrade/pull of a new tag (and once for the changed systemd unit, F16). The flow for deploy, upgrade and reconfigure is config-host (loadsconfigfs,tasks/config-host.yml:1-10) → config → check-containers → flush (tasks/deploy.yml,upgrade.yml= deploy). - Host interaction: bootstrap-servers removes host
open-iscsion Debian/Ubuntu always, andiscsi-initiator-utilson RedHat only whenenable_nova_libvirt_container(ansible-collection-kollaroles/packages/defaults/main.yml:25-34, applied byroles/packages/tasks/main.yml:17-21). NOT AVAILABLE: whether computes then have no iscsid at all whenenable_iscsidis false (no cinder iSCSI backend, for example a third-party iSCSI SAN). - VM impact (NOT AVAILABLE): whether iSCSI sessions survive an iscsid restart, and whether session recovery or
re-login after a path or portal failure, and os-brick attach/detach from nova-compute, fail or stall while iscsid is
down. The
iscsi_infonamed volume holds the initiator IQN (kolladocker/iscsid/extend_start.sh:3-7). NOT AVAILABLE: whether recreating the volume (for example afterdestroy) changes the IQN and breaks array ACLs. - tgtd restart (consequences NOT AVAILABLE): the command is only
tgtd -d 1 -f --iscsi portal=...with emptyconfig_files(templates/tgtd.json.j2:2-3); new in 2024.1, the portal address is the variabletgtd_interface_address, defaultapi_interface_address(defaults/main.yml:22). The image adds no start script of its own. It only insertsinclude /var/lib/cinder/volumes/*into/etc/tgt/targets.conf(kolladocker/tgtd/Dockerfile.j2:19), and nothing runstgt-adminat start. The tgtd container does not mount thecindervolume (ansible/roles/iscsi/defaults/main.yml:50-58), so/var/lib/cinder/volumesinside it is not the cinder-volume data. NOT AVAILABLE: whether tgtd keeps its target table only in memory, so that a restart drops every exported LVM target and its sessions until cinder-volume (for example at its own restart) re-creates them, and whether I/O to attached LVM volumes stalls or fails meanwhile. The precheck fails on RedHat-family hosts when tgtd is enabled (tasks/precheck.yml:29-35). See Kolla images §5. - Precheck oddity, still present: it checks that
iscsi_port3260 is free on the iscsid hosts (tasks/precheck.yml:17-27), but 3260 is the target (tgtd) port. The probed address is nowtgtd_interface_address(:19), which makes the mismatch plainer.
5.3 multipathd¶
- Play:
ansible/site.yml:454-466. Defaultenable_multipathd: "no"(all.yml:809). No role has a meta dependency on common. - Container: privileged, ipc host, with
/dev,/run(:sharedwith Docker),/lib/modules:ro,/sys/kernel/config:/configfs(ansible/roles/multipathd/defaults/main.yml:2-29). It does not mountiscsi_infoorcinder. There are still nodimensions. - Config:
multipath.confis first-found from{{node_custom_config}}/multipath/<host>/multipath.conf, then{{node_custom_config}}/multipath.conf, then the role template (tasks/config.yml:28-44). The role template isuser_friendly_names no,find_multipaths yes,skip_kpartx yes, and an empty blacklist (templates/multipath.conf.j2:1-8). - The
dm-multipathmodule is loaded by config-host (tasks/config-host.yml:1-7). - Restart (handler
:2-16): on config.json or multipath.conf change, or on image/volume change. - VM impact (NOT AVAILABLE): whether dm maps persist across a restart, whether path checking and failback pause
while multipathd is down, and whether a path failure in that window causes I/O errors or queueing depending on
no_path_retry. Restarts run in the same handler step on all notified multipathd hosts of the batch:flush_handlersputs the notified hosts in the handlers state and the linear strategy queues each handler like a task for every such host (ansible-core 2.16.20lib/ansible/plugins/strategy/__init__.py:952-963,lib/ansible/plugins/strategy/linear.py:90-91,129-198); with serial 0 by default the batch is every host (lib/ansible/executor/playbook_executor.py:300-302).
5.4 collectd and telegraf¶
- collectd: play
site.yml:342-354. Privileged,/sysand/devro (ansible/roles/collectd/defaults/main.yml:2-29). collectd.conf is first-found (host → global → role template), with no merge (tasks/config.yml:41-58). The default sends to{{ 'api' | kolla_address }}:{{ collectd_udp_port }}, i.e. its own host (templates/collectd.conf.j2:9-11), where telegraf'ssocket_listenerreceives it if telegraf is enabled (ansible/roles/telegraf/templates/telegraf.conf.j2:29-35). Per-host only. - telegraf: play
site.yml:370-382.pid_mode: host,/procand/sysro,/var/run/docker.sock:ro(ansible/roles/telegraf/defaults/main.yml:2-10,20-27). telegraf.conf is first-found, not merged (tasks/config.yml:32-48). It loops overgroups['influxdb'](telegraf.conf.j2:14-24), so adding or removing an influxdb host restarts telegraf on every host. The haproxy and elasticsearch inputs test theloadbalancerandopensearchgroups (telegraf.conf.j2:50,58); new in 2024.1, the outward-rabbitmq input is gone with outward RabbitMQ.- Latent bug, still present: the "Copying over telegraf plugin files" task tests
item.value.enabledinside awith_fileglobloop of path strings (tasks/config.yml:50-64). The glob is currently empty becausetemplates/config/does not exist, but adding files there would break the task. - check-containers does not pass the
HOST_PROC/HOST_SYSenv, while the handler does (handlers/main.yml:12-14;tasks/check-containers.yml:2-16).
- Latent bug, still present: the "Copying over telegraf plugin files" task tests
5.5 prometheus (node-exporter, cadvisor, libvirt-exporter)¶
- Play:
site.yml:416-437. Hosts includeprometheus-node-exporter,prometheus-cadvisorandprometheus-libvirt-exporter, which contain compute (multinode:543-548,556-561,575-576). - node_exporter:
pid_mode: host,/:/host:ro,rslave, listens onapi_interface_address:9100(ansible/roles/prometheus/defaults/main.yml:25-32,260-265;templates/prometheus-node-exporter.json.j2:2;all.yml:565). - cadvisor:
/var/run:/var/run:rw(this includes docker.sock read-write, which makes the container root-equivalent),/var/lib/docker:ro,/sys:ro, plus/:/rootfs:roand/dev/disk:ro(defaults/main.yml:271-280). The command is/opt/cadvisor --port={{ prometheus_cadvisor_port }}(18080,all.yml:571) with no listen address (templates/prometheus-cadvisor.json.j2:2). NOT AVAILABLE: whether it listens on all interfaces. - libvirt-exporter:
/run/libvirt:/run/libvirt:ro, listening onapi_interface_address:9177(defaults/main.yml:118-124,302-306;templates/prometheus-libvirt-exporter.json.j2:2;all.yml:573,1227). - Compute-side config is config.json only (per-host). The server config
prometheus.yml.j2loops over node-exporter, cadvisor, libvirt-exporter and other groups (templates/prometheus.yml.j2:25-257). Adding a compute changes prometheus.yml and restartsprometheus_server(tasks/config.yml:79-98), and rendering it needs facts for every compute. Changed in 2024.1: the haproxy exporter container is gone; prometheus.yml scrapes theloadbalancerhosts directly (prometheus.yml.j2:63), and the server now has a basic-authweb.yml(tasks/config.yml:100-116). - Scaling: "Find prometheus host config overrides" runs
delegate_to: localhostper host, explicitlyrun_once: false(tasks/config.yml:68-77), even on hosts that are not prometheus servers.
5.6 masakari (monitors)¶
- Play:
site.yml:1030-1045. Hosts are masakari-api, masakari-engine,masakari-hostmonitor(= control) andmasakari-instancemonitor(= compute) (multinode:581-591). masakari-instancemonitorandmasakari-hostmonitorare deployed (ansible/roles/masakari/defaults/main.yml:31-46), both enabled withenable_masakari(all.yml:806-807). The hostmonitor runs on the control hosts withrestrict_to_remotes = Trueandpacemaker_node_type = cluster(templates/masakari-monitors.conf.j2:21-43), and the computes runhacluster_pacemaker_remote(5.15). There is still no processmonitor (noprocessmonitorinansible/). NOT AVAILABLE: whether and how fast the hostmonitor detects a failed compute (masakari-monitors and Pacemaker code are not in the sources).- Config:
masakari-monitors.confmerge chain (tasks/config.yml:88-110). It is per-host and has no rabbit, memcached or DB. Libvirt is reached overconnection_uri = "qemu+tcp://{{ migration_interface_address }}/system"(templates/masakari-monitors.conf.j2:16-19).- Default libvirt auth (outcome NOT AVAILABLE): nova-cell defaults to
libvirt_enable_sasl: true(ansible/roles/nova-cell/defaults/main.yml:569), givingauth_tcp = "sasl"(ansible/roles/nova-cell/templates/libvirtd.conf.j2:9-13).auth.confwith the SASL credentials is copied to masakari-instancemonitor whenlibvirt_enable_sasl(ansible/roles/masakari/tasks/config.yml:141-157) and placed at/var/lib/masakari/.config/libvirt/auth.conf(templates/masakari-instancemonitor.json.j2:9-15). Withlibvirt_tls: true,listen_tcp = 0(libvirtd.conf.j2:1-3; defaultlibvirt_tls: false,nova-cell/defaults/main.yml:552). NOT AVAILABLE: whether masakari-instancemonitor connects to libvirt with these defaults.
- Default libvirt auth (outcome NOT AVAILABLE): nova-cell defaults to
- Bug, still present: "Copying over existing policy file" uses
dest: "{{ node_config_directory }}/{{ item }}/..."withwith_dict(tasks/config.yml:34-47, dest at:37). When a custom policy exists, the dest path renders a dict. - Bootstrap (DB sync) runs on every deploy/reconfigure (
tasks/deploy.yml:11,bootstrap_service.yml).
5.7 zun (zun-compute, zun-cni-daemon)¶
- Play:
site.yml:998-1013. The precheck requires kuryr (ansible/roles/zun/tasks/precheck.yml:52-58). - Containers (
defaults/main.yml:44-61,184-204): zun_compute is privileged with/run(:sharedwith Docker),/usr/lib/docker,/var/lib/docker,/dev,/lib/modules:ro,iscsi_info:/etc/iscsiwhen iscsid is enabled, and/var/lib/zun:sharedwhenzun_configure_for_cinder_ceph. zun_cni_daemon is privileged. - Config:
zun.confis rendered for all zun services including compute (tasks/config.yml:56-74). It containstransport_url = rpc_transport_url(templates/zun.conf.j2:10), the full[database] connectionwith password (:21-22), memcached ×2 whenenable_memcached(:46,69), anddocker api_url = tcp://<api_ip>:2375(:121-124).- The docker daemon on zun-compute hosts is opened on TCP 2375 without TLS or auth by bootstrap-servers when
docker_configure_for_zunis set (ansible/group_vars/all.yml:123-124; ansible-collection-kollaroles/docker/templates/docker_systemd_service.j2:13). - Changed in 2024.1:
docker_zun_configis now{}(all.yml:125), so bootstrap-servers no longer merges an etcdcluster-storehost list into/etc/docker/daemon.jsonon zun-compute hosts (ansible-collection-kollaroles/docker/tasks/config.yml:35-37). - New in 2024.1: the Ceph files for zun-compute are now
<zun_ceph_cluster>.confand<zun_ceph_cluster>.<keyring>(zun_ceph_cluster: "ceph",defaults/main.yml:251;tasks/external_ceph.yml).
- The docker daemon on zun-compute hosts is opened on TCP 2375 without TLS or auth by bootstrap-servers when
- Handlers: check-containers notifies
"<svc> container changed", and handlerslistenon that as well as being named "Restart <svc> container" (tasks/check-containers.yml:17-21;handlers/main.yml:40-76). On a zun-cni-daemon container change, the loopback binary is copied to the host/opt/cni/bin(:78-88). - NOT AVAILABLE: whether restarting zun_compute stops user containers, and whether a zun_cni_daemon restart breaks CNI operations during the window.
5.8 kuryr¶
- Play:
site.yml:695-707. Hosts arecomputedirectly, and the service group is hard-coded tocompute(ansible/roles/kuryr/defaults/main.yml:10-21). - The container is privileged with
cap_add: NET_ADMIN,/run(:sharedwith Docker) and/usr/lib/docker(:16-19,50-59).kuryr.specis placed into/usr/lib/docker/plugins/kuryr/(templates/kuryr.json.j2:10-15). - Config is per-host:
kuryr_uriuses its own api IP (templates/kuryr.conf.j2:2,defaults/main.yml:95), and kuryr.conf has no rabbit, memcached or DB (templates/kuryr.conf.j2:1-29). Changed in 2024.1:capability_scopeis nowlocalinstead ofglobal(kuryr.conf.j2:6), so kuryr.conf changes, and kuryr restarts, on every compute at the first 2024.1 run. NOT AVAILABLE: what the scope change means for existing Docker networks (kuryr-libnetwork and Docker code are not in the sources). - Keystone user registration runs from the compute-host play (
tasks/register.yml,tasks/deploy.yml:2). - NOT AVAILABLE: whether docker network create/attach fails while kuryr is restarting, and whether existing veth wiring persists.
5.9 cyborg (agent)¶
- Play:
site.yml:869-883still usesserial: '{{ serial|default("0") }}'(:876), notkolla_serial.ANSIBLE_SERIALis therefore ignored for cyborg on reconfigure and upgrade. This oddity is still present in 2024.1. cyborg-agentruns on compute and is privileged (ansible/roles/cyborg/defaults/main.yml:11-19).cyborg.confis rendered for all services (tasks/config.yml:65-83). It contains DB connection credentials (templates/cyborg.conf.j2:12-13), placement service credentials (:34-44), nova service credentials (:46-56), rabbit rpc and notify (:6,64) and memcached (:21). Global, and it carries DB, placement and nova credentials on compute.- Bugs and oddities:
cyborg_serviceshas nohaproxykey (defaults/main.yml:2-27), soloadbalancer.ymlconfigures nothing, yet the keystone endpoints are built for port 6666 (all.yml:369) behind an FQDN.- New in 2024.1: the endpoints are
cyborg_internal_fqdn | kolla_url(...)andcyborg_external_fqdn | kolla_url(...)(defaults/main.yml:147-148), but no file of the 2024.1 tree definescyborg_internal_fqdnorcyborg_external_fqdn(a grep of the whole kolla-ansible tree finds only these two uses;group_vars/all.ymldefines<svc>_internal_fqdnfor other services, for examplemanila_internal_fqdnat:461). They are used by keystone registration (tasks/register.yml,defaults/main.yml:169-175). NOT AVAILABLE (not run): whetherdeploythen fails at cyborg registration unless the operator defines both variables; reviewer to confirm. - The keystone service type is
"accelerator"(defaults/main.yml:169-175).
5.10 skydive (agent)¶
Still does not exist in 2024.1. ansible/roles/ has no skydive role, site.yml has no skydive play, and
inventory/multinode has no skydive-agent or skydive-analyzer group (a grep for skydive in ansible/ finds
nothing).
5.11 monasca (agents)¶
No longer anything in 2024.1. The 2023.1 leftovers are gone too: ansible/roles/monasca (with its
tasks/cleanup.yml), ansible/monasca_cleanup.yml and the monasca_cleanup command of tools/kolla-ansible do not
exist, the monasca-agent* groups are gone from inventory/multinode, and a grep for monasca in ansible/ and
tools/kolla-ansible finds nothing. Containers that an older release left behind are therefore not cleaned up by any
2024.1 playbook.
5.12 chrony¶
Still does not exist in 2024.1 (see Chrony): ansible/roles/ has no chrony role,
site.yml has no chrony play, and inventory/multinode has no chrony or chrony-server group. There is no chrony
container or chrony.conf to restart or reconfigure on computes. Time sync is left to the host;
kolla-ansible prechecks checks it (ansible/roles/prechecks/tasks/timesync_checks.yml, see the Chrony page). NOT
AVAILABLE: which time source the hosts use.
5.13 nova-hyperv¶
Still does not exist in 2024.1: ansible/roles/ has no nova-hyperv role, site.yml has no hyperv play, and
inventory/multinode has no hyperv group (a grep for hyperv in ansible/ finds only hypervisor_inspector and a
comment, ansible/roles/ceilometer/templates/ceilometer.conf.j2:9, ansible/roles/nova-cell/tasks/deploy.yml:21).
5.14 Cross-reference: ovn-controller on compute¶
The ovn-controller role's setup-ovs.yml sets external_ids:ovn-remote={{ ovn_sb_connection }} in the local OVSDB
through kolla_toolbox openvswitch_db (ansible/roles/ovn-controller/tasks/setup-ovs.yml:13-40, ovn-remote at
:33), which is a loop over groups['ovn-sb-db'] (all.yml:549). The task runs on every deploy
(tasks/deploy.yml:6). A neutron-ovn-metadata-agent ini on compute also lists the NB and SB DBs
(ansible/roles/neutron/templates/neutron_ovn_metadata_agent.ini.j2:17-18).
5.15 hacluster (pacemaker-remote)¶
- Play:
site.yml:709-722, hostshacluster(= control) andhacluster-remote(= compute,multinode:78-82),&enable_hacluster_True.enable_haclusterdefaults toenable_masakari_hostmonitor, which defaults toenable_masakari(all.yml:771,807): enabling masakari puts ahacluster_pacemaker_remotecontainer on every compute. - Container:
hacluster_pacemaker_remote,ipc_mode: host(ansible/roles/hacluster/defaults/main.yml:26-33). Its config is config.json and the Pacemakerauthkey, which the role generates once on the deploy host withdd if=/dev/urandomwhen it is missing (tasks/bootstrap.yml:36-41) and copies to every remote (tasks/config.yml:81-96); a change of either restarts the container (handlers/main.yml:36-50). - Each compute is added as a Pacemaker
remoteresource named after itsnodename, with its api address, by acibadmin --modifydelegated to the firsthaclusterhost (tasks/bootstrap_service.yml:10-32). - NOT AVAILABLE: what a
hacluster_pacemaker_remoterestart or a compute removal does to the Pacemaker cluster and to masakari host-failure handling (Pacemaker semantics are not in the sources); no task removes theremoteresource of a deleted compute (the onlycibadmininansible/roles/hacluster/is the--modifyattasks/bootstrap_service.yml:16).
6. Per-cluster host lists in templates¶
6.1 grep for host in groups over ansible/roles/*/templates¶
Memcached loops (groups['memcached']) appear in: aodh, barbican, blazar, ceilometer, cinder, cloudkitty,
cyborg, designate, glance, gnocchi, heat (×2), horizon, ironic (×2), keystone (×2), magnum, manila (×4), masakari
(api/engine only), mistral, neutron, nova (×2), octavia (×2), placement (×2), swift (×2), tacker, venus, watcher,
zun (×2) (27 roles in 2024.1; the 2023.1 freezer, murano, sahara, senlin, solum and vitrage roles no longer exist).
Other group loops:
- certificates
tls-backendandcompute; designatedesignate-backend-bind9/designate-worker; opensearchopensearch; haproxy-config backends (groups[host_group]); loadbalancer (keepalived)loadbalancer; new in 2024.1: letsencryptloadbalancer(letsencrypt/templates/crontab.j2:4,7,letsencrypt-lego-run.sh.j2:6,9); keystone fernet-pushkeystone; galeramariadb(per shard group); hacluster corosynchacluster(hacluster/templates/hacluster_corosync.conf.j2:13); octaviaoctavia-health-managerand, new in 2024.1,redis(octavia.conf.j2:82,180); prometheus.yml (13 groups, includingprometheus-node-exporter,prometheus-cadvisor,prometheus-libvirt-exporterand nowloadbalancer); prometheus-alertmanager; rabbitmq; telegrafinfluxdb. - Non-loop group references:
cinder.conf.j2:20andnova/templates/nova.conf.j2:62usegroups['glance-api'] | length(the nova-cellnova.conf.j2hasnum_retries = 3,:109). Cinder coordination isredis_connection_stringor an etcd URL onkolla_internal_fqdn, depending oncinder_coordination_backend(cinder.conf.j2:271-281,group_vars/all.yml:1047). - Host lists defined as variables rather than template loops:
rpc_transport_url/notify_transport_url(all.yml:248,258),nova_cell_rpc/notify_transport_url(ansible/roles/nova-cell/defaults/main.yml:206-207, groups default toom_rpc_groupat:191,200),ovn_nb/sb_connection(all.yml:548-549), andredis_connection_string(all.yml:893). Changed in 2024.1:docker_zun_configno longer holds an etcd list (all.yml:125).
6.2 Config files on COMPUTE hosts that embed per-cluster host lists¶
File on compute (/etc/kolla/...) |
Container restarted by its handler | Lists embedded | Evidence |
|---|---|---|---|
nova-compute/ (and the other nova-cell compute services) |
nova_compute, etc. (cross-ref) | cell rabbit rpc and notify | nova-cell/; nova-cell/ |
neutron-openvswitch-agent/ (also linuxbridge, ovn-metadata, ovn-agent) |
neutron agents (cross-ref) | rabbit rpc and notify, memcached | neutron/; rendered for agent services at neutron/ |
neutron-ovn-metadata-agent/ |
ovn metadata agent (cross-ref) | OVN NB and SB DB hosts | neutron_ovn_metadata_agent.ini.j2:17-18 |
OVS DB external_ids:ovn-remote (not a file) |
none (live setting) | OVN SB hosts | ovn-controller/ |
ceilometer-compute/, ceilometer-ipmi/ |
ceilometer_compute, ceilometer_ipmi | rabbit rpc and notify, memcached | ceilometer.conf.j2:6, |
zun-compute/, zun-cni-daemon/ |
zun_compute, zun_cni_daemon | rabbit rpc, memcached ×2 | zun.conf.j2:10, |
cyborg-agent/ |
cyborg_agent | rabbit rpc and notify, memcached | cyborg.conf.j2:6, |
skydive-agent/ |
– | no skydive in 2024.1 (5.10) | – |
telegraf/ |
telegraf | influxdb hosts | telegraf.conf.j2:14-24 |
/etc/ (zun-compute, bootstrap-servers only) |
– | no longer any host list in 2024.1 (docker_zun_config: {}, all.yml:125) |
ansible-collection-kolla roles/ |
Windows nova.conf and neutron_hyperv_agent.conf (hyperv group) |
– | no nova-hyperv in 2024.1 (5.13) | – |
Compute-side files with no cluster lists (per-host only): iscsid and tgtd config.json, multipath.conf,
collectd.conf, node-exporter, cadvisor and libvirt-exporter config.json, masakari-monitors.conf, the
hacluster-pacemaker-remote config.json and authkey, kuryr.conf, and the common/fluentd outputs (these use
elasticsearch_address and opensearch_address, which default to the internal FQDN, all.yml:885;
common/templates/conf/output/01-es.conf.j2:5, 03-opensearch.conf.j2:5).
Implication (consequence NOT AVAILABLE): adding or removing a host in rabbitmq or memcached (normally
controllers), then running deploy or reconfigure across the fleet:
- re-templates the files above on every compute;
- fires
recreate_or_restart_containerfor ceilometer_compute, ceilometer_ipmi, zun_compute, zun_cni_daemon, cyborg_agent, the neutron agents, nova_compute (cross-ref) and others; - runs those restarts in the same handler step on all notified hosts of the batch (ansible-core 2.16.20
lib/ansible/plugins/strategy/__init__.py:952-963,lib/ansible/plugins/strategy/linear.py:90-91,129-198); with the defaultkolla_serial=0the batch is every host (lib/ansible/executor/playbook_executor.py:300-302). NOT AVAILABLE: whether this causes a fleet-wide reconnect storm on RabbitMQ.
The same fleet-wide re-template happens once on the first 2024.1 run for a cloud that kept the 2023.1 classic-queue
default, because rabbit_quorum_queue = true is new in these files (F17).
Rendering each of these files also requires facts (kolla_address(host)) for every host in rabbitmq and memcached
(kolla_ansible/kolla_address.py:56-100). This matters for --limit runs (cross-ref gather-facts.yml).
Conversely, adding a compute changes non-compute configs: prometheus.yml (prometheus_server restart,
prometheus.yml.j2:37,102,198) and, with masakari, the Pacemaker CIB (a new remote resource, 5.15).
7. Consolidated bugs and oddities¶
Status against 2024.1 in each item.
- Swift
stop.yml:23-31: account-auditor guard checksswift_object_auditor. Still present. - Swift precheck checks rsync port 873, but rsyncd uses 10873 (
swift/tasks/precheck.yml:86-95vsall.yml:628). Still present. - Swift
{% if enable_swift_recon %}without| boolagainst the string"no"(account.conf.j2:15, etc.;all.yml:844). Still present. NOT AVAILABLE: whether recon is then always on. - Swift deploy never restarts on config or ring change (no notify anywhere; no-op handler at
swift/handlers/main.yml:2-4). Reconfigure fails if any swift container is down (reconfigure.yml:32-42). The "rolling" upgrade stops all storage at once (rolling_upgrade.yml:31-39). All still present. - Octavia:
heartbeat_key = insecureis still hard-coded (octavia.conf.j2:81). The amp network, secgroup and flavor are filled in by default from the resources thatoctavia_auto_configurecreates, and stay unset with no precheck only when it is off (octavia.conf.j2:91-108,all.yml:1323). The HM precheck still usesapi_interface_address(precheck.yml:29-38). The CA private key is still copied to 3 services (config.yml:120-171). New in 2024.1: the jobboard is on by default with the amphora provider but needs Redis, which is off by default, soprechecksfails with the defaults (all.yml:832,840;octavia/tasks/precheck.yml:67-73). - Manila: policy copy still has no
become(manila/tasks/config.yml:101-112). Service-instance password still defaults tomanila(defaults/main.yml:212). Neutron host_in_groups for manila-share are still asymmetric (OVS: generic only,neutron/defaults/main.yml:36; LB: any backend,:60). New in 2024.1: the CephFS config moved to aceph/sub-directory with per-cluster files (manila/tasks/external_ceph.yml:2-42), which changes the copied config on the first run. - Masakari: policy-file dest still uses a dict
item(masakari/tasks/config.yml:37). A hostmonitor exists (5.6, 5.15); there is still no processmonitor. instancemonitor uses qemu+tcp and receives the SASLauth.confwhen SASL is on (masakari/tasks/config.yml:141-157;masakari-monitors.conf.j2:18;nova-cell/defaults/main.yml:569). - Cyborg: the play still uses
serialinstead ofkolla_serial(site.yml:876). There is still no haproxy entry (cyborg/defaults/main.yml:2-27). New in 2024.1: the endpoints usecyborg_internal_fqdn/cyborg_external_fqdn, which nothing defines (defaults/main.yml:147-148). DB credentials (and placement and nova credentials) are on compute. - Telegraf:
item.value.enabledin awith_fileglobloop (telegraf/tasks/config.yml:50-64). Still present. - Ceilometer: pollsters dir deleted every run without become (
ceilometer/tasks/config.yml:77-80). Many localhost stats lackrun_once(one fewer than in 2023.1, since event_pipeline is now merged). The bootstrap is explicitly non-idempotent (tasks/bootstrap.yml:2). All still present. - iscsid runs at debug
-d 8(iscsi/templates/iscsid.json.j2:2). The precheck tests port 3260 on iscsid hosts, now ontgtd_interface_address(iscsi/tasks/precheck.yml:17-27). Both still present. - multipathd has no meta and no dimensions (
multipathd/defaults/main.yml:2-10); config overrides are first-found (multipathd/tasks/config.yml:28-44). Still as in 2023.1. - nova-hyperv: does not apply, the role does not exist in 2024.1 (5.13).
- Security spread onto compute: DB credentials (zun
zun.conf.j2:21-22, cyborgcyborg.conf.j2:12-13); unauthenticated docker TCP 2375 for zun (all.yml:124); cadvisor with rw/var/run(docker.sock) (prometheus/defaults/main.yml:277); the shared Pacemakerauthkeyon every compute when masakari is enabled (hacluster/tasks/config.yml:81-96). No skydive or chrony in 2024.1. - Barbican and other bootstraps (octavia, manila, ceilometer, masakari, zun, cyborg) re-run their DB sync container on
every deploy/reconfigure (bootstrap containers are auto-removed,
kolla_container.py:314). Still present. - New in 2024.1: kuryr's
capability_scopechanged fromglobaltolocal(kuryr/templates/kuryr.conf.j2:6), so every kuryr container restarts on the first 2024.1 run; the effect on existing networks is NOT AVAILABLE.
8. NOT AVAILABLE list¶
- NOT AVAILABLE: whether the swift containers serve the new rings once
reconfigurehas restarted them. A changed ring file makes--checkreturn rc 1 (the rings areconfig_filesentries,swift/templates/swift-object-server.json.j2:4-15; kolladocker/base/set_configs.py:165-198,220-242; Kolla images §3.3), andreconfigurethen restarts the container under COPY_ALWAYS (swift/tasks/reconfigure.yml:91-105). Thatdeployleaves the old rings in running containers is settled in code (§3.4). - NOT AVAILABLE: the effective value of
enable_swift_reconin the renderedobject-server.conf. - NOT AVAILABLE: what an iscsid, tgtd or multipathd restart does to attached volumes under I/O (session state, path failover, the guest view); for tgtd, whether the LVM targets come back without a cinder-volume restart.
- NOT AVAILABLE: whether masakari-instancemonitor can connect to libvirt with SASL on (it receives
auth.conf, 5.6). - NOT AVAILABLE: whether the fleet-wide restarts after adding a rabbitmq or memcached host cause a reconnect storm, with
kolla_serial=0versus a setANSIBLE_SERIAL. - NOT AVAILABLE: Octavia amphora behaviour after a change in the health-manager host list.
- NOT AVAILABLE: whether ceilometer-ipmi can reach local IPMI without privileged mode or
/dev. - NOT AVAILABLE: what
octavia_auto_configuredoes during an upgrade from manually registered Ussuri resources with other names (2.3). - NOT AVAILABLE: whether and how fast masakari-hostmonitor detects a failed compute, and what a
hacluster_pacemaker_remoterestart or a removed compute does to the Pacemaker cluster (5.6, 5.15). - NOT AVAILABLE (new in 2024.1): what Octavia does when the jobboard Redis is unreachable or a Redis host is replaced (2.5, 2.6).
- NOT AVAILABLE (new in 2024.1): whether cyborg registration fails because
cyborg_internal_fqdn/cyborg_external_fqdnare undefined (5.9). - NOT AVAILABLE (new in 2024.1): what kuryr's
capability_scope = localmeans for existing Docker networks (5.8).