Known bugs and limitations¶
The implementation limitations and bugs found in Kolla-Ansible 2023.1-eol, by area, with the evidence and the status
of each row. The rows carried over from the Ussuri study keep their area; where an issue no longer holds in 2023.1, the
row says so, with the code that shows it. Where a Status cell adds NOT AVAILABLE, it names what the sources do not
establish about the issue.
Note
Key answers
- Implementation limitations and bugs of
2023.1-eol, by area: CLI,site.yml,kolla_docker, the kolla images and individual roles (table below). - No longer holding in 2023.1: the ignored stop and client timeouts (F3), the
-t commonno-op (F2), the always-changed OVSSet system-id, the neutron-dynamic-routing upgrade gap, the haproxy upgrade bug, the octavia items, the masakari SASL mismatch,docker_custom_optionand theovs-dpdkpullbug.
| Area | Issue | Evidence | Status |
|---|---|---|---|
| CLI | -e kolla_serial=N is overridden by ANSIBLE_SERIAL (default 0) for reconfigure/upgrade; it is not passed for deploy |
tools/ |
CODE |
| CLI | --ask-vault-pass overwrites VERBOSITY with a copy of EXTRA_OPTS |
tools/ |
CODE |
| CLI | --include-dev: declared with an argument but used as a flag; the command line is executed unquoted |
tools/ |
CODE fact sheet E |
| site.yml | cyborg play uses serial instead of kolla_serial |
site.yml:784 |
CODE |
| site.yml | placement loadbalancer include lacks when: enable_placement |
site.yml:243-246 |
CODE |
| kolla_docker | No longer holds in 2023.1 (F3): graceful_timeout and client_timeout from docker_common_options are now used. When the task does not set them, generate_module takes them from common_options (strings of digits become integers); client_timeout is the Docker client timeout, and graceful_timeout (docker_graceful_timeout, 60 s) goes into the container's systemd unit as docker stop -t. The FIXME "restart_policy is not checked in the container" remains, but the policy is written into the unit, whose changes compare_container detects (Repository architecture §4.3) |
library/; group_vars/; module_utils/; module_utils/ |
CODE |
| kolla images | kolla_set_configs --check compares only config_files entries (F14): changes confined to config.json (command, permissions; all of memcached's settings) are never detected, so neither genconfig + deploy-containers nor a later run applies them until the container restarts for another reason; preserve_properties entries (prometheus extras) and directory sources with subdirectories report drift on every run. Changed in 2023.1: a newly added optional file is drift (exit 1, the container restarts) instead of an exit 2 that failed check-containers; and a restart now deletes, or resets to the original, the destinations of entries dropped from config.json (glob sources excepted), which --check still does not report (Kolla images §3, §6 items 1–4) |
kolla docker/; module_utils/ |
CODE |
| common | No longer holds in 2023.1 (F2): common is its own play, tagged common, so -t common runs the role; roles/ is now a single include_tasks: "{{ kolla_action }}.yml" |
site.yml:86-97; roles/; Tags §2.1 |
CODE |
| haproxy | Now the loadbalancer role. Restart instead of reload (still); stale services.d files never removed (still; the only removal is mariadb.cfg when ProxySQL is enabled); keepalived priority from inventory order (still). The Ussuri upgrade "Stopping all slave keepalived" task is gone: upgrade.yml imports deploy.yml |
roles/; roles/; roles/; roles/; roles/; HAProxy and keepalived; fact sheet A §1.8 |
CODE |
| openvswitch | The Ussuri bug no longer holds as written: the command that always reported changed is replaced by a kolla_toolbox call of the openvswitch_db module (external_ids:system-id, other_config:hw-offload), whose changed result kolla_toolbox passes through and which notifies Restart openvswitch-vswitchd container |
roles/; library/; kolla docker/ |
CODE; whether openvswitch_db (collection openvswitch.openvswitch, not in these sources) reports changed on an unchanged host, and so restarts openvswitch_vswitchd on every run, NOT AVAILABLE |
| neutron | Changed in 2023.1: upgrade with the default neutron_enable_rolling_upgrade: "yes" expands/contracts neutron plus every enabled subproject of neutron_subprojects (networking-sfc, neutron-dynamic-routing, neutron-vpnaas, vmware-nsx), the same list the KOLLA_BOOTSTRAP container migrates on deploy, so the Ussuri neutron-dynamic-routing gap is closed. Still: the image's neutron-db-manage calls pass no --config-file; has_offline_migrations runs on the first neutron-server host with ignore_errors, and any non-zero rc (a failed docker exec included) stops neutron_server on every neutron-server host in the play until the handler flush after the contract container. New: the check loops over the enabled subprojects only, never neutron itself, so with no subproject enabled no server is stopped before upgrade --contract |
roles/; roles/; roles/; roles/; kolla docker/ |
CODE; impact on the Neutron API, and whether a neutron contract migration is safe against running servers, NOT AVAILABLE |
| ovs-dpdk | pull bug fixed in 2023.1 (pull imports service-images-pull, which reads ovsdpdk_services through kolla_role_name: "ovsdpdk"); the NIC mapping file is still written once; the 300 s wait handler still has no port |
roles/, roles/, roles/; roles/; roles/; fact sheet A §6 |
CODE |
| rabbitmq | outward RabbitMQ pull looks up outward_rabbitmq_services |
fact sheet B §2.8 (item 6); rabbitmq.yml:60-80 (project_name: outward_rabbitmq, :71), roles/, roles/; ansible-core 2.14.18 lib/ |
CODE: with outward RabbitMQ enabled, that variable is not defined anywhere (the role defines only rabbitmq_services, roles/), the vars lookup raises an undefined-variable error and "Pull images" fails on the outward-rabbitmq hosts |
| rabbitmq | the image writes .erlang.cookie only in the bootstrap container (new volume); a cookie change recreates the brokers (env compare) but they keep the old cookie, while kolla_toolbox gets the new one |
kolla docker/; roles/; roles/; roles/, roles/ |
CODE; consequences, and whether rabbitmq-server uses RABBITMQ_CLUSTER_COOKIE or only the cookie file, NOT AVAILABLE |
| mariadb | the restart batches have no any_errors_fatal; recovery stops healthy nodes; clustercheck started even when disabled. Changed in 2023.1: the restarts run in separate plays with serial: "33%"; deploy and reconfigure notify the clustercheck handler only for enabled items, so only recovery still starts it when disabled |
fact sheet B §1.5, §1.8 (items 5–6); mariadb.yml:14-40; roles/; roles/; roles/, check-containers.yml:13-18; roles/; ansible-core 2.14.18 lib/, lib/ |
CODE: a node that fails its port or WSREP wait marks only its own host failed; when that is every host of the batch (always so with three nodes, one host per batch) the playbook ends and later batches are not restarted; when other hosts of the batch succeed, the next batch still restarts its nodes (R9); whether the failed node is then still joining, and how large the quorum and availability risk is, NOT AVAILABLE |
| keystone | fernet cron generator hour calculation; precheck vs generator expiry limits; push script stops at the first unreachable peer (all three unchanged in 2023.1) | fact sheet B §3.9 (items 3–5); roles/; roles/, roles/; roles/ |
CODE |
| glance | global log_bin_trust_function_creators toggle on every deploy/reconfigure; file backend = single host; backend switches silently when Swift/Ceph is enabled |
roles/, bootstrap.yml:38, deploy.yml:12, reconfigure.yml:2; group_vars/; fact sheet C §1 |
CODE |
| cinder | no host/backend_host/cluster (still); etcd coordination uses only the first etcd host (no longer holds as written: the etcd backend_url points at kolla_internal_fqdn, the internal VIP, where the etcd role has a HAProxy frontend); Ceph backup driver default without a Ceph config (still) |
roles/; roles/; group_vars/; roles/; Cinder; fact sheet C §3 |
CODE |
| heat | bootstrap_heat creates heat_user_domain/heat_domain_admin only when the domain is absent: a changed heat_domain_admin_password reaches heat.conf but never Keystone |
kolla docker/; roles/; roles/ |
CODE; impact NOT AVAILABLE |
| swift | no handler is ever notified (deploy does not apply config/ring changes; the only handler is a no-op debug); stop.yml wrong container name; precheck probes rsync port 873 vs 10873 |
roles/ (no notify in roles/); roles/; roles/, group_vars/; fact sheet D |
CODE; runtime effects NOT AVAILABLE |
| octavia | No longer holds in 2023.1: upgrade.yml has no task that removes the octavia admin role (it is config, check-containers, bootstrap_service, flush), and with octavia_auto_configure (on by default: amphora is always in octavia_provider_drivers) deploy creates the amphora flavor, keypair, security groups and management network |
roles/; roles/, prepare.yml:2-146; group_vars/, roles/; Octavia; fact sheet D |
CODE |
| masakari | The Ussuri SASL mismatch no longer holds: with libvirt_enable_sasl (default true in the masakari role's own defaults, as in nova-cell's) the role renders auth.conf with the libvirt SASL credentials into the instancemonitor. The connection_uri is still hard-coded qemu+tcp, while libvirt_tls: true makes libvirtd listen on TLS only (listen_tcp = 0) |
roles/; roles/, roles/, masakari-instancemonitor.json.j2:9-12; roles/; roles/; roles/ |
CODE; effect on the instancemonitor with libvirt_tls NOT AVAILABLE |
| baremetal | Now ansible-collection-kolla roles. Docker restarted on daemon.json change on all hosts at once (still: bootstrap-servers uses kolla_serial, default 0, and the handler restarts unless docker_systemd_reload); docker_custom_option no longer exists: docker_custom_config is merged into daemon.json, so its changes are applied (with that restart); Docker masked during re-runs on Debian/Ubuntu (still, when /etc/ is absent; unmasked at the end of the docker config tasks); /etc/hosts lists every baremetal host (O(N²) templating, stale with --limit, still) |
kolla-host.yml:4-10; ansible-collection-kolla roles/, roles/, roles/, roles/, roles/; Risk register R14; fact sheet E §1–5 |
CODE; runtime effects, and whether the path is absent at a re-run (it depends on how systemd unmasks), NOT AVAILABLE |
| certificates | test CA only; certs never renewed (every key, CSR and certificate task has creates:); new backend hosts not added |
roles/, generate.yml:28,, generate-backend.yml:21,; fact sheet A §8, E |
CODE |
Related¶
- Troubleshooting — the symptom index, which points to these rows by what an operator sees
- Findings — the headline findings; the rows
kolla_docker,kolla imagesandcommoncite F3, F14 and F2 - Per-role fact sheets — the fact sheets that the role rows cite as evidence, and their corrections
- Fact sheet D §7 — the consolidated bugs and oddities of the roles of fact sheet D
- Kolla images §3 — what
kolla_set_configs --checkcompares and what it never compares (rowkolla images) - What is not known — the effects that the Status column marks NOT AVAILABLE, by topic
- Open questions — the open questions Q1 to Q14
To remember
Lumo: if you keep only 5 things from this page, keep these.
-e kolla_serial=Nis overridden byANSIBLE_SERIAL(default 0) forreconfigure/upgrade; it is not passed fordeploy. CODEkolla_set_configs --checkcompares onlyconfig_filesentries: changes confined toconfig.jsonare not applied until the container restarts for another reason. CODE- A changed
heat_domain_admin_passwordreachesheat.confbut never Keystone. CODE - Certificates are never renewed (every key, CSR and certificate task has
creates:), the CA is a test CA only, and new backend hosts are not added. CODE - No longer holding in 2023.1:
graceful_timeoutandclient_timeoutfromdocker_common_optionsare now used, and-t commonruns the role. CODE
Check yourself
Lumo: 3 quick questions to check the main ideas of this page. Choose an answer to see the explanation and the section that gives the evidence. Your score counts your first choices and stays in this browser only.
Lumo: ready for the next step? Take your time: you can come back to this page whenever you need it.
Lumo: nicely done! One more page on your journey.