Deployment execution flow¶
How Kolla-Ansible 2023.1 decides what runs when: play order, the steps inside each role, handler flush points,
batching, and how deploy, reconfigure, upgrade, deploy-containers, genconfig, pull and stop differ.
Note
Key answers
- There is no dependency resolution. Order comes from play order (
commonis now a play of its own, not a role dependency), task order inside a role, handler flush points,serial, and the restart plays that MariaDB and RabbitMQ handlers fill throughgroup_byCODE (§1). - What each action does (
deploy,reconfigure,upgrade,deploy-containers,genconfig,pull,stop,precheck,validate-config; the wrapper no longer offerscheck): the action matrix (§4).genconfig, thendeploy-containers --limit <batch>, is the only built-in way to separate "write the config" from "apply it" (§4). - Batching: a batch runs the complete play, handler restarts included, before the next batch starts, but each play
is still traversed batch by batch for the whole fleet before the next play starts. MariaDB and RabbitMQ apply their
role to all hosts in one batch, then restart the notified hosts in separate plays with
serial: "33%"(§5). reconfigureon a running cloud still runs the DB sync containers of keystone, glance, cinder, placement, nova, neutron, heat and others CODE. In 2023.1 Neutron's DB sync is no longer skipped (§6).- What one compute host goes through in a full
deploy, in order (§2.4). On a fresh compute,nova_computeis started beforeopenvswitchandneutron_openvswitch_agentexist on that host. - Restart order is handler definition order, not notification order CODE, and the handlers of a play run on all notified hosts in parallel (§7).
Deployment order (this page) ≠ runtime communication (see Component dependencies). Arrows in the diagrams below mean "runs before", unless a diagram explicitly says it shows runtime calls.
1. How the order is decided¶
There is no dependency resolution in Kolla-Ansible 2023.1. Order comes from these mechanisms only CODE:
| Mechanism | Effect | Evidence |
|---|---|---|
Play order in site.yml / nova.yml / mariadb.yml / rabbitmq.yml |
Services are applied strictly in file order. Ansible's default linear strategy runs play N on all its hosts (per batch) before play N+1 |
ansible/, generated list in Generated inventory §1 |
Role meta dependency on common |
None in 2023.1. No role has a meta/ directory. common is the Apply role common play, run after the prechecks play and before the loadbalancer play, on the cron, fluentd, kolla-logs and kolla-toolbox hosts, tagged common |
site.yml:86-97, roles/ |
| Task order inside a role | Fixed sequence: register → config(-host) → check-containers → bootstrap → flush handlers (§3) | roles/ |
| Handler flush points | Restarts happen at meta: flush_handlers (end of most deploy.yml) or at the end of the play. The loadbalancer role deliberately defers to the end of its play |
e.g. roles/; roles/ |
| Restart plays filled by handlers | MariaDB and RabbitMQ handlers only put hosts into groups (group_by); the following plays of mariadb.yml / rabbitmq.yml restart those groups with serial: "33%" |
roles/, mariadb.yml:14-52; roles/, rabbitmq.yml:26-41 |
serial / kolla_serial |
Splits a play into batches; each batch completes the whole play, handlers included, before the next starts | serial: '{{ kolla_serial|default("0") }}' on the gather-facts plays, the common play and every service play, except the plays of mariadb.yml and rabbitmq.yml (no serial, or a fixed "33%" on the restart plays) and cyborg, which reads serial: '{{ serial|default("0") }}' (site.yml:784), so kolla_serial never batches it. The group_by and prechecks plays have no serial key either |
Conditions only remove work, never reorder it:
when: enable_<svc>on the role;- the host-pattern intersection
&enable_<svc>_True; - per-task
when:onkolla_action; - placement filters (
inventory_hostname in groups[...]/host_in_groups).
2. The five diagrams¶
2.1 Infrastructure services (deployment order)¶
CODE (gather-facts.yml, site.yml:1-440, mariadb.yml, rabbitmq.yml)
flowchart TB
F["gather-facts (all hosts; delegated when --limit)"] --> G["group_by: kolla_action, enable_* flags"]
G --> P["prechecks (only kolla_action=precheck)"]
P --> CO["common (own play, tag common: fluentd, kolla_toolbox, cron)"]
CO --> HA["loadbalancer: haproxy + keepalived (+ proxysql)<br/>+ 44 per-service loadbalancer includes<br/>(handlers flushed at END of this play)"]
HA --> MON["collectd → influxdb → telegraf → redis"]
MON --> DB["mariadb.yml: apply role (no serial) → restart / start plays (serial 33%) → bootstrap-host restart → post-config"]
DB --> MC["memcached"]
MC --> PR["prometheus → iscsi → multipathd"]
PR --> MQ["rabbitmq.yml: rabbitmq → restart (serial 33%) → post-config; same for outward rabbitmq"]
MQ --> QD["etcd"]
QD --> KS["keystone ⇒ control plane (2.2)"]
style HA fill:#ffe9b3
style DB fill:#ffd6d6
style MQ fill:#ffd6d6
The Ussuri plays chrony, zookeeper, elasticsearch, kibana and qdrouterd no longer exist in site.yml;
OpenSearch now comes after Keystone (site.yml:460-468).
2.2 OpenStack control plane (deployment order)¶
CODE (site.yml:450-968, nova.yml)
flowchart LR
KS["keystone"] --> OS["opensearch"] --> SW["swift → ceph-rgw"] --> GL["glance"] --> IR["ironic"] --> CI["cinder"] --> PL["placement"] --> NOVA["nova.yml (2.3)"]
NOVA --> NET["openvswitch / ovs-dpdk → ovn-controller → ovn-db → neutron → kuryr (2.5)"]
NET --> HC["hacluster"] --> HE["heat"] --> HO["horizon"] --> MISC["murano → solum → magnum → mistral → sahara"]
MISC --> MA["manila"] --> TEL["gnocchi → ceilometer → aodh"] --> BA["barbican"] --> MORE["cyborg → designate → trove → watcher → grafana → cloudkitty → freezer → senlin → tacker"]
MORE --> OC["octavia"] --> TAIL["zun → vitrage → blazar → masakari → venus → skyline"]
Notable order facts CODE:
- Keystone comes after MariaDB, RabbitMQ and memcached, but before any service that registers with it.
service-ks-registerusesopenstack_interface(internal) and authenticates againstkeystone_internal_url(roles/service-ks-register/defaults/main.yml:5,group_vars/all.yml:609,865,895-896). - Placement, Glance and Cinder come before Nova. Neutron comes after Nova (see §2.5).
- Barbican comes after Nova, Cinder and Glance, even though they can be configured to use it as a key manager.
With
enable_barbican, Nova and Cinder render a[barbican]section (roles/nova/templates/nova.conf.j2:204-207,roles/nova-cell/templates/nova.conf.j2:246-249,roles/cinder/templates/cinder.conf.j2:238-241), and their plays run before the barbican play (site.yml:519,544,765). On a first deploy they therefore start configured with a[barbican]section before Barbican exists. NOT AVAILABLE Whether this has any effect before encrypted volumes are used. - Octavia is near the end. With
octavia_auto_configure(default: true when the amphora provider is enabled,group_vars/all.yml:1228), its play creates the amphora flavor, the keypair, the security groups and the management network (roles/octavia/tasks/deploy.yml:4-5,prepare.yml). No task of the role uploads the amphora image, and the certificates come from the separateoctavia-certificatescommand (tools/kolla-ansible:536) (fact sheet D).
2.3 Nova deployment (ansible/nova.yml)¶
flowchart TB
subgraph P1["Play 1: Bootstrap nova API databases — hosts nova-api (tags nova, nova-bootstrap, nova-api, nova-api-bootstrap)"]
A1["mysql_db nova_api + nova_cell0 (run_once, kolla_toolbox)"] --> A2["mysql_user grants (run_once)"] --> A3["config_bootstrap: /etc/kolla/nova-api-bootstrap (run_once)"]
A3 --> A4["nova_api_bootstrap container: api_db sync + db sync (run_once)"] --> A5["nova_api_map_cell0 + list_cells for cell0 (+ update cell0 if its DB URL differs) (delegate groups['nova-api'][0])"]
A5 -->|"map_cell0 changed"| A6["nova_api_bootstrap again"]
end
subgraph P2["Play 2: Bootstrap nova cell databases — hosts nova-conductor, any_errors_fatal"]
B1["cell DB + user (on groups[nova_cell_conductor_group][0])"] --> B2["RabbitMQ vhost/user (service-rabbitmq, delegated to the cell RPC group's first host)"] --> B3["nova-cell-bootstrap config"] --> B4["nova_cell_bootstrap container (cell db sync)"] --> B5["list_cells → create_cell or update_cell"]
end
subgraph P3["Play 3: Apply role nova — nova-api, nova-scheduler, nova-super-conductor"]
C1["service-ks-register (run_once)"] --> C2["config.yml"] --> C3["check-containers"] --> C4["flush: super-conductor → scheduler → api"]
end
subgraph P4["Play 4: Apply role nova-cell — compute, nova-compute-ironic, nova-conductor, consoles"]
D1["config-host (br_netfilter, sysctl, udev, qemu-kvm mask)"] --> D2["config.yml (nova.conf, libvirt, ssh, ceph)"] --> D3["check-containers"] --> D4["flush: conductor → consoles → ssh → libvirt → libvirt ready check + SASL user → compute"] --> D5["wait for nova-compute registration (first compute of batch, delegated to conductor[0], 20×10s)"] --> D6["list_cells + discover_hosts --by-service (delegated to conductor[0])"] --> D7["post-config: remove old nova_libvirt_secrets volume"]
end
subgraph P5["Play 5: Refresh scheduler cell cache — kill -HUP nova_scheduler (deploy/reconfigure)"]
end
subgraph P6["Plays 6-10: reloads + online data migrations — upgrade only"]
end
P1 --> P2 --> P3 --> P4 --> P5 --> P6
CODE (nova.yml:29-156; roles/nova/tasks/bootstrap.yml:49-56, map_cell0.yml:2-67;
roles/nova-cell/tasks/bootstrap.yml:37-43,
rabbitmq.yml:2-24, create_cells.yml:2-55; roles/nova-cell/tasks/deploy.yml:5-39,
wait_discover_computes.yml:27-28,42-43,87-90,
discover_computes.yml:5,16-21, post-config.yml:3-7; handler order roles/nova/handlers/main.yml:2,20,38,
roles/nova-cell/handlers/main.yml:2-151; roles/nova/tasks/refresh_scheduler_cell_cache.yml:10)
Details and Nova runtime dependencies: Nova deep dive.
2.4 Compute node deployment (what one compute host experiences)¶
Sequence on one compute host during a full deploy, with the default enable flags and the multinode inventory
groups (iscsi only with enable_iscsid) CODE (facts and group_by first (site.yml:2,7-77); play order
site.yml:86-97,417-426, nova.yml:116-138 (imported at site.yml:544), site.yml:546-555,590-608; task and
handler order in each role's tasks/deploy.yml and handlers/main.yml; common in its own play
(site.yml:86-97, roles/common/tasks/deploy.yml:2-9); discovery on the first compute of the batch only, delegated
to the first conductor, roles/nova-cell/tasks/wait_discover_computes.yml:42-43,87-90 and
discover_computes.yml:21):
sequenceDiagram
autonumber
participant D as Deploy host
participant C as compute001
D->>C: setup (facts), group_by (always)
Note over D,C: Play "common" (compute is in the common group)
D->>C: common: config dirs, config.json ×3, td-agent.conf, cron logrotate config, toolbox Erlang cookie, rabbitmq-env.conf, erl_inetrc
D->>C: common: check containers (fluentd, kolla_toolbox, cron), create kolla_logs volume, link /var/log/kolla
D->>C: handlers: Restart fluentd → kolla_toolbox (+ toolbox init) → cron
Note over D,C: Play "iscsi" (only if enable_iscsid)
D->>C: iscsid config/check → Restart iscsid
Note over D,C: Play "nova-cell" (after controllers' nova plays have bootstrapped DBs/cell)
D->>C: br_netfilter, sysctl, udev kvm rule, config dirs, config.json, nova.conf, qemu.conf, libvirtd.conf, SASL, ssh keys
D->>C: check nova-cell containers
D->>C: handlers: Restart nova-ssh → nova-libvirt → libvirt ready check → Create libvirt SASL user → nova-compute
D->>C: (first compute of batch only) wait for nova-compute services, discover_hosts — delegated to conductor[0]
D->>C: post-config: remove old nova_libvirt_secrets volume
Note over D,C: Play "openvswitch"
D->>C: load openvswitch module, config, check → Restart openvswitch_db → wait ovsdb → Restart vswitchd
D->>C: post-config: set system-id and hw-offload through openvswitch_db (notifies vswitchd restart), bridge ensure only if computes need external bridge
Note over D,C: Play "neutron"
D->>C: ip6_tables module, sysctl (only on l3 hosts), config (neutron.conf, openvswitch_agent.ini), check
D->>C: handler: Restart neutron-openvswitch-agent
The common steps are roles/common/tasks/config.yml, check-containers.yml and bootstrap.yml:2-16. The
openvswitch post-config.yml:3-22 runs after the role's flush, so a vswitchd restart it notifies runs at the end of
the play. The neutron module and sysctl steps are roles/neutron/tasks/config-host.yml:1-42.
NOT AVAILABLE How many task results, skips and handlers a compute host or a controller produces in a full
deploy: that depends on the inventory, the enable flags and the when: conditions met at run time. The first
controller also runs the run_once and delegated bootstrap tasks (e.g. roles/nova/tasks/map_cell0.yml:63-64).
Ordering consequence: on a fresh compute, nova_compute is started (and its registration awaited) before
openvswitch and neutron_openvswitch_agent exist on that host. NOT AVAILABLE Whether instances can be scheduled
there successfully before the later plays finish (Nova deep dive
§3 item 6). On a running cloud this
ordering matters when you add computes with --limit (see Add a compute node
§2).
2.5 Networking components¶
flowchart TB
subgraph Order["Deployment order"]
direction TB
O1["loadbalancer play: neutron-server frontend"] --> O2["openvswitch play: openvswitch_db → vswitchd (network, compute, manila-share)"]
O2 --> O2b["post-config: system-id, hw-offload; br-ex + external interface on network nodes"]
O2b --> O3["ovs-dpdk play (instead of openvswitch when enable_ovs_dpdk)"]
O3 --> O4["ovn-controller play (if neutron_plugin_agent=ovn): br-int, OVN settings in OVSDB, ovn_controller"]
O4 --> O4b["ovn-db play: NB/SB DB cluster, northd"]
O4b --> O5["neutron play: register → config-host → config → check → bootstrap (DB/user and DB-sync container) → flush"]
O5 --> O6["kuryr (if enabled)"]
end
subgraph Within["Inside the neutron play (all hosts in parallel)"]
N1["neutron-server (controllers)"]
N2["l3 / dhcp / metadata / ovs agents (network nodes)"]
N3["ovs agent (+ l3/metadata if DVR) or ovn-metadata agent (computes)"]
end
O5 --> Within
CODE (site.yml:546-608; roles/openvswitch/tasks/deploy.yml, post-config.yml:3-37;
roles/ovn-controller/tasks/deploy.yml, setup-ovs.yml; roles/ovn-db/tasks/deploy.yml;
roles/neutron/tasks/deploy.yml:2-23, bootstrap.yml:38, bootstrap_service.yml:2-21)
In 2023.1 the OVN controller play runs before the OVN database play. The neutron play deploys the server and all agents in a single play and a single handler flush. Agents are not deployed after the server has been verified. NOT AVAILABLE Whether the RPC layer tolerates the agents starting first.
3. Inside a role: the standard deploy sequence¶
Typical deploy.yml (Nova shown; Neutron adds config-host and runs its bootstrap before the flush):
| Step | Typical task file | Scope | Runs on reconfigure? |
|---|---|---|---|
| 1. Keystone registration | register.yml → service-ks-register |
run_once (per batch) via kolla_toolbox on the first host of the play |
yes |
| 2. Host configuration | config-host.yml (sysctl, modules, udev) |
per host | yes |
| 3. Config generation | config.yml (dirs, config.json, INI merges, policy, certs) |
per host × per service × per file | yes |
| 4. Container check | check-containers.yml (compare_container: container parameters, systemd unit file, kolla_set_configs --check) |
per host × per container | yes |
| 5. Bootstrap | bootstrap.yml (DB, user, grants) + bootstrap_service.yml (one-shot container: DB sync) |
run_once / first host of the group |
yes for most roles, Neutron included: its bootstrap_neutron DB-sync container has no kolla_action or neutron_enable_rolling_upgrade condition in 2023.1 (roles/, bootstrap_service.yml:2-21) |
| 6. Flush handlers | meta: flush_handlers |
notified hosts, in handler definition order | yes |
| 7. Post steps | e.g. openvswitch post-config.yml, nova-cell discovery and post-config.yml |
varies | yes |
CODE (roles/nova/tasks/deploy.yml:2-9; roles/neutron/tasks/deploy.yml:2-23;
module_utils/kolla_container_worker.py:59-66)
Order caveats:
- Some roles bootstrap before containers start, some after. Nova bootstraps in separate plays before the service
plays (
nova.yml:29-96). Neutron runs its bootstrap after check-containers but before the flush (roles/neutron/tasks/deploy.yml:13-23). Keystone bootstraps before its flush and registers last, after Fernet key distribution (roles/keystone/tasks/deploy.yml:2-21). Glance and Cinder vary (fact sheets B, C). - The loadbalancer role does not flush in
deploy.yml. The NOTE inroles/loadbalancer/tasks/deploy.yml:8-11explains that site.yml handles all haproxy changes in that play, so the restart happens once, after all per-service frontends are rendered.
4. Action matrix: what each kolla_action does¶
| Action (CLI) | Config written | Containers compared | Handlers (restarts) | Bootstrap / DB sync | Keystone registration | Images pulled | Notes |
|---|---|---|---|---|---|---|---|
deploy |
yes | yes | yes | yes | yes | only if absent | Full flow |
reconfigure |
yes | yes | yes | yes (most roles) | yes | only if absent | Identical to deploy in 57 of the 59 roles that have a reconfigure.yml (swift and bifrost are the exceptions) (Repository architecture §4.2). kolla_serial passed from ANSIBLE_SERIAL (tools/) |
upgrade |
yes | yes | yes | yes: most roles re-run the deploy KOLLA_BOOTSTRAP one-shot with the new image (cinder, heat, barbican, octavia, manila, horizon, placement, legacy glance/neutron). Upgrade-only one-shots: keystone KOLLA_UPGRADE then KOLLA_FINISH_UPGRADE, run as handlers (roles/); neutron rolling (default) NEUTRON_DB_EXPAND/NEUTRON_DB_CONTRACT; glance rolling (opt-in, glance_enable_rolling_upgrade: "no", group_vars/) GLANCE_DB_EXPAND/MIGRATE/CONTRACT; mariadb upgrade_mariadb (roles/); KOLLA_OSM online migrations (nova, placement, cinder, rolling ironic). Nova bootstrap_upgrade re-runs its deploy db-sync container, minus DB creation and cell0 mapping (Kolla images §4) |
yes (some roles) | only if absent → run pull first |
Nova reload plays and the separate KOLLA_OSM online-migration containers are active only here. kolla_serial passed from ANSIBLE_SERIAL (tools/). Placement's bootstrap container also runs placement-manage db online_data_migrations on every deploy/reconfigure (kolla docker/); so does Ironic's with ironic_enable_rolling_upgrade: no (roles/, kolla docker/) |
deploy-containers |
no | yes (incl. drift of the files listed in config_files; not command/permissions changes inside config.json, see below) |
yes | no | no | only if absent | tasks/ = check-containers.yml (e.g. roles/). Loadbalancer includes skipped (site.yml:339-341) |
genconfig (config) |
yes | no | all container restarts skipped (when: kolla_action != "config"; the unguarded handlers are waits, checks and bootstrap steps chained from guarded handlers, plus swift's no-op debug handler; the Ussuri exception, the nova-hyperv role, no longer exists, Pre-execution checklist) |
no | no | no | tools/. The Ussuri fluentd label check, which pulled the fluentd image, is gone from roles/. With octavia_auto_configure, octavia's config tasks query the cloud through kolla_toolbox (roles/) |
pull |
no | no | no | no | no | yes, every enabled container mapped to the host, plus common (in its own play) |
service-images-pull (roles/) |
stop |
no | no | no | no | no | no | stop_container for every enabled container on the host, honouring skip_stop_containers (roles/) |
precheck |
no | no | no | no | no | no | prechecks role + each role's precheck.yml (port checks, group validation) |
validate-config (config_validate, new) |
no | no | no | no | no | no | tools/. Runs oslo-config-validator inside each running container that has a validation list, and writes the errors under service_config_validate_output_dir on the deploy host (roles/, validate.yml:2-49) |
check |
– | – | – | – | – | – | No longer a kolla-ansible command in 2023.1: the wrapper has no check case (tools/). The roles still carry a check.yml, reached only if kolla_action=check is passed to the playbooks some other way |
The combination genconfig → (review the diff) → deploy-containers --limit <batch> is the only built-in way to
separate "write the config" from "apply it", and to apply in controlled batches. It works because compare_container
detects drift in the files listed in each config.json's config_files (content, mode, owner) through
kolla_set_configs --check CODE (module_utils/kolla_container_worker.py:59-66,
module_utils/kolla_docker_worker.py:155-190, kolla docker/base/set_configs.py:165-198,220-242,556-559).
Changes confined to config.json itself (command, permissions; e.g. every memcached setting,
roles/memcached/templates/memcached.json.j2:2-3) are NOT detected. deploy-containers therefore does not apply them
after genconfig, and a later deploy/reconfigure does not either: the config tasks that notify the restart, the
config.json template task included (memcached's only one, roles/memcached/tasks/config.yml:13-26), are then
unchanged. Such changes take effect only at the next restart for another reason.
An optional file that newly appears, and whose destination does not exist in the container yet, now counts as drift:
--check exits 1 and the restart copies the file. In Ussuri the same case made the check exit 2 and the task fail
CODE (kolla docker/base/set_configs.py:36-39,165-171,220-242,576-578, module_utils/kolla_docker_worker.py:178-190).
See
Write config first, apply in controlled batches
§2 and Kolla images
§3.
5. Batching (serial) and its interaction with order¶
gantt
dateFormat X
axisFormat %s
title Play execution with kolla_serial=2 (illustrative, 4 computes)
section nova-cell play
batch 1 (compute001-002) config→check→flush→discover :a1, 0, 30s
batch 2 (compute003-004) config→check→flush→discover :a2, after a1, 30s
section openvswitch play
batch 1 :b1, after a2, 15s
batch 2 :b2, after b1, 15s
The bar lengths are illustrative only; no duration is known from the code.
- A batch runs the complete play before the next batch starts. That includes handler restarts and the compute discovery. Batches therefore bound the number of simultaneous restarts per play, but each play is still traversed batch by batch for the whole fleet before the next play starts. The neutron agents on compute001 are restarted only after all computes have finished the nova-cell and openvswitch plays.
run_oncetasks run once per batch (Tags §2.5). Examples: DB sync, Keystone registration, compute discovery (wait_discover_computes.ymlruns forall_computes_in_batch[0]).- MariaDB and RabbitMQ are not batched by
kolla_serial. Their role plays run all hosts in one batch; their handlers only add the hosts to restart groups, and the next plays restart those groups withserial: "33%"(mariadb.yml:14-40,rabbitmq.yml:26-41,82-98). With three nodes that is one node per batch (ansible-core 2.14.18lib/ansible/utils/helpers.py:32). Each RabbitMQ node is put into maintenance mode before its restart (roles/rabbitmq/tasks/restart_services.yml:9-17) (fact sheet B). cyborg is not batched bykolla_serialeither: its play readsserialinstead. - How
kolla_serialis set depends on the action. Forreconfigureandupgradethe wrapper sets it fromANSIBLE_SERIAL(tools/kolla-ansible:290,508,517), after your own-e(tools/kolla-ansible:360-362), so use the environment variable. Fordeploythe wrapper does not set it (tools/kolla-ansible:476-478): pass-e kolla_serial=Nyourself.
6. Initial deployment vs reconfiguration: what actually differs¶
| Aspect | Initial deploy (empty hosts) |
reconfigure on a running cloud |
|---|---|---|
| Containers | All create+start: the container check finds no container, and the restart handler then starts a new one under its systemd unit CODE (module_utils/, module_utils/). Exception: on the bootstrap host (groups[mariadb_shard_group][0]), mariadb runs first as the one-shot bootstrap_mariadb, then a handler starts it with --wsrep-new-cluster and waits for its port and WSREP sync, and the "Restart bootstrap mariadb service" play runs recreate_or_restart_container on it (roles/, bootstrap_cluster.yml:2-27, roles/, mariadb.yml:42-52, tasks/). NOT AVAILABLE Container counts, which depend on the inventory and the enable flags |
Restart (config) or recreate (image or params) only if drift is detected; otherwise none CODE (module_utils/, module_utils/) |
| One-shot containers | DB bootstrap for every service, MariaDB and RabbitMQ bootstrap, fernet key generation, cell creation | DB sync containers still run for keystone, glance, cinder, placement, nova (API and cell DB sync, plus map_cell0, list_cells for cell0 and list_cells ×2), neutron, heat, plus barbican, octavia and others CODE (in these roles reconfigure.yml imports deploy.yml and the bootstrap containers have no kolla_action guard, e.g. roles/; nova: nova.yml:45-51,, roles/, list_cells from roles/ and discover_computes.yml:5; neutron: roles/). MariaDB/RabbitMQ bootstrap and the keystone Fernet bootstrap (when: groups['keystone_fernet_running'] is not defined, roles/) are not re-run |
| Handler order effects | Every handler fires | Only notified handlers fire, but loop-notify widens them to all containers of the role on that host (Tags §2.4) |
Neutron DB sync (bootstrap_neutron) |
runs | runs too in 2023.1 (no condition on kolla_action or neutron_enable_rolling_upgrade, roles/). The DB/user/grant tasks run as well |
| Nova cell | create_cell |
update_cell only if the transport URL or DB URL changed (roles/) |
| Fernet keys | generated | re-distributed from the first running keystone_fernet node to the others (fact sheet B §3.6, task flow in §3.3) |
| MariaDB | Galera bootstrap on the first host of the shard | cluster lookup; if config changed, restart plays with serial: "33%" (mariadb.yml:14-40) |
| Duration drivers | image pulls, first container starts | per-task SSH overhead across all hosts, compare_container exec per container (see Compute scaling). NOT AVAILABLE Any duration |
7. Dependencies introduced by handlers¶
Handlers create ordering that is not visible in task files:
- Within a role, restart order = handler definition order, not notification order.
- nova-cell restarts
nova_ssh→nova_libvirt(→ libvirt ready check, SASL user) →nova_compute(roles/nova-cell/handlers/main.yml:74-151). - openvswitch restarts
openvswitch_db→ waits →openvswitch_vswitchd(roles/openvswitch/handlers/main.yml:2-29). - CODE ansible-core 2.14.18 copies the play's handler list in definition order into each host's state and walks it
in that order, running the handlers notified for the host
(
lib/ansible/executor/play_iterator.py:192-197,430-452).
- nova-cell restarts
- Handlers of a play run on all notified hosts in parallel (ansible-core 2.14.18: the linear strategy puts all
hosts in lockstep on the same handler,
lib/ansible/plugins/strategy/linear.py:84-123). Withserial: 0, every compute restartsnova_compute(and, through loop-notify,nova_libvirtandnova_ssh) in the same step. - Handler chains:
- Restart nova-libvirt → Checking libvirt container is ready → Create libvirt SASL user (with
libvirt_enable_sasl,roles/nova-cell/handlers/main.yml:96,116). - Restart openvswitch-db → Waiting for openvswitch_db.
- Restart kolla-toolbox → Initializing toolbox (
roles/common/handlers/main.yml:34-37). - Restart haproxy / proxysql / keepalived container → handlers that
listento these names: check the API interface addresses, group the hosts by whether they hold the VIP, stop and start the backup hosts' containers, then the master's, then wait for HAProxy (and ProxySQL) to listen on the VIP (roles/loadbalancer/handlers/main.yml:9-329). - MariaDB and RabbitMQ: restart handlers only
group_bythe host; the restart itself is a later play (item 5).
- Restart nova-libvirt → Checking libvirt container is ready → Create libvirt SASL user (with
- Handlers can cross roles.
haproxy-config(imported throughloadbalancer-configby every service'sloadbalancer.yml) notifies the loadbalancer role'sRestart haproxy container(roles/haproxy-config/tasks/main.yml:22-23). This is why the loadbalancer play includes the loadbalancer role even for-t <svc>. - The MariaDB restart is a sequence of plays, not of handler waves. The handlers put running members into
mariadb_restartand new members intomariadb_start(roles/mariadb/handlers/main.yml:56-73); the "Restart mariadb services" and "Start mariadb services" plays then runrestart_services.yml(restart, port wait, WSREP sync wait) withserial: "33%"(mariadb.yml:14-40,roles/mariadb/tasks/restart_services.yml:2-51). The Ussuriindex % 4waves no longer exist (fact sheet B §1.5).
Related¶
- Component dependencies (runtime dependencies)
- Tags §2.5 (
run_oncewithserial) - Nova deep dive (the nova plays)
- Write config first, apply in controlled batches
§2 (
genconfigthendeploy-containers) - Glossary (glossary)
To remember
Lumo: if you keep only 5 things from this page, keep these.
- There is no dependency resolution: order comes from play order, task order inside a role, handler flush points,
serialand the MariaDB and RabbitMQ restart plays. CODE genconfig, thendeploy-containers --limit <batch>, is the only built-in way to separate "write the config" from "apply it".- A batch runs the complete play, handler restarts included, before the next batch starts, but each play is traversed batch by batch for the whole fleet first.
reconfigureon a running cloud still runs the DB sync containers of keystone, glance, cinder, placement, nova, neutron, heat and others. CODE- Restart order is handler definition order, not notification order. CODE
Check yourself
Lumo: 3 quick questions to check the main ideas of this page. Choose an answer to see the explanation and the section that gives the evidence. Your score counts your first choices and stays in this browser only.
Lumo: ready for the next step? Take your time: you can come back to this page whenever you need it.
Lumo: nicely done! One more page on your journey.