Ten things to know before a production run¶
Ten things to know before running Kolla-Ansible 2023.1 on production. Most items link to the section that gives the evidence.
reconfigureisdeployfor 57 of the 59 roles that have areconfigure.yml(swift and bifrost are the exceptions). It re-runs Keystone registration, DB creation and the DB-sync bootstrap containers for most services, Neutron's included in 2023.1. The only lightweight path isgenconfigfollowed bydeploy-containers(Execution flow §4).- Config is applied by a full stop/start, never a reload, and the stop timeout is 60 s. Containers run under
systemd units whose stop is
docker stop -t <docker_graceful_timeout>, 60 s by default; a service still running then is killed (R24, Repository architecture §6). The only signal Kolla sends to a running container is aSIGHUPtonova_scheduler, which refreshes its cell cache. - One changed container restarts all containers of its role on that host (ansible-core loop-notify). With the
default Nova settings, a nova-compute config change restarts nova_libvirt and nova_ssh with
nova_computeon every compute (Tags §2.4). - Without batching, handlers restart a service on every host of the play at once. Batching is
ANSIBLE_SERIALforreconfigure/upgradeand-e kolla_serial=Nfordeploy, which ignoresANSIBLE_SERIAL. Only MariaDB and RabbitMQ roll by themselves: their restarts run in separate plays withserial: "33%"(one node at a time with three nodes), and each RabbitMQ node is drained first (Execution flow §5). - A service tag is never just that service. For a service with a frontend it also runs the haproxy frontend config
(and can restart HAProxy). For API services it runs Keystone registration and DB sync. In 2023.1 it no longer runs
common(Tags §3, Reconfigure a single service §2). -t commonnow does the work, and only it does.commonis its own play, taggedcommon, so fluentd, cron and kolla_toolbox changes need-t commonor a run without tags; a service tag leaves them pending (Tags §2.1, Applycommonchanges). The Ussuri workaround-t chronyno longer exists.- Global settings land in per-host files.
enable_*, RabbitMQ/memcached membership and passwords end up in compute-side configs, so small central changes cause fleet-wide restarts (Why a small change creates a large workload). deploynever pulls an image that is already present. Rebuilt images under the same tag needpull. A tag change applied withdeployrecreates the entire fleet at once (Update container images).--limitdoes not contain everything. Delegated tasks still run ongroups[x][0]controllers, and all hosts' facts are gathered serially by the limited hosts (Compute scaling, Add a compute node §2).--list-taskscannot predict a run. Usegenconfigpluskolla_set_configs --check, or staging (Predict and inspect a run). Two things to know about the check:- It compares only the files listed in
config.json, not itscommandorpermissions: memcached lists no files, so a change to its command line never shows as drift. A plainreconfigurestill applies it (the changedconfig.jsonnotifies the restart handler), but oncegenconfighas written it, no later run restarts the container for it; it takes effect at the next restart for another reason (Write config first, apply in controlled batches §1.1). - A newly rendered optional file whose destination is not yet in the container makes
--checkexit 1, so the container restarts and the file is copied. In Ussuri the same case made the check exit 2 and the task fail (Kolla images §3).
- It compares only the files listed in
Related¶
- Find it fast — typical operator questions and the section that answers them
- Pre-execution checklist — what to check before a run
- Risk register — the risks R1–R25, with their precautions (R25 does not apply to 2023.1)
- Technical findings — the fifteen headline findings, each with its evidence
- Operations — the operation pages, each with its commands
To remember
Lumo: if you keep only 5 things from this page, keep these.
reconfigureisdeployfor 57 of the 59 roles that have areconfigure.yml; the only lightweight path isgenconfigfollowed bydeploy-containers.- Config is applied by a full stop/start, never a reload, and the stop timeout is 60 s.
- One changed container restarts all containers of its role on that host.
- Without batching, handlers restart a service on every host of the play at once;
deployignoresANSIBLE_SERIALand needs-e kolla_serial=N. deploynever pulls an image that is already present: rebuilt images under the same tag needpull.
Check yourself
Lumo: 3 quick questions to check the main ideas of this page. Choose an answer to see the explanation and the section that gives the evidence. Your score counts your first choices and stays in this browser only.
Lumo: ready for the next step? Take your time: you can come back to this page whenever you need it.
Lumo: nicely done! One more page on your journey.