Skip to content

Ten things to know before a production run

Ten things to know before running Kolla-Ansible Ussuri on production. Most items link to the section that gives the evidence.

  1. reconfigure is deploy for 70 of the 72 roles that have a reconfigure.yml. It re-runs Keystone registration, DB creation and the DB-sync bootstrap containers for most services; Neutron's DB sync is the default exception. The only lightweight path is genconfig followed by deploy-containers (Execution flow §4).
  2. Config is applied by a full stop/start, never a reload, and the effective SIGKILL timeout is 10 s. docker_graceful_timeout is silently ignored (Repository architecture §6). The only signal Kolla sends is a SIGHUP to nova_scheduler, which refreshes its cell cache.
  3. One changed container restarts all containers of its role on that host (Ansible 2.9 loop-notify). With the default Nova settings, a nova-compute config change restarts nova_libvirt on every compute (Tags §2.4).
  4. Without batching, handlers restart a service on every host of the play at once. Batching is ANSIBLE_SERIAL for reconfigure/upgrade and -e kolla_serial=N for deploy, which ignores ANSIBLE_SERIAL. Only MariaDB (4 waves) and RabbitMQ (one by one) roll by themselves.
  5. A service tag is never just that service. For a service with a frontend it also runs the haproxy frontend config (and can restart HAProxy). It runs the common role on every host of the play (memcached and multipathd excepted) and, for API services, Keystone registration and DB sync (Tags §3, Reconfigure a single service §2).
  6. -t common does nothing (Tags §2.1). Use -t chrony or a service tag to push fluentd/cron changes.
  7. Global settings land in per-host files. enable_*, RabbitMQ/memcached membership and passwords end up in compute-side configs, so small central changes cause fleet-wide restarts (Why a small change creates a large workload).
  8. deploy never pulls an image that is already present. Rebuilt images under the same tag need pull. A tag change applied with deploy recreates the entire fleet at once (Update container images).
  9. --limit does not contain everything. Delegated tasks still run on groups[x][0] controllers, and all hosts' facts are gathered serially by the limited hosts (Compute scaling, Add a compute node §2).
  10. --list-tasks cannot predict a run. Use genconfig plus kolla_set_configs --check, or staging (Predict and inspect a run). Two things to know about the check:
    • It compares only the files listed in config.json, not its command or permissions: memcached lists no files, so a change to its command line never shows as drift. A plain reconfigure still applies it (the changed config.json notifies the restart handler), but once genconfig has written it, no later run restarts the container for it; it takes effect at the next restart for another reason (Write config first, apply in controlled batches §1.1).
    • A newly rendered optional file whose destination is not yet in the container makes --check exit 2, unless an entry listed before it already differs (then it exits 1 and the container restarts). Exit 2 is real drift, but the check task fails instead of restarting the container (Kolla images §3).

To remember

Lumo: if you keep only 5 things from this page, keep these.

  • reconfigure is deploy for 70 of the 72 roles that have a reconfigure.yml; the only lightweight path is genconfig followed by deploy-containers.
  • Config is applied by a full stop/start, never a reload, and the effective SIGKILL timeout is 10 s: docker_graceful_timeout is silently ignored.
  • One changed container restarts all containers of its role on that host.
  • Without batching, handlers restart a service on every host of the play at once; deploy ignores ANSIBLE_SERIAL and needs -e kolla_serial=N.
  • deploy never pulls an image that is already present: rebuilt images under the same tag need pull.

Check yourself

Lumo: 3 quick questions to check the main ideas of this page. Choose an answer to see the explanation and the section that gives the evidence. Your score counts your first choices and stays in this browser only.

1. What is the only lightweight path to apply config?
Show the answer

Answer: genconfig followed by deploy-containers. reconfigure is deploy for 70 of the 72 roles that have a reconfigure.yml. The only lightweight path is genconfig followed by deploy-containers.

2. Config is applied by a full stop/start. What is the effective SIGKILL timeout?
Show the answer

Answer: 10 s: docker_graceful_timeout is silently ignored. Config is applied by a full stop/start, never a reload, and the effective SIGKILL timeout is 10 s. docker_graceful_timeout is silently ignored.

3. What does -t common do in Ussuri?
Show the answer

Answer: Nothing: use -t chrony or a service tag. -t common does nothing. Use -t chrony or a service tag to push fluentd/cron changes.