Common: fluentd, cron, kolla_toolbox¶
This page is the starting point for the common role: the containers fluentd, kolla_toolbox and cron, on the
compute, control, monitoring, network and storage nodes of the example inventory. In 2024.1 the role is still no meta
dependency of any other role (no roles/*/meta directory): it runs only in its own play, "Apply role common", on the
groups cron, fluentd, kolla-logs and kolla-toolbox, with the play tag common (site.yml:88-103). -t common
therefore runs it, and a service tag such as -t nova does not CODE. New in 2024.1, its config step also makes sure
the fluentd image is on the host and reads two labels from it (roles/common/tasks/config.yml:42-91), so even a
config-only run of the role can pull that image CODE. The lists below say where the study covers how to operate it,
its impact, its dependencies and the evidence, what to check around a run and what stays open; the service card
follows them.
Where to look¶
Operate¶
- Scenario: Apply
commonchanges (fluentd, cron, kolla_toolbox) - Command: Apply
commonchanges (fluentd, cron, kolla_toolbox) §1.1:kolla-ansible -i multinode -t common reconfigure, optionally with--limit
Impact and risk¶
- What the tag executes (
common): Tags §3 and Tags §3.1, rowscommon; why the tag reaches the included tasks: Tags §2.1 - Risks: R2; R25 does not apply to 2024.1 (the risk register keeps the entry and says why)
Dependencies¶
- Dependencies: Component dependencies §1, Component dependencies §2 (row "Fluentd"), Component dependencies §5
- Image registry, new in 2024.1: with
enable_fluentdon (defaultyes,group_vars/all.yml:765), the config step runskolla_containerensure_imageon the fluentd image, which pulls it when it is not on the host (roles/common/tasks/config.yml:42-51;module_utils/kolla_docker_worker.py:501-503,module_utils/kolla_podman_worker.py:675-677), then reads its labels withdocker_image_infoorpodman_image_info(config.yml:53-91), modules of thecommunity.dockerandcontainers.podmancollections (ansible-core 2.16.20lib/ansible/config/ansible_builtin_runtime.yml:61-62,733-734;requirements-core.yml:18-23) CODE.
Evidence¶
- Fact sheet: – (see Repository architecture
§5.1, which also records that the
common_runguard is gone) - Image side: Kolla images §5 (rows
fluentd,cron,kolla-toolbox) - Also: Apply
commonchanges (fluentd, cron, kolla_toolbox); handlers: Tags §5
What changed in 2024.1¶
Read in the 2024.1 sources and compared with 2023.1 CODE:
| Item | 2024.1 | Evidence |
|---|---|---|
| Play | Same hosts, tag and serial; new max_fail_percentage, common_max_fail_percentage, then kolla_max_fail_percentage, then 100 |
site.yml:88-103 |
| fluentd command and user | Read from the image labels fluentd_binary and fluentd_user, td-agent when a label is missing; the 2024.1 kolla image sets both to fluentd. The command is now <fluentd_cmd> -c /etc/, and td-agent.conf is copied to /etc/. In 2023.1 the template hard-coded td-agent |
roles/; roles/; kolla docker/ |
| fluentd read positions | pos_file /var/ instead of /var/; the image creates /var/ |
roles/; kolla docker/ |
| fluentd volumes | Always mounts /var/, whatever enable_fluentd_systemd says |
roles/ |
| New fluentd inputs | 11-letsencrypt.conf.j2 with enable_letsencrypt; 12-systemd.conf.j2 (the journal) with enable_fluentd_systemd, which defaults to enable_fluentd and enable_central_logging (default no); OVN log files with enable_ovn, on the OVN hosts only |
roles/; group_vars/; roles/ |
| fluentd outputs | bulk_message_request_threshold (20M) and chunk_limit_size (8M) on the Elasticsearch and OpenSearch outputs |
roles/; roles/ |
| kolla_toolbox | Command kolla_toolbox instead of sleep infinity: a sleep 1 loop that exits 0 on TERM or QUIT. /run is mounted :shared only with Docker, plus run_default_volumes_<engine> (empty for Docker). The image ships ansible-core==2.16.* |
roles/; kolla docker/, docker/; roles/; group_vars/ |
| Service lists | fluentd input list: placement added; freezer, murano, sahara, senlin and solum gone. Logrotate list: redis added; freezer, murano, outward-rabbitmq, sahara, senlin, solum and vitrage gone |
roles/; roles/ |
| Podman | With kolla_container_engine: podman only: /etc/ and systemd-tmpfiles --create on the host |
roles/ |
Before and after a run¶
- Pre-execution checklist, step 7 (scope):
-t commonruns the common role in its own play, and service tags do not run it (site.yml:88-103,roles/common/tasks/main.yml:2). - R2, precautions: treat
enable_*changes as fleet-wide changes for thecommonplay; roll out with--limitbatches orANSIBLE_SERIAL(the play hasserial: kolla_serial,site.yml:95). - Registry: the run needs the fluentd image of the configured tag on every host of the play, or access to the
registry to pull it, even for
genconfig(roles/common/tasks/config.yml:42-51) CODE. - Apply
commonchanges (fluentd, cron, kolla_toolbox) §4, verification: before,genconfig, thenkolla_set_configs --checkin each offluentd,cronandkolla_toolbox(1 in any of them = all three will restart, by loop-notify); the check does not cover image, volume orcommandchanges. After,docker psshows fresh start times,docker logs fluentdhas no parse errors and logs still reach the configured output. - Tags §3.1, row
common: config files that can change:config.json,td-agent.conf, the cron logrotate config and the kolla_toolbox files, on every host ofcommonCODE (roles/common/tasks/config.yml:93-323); safe on its own: "yes; the way to push common config fleet-wide".
Known limits and open points¶
- F2, "
-t commonis a no-op": does not hold in 2024.1. The tagcommonis on the play (site.yml:100-101), and the tasks of the dynamic include inroles/common/tasks/main.yml:2keep the include's parent (ansible-core 2.16.20lib/ansible/plugins/strategy/__init__.py:831-842), whose tags extend those of the play (lib/ansible/playbook/block.py:352-361;tagsis anextend=Truefield,lib/ansible/playbook/taggable.py:42) CODE (Applycommonchanges §2). The reverse surprise stays: a tagged service run does not apply pendingcommonchanges, which wait for an untagged run or-t common. - F1, "Loop-notify amplification": still holds.
Check common containersloops overcommon_serviceswithnotify: "Restart {{ item.key }} container"(roles/common/tasks/check-containers.yml:2-17), and so does theconfig.jsontask (roles/common/tasks/config.yml:93-102). In ansible-core 2.16.20 the loop result is changed when any item changed (lib/ansible/executor/task_executor.py:124-130), every executed item carries the notification (:816-820), and the strategy saves the notifications of every item of a changed result (lib/ansible/plugins/strategy/__init__.py:648-673): a change for one container restarts all three on that host CODE. - F16: a common container whose systemd unit was written by 2023.1 code is reported as
changed at its first 2024.1 check, and the fluentd container also differs by its new
/var/log/journalvolume (roles/common/defaults/main.yml:124) CODE. - Q10 compares Ansible 2.8.20 and 2.9.27 and does not cover ansible-core 2.16.20; for 2024.1, F1 and F2 are settled by the code cited in the items above.
- F6, "Global inputs are rendered into per-host files": for
common, theenable_*flags of the OpenStack services (fluentd_input_openstack_services,roles/common/defaults/main.yml:154-210) and the logrotate services list (roles/common/tasks/config.yml:212-261) end up intd-agent.confandlogrotate.confon every common host CODE. New flags in 2024.1:enable_redis(defaultno,group_vars/all.yml:840) changeslogrotate.confon every cron host (config.yml:254), nottd-agent.conf(redis is not in the fluentd list). The Octavia precheck requires it when the jobboard is on (roles/octavia/tasks/precheck.yml:67-72), and the jobboard is on by default with Octavia and the amphora driver (group_vars/all.yml:832);enable_letsencrypt(defaultno,group_vars/all.yml:796) adds the11-letsencrypt.conf.j2input totd-agent.confon every fluentd host (config.yml:166-167). Acron-logrotate-letsencrypt.conf.j2template exists, butletsencryptis not in the logrotate list (config.yml:212-261), so it is never rendered;enable_ovn(group_vars/all.yml:834) adds OVN log inputs totd-agent.conf, but only on the hosts of theovn-controller,ovn-nb-db,ovn-northdandovn-sb-dbgroups (roles/common/templates/conf/input/10-openvswitch.conf.j2:31-67).
- OVN read positions: the OVN inputs write their positions to
/var/run/fluentd/, hard-coded, while every other input uses/var/run/<fluentd_user>/(roles/common/templates/conf/input/10-openvswitch.conf.j2:36,54,:4,19). With the 2024.1 image both are/var/run/fluentd(kolladocker/fluentd/Dockerfile.j2:3,41-42); with an image whose labels are missing,fluentd_userfalls back totd-agent(roles/common/tasks/config.yml:65-71) and the two paths differ CODE. What fluentd then does is NOT AVAILABLE. - What is not known: the sources do not
establish whether restarts that touch none of the containers the study traces to running VMs (the
commoncontainers, the control-plane services) affect running VMs, APIs or traffic another way. - What is not known,
row "fluentd": the sources do not establish whether the tail read positions (in
/var/run/fluentd/with the 2024.1 image, on no volume,roles/common/templates/conf/input/00-global.conf.j2:12,roles/common/defaults/main.yml:118-124) survive a config-only restart, and whether they are lost on a recreate. - Journal input: with
enable_fluentd_systemd,config.jsonaskskolla_set_configsto chown/var/log/journalrecursively to the fluentd user (roles/common/templates/fluentd.json.j2:27-33), while the volume is mounted read-only (roles/common/defaults/main.yml:124). A failed chown is logged and does not stop the start (kolladocker/base/set_configs.py:351-355) CODE; whether fluentd can then read the journal is NOT AVAILABLE. - Kolla images §6, item 9: a changed
rabbitmq_cluster_cookierecreates the brokers but their cookie file keeps the old value, whilekolla_toolboxgets the new one CODE (consequences NOT AVAILABLE).
Service card¶
The rows of the card are explained in How to read a service card.
| Item | Value |
|---|---|
| Play | site.yml:88 |
| Tags | common |
| Play host patterns | cron, fluentd, kolla-logs, kolla-toolbox |
| Node types in play (example inventory) | compute, control, monitoring, network, storage |
| Node types running its containers | compute, control, monitoring, network, storage |
| Containers | fluentd, kolla_toolbox, cron |
deploy/reconfigure flow (tasks/) |
config → check-containers → bootstrap → flush handlers |
| Task files | bootstrap.yml, check-containers.yml, check.yml, config.yml, config_validate.yml, copy-certs.yml, deploy-containers.yml, deploy.yml, main.yml, precheck.yml, pull.yml, reconfigure.yml, stop.yml, upgrade.yml |
| Templates | admin-openrc-system.sh.j2, admin-openrc.sh.j2, clouds.yaml.j2, conf/, conf/, conf/, conf/, conf/, conf/, conf/, conf/, conf/, conf/, conf/, conf/, conf/, conf/, conf/, conf/, conf/, conf/, cron-logrotate-ansible.conf.j2, cron-logrotate-aodh.conf.j2, cron-logrotate-barbican.conf.j2, cron-logrotate-blazar.conf.j2 … (+57) |
| Handlers (definition order) | Restart fluentd container; Restart kolla-toolbox container; Initializing toolbox container using normal user; Restart cron container |
| Key variables | enable_fluentd, enable_* (all, via td-agent/, elasticsearch_address, enable_monasca |
| Role dependencies | none |
Related¶
- Apply
commonchanges (fluentd, cron, kolla_toolbox) — the commands that pushcommonchanges, what restarts and how to verify - Chrony —
-t chrony, the Ussuri way to pushcommonchanges; no chrony role or play in 2024.1 - Tags §2.1 — the tag semantics that make
-t commonwork in 2024.1 - Repository architecture §5.1 —
the
common_runguard (gone since 2023.1) and the globalfluentdandcronconfigs - Findings F16 — the first 2024.1 check of a container whose unit 2023.1 wrote
- Why a small change creates a large workload —
enable_<new service>: yeschanges fluentd/logrotate on every host - Components — the other component pages