Keystone¶
This page is the starting point for Keystone, the keystone role: the containers keystone, keystone_ssh and
keystone_fernet, on the control nodes of the example inventory (inventory/multinode:111-112), selected by the tag
keystone (site.yml:450-458). Keystone is a hard deploy-time dependency of the 37 roles whose tasks use
service-ks-register (keystone itself included; grep of roles/*/tasks): registration runs run_once
(roles/service-ks-register/tasks/main.yml:136) with openstack_interface (internal) and authenticates against
keystone_internal_url; 2023.1 has no separate admin endpoint, keystone_admin_url is keystone_internal_url
(roles/service-ks-register/defaults/main.yml:5, group_vars/all.yml:609,864-865,895-896) CODE (Component
dependencies §1). The
lists below say where the study covers how to operate it, its impact, its dependencies and the evidence, what to check
around a run and what stays open; the service card follows them.
Where to look¶
Operate¶
- Scenario: Reconfigure a single service §2, Reconfigure a single service §3
- Command: Reconfigure a single service §1.1
Impact and risk¶
- What the tag executes (
keystone): Tags §3, row<svc>; Tags §3.1, row<svc>API service - Risks: R19, R8, R24
Dependencies¶
- Dependencies: Component dependencies §1, Component dependencies §3
Evidence¶
- Fact sheet: fact sheet B §3
- Image side: Kolla images
§4 (row
keystone), Kolla images §4.1, Kolla images §5 - Also: handlers: Tags §5
Before and after a run¶
- Tags §3.1, row
<svc>API service: safe on its own "yes, it is the intended unit; it still runs Keystone registration and DB sync"; batch withANSIBLE_SERIAL. Prerequisites: MariaDB, RabbitMQ, memcached, Keystone (registration), HAProxy VIP. - R19, precautions for a Keystone
reconfigure:ANSIBLE_SERIAL; verify fernet key consistency before and after. - fact sheet B
§3.9,
item 1: with the default serial 0, a reconfigure with a config change restarts
keystone_ssh, thenkeystone_fernet, thenkeystone, each on all keystone hosts simultaneously. SetANSIBLE_SERIALfor a rolling reconfigure. - R8, precautions for a HAProxy config change:
ANSIBLE_SERIAL=1for haproxy changes;--skip-tags haproxywhen you don't intend to change frontends. - R24, container stop timeout: in 2023.1 the stop goes through the container's systemd
unit with
docker stop -tdocker_graceful_timeout(60 s,group_vars/all.yml:140), no longer 10 s. Precautions: drain long operations before restarts; raisedocker_graceful_timeoutif needed. - R22, precautions for an inventory reordering (the keystone group is one of the groups it names): treat inventory order as configuration; never reorder casually.
- Reconfigure a single service
§3,
verification of
-t <svc> reconfigure: before,kolla-ansible -t <svc> genconfig, thenkolla_set_configs --checkin each<svc>container and a diff of/etc/kolla/haproxy/services.d/<svc>*.cfgagainst the previous version; after,openstack endpoint list --service <name>is unchanged for each Keystone service the role registers, the API answers through the VIP, the container start times show which hosts restarted, anddocker ps --filter name=haproxyshows whether HAProxy restarted.
Known limits and open points¶
- Known bugs and limitations, row
keystone: fernet cron generator hour calculation; precheck vs generator expiry limits; push script stops at the first unreachable peer. Status: all three unchanged in 2023.1 CODE: the generator computeshour = time % HOUR_SPAN(roles/keystone/files/fernet_rotate_cron_generator.py:69, the file is identical to Ussuri apart from a Python 2 import); the precheck assertsfernet_token_expiry <= 604800(roles/keystone/tasks/precheck.yml:43-53) while the generator is fedfernet_key_rotation_interval(roles/keystone/tasks/config.yml:195-200);fernet-push.shruns underset -o errexitand rsyncs to each peer in turn (roles/keystone/templates/fernet-push.sh.j2:3,17-22). - Q11: "Fernet key distribution failure modes". Status: NOT AVAILABLE.
- What is not
known, row
"Keystone": the sources do not establish whether authentication, and so the other services' API requests, fails while
keystone restarts or is down (R19); Fernet key distribution failure modes
(Q11); whether Keystone still reads a removed policy file after the
restart; the upstream semantics of
keystone-manage bootstrap; whether thekeystone_fernetcontainer restarts when key 0 is missing. - What is not known:
the sources do not establish whether the SSH clients accept the new
nova_sshandkeystone_sshhost keys after a recreate without failing. - F5, "
reconfigure=deploy": a reconfigure re-runs Keystone registration, DB creation/grants and the DB-sync bootstrap containers of most services CODE. - fact sheet B §3.9, "Operationally notable behaviour, risks and oddities", ten items: reconfigure with a config change; what restarts keystone indirectly; Fernet push has no partial-failure handling; bug in the cron generator hour calculation; the precheck validates the wrong variable; under-rotation by design; shared SSH key; domain config cleanup; upgrade; reconfigure re-asserts identity state.
- fact sheet B §3.10: NOT AVAILABLE: whether the API services whose
[keystone_authtoken]sections point keystonemiddleware at Keystone fail to authenticate requests while Keystone or memcached is unavailable. - Kolla images §5, row
keystone-fernet: without key 0 on the node the start waits up to 10 × 60 s and the script exits 1 before the cron command is exec'd, so no rotation cron runs there meanwhile CODE; whether the container then restarts is NOT AVAILABLE. - Kolla images §6, item 18: SSH host keys of
nova_sshandkeystone_sshchange on every recreate, because/etc/sshis on no volume CODE; whether the SSH client then accepts the new keys without failing is NOT AVAILABLE (OpenSSH semantics). - Kolla images §7: not available from the
sources: the upstream semantics of
keystone-manage bootstrap, in particular whether it resets the admin password tokeystone_admin_password.
Service card¶
The rows of the card are explained on the Components page.
| Item | Value |
|---|---|
| Play | site.yml:450 |
| Tags | keystone |
| Play host patterns | keystone |
| Node types in play (example inventory) | control |
| Node types running its containers | control |
| Containers | keystone, keystone_ssh, keystone_fernet |
deploy/reconfigure flow (tasks/) |
config → check-containers → clone (if) → bootstrap → flush handlers → distribute_fernet → register → register_identity_providers (if) |
| Task files | bootstrap.yml, bootstrap_service.yml, check-containers.yml, clone.yml, config-federation-oidc.yml, config.yml, config_validate.yml, copy-certs.yml, deploy-containers.yml, deploy.yml, distribute_fernet.yml, loadbalancer.yml, main.yml, precheck.yml, pull.yml, reconfigure.yml, register.yml, register_identity_providers.yml, stop.yml, upgrade.yml |
| Templates | crontab.j2, fernet-healthcheck.sh.j2, fernet-node-sync.sh.j2, fernet-push.sh.j2, fernet-rotate.sh.j2, keystone-fernet.json.j2, keystone-ssh.json.j2, keystone-startup.sh.j2, keystone.conf.j2, keystone.json.j2, ssh_config.j2, sshd_config.j2, wsgi-keystone.conf.j2 |
| Handlers (definition order) | Init keystone database upgrade; Restart keystone-ssh container; Restart keystone-fernet container; Restart keystone container; Finish keystone database upgrade |
| Key variables | enable_keystone, keystone_token_provider, fernet_token_expiry, fernet_key_rotation_interval |
| Role dependencies | none |
Related¶
- Reconfigure a single service —
-t keystone reconfigure, the command example of the page: scope, restarts and one-shot - HAProxy + Keepalived — the loadbalancer play renders the keystone frontend under
-t keystone(site.yml:198-201), and HAProxy restarts if it changed - Memcached — Keystone's token cache
- Component dependencies §1 — Keystone in the shared substrate that every OpenStack service uses
- fact sheet B §3.10 — the runtime dependency of other services on Keystone, counted by role
- Pre-execution checklist — the eight checks before every production run
- Components — the other component pages