Logs: where they are and how to read them¶
How to find, read and interpret the logs of OpenStack services and of the system around them (infrastructure services, hypervisor, kernel), so that an operator can go from a symptom to the line that explains it. Each page cites the upstream documentation or source code it comes from; real log lines come from lab runs and are redacted.
These pages explain how to read; the fixes are in the runbooks of the other folders, which the pages link to.
Pages¶
| Page | Covers | Releases | Status |
|---|---|---|---|
| Where the OpenStack logs are, and how to read one log line | Kolla-Ansible and package installs, the oslo.log line format, request ids, levels, debug on and off | 2026.1 | lab-verified |
| Reading the Keystone logs, and the authentication lines every service writes | Keystone's refused logins and key repository lines, keystonemiddleware's 401 and 503 paths, what default log levels hide | 2026.1 | draft |
| Reading the Cinder logs | cinder-volume driver start and heartbeat, create failures, cinder-scheduler filtering, os-brick iSCSI lines on the compute | 2026.1 | draft |
| Following one request, instance, port or volume through the logs of every service and host | Which id links which hop, grep and journalctl recipes across hosts, the port-id chain of network-vif-plugged, time zones and clock offset | 2026.1 | draft |
| Reading the MariaDB and Galera logs, and the oslo.db lines that go with them | Error log format, WSREP state changes, quorum and membership, IST and SST, client errors, oslo.db exceptions | 2026.1 | lab-verified |
| Reading the Glance logs, and the image lines in Nova's logs | Store refusals at upload (full, quota, permissions), Nova's download and snapshot lines | 2026.1 | draft |
| Reading the Nova logs. What normal looks like, and what the common lines mean | nova-api, nova-scheduler, nova-conductor, nova-compute: start-up, periodic lines, common error lines and what to read next | 2026.1 | draft |
| Reading the HAProxy, keepalived and memcached logs | Backend DOWN and UP lines, 503s and termination states in request lines, VRRP state changes and the check_alive script, memcached counters | 2026.1 | lab-verified |
| Reading the Octavia logs | Amphora heartbeats, stale amphorae and failover batches, slow heartbeat processing, worker connection retries | 2026.1 | draft |
| Reading the RabbitMQ logs, and the oslo.messaging lines that go with them | Broker log format, Khepri or Mnesia, nodes down and partitions, memory and disk alarms, flow control, connection churn and heartbeats, client reconnect lines | 2026.1 | lab-verified |
| Reading the Placement logs, and the Placement lines in Nova's logs | The request line, 409s and their warnings (over capacity, unit constraints, retries), the report client's lines with Placement request ids | 2026.1 | draft |
| Reading the Heat logs | Stack and resource failure lines, the ResourceFailure status reason, following the physical id into the service that failed | 2026.1 | draft |
| Reading the Neutron logs with ML2/OVS | neutron-server, RPC server and periodic workers, OVS, L3, DHCP and metadata agents: heartbeats, port binding, the vif-plugged notification, common lines | 2026.1 | draft |
| Reading the OVN logs with Neutron | ovn-controller port claims, database connections and probes, ovn-northd's active lock, SB relays, the Neutron OVN driver's binding and status lines | 2026.1 | draft |
| Reading the kernel log of a compute (dmesg, journalctl -k) | journalctl -k and dmesg, then the OOM killer, KVM, NIC link flaps, bonding and LACP, over-MTU drops, disk and filesystem errors, conntrack table full, hung tasks | 2026.1 | draft |
| Reading the libvirt and QEMU logs of a compute, and the guest console log | libvirtd and virtqemud logs, filters and debug on and off, the per-instance QEMU log (start, migration, shutdown reasons, QEMU and KVM messages), the guest console log | 2026.1 | draft |
| Reading the Open vSwitch logs (ovs-vswitchd, ovsdb-server) and changing their levels with ovs-appctl | Where the logs are, the vlog line format, messages about ports, controller and database connections and an overloaded daemon, ovs-appctl vlog/list and vlog/set | 2026.1 | draft |
| Reading the systemd journal and the container engine on an OpenStack host | journalctl units, priorities, time ranges and boots, the kolla-<container>-container units, restart loops as systemd and Docker show them, Docker and Podman logs, a persistent journal | 2026.1 | lab-verified |
| Reading the time synchronisation logs (chrony, systemd-timesyncd), and how clock skew shows in OpenStack logs | chronyc tracking and sources, chrony and timesyncd log lines, and the Nova heartbeat, Neutron agent registration and Keystone token expiry lines a clock offset produces | 2026.1 | draft |
| Reading the storage logs of a compute. iscsid, multipathd, NFS client messages and Ceph client (librbd) logs | iscsid and kernel iSCSI lines with error codes and timers, multipathd path and queueing lines, NFS not responding lines, the Ceph client admin socket and log on computes | 2026.1 | draft |
| Error messages: scheduling | No valid host was found (no allocation candidates, a filter returned 0 hosts) and Exceeded maximum number of retries (the computes that tried): meaning, causes, next log, runbook | 2026.1 | draft |
| Error messages: messaging and database | MessagingTimeout, AMQP unreachable, closed connection and heartbeat lines, Nova conductor lines, DBConnectionError by MySQL code, SQL connection retries, DBDeadlock: meaning, causes, next log, runbook | 2026.1 | draft |
Planned¶
- More error-message dictionaries (authentication, compute and libvirt, networking), in the format of the two above.