Skip to content

← openstack-vm-runbooks

Logs: where they are and how to read them

How to find, read and interpret the logs of OpenStack services and of the system around them (infrastructure services, hypervisor, kernel), so that an operator can go from a symptom to the line that explains it. Each page cites the upstream documentation or source code it comes from; real log lines come from lab runs and are redacted.

These pages explain how to read; the fixes are in the runbooks of the other folders, which the pages link to.

Pages

Page Covers Releases Status
Where the OpenStack logs are, and how to read one log line Kolla-Ansible and package installs, the oslo.log line format, request ids, levels, debug on and off 2026.1 lab-verified
Reading the Keystone logs, and the authentication lines every service writes Keystone's refused logins and key repository lines, keystonemiddleware's 401 and 503 paths, what default log levels hide 2026.1 draft
Reading the Cinder logs cinder-volume driver start and heartbeat, create failures, cinder-scheduler filtering, os-brick iSCSI lines on the compute 2026.1 draft
Following one request, instance, port or volume through the logs of every service and host Which id links which hop, grep and journalctl recipes across hosts, the port-id chain of network-vif-plugged, time zones and clock offset 2026.1 draft
Reading the MariaDB and Galera logs, and the oslo.db lines that go with them Error log format, WSREP state changes, quorum and membership, IST and SST, client errors, oslo.db exceptions 2026.1 lab-verified
Reading the Glance logs, and the image lines in Nova's logs Store refusals at upload (full, quota, permissions), Nova's download and snapshot lines 2026.1 draft
Reading the Nova logs. What normal looks like, and what the common lines mean nova-api, nova-scheduler, nova-conductor, nova-compute: start-up, periodic lines, common error lines and what to read next 2026.1 draft
Reading the HAProxy, keepalived and memcached logs Backend DOWN and UP lines, 503s and termination states in request lines, VRRP state changes and the check_alive script, memcached counters 2026.1 lab-verified
Reading the Octavia logs Amphora heartbeats, stale amphorae and failover batches, slow heartbeat processing, worker connection retries 2026.1 draft
Reading the RabbitMQ logs, and the oslo.messaging lines that go with them Broker log format, Khepri or Mnesia, nodes down and partitions, memory and disk alarms, flow control, connection churn and heartbeats, client reconnect lines 2026.1 lab-verified
Reading the Placement logs, and the Placement lines in Nova's logs The request line, 409s and their warnings (over capacity, unit constraints, retries), the report client's lines with Placement request ids 2026.1 draft
Reading the Heat logs Stack and resource failure lines, the ResourceFailure status reason, following the physical id into the service that failed 2026.1 draft
Reading the Neutron logs with ML2/OVS neutron-server, RPC server and periodic workers, OVS, L3, DHCP and metadata agents: heartbeats, port binding, the vif-plugged notification, common lines 2026.1 draft
Reading the OVN logs with Neutron ovn-controller port claims, database connections and probes, ovn-northd's active lock, SB relays, the Neutron OVN driver's binding and status lines 2026.1 draft
Reading the kernel log of a compute (dmesg, journalctl -k) journalctl -k and dmesg, then the OOM killer, KVM, NIC link flaps, bonding and LACP, over-MTU drops, disk and filesystem errors, conntrack table full, hung tasks 2026.1 draft
Reading the libvirt and QEMU logs of a compute, and the guest console log libvirtd and virtqemud logs, filters and debug on and off, the per-instance QEMU log (start, migration, shutdown reasons, QEMU and KVM messages), the guest console log 2026.1 draft
Reading the Open vSwitch logs (ovs-vswitchd, ovsdb-server) and changing their levels with ovs-appctl Where the logs are, the vlog line format, messages about ports, controller and database connections and an overloaded daemon, ovs-appctl vlog/list and vlog/set 2026.1 draft
Reading the systemd journal and the container engine on an OpenStack host journalctl units, priorities, time ranges and boots, the kolla-<container>-container units, restart loops as systemd and Docker show them, Docker and Podman logs, a persistent journal 2026.1 lab-verified
Reading the time synchronisation logs (chrony, systemd-timesyncd), and how clock skew shows in OpenStack logs chronyc tracking and sources, chrony and timesyncd log lines, and the Nova heartbeat, Neutron agent registration and Keystone token expiry lines a clock offset produces 2026.1 draft
Reading the storage logs of a compute. iscsid, multipathd, NFS client messages and Ceph client (librbd) logs iscsid and kernel iSCSI lines with error codes and timers, multipathd path and queueing lines, NFS not responding lines, the Ceph client admin socket and log on computes 2026.1 draft
Error messages: scheduling No valid host was found (no allocation candidates, a filter returned 0 hosts) and Exceeded maximum number of retries (the computes that tried): meaning, causes, next log, runbook 2026.1 draft
Error messages: messaging and database MessagingTimeout, AMQP unreachable, closed connection and heartbeat lines, Nova conductor lines, DBConnectionError by MySQL code, SQL connection retries, DBDeadlock: meaning, causes, next log, runbook 2026.1 draft

Planned

  1. More error-message dictionaries (authentication, compute and libvirt, networking), in the format of the two above.