id: PG-23
category: compute
title: Live migration never completes?
situation: >-
  The instance stays `MIGRATING` far longer than its peers, then is `ACTIVE` again on the source. It writes
  memory faster than it can be copied.

diagram:
  type: branches
  head: [What you observe, It means]
  split: 50
  rows:
    - see: "`memory_remaining_bytes` falls steadily, slowly"
      means: A slow link
      sub: Big instance, slow or shared migration network
      tone: warn
    - see: "`memory_remaining_bytes` falls, then hovers"
      means: It will not converge
      sub: Memory is dirtied as fast as it is copied
      tone: stop
    - see: "Source log: `live migration not completed after <n> sec`"
      means: Nova's timeout fired
      sub: 800 s per GiB by default, disk included
      tone: mute
  note: The instance keeps running on the source the whole time. Its users see nothing until the end.

checks:
  - label: The migration record and its id
    hint: openstack server migration list --server <id>
  - label: Its progress, a few times
    hint: "server migration show <id> <mig-id> (compute API 2.24 or later)"
  - label: The size of the instance
    hint: openstack server show <id> -c flavor
  - label: When Nova gave up
    hint: grep "live migration not completed" nova-compute.log (source)

caution:
  - level: state-changing
    text: >-
      Force-complete pauses the instance until its memory is copied. Not every application tolerates it:
      agree it with the owner first.
  - level: know
    text: >-
      Auto-converge slows the instance down, with no guarantee. Post-copy completes, but a network break
      during it reboots the instance.

lesson: Watch whether the remaining memory goes down. If it hovers, waiting will not help.

runbook:
  path: migration/live-migration-never-completes-or-times-out.md
  status: draft
  release: "2026.1"
  also:
    - migration/moving-instances-off-a-host-for-maintenance.md

scope: >-
  The runbook is a draft, written from the Nova 2026.1 live migration guides and not reproduced on a lab. The
  timeout is Nova's default ([libvirt] live_migration_completion_timeout); a deployment may have changed it.
  The progress fields need compute API 2.24 or later. Auto-converge and post-copy must be allowed on all
  computes. A migration refused at once by a pre-check is a different case, as are PCI devices, vGPU and
  NUMA-pinned instances.

linkedin:
  title: "A live migration that never completes: watch one number"
  caption: |
    A live migration starts fine. Then it runs. And runs. Much longer than the others. Eventually the instance is ACTIVE again, on the source, and the migration is marked failed.

    Live migration copies memory while the instance keeps running. Every page written again must be copied again. If the instance dirties memory faster than the network copies it, the migration cannot converge.

    One number tells you which case you are in, memory_remaining_bytes:
    → it falls steadily, slowly: a slow or shared migration network
    → it falls, then hovers: it will not converge without help

    The ways out all have a price. Force-complete pauses the instance. Auto-converge throttles its CPU. Post-copy completes, but a network break during that phase reboots the instance.

    Source runbook status: draft, not yet reproduced on a lab.
  hashtags: [OpenStack, Nova, LiveMigration, Libvirt, CloudOperations, OzarLand]
