Neutron¶
This page is the starting point for Neutron, the neutron role: neutron_server and fifteen other containers (the
agents, ironic_neutron_agent and neutron_tls_proxy, roles/neutron/defaults/main.yml:2-210), on the compute,
control and network nodes of the example inventory, selected by the tag neutron. The neutron play includes every
compute and manila-share (site.yml:671-693), and Open vSwitch is not touched by -t neutron: it has a separate
play and tag (site.yml:611-624; Reconfigure Neutron). common is not run by
-t neutron: it is its own play, tagged common (site.yml:88-104). The lists below say where the study covers how
to operate it, its impact, its dependencies and the evidence, what to check around a run and what stays open; the
service card follows them.
Where to look¶
Operate¶
- Scenario: Reconfigure Neutron
- Command: Reconfigure Neutron §1.1
Impact and risk¶
- What the tag executes (
neutron): Tags §3 and Tags §3.1, rowsneutron - Risks: R5, R6, R7; membership changes: R3, R4
Dependencies¶
- Dependencies: Component dependencies §3, Component dependencies §4
Evidence¶
- Fact sheet: –
- Image side: Kolla images
§4 (row
neutron-server), Kolla images §5, Kolla images §5.3 - Also: Reconfigure Neutron; handlers: Tags §5; the container module: Repository architecture §6
Before and after a run¶
- Pre-execution checklist, step 5 (health gates): before the run, check
openstack compute service listandopenstack network agent listall up. Repeat after each batch. - Tags §3.1, row
neutron: safe on its own "risky without batching" (R5, R6); prerequisites: Open vSwitch or OVN running, MariaDB, RabbitMQ, Keystone. - Agents on computes, still holds in 2024.1 CODE: with the default
neutron_plugin_agent: "openvswitch"(group_vars/all.yml:325), every compute runsneutron_openvswitch_agent(roles/neutron/defaults/main.yml:26-46). Withenable_neutron_dvr(defaultno,group_vars/all.yml:813) computes also runneutron_l3_agentandneutron_metadata_agent(roles/neutron/defaults/main.yml:85-89, :122-126); with SR-IOV or Mellanox enabled, theneutron_sriov_agent,neutron_mlnx_agentandneutron_eswitchdcontainers run on computes only (:93-114). - R5, still narrow in 2024.1: an
ml2_conf.inioverride reaches onlyneutron_server(andneutron_infoblox_ipam_agent), plus, through loop-notify in the check step, the other neutron containers of a host that also runs neutron-server (roles/neutron/tasks/config.yml:159-179,check-containers.yml:2-19); put agent options inneutron/openvswitch_agent.ini(config.yml:198-213) and check the rendered diff. - R6, precautions for any restart of
neutron_l3_agentwithenable_neutron_agent_ha: yes: restart network nodes one at a time; fail routers over first. The handler is unchanged in 2024.1 apart from the module name (kolla_container): stopped L3 agents are started first, then each host loops over the hosts with a running L3 agent and restarts its own container on its own item, with a pause ofneutron_l3_agent_failover_delayseconds (default0) between items (roles/neutron/handlers/main.yml:90-157,roles/neutron/defaults/main.yml:651; ansible-corelib/ansible/executor/task_executor.py:318-322) CODE. - R3, precautions for a change of RabbitMQ group membership, password or port: plan as a
maintenance; batch with
--limit; verify RabbitMQ cluster health first. - R4, precautions for a change of memcached membership: plan as a dataplane change, like
R3: batch with
--limitorANSIBLE_SERIAL, network nodes one at a time; low-traffic window. - R12, precautions for changing
openstack_tag/*_tagand runningdeployinstead ofupgrade: usepullthenupgrade; per-service tags;ANSIBLE_SERIAL. - Reconfigure Neutron §2, still holds in 2024.1: the
bootstrap step always runs the Neutron DB and user/grant tasks (skipped only with
use_preconfigured_databases) and thebootstrap_neutronDB-sync container on neutron-server[0] on everydeployandreconfigure(roles/neutron/tasks/bootstrap.yml:2-38,bootstrap_service.yml:2-21, nowhen) CODE, so MariaDB must be reachable. - Host settings, changed in 2024.1: the L3-host
gc_threshsysctl values now go through the newsysctlrole, which runs its own IPv6 check and skips thenet.ipv6.*keys when IPv6 is disabled (roles/neutron/tasks/config-host.yml:20-34,roles/sysctl/tasks/main.yml:1-20). The neutron role's ownCheck IPv6 supporttask still runs on every host of the play, and no task of the role reads its result any more (config-host.yml:15-18) CODE (Reconfigure Neutron §2). - FWaaS and TaaS, new in 2024.1:
enable_neutron_fwaasandenable_neutron_taas(both defaultno,group_vars/all.yml:814, :822) addfwaas_driver.ini(neutron-server and L3 agents) andneutron_taas.conf(neutron-server and the Open vSwitch agents, so every compute) (roles/neutron/tasks/config.yml:327-347, :526-547), a service plugin inneutron.conf(roles/neutron/defaults/main.yml:745-775,templates/neutron.conf.j2:49) and a subproject thatbootstrap_neutronmigrates (defaults/main.yml:674-686, :727) CODE. Turning either on therefore restarts every neutron container that rendersneutron.conf(Reconfigure Neutron §3). - DNS domain, changed in 2024.1:
neutron.confalways rendersdns_domain = {{ neutron_dns_domain }}(defaultopenstacklocal), andexternal_dns_driver = designateonly withenable_designateandneutron_dns_integration(defaultenable_designate) (roles/neutron/templates/neutron.conf.j2:64-76,roles/neutron/defaults/main.yml:956-958) CODE. 2023.1 wrotedns_domainonly with Designate, fromdesignate_ns_record. The new precheck fails when DNS integration is on andneutron_dns_domainis empty, lacks a final.or isopenstacklocal(roles/neutron/tasks/precheck.yml:57-68); it runs only inprechecks, not inreconfigureCODE. - Reconfigure Neutron §5, verification: before,
openstack network agent list(all alive), identify the network nodes that host HA router masters, and render withgenconfig -t neutronand diffopenvswitch_agent.iniandneutron.confon a compute (if either changed, every compute will restart its agent). Apply: network nodes one by one (--limit network01, thennetwork02), computes in batches. After: the agent list, anip netnscount on network nodes that matches the number of routers/networks, and a continuous ping of a VM's floating IP during each batch.
Known limits and open points¶
- Known bugs and limitations, row
neutron. Still holds in 2024.1:upgradewith the defaultneutron_enable_rolling_upgrade: "yes"expands/contractsneutronplus every enabled subproject ofneutron_subprojects, the same list the bootstrap container migrates; changed in 2024.1, the list now also holds neutron-fwaas and tap-as-a-service (roles/neutron/defaults/main.yml:674-686, :727, :733). Thehas_offline_migrationscheck loops over the enabled subprojects only, neverneutronitself: any non-zero rc (or a failedexec,ignore_errors) stopsneutron_serveron every neutron-server host in the play until the handler flush after the contract container, and with no subproject enabled no server is stopped beforeupgrade --contract(roles/neutron/tasks/rolling_upgrade.yml:38-89) CODE. Impact on the Neutron API, and whether a neutron contract migration is safe against running servers: NOT AVAILABLE (Kolla images §6, item 8). - New in 2024.1:
registerandupgradegrant the neutron user theservicerole in theserviceproject (roles/neutron/tasks/register.yml:8,upgrade.yml:10-15,roles/neutron/defaults/main.yml:923-926,roles/service-ks-register/tasks/main.yml:103-117) CODE. - Q2: "Data-plane loss during a
neutron_openvswitch_agentrestart". Status: NOT AVAILABLE. - Q4: "L3 agent restart with and without
enable_neutron_agent_ha". Status: NOT AVAILABLE. - Q14, answered from the source CODE: in 2024.1 neutron-server's
KOLLA_BOOTSTRAPandKOLLA_UPGRADEbranches both migrateneutron, then each subproject passed inNEUTRON_BOOTSTRAP_SERVICES/NEUTRON_ROLLING_UPGRADE_SERVICES, which are the same enabled-subproject list (kolladocker/neutron/neutron-server/extend_start.sh:5-34;roles/neutron/defaults/main.yml:727, :733). Changed in 2024.1: neutron-fwaas is back in the list, and tap-as-a-service is new (roles/neutron/defaults/main.yml:674-686); the kolla images installtap-as-a-servicein neutron-base and carry aneutron-fwaasplugin source (kolladocker/neutron/neutron-base/Dockerfile.j2:59-63,kolla/common/sources.py:224-227); whether a given built image contains them is NOT AVAILABLE. Agent, OVS and OVN images have no one-shot branch (noKOLLA_BOOTSTRAP/KOLLA_UPGRADEin kolladocker/neutron/*,docker/openvswitch/*,docker/ovn/*other than neutron-server). - What is not known, group "Network data plane": what the
sources do not establish about a restart of
neutron_openvswitch_agent,neutron_l3_agent,neutron_dhcp_agent,neutron_metadata_agentandneutron_server, about the processes that agent containers spawn, and about a change ofneutron_legacy_iptables. - What is not known: the sources do not
establish whether
neutron-db-manage … upgrade head, run by a tag change applied withdeploy, applies contract migrations under live old-version servers, and what that does to them. - What is not known:
the sources do not establish whether services started by a shell wrapper that does not
execthem (neutron-l3-agentamong them,roles/neutron/templates/neutron-l3-agent-wrapper.sh.j2:16-20) receive SIGTERM at all before the container stops. - Update container images §4: with a tag change applied by
deployinstead ofupgrade,bootstrap_neutronrunsneutron-db-manage … upgrade headwith the new image before the handlers restart the oldneutron_servercontainers (roles/neutron/tasks/deploy.yml:13-23) CODE. - F1, "Loop-notify amplification": a change on a network node restarts all neutron agents
there (
roles/neutron/tasks/check-containers.yml:2-19) CODE. - F5, "
reconfigure=deploy": still holds in 2024.1:reconfigure.ymlimportsdeploy.yml, andbootstrap_neutronhas nowhen, so a reconfigure re-runs Neutron's DB create/grant tasks and its DB sync;neutron_enable_rolling_upgradeis read only byupgrade.yml:5, :8(roles/neutron/tasks/reconfigure.yml:2,bootstrap_service.yml:2-21) CODE. - F6, "Global inputs are rendered into per-host files":
enable_*flags, RabbitMQ/memcached membership, service passwords and the external FQDN end up in compute-side files;neutron.conf, with the transport URLs and the memcached list, is rendered for the Open vSwitch agent too (roles/neutron/templates/neutron.conf.j2:58, :123, :126,roles/neutron/tasks/config.yml:87-122) CODE. - F8, "
--limitdoes not confine actions": withdeploy --limitset to a compute host,control01runs the Neutron DB create/grant and thebootstrap_neutroncontainer (roles/neutron/tasks/bootstrap.yml:13-14, :33-34,bootstrap_service.yml:20-21) CODE. - F11, "The
openvswitch_agent.inimerge": no longer applies, as in 2023.1. The merge reads onlyopenvswitch_agent.ini.j2and theneutron/openvswitch_agent.inioverrides, andml2_conf.iniis rendered only for neutron-server and neutron-infoblox-ipam-agent, so server-side ML2 changes do not restart the OVS agents (roles/neutron/tasks/config.yml:159-179, :198-213) CODE. - F12, "The L3 agent wrapper runs
neutron-netns-cleanup --force --agent-type l3at start" whenenable_neutron_agent_ha: yes; changed in 2024.1, the cleanup and the agent also readfwaas_driver.iniwhen FWaaS is enabled, and the agent readsneutron_vpnaas.confonly when VPNaaS is enabled (roles/neutron/templates/neutron-l3-agent-wrapper.sh.j2:5-20) CODE; NOT AVAILABLE whether an L3 agent restart interrupts router traffic on that node. - F13, "The Nova deploy order" puts nova-compute before OVS and the Neutron agents on
new computes (
site.yml:609importsnova.ymlbefore the openvswitch play at:611and the neutron play at:671) CODE; NOT AVAILABLE whether instances can be scheduled on such a compute before the later plays finish. - F15, "What the bootstrap containers run differs per image": the bootstrap container
and the rolling
upgrademigrate the same enabled subprojects, neutron-dynamic-routing included whenenable_neutron_bgp_dragentis set and, new in 2024.1, neutron-fwaas and tap-as-a-service when enabled (roles/neutron/defaults/main.yml:674-686, :727, :733) CODE. - Component dependencies §4, "Runtime
facts" CODE: the L3 agent wrapper runs a namespace cleanup on start;
/run/netnsis shared with the host (roles/neutron/defaults/main.yml:514-523); the OVS bridge layout is set in a post-config step of theopenvswitchrole. - Kolla images §6, items 8 and 12: the contract-step traps of the
Neutron upgrade CODE; the L3 agent wrapper does not
execthe agent, and NOT AVAILABLE whether SIGTERM then reaches only bash, so that the agent is killed without a graceful shutdown.
Service card¶
The rows of the card are explained on the Components page.
| Item | Value |
|---|---|
| Play | site.yml:671 |
| Tags | neutron |
| Play host patterns | neutron-server, neutron-dhcp-agent, neutron-l3-agent, ironic-neutron-agent, neutron-metadata-agent, neutron-ovn-metadata-agent, neutron-metering-agent, neutron-bgp-dragent, neutron-infoblox-ipam-agent, compute, manila-share |
| Node types in play (example inventory) | compute, control, network |
| Node types running its containers | compute, control, network |
| Containers | neutron_server, neutron_openvswitch_agent, neutron_linuxbridge_agent, neutron_dhcp_agent, neutron_l3_agent, neutron_sriov_agent, neutron_mlnx_agent, neutron_eswitchd, neutron_metadata_agent, neutron_ovn_metadata_agent, neutron_bgp_dragent, neutron_infoblox_ipam_agent, neutron_metering_agent, ironic_neutron_agent, neutron_tls_proxy, neutron_ovn_agent |
deploy/reconfigure flow (tasks/) |
neutron_plugin_agent_check → register → clone (if) → config-host → config → check-containers → config-neutron-fake (if) → bootstrap → flush handlers |
| Task files | bootstrap.yml, bootstrap_service.yml, check-containers.yml, check.yml, clone.yml, config-host.yml, config-neutron-fake.yml, config.yml, config_validate.yml, copy-certs.yml, deploy-containers.yml, deploy.yml, legacy_upgrade.yml, loadbalancer.yml, main.yml, neutron_plugin_agent_check.yml, precheck.yml, pull.yml, reconfigure.yml, register.yml, rolling_upgrade.yml, stop.yml, upgrade.yml |
| Templates | bgp_dragent.ini.j2, dhcp_agent.ini.j2, dnsmasq.conf.j2, eswitchd.conf.j2, fwaas_driver.ini.j2, ironic-neutron-agent.json.j2, ironic_neutron_agent.ini.j2, l3_agent.ini.j2, linuxbridge_agent.ini.j2, metadata_agent.ini.j2, metering_agent.ini.j2, ml2_conf.ini.j2, mlnx_agent.ini.j2, neutron-bgp-dragent.json.j2, neutron-dhcp-agent.json.j2, neutron-eswitchd.json.j2, neutron-infoblox-ipam-agent.json.j2, neutron-l3-agent-wrapper.sh.j2, neutron-l3-agent.json.j2, neutron-linuxbridge-agent.json.j2, neutron-metadata-agent.json.j2, neutron-metering-agent.json.j2, neutron-mlnx-agent.json.j2, neutron-openvswitch-agent-xenapi.json.j2, neutron-openvswitch-agent.json.j2 … (+14) |
| Handlers (definition order) | Restart neutron-server container; Restart neutron-openvswitch-agent container; Restart fake neutron-openvswitch-agent container; Restart neutron-linuxbridge-agent container; Restart neutron-dhcp-agent container; Get container facts; Group hosts; Start stopped neutron-l3-agent container; Wait if container starting; Restart running neutron-l3-agent container; Restart neutron-sriov-agent container; Restart neutron-mlnx-agent container; Restart neutron-eswitchd container; Restart neutron-tls-proxy container; Restart neutron-metadata-agent container; Restart neutron-ovn-metadata-agent container; Restart neutron-bgp-dragent container; Restart neutron-infoblox-ipam-agent container; Restart neutron-metering-agent container; Restart ironic-neutron-agent container; Restart neutron-ovn-agent container |
| Key variables | enable_neutron, neutron_plugin_agent, enable_neutron_dvr, enable_neutron_agent_ha, enable_neutron_provider_networks, neutron_enable_rolling_upgrade |
| Role dependencies | none |
Related¶
- Reconfigure Neutron — what
-t neutron reconfigureruns on controllers, network nodes and every compute - Open vSwitch —
openvswitch_dbandopenvswitch_vswitchdforward all VM traffic and have their own play and tag - OVN — the roles used if
neutron_plugin_agent: ovn - Change control-plane membership — in the default inventory a
controller change (RabbitMQ and memcached membership) restarts all nova and neutron containers on all computes; a
memcached-only change still restarts the neutron agents on computes - Update container images §4 — what a tag change applied with
deployruns against the Neutron DB - Pre-execution checklist — the checks before every production run
- Components — the other component pages