在早期的OpenStack中 , 网络是由Nova的“nova-network”模块实现的。之后随着网络功能日趋复杂 , 一个叫作Quantum的新项目被创建来替代“nova-network”模块 , 它就是Neutron的前身。其后 , 由于发现Quantum这一名称已经被注册 , 因此项目被改名为Neutron。Neutron在OpenStack的H版本中首次出现。

1. Neutron 简介

Neutron是Openstack的网络组件 , Neutron的前身是Quantum , Quantom的前身是Nova-Network , 经过多个版本的演进 , Neutron现如今是openstack的重要组件之一。

在openstack的网络中 , 计算机机网络的相关概念稍有变动。在实际的计算机网络中 , 网络连接的是一个个的主机 , 而在openstack中 , 网络连接的是一个个的云主机。

neutron的主要作用是在openstack中为启动虚拟机实例提供网络服务 , 对于neutron来讲 , 它可以提供两种类型的网络 ; 第一种是provider network , 这种网络就是我们常说的桥接网络 , 虚拟机内部网络通常是通过bridge的方式直接桥接到宿主机的某块物理网卡上 , 从而实现虚拟机可以正常的访问外部网络 , 同时虚拟机外部网络也可以访问虚拟机的内部网络 ; 第二种是self-service networks , 这种网络就是nat网络 ; nat网络的实现是通过在虚拟机和宿主机之间实现了虚拟路由器 , 在虚拟机内部可以是一个私有地址连接至虚拟路由器的一个接口上 , 而虚拟路由器的另外一端通过网桥桥接到宿主机的某一张物理网卡 ; 所以nat网络很好的隐藏了虚拟机的地址 , 它能够实现虚拟机访问外部网络 , 而外网用户是不能够直接访问虚拟机的 ; 但在openstack中 , 它能够实现虚拟机和外部的网络做一对一nat绑定 , 从而实现从虚拟机外部网络访问虚拟机 ;

1.1 provide network

self-service network 示意图
self-service network 和provide network最大的区别是自服务网络中有虚拟路由器 ; 有路由器就意味着虚拟机要和外网通信 , 网络报文要走三层 , 而对于provide network 来讲 , 它的网络报文就可以直接走二层网络 ; 所以在openstack上这两种类型的网络实现方式和对应的组件也有所不同 ;
provide network 实现所需组件
Provider networks - Overview
Provider networks 连接示意图
桥接网络也叫共享网络 , 虚拟机实例网络是通过桥接的方式直接共享宿主机网络 ; 虚拟机和宿主机通信 , 就类似宿主机同局域网的其他主机通信一样 ; 所以虚拟机和宿主机通信报文都不会到三层 , 所以这里面就不涉及三层网络相关的操作和配置 ;

1.2 self-service network

self-service network实现所需组件
Self-service networks - Overview
Self-service networks连接示意图

对比上面两种网络的实现所需组件 , 我们可以发现self-service network的实现要比provide network要多一个networking L3 Agent插件 ; 这个插件用作实现3层网络功能 , 比如 , 提供或管理虚拟路由器 ; 从上面的两种网络连接示意图也可以看出 , self-service network是包含provide network , 也就是说我们选择使用self-service network这种类型的网络结构 , 我们即可以创建自服务网络 , 也可以创建桥接网络 ; 对于自服务网络来讲 , 我们在计算节点启动的虚拟机 , 虚拟机想要访问外部网络 , 它会通过计算节点的vxlan接口 , 这里的vxlan我们可以理解为在计算节点内部实现的虚拟交换机 , 各虚拟机实例通过连接不同的vni(网络标识符) , 类似vlan id一样)的vxlan来实现网络的隔离 , 同时vxlan这个虚拟接口通常是桥接在本地管理网络接口上 , 这个管理网络一般是不能够和外部网络通信 ; 虚拟机访问外部网络 , 通过vxlan接口实现的vxlan隧道 , 这个隧道是一头是和计算节点的管理网络接口连接 , 一头是和控制节点的管理网络接口连接 ; 虚拟机访问外部网络是通过vxlan隧道 , 再通过控制节点中的虚拟路由器 , 将请求通过路由规则 , 路由到控制节点能够上外网的接口上 , 然后发出去 , 从而实现虚拟机能够和外部网络进行交互 ; 而对于外部网络要访问虚拟机 , 在openstack上是通过一对一nat绑定实现 ; 也就说在控制节点能够上外网的接口上配置很多ip地址 , 这些IP地址都是可以正常访问外部网络的 , 在虚拟机访问外部网络时 , 在控制节点的虚拟机路由器上就固定的把计算节点的某个虚拟机的流量通过固定SNAT的方式进行数据发送 , 对于这个固定地址在控制节点上再做固定的DNAT , 从而实现外部网络访问控制节点上的这个固定ip , 通过DNAT规则把外部流量引入到虚拟机 , 从而实现外部网络和虚拟机通信 ;

1.3 neutron工作流程

neutron服务主要由neutron-server、neutron agents、neutron plugins这三个组件组成，这三者都依赖消息队列服务；其中neutron server主要用来接收用户的请求，比如创建或管理网络；当neutron server接收到客户端 ( openstack其他服务，如nova，neutron专有客户端 ) 请求后，它会把请求丢到消息队列中去，然后neutron agents负责从消息队列中取出客户端的请求，在本地完成网络创建或管理，并把对应的操作的结果写到neutron 数据库中进行保存；这里需要说明一点neutron agents是指很多agent，每个agent都负责完成一件事，比如DHCP agent负责分配ip地址，network manage agent负责管理网络；而对于neutron plugins 主要用来借助外部插件的方式提供某种服务；比如ML2 plugin用来提供2层虚拟网络服务的；如果neutron agents在创建或管理网络需要用到某个插件服务时，它会把请求插件的消息丢到消息队列，然后neutron plugins 从消息队列取出消息，并响应请求，把结果丢到消息队列，同时也会写到数据库中；

2. Neutron 功能

Neutron是OpenStack云计算平台的核心组件之一 , 负责虚拟网络设备的创建和管理。这些虚拟的网络设备包括网桥、网络、端口等。

虚拟网络设备

网桥 ( Bridge )  
#网桥类似于交换机 , 用于连接不同的网络设备。Neutron把网桥分为内部网桥 ( bridge-internal , br-int ) 和外部网桥 ( bridge-external , ex-int ) 两类。内部网桥即实现内部网络功能的网桥。外部网桥即负责跟外部网络通信的网桥。
网络 ( Network )  
#一个隔离的二层网段 , 类似于一个虚拟局域网 ( Virtual Local Area Network , VLAN ) 。网络之间是隔离的 , 不同网络中的IP地址可以重复。子网和端口都挂接在某个网络上。
子网 ( Subnet )  
#子网是一个IPv4 或者 IPv6段。子网中的云主机的IP地址从该地址段中进行分配。子网必须关联一个网络。网络与地址子网是一对多关系 , 一个子网只能属于某个网络 ; 一个网络可以有多个子网。
端口 ( Port ) 
#端口可以看作虚拟交换机上的一个端口。端口上定义了硬件物理地址 ( MAC地址 ) 和IP地址 , 当云主机的虚拟网卡 ( Virtual Interface , VIF ) 绑定到某个端口时 , 端口就会将MAC地址和IP地址分配给虚拟网卡。子网与端口是一对多关系 , 一个端口必须属于某个子网 ; 一个子网可以有多个端口。

3. Neutron架构

Neutron由对外提供服务的Neutron服务模块“neutron-server”、任意数量的插件“neutron-plugin”和与插件相对应的代理“neutron-agent”组成。

neutron-server : Neutron的服务模块 , 对外提供OpenStack网络API , 接收请求 , 并调用插件处理请求。
neutron-plugin : Neutron的插件对应某个具体功能 , 各个厂商可以开发自己的插件放入Neutron中。插件做的事情主要有两件：在数据库中创建资源和发送请求给具体的“neutron-agent”。
neutron-agent : Neutron的代理可以理解为插件在物理设备上的对应代理 , 插件要实现具体功能必须要通过代理。代理接收到“neutron-plugin”通知的业务操作和参数 , 然后在网络提供者 ( 如一块真实的网卡 ) 上实现各种网络功能 , 如创建网络、子网、网桥等。当设备发生问题时 , “neutron-agent”会将情况通知给“neutron-plugin” 。

Neutron的网络分层模型

开放系统互连 ( Open System Interconnection , OSI ) 参考模型定义了著名的七层网络模型 , Neutron在其第二层到第七层都提供了插件来支持各种不同的网络设备和网络服务。这些插件按照其功能分为两类：核心插件 ( Core-plugin ) 和服务插件 ( Service-pluging ) 。

核心插件
Neutron中的核心插件即为二层模块 ( Modular Layer 2 , ML2 ) , 它负责管理OSI第二层的网络连接。ML2中主要包括网络、子网、端口这3类核心资源。在Neutron服务中必须包括核心插件 , 因此在Neutron配置文件中必须配置ML2 , 否则无法启动Neutron服务。
服务插件
服务插件是除核心插件以外其他的插件的统称 , 主要实现OSI模型的第三层到第七层的网络服务。与核心插件不同的是 , 服务插件通常不会影响Neutron服务运行 , 因此在Neutron的配置文件中可以不用配置此类插件的信息。

4. Neutron 的基本工作流程

第1步 , “neutron-server”服务模块收到要创建虚拟网络的请求 , 然后将该请求通过消息队列通知给对应的“neutron-plugin”插件 , 假设网络提供者为开放虚拟交换机 ( Open vSwitch , OVS ) , 那么这里的插件对应的就是OVS的插件。
第2步 , 当OVS插件收到请求后 , 将需要创建的虚拟网络的信息 ( 名称、ID等 ) 保存到数据库中并通过消息队列通知运行在各个节点上的 “neutron-agent”代理。
第3步 , “neutron-agent”代理收到消息后会在节点上创建对应设备 , 例如VLAN设备。

5. Neutron支持的网络模式

Flat网络模式
扁平 ( Flat ) 网络模式在OSI模型第二层上用虚拟网桥建立了云主机网卡与物理网卡之间的联系 , 所有虚拟机的网卡IP地址和物理机的网卡IP地址同处一个网段中。
在Flat网络模式中 , 由于云主机和物理机属于同一个网段 , 因此云主机和物理机可以直接通信。其缺点是所有的云主机都在同一个网络内 , 没有进行网络隔离 , 容易产生广播风暴。而且每台虚拟机都要用到宝贵的局域网IP地址资源 , 这就注定Flat网络模式的网络中能容纳的云主机数量不会太多。
VLAN网络模式
VLAN是一种将局域网 ( Local Area Network , LAN ) 设备从逻辑上划分成更小的局域网。VLAN网络模块是指将若干云主机按逻辑划分属于不同的VLAN , 此时只有同一个VLAN中的虚拟机可以相互访问。
共享同一个物理网络的多个VLAN是相互隔离的 , 不同的VLAN默认不能相互访问 , 因此可以有效避免Flat网络模式中的广播风暴问题。如图物理网络“ens32.101”“ens34.102”中的“101”“102”被称为“VLAN ID” , 用于标注不同的网络。“VLAN ID”的有效值为0～4095 , 其中0和4095一般被保留不用 , 所以最多能标识4094个网络 , 故VLAN网络模式无法满足公有云超大规模用户的需求 , 只在私有云中应用较多。
VXLAN与GRE网络模式
虚拟扩展局域网 ( Virtual Extensible Local Area Network , VXLAN ) 和通用路由封装 ( Generic Routing Encapsulation , GRE ) 这两种延展网络 ( Overlay Network ) 。所谓延展网络 , 是指在其他网络上扩建而来的网络。
都是建立在隧道技术上的网络模型 , 只是VXLAN构建的隧道是“VXLAN Tunnel” , 而GRE构建的隧道是“GRE Tunnel”。与VLAN相比 , VXLAN与GRE支持更多的网段。VXLAN与GRE可以支持的网段数超过1600万 , 因此它们更适合用于公有云。

6. Neutron组件安装与配置

6.1 控制节点配置

6.1.1 环境准备

登陆数据库，并创建neutron用户，并授权

[root@node02 ~]# mysql -uroot -p
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 9234
Server version: 10.5.22-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> CREATE DATABASE neutron;
Query OK, 1 row affected (0.001 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'%'  IDENTIFIED BY 'neutron123';
Query OK, 0 rows affected (0.009 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.002 sec)

在控制端使用mysql测试是否能够连接

[root@node01 nova]# mysql -uneutron -pneutron123 -hnode02
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 9245
Server version: 10.5.22-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| neutron            |
+--------------------+
2 rows in set (0.002 sec)

要创建服务凭证，请完成以下步骤

创建neutron用户 ,导出admin环境变量，创建neutron用户，设置其密码为neutron

[root@node01 ~]# . keystone_admin.rc
[root@node01 ~]# openstack user create --domain default --password-prompt neutron
User Password:
Repeat User Password:
+---------------------+----------------------------------+
| Field               | Value                            |
+---------------------+----------------------------------+
| domain_id           | default                          |
| enabled             | True                             |
| id                  | 9392d1808831448291f0e565411ff97d |
| name                | neutron                          |
| options             | {}                               |
| password_expires_at | None                             |
+---------------------+----------------------------------+

将neutron用户添加至service项目，并授权为admin角色

[root@node01 ~]# openstack role add --project service --user neutron admin

创建neutron服务

[root@node01 ~]# openstack service create --name neutron --description "OpenStack Networking" network
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | OpenStack Networking             |
| enabled     | True                             |
| id          | c550130289a244a99f3c0f71801da742 |
| name        | neutron                          |
| type        | network                          |
+-------------+----------------------------------+

创建网络服务API端点

公共端点

[root@node01 ~]# openstack endpoint create --region RegionOne  network public http://openstack-vip.net:9696
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | 49566f6b5b7a4d959bd74334fe4d1b3d |
| interface    | public                           |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | c550130289a244a99f3c0f71801da742 |
| service_name | neutron                          |
| service_type | network                          |
| url          | http://openstack-vip.net:9696    |
+--------------+----------------------------------+

私有端点

[root@node01 ~]# openstack endpoint create --region RegionOne network internal http://openstack-vip.net:9696
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | d17b22d103c94a939cc64f2b33e3c1b3 |
| interface    | internal                         |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | c550130289a244a99f3c0f71801da742 |
| service_name | neutron                          |
| service_type | network                          |
| url          | http://openstack-vip.net:9696    |
+--------------+----------------------------------+

管理端点

[root@node01 ~]# openstack endpoint create --region RegionOne network admin http://openstack-vip.net:9696
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | 006e57626b6b44ab8f2e9e13edac8a9f |
| interface    | admin                            |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | c550130289a244a99f3c0f71801da742 |
| service_name | neutron                          |
| service_type | network                          |
| url          | http://openstack-vip.net:9696    |
+--------------+----------------------------------+

配置haproxy节点中的haproxy配置文件

listen openstack_neutron_port
    bind 192.168.71.250:9696
    mode tcp
    log global
    server 192.168.71.11 192.168.71.11:9696 check inter 3000 fall 3 rise 5

后续有两种网络:provider和self-service两种安装网络配置

6.1.2 配置Provider 网络

控制节点安装组件

[root@node01 ~]# yum install openstack-neutron openstack-neutron-ml2 openstack-neutron-openvswitch

配置服务组件

编辑 /etc/neutron/neutron.conf 文件并完成以下操作

在 [database] 部分，配置数据库访问权限：

[database]
# ...
connection = mysql+pymysql://neutron:neutron123@openstack-vip.net/neutron

在 [DEFAULT] 部分，启用模块化第二层(ML2)插件、路由器服务和重叠IP地址：

[DEFAULT]
# ...
core_plugin = ml2
service_plugins =

在 [DEFAULT] 部分，配置 RabbitMQ 消息队列访问权限：

[DEFAULT]
# ...
transport_url = rabbit://openstack:openstack123@openstack-vip.net

在 [DEFAULT] 和 [keystone_authtoken] 部分，配置身份服务访问权限：

[DEFAULT]
# ...
auth_strategy = keystone

[keystone_authtoken]
# ...
www_authenticate_uri = http://openstack-vip.net:5000
auth_url = http://openstack-vip.net:5000
memcached_servers = openstack-vip.net:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = neutron
password = neutron

在 [DEFAULT] 和 [nova] 部分，配置网络服务以通知计算服务网络拓扑变化：

[DEFAULT]
# ...
notify_nova_on_port_status_changes = true
notify_nova_on_port_data_changes = true

[nova]
# ...
auth_url = http://openstack-vip.net:5000
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = service
username = nova
password = nova

在 [oslo_concurrency] 部分，配置锁路径：

[oslo_concurrency]
# ...
lock_path = /var/lib/neutron/tmp

配置ml2插件
编辑配置文件/etc/neutron/plugins/ml2/ml2_conf.ini
在[ml2]配置段配置支持flat（平面网络），vlan和vxlan
```
[ml2]
# ...
type_drivers = flat,vlan
```
提示：配置ML2插件之后，删除type_drivers选项中的值可能会导致数据库不一致；意思是初始化数据库后，如果在删除上面的值，可能导致数据库不一致的情况；
在[ml2]配置段开启租户网络类型为``
```
[ml2]
# ...
tenant_network_types = 
```
在[ml2]配置段启用Linux桥接
```
[ml2]
# ...
mechanism_drivers = openvswitch
```
在[ml2]配置段中启用端口安全扩展驱动程序
```
[ml2]
# ...
extension_drivers = port_security
```
在[ml2_type_flat]配置段配置flat_networks = provider, 这里设置为external以作为区分
```
[ml2_type_flat]
# ...
flat_networks = external
```
这里主要是指定平面网络的名称，就是虚拟机内部网络叫什么名，这个名称可以自定义，但后面会用到把该网络桥接到物理网卡中的配置，以及后续的创建网络都要用到这名称，请确保后续的名称和这里的名称保持一致；

添加br-ex网桥

ovs-vsctl add-br br-ex
ovs-vsctl add-port br-ex ens4

配置Open vSwitch agent
Linux 桥接代理构建了用于实例的第二层（桥接和交换）虚拟网络基础设施，并处理安全组。
编辑 /etc/neutron/plugins/ml2/openvswitch_agent.ini 文件并完成以下操作：
在 [ovs] 部分，将提供商虚拟网络映射到提供商物理网络接口：
```
[ovs]
bridge_mappings = external:br-ex
#br-ex这个名称是自定义的，须与ovs-vsctl add-br 相关操作保持统一
```
提示：这里主要是配置把把虚拟机内部的那个网络和物理接口的桥接映射，请确保虚拟机内部网络名称和这里配置的保持一致；冒号前指定虚拟机内部网络名称，冒号后面指定要桥接的物理网卡接口名称；
在[securitygroup]配置段配置启用安全组并配置Linux bridge iptables防火墙驱动程序
```
[securitygroup]
# ...
enable_security_group = true
firewall_driver = openvswitch
```

确定br_netfilter内核模块是加载启用，若没加载，加载内核模块并配置相关内核参数

[root@node01 ~]# lsmod |grep br_netfilter
[root@node01 ~]# modprobe br_netfilter
[root@node01 ~]# lsmod |grep br_netfilter
br_netfilter           22209  0
bridge                136173  1 br_netfilter
[root@node01 ~]#

配置相关内核参数

[root@node01 ~]# cat /etc/sysctl.conf 
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
#
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1

[root@node01 ~]# sysctl -p
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1

编辑文件/etc/neutron/dhcp_agent.ini
在 [DEFAULT] 部分，配置 Linux 桥接接口驱动程序，Dnsmasq DHCP 驱动程序，并启用隔离的元数据服务，以便在提供商网络上的实例可以通过网络访问元数据：
```
[DEFAULT]

interface_driver = openvswitch
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
enable_isolated_metadata = true
```

6.1.3 配置self-service选项

安装neutron服务软件包

[root@node01 ~]# dnf install openstack-neutron openstack-neutron-ml2 openstack-neutron-openvswitch ebtables

配置服务组件

编辑 /etc/neutron/neutron.conf 文件并完成以下操作

在 [database] 部分，配置数据库访问权限：

[database]
# ...
connection = mysql+pymysql://neutron:neutron123@openstack-vip.net/neutron

在 [DEFAULT] 部分，启用模块化第二层(ML2)插件、路由器服务和重叠IP地址：

[DEFAULT]
# ...
core_plugin = ml2
service_plugins = router

在 [DEFAULT] 部分，配置 RabbitMQ 消息队列访问权限：

[DEFAULT]
# ...
transport_url = rabbit://openstack:openstack123@openstack-vip.net

在 [DEFAULT] 和 [keystone_authtoken] 部分，配置身份服务访问权限：

[DEFAULT]
# ...
auth_strategy = keystone

[keystone_authtoken]
# ...
www_authenticate_uri = http://openstack-vip.net:5000
auth_url = http://openstack-vip.net:5000
memcached_servers = openstack-vip.net:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = neutron
password = neutron

在 [DEFAULT] 和 [nova] 部分，配置网络服务以通知计算服务网络拓扑变化：

[DEFAULT]
# ...
notify_nova_on_port_status_changes = true
notify_nova_on_port_data_changes = true

[nova]
# ...
auth_url = http://openstack-vip.net:5000
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = service
username = nova
password = nova

在 [oslo_concurrency] 部分，配置锁路径：

[oslo_concurrency]
# ...
lock_path = /var/lib/neutron/tmp

配置ml2插件
编辑配置文件/etc/neutron/plugins/ml2/ml2_conf.ini
在[ml2]配置段配置支持flat（平面网络），vlan和vxlan
```
[ml2]
# ...
type_drivers = flat,vlan,vxlan
```
提示：配置ML2插件之后，删除type_drivers选项中的值可能会导致数据库不一致；意思是初始化数据库后，如果在删除上面的值，可能导致数据库不一致的情况；
在[ml2]配置段开启租户网络类型为vxlan
```
[ml2]
# ...
tenant_network_types = vxlan
```
在[ml2]配置段启用Linux桥接和二层填充机制
```
[ml2]
# ...
mechanism_drivers = openvswitch,l2population
```
在[ml2]配置段中启用端口安全扩展驱动程序
```
[ml2]
# ...
extension_drivers = port_security
```
在[ml2_type_flat]配置段配置flat_networks = provider, 这里设置为external以作为区分
```
[ml2_type_flat]
# ...
flat_networks = external
```
这里主要是指定平面网络的名称，就是虚拟机内部网络叫什么名，这个名称可以自定义，但后面会用到把该网络桥接到物理网卡中的配置，以及后续的创建网络都要用到这名称，请确保后续的名称和这里的名称保持一致；
在[ml2_type_vxlan]配置段中配置vxlan的标识范围
```
[ml2_type_vxlan]
# ...
vni_ranges = 1:1000
```
在[securitygroup]配置段启用ipset
```
[securitygroup]
enable_ipset = true
```
配置Open vSwitch agent
Linux 桥接代理构建了用于实例的第二层（桥接和交换）虚拟网络基础设施，并处理安全组。
编辑 /etc/neutron/plugins/ml2/openvswitch_agent.ini 文件并完成以下操作：
在 [ovs] 部分，将提供商虚拟网络映射到提供商物理网络接口：
```
[ovs]
bridge_mappings = external:br-ex
local_ip = 192.168.71.11
#br-ex这个名称是自定义的，须与ovs-vsctl add-br 相关操作保持统一
```
提示：这里主要是配置把把虚拟机内部的那个网络和物理接口的桥接映射，请确保虚拟机内部网络名称和这里配置的保持一致；冒号前指定虚拟机内部网络名称，冒号后面指定要桥接的物理网卡接口名称；
提示：local_ip写控制节点的管理ip地址（如果有多个ip地址的话）；
在[vxlan]配置段配置启用vxlan，并配置本地管理ip地址和开启l2_population
```
[agent]
tunnel_types = vxlan
l2_population = true
```
在[securitygroup]配置段配置启用安全组并配置Linux bridge iptables防火墙驱动程序
```
[securitygroup]
# ...
enable_security_group = true
firewall_driver = openvswitch
```

配置虚拟机流量的桥接口。这里需要注意，每台主机里面有两个网口，其中一个网口配置有IP, 另外一个网口只开启, 不配置任何IP。以下是node01节点的网口配置

[root@node01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens3 
DEVICE=ens3
TYPE=Ethernet
ONBOOT="yes"
BOOTPROTO="none"
IPADDR=192.168.71.11
PREFIX=24
GATEWAY=192.168.71.254
[root@node01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens4
DEVICE=ens4
TYPE=Ethernet
ONBOOT="yes"
BOOTPROTO="static"

之后用ovs-vsctl命令添加桥接口

[root@node01 ~]# ovs-vsctl add-br br-ex
[root@node01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether fa:16:3e:50:18:0d brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet 192.168.71.11/24 brd 192.168.71.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe50:180d/64 scope link 
       valid_lft forever preferred_lft forever
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether fa:16:3e:3a:43:e6 brd ff:ff:ff:ff:ff:ff
    altname enp0s4
    inet6 fe80::f816:3eff:fe3a:43e6/64 scope link 
       valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 7a:e4:de:b9:90:d0 brd ff:ff:ff:ff:ff:ff
5: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 32:85:24:55:37:48 brd ff:ff:ff:ff:ff:ff
6: br-ex: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 3e:ea:e7:70:ee:49 brd ff:ff:ff:ff:ff:ff
[root@node01 ~]# ovs-vsctl add-port br-ex ens4
[root@node01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether fa:16:3e:50:18:0d brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet 192.168.71.11/24 brd 192.168.71.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe50:180d/64 scope link 
       valid_lft forever preferred_lft forever
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000
    link/ether fa:16:3e:3a:43:e6 brd ff:ff:ff:ff:ff:ff
    altname enp0s4
    inet6 fe80::f816:3eff:fe3a:43e6/64 scope link 
       valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 7a:e4:de:b9:90:d0 brd ff:ff:ff:ff:ff:ff
5: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 32:85:24:55:37:48 brd ff:ff:ff:ff:ff:ff
6: br-ex: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 3e:ea:e7:70:ee:49 brd ff:ff:ff:ff:ff:ff

另外可以手动创建网卡配置文件，实现这个功能

[root@node01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-3
# Generated by dracut initrd
NAME="ens3"
DEVICE="ens3"
ONBOOT="yes"
NETBOOT="yes"
UUID="1d6a3ff0-3d89-4ec9-9979-dca4e9ff3646"
IPV6INIT="no"
BOOTPROTO="no"
#TYPE="Ethernet"
TYPE="OVSPort"
DEFROUTE="yes"
PEERDNS="yes"
PEERROUTES="yes"
IPV4_FAILURE_FATAL="no"
IPADDR=192.168.71.11
PREFIX=24
GATEWAY=192.168.71.254
#以下三行为主要配置内容
OVS_BRIDGE=br-ex
DEVICETYPE=ovs
TYPE="OVSPort"

[root@node01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-br-ex 
DEVICE=br-ex
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
ONBOOT=yes

systemctl restart network
systemctl restart openvswitch

其实这里每个节点可以只有一个网口, 只不过需要将对应的IP, 配置在br-ex上面。以下为类似的配置示例

[root@hyhive01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp6s0 
# Generated by dracut initrd
NAME="enp6s0"
DEVICE="enp6s0"
ONBOOT="yes"
NETBOOT="yes"
UUID="1d6a3ff0-3d89-4ec9-9979-dca4e9ff3646"
IPV6INIT="no"
BOOTPROTO="no"
#TYPE="Ethernet"
TYPE="OVSPort"
DEFROUTE="yes"
PEERDNS="yes"
PEERROUTES="yes"
IPV4_FAILURE_FATAL="no"
##以下为需要注意内容
OVS_BRIDGE=br-ex
DEVICETYPE=ovs
TYPE="OVSPort"

[root@hyhive01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-br-ex 
DEVICE=br-ex
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.10.213
PREFIX=24
GATEWAY=192.168.10.1
DNS1=114.114.114.114


ifup  enp6s0
ifup br-ex
systemctl restart openvswitch

确定br_netfilter内核模块是加载启用，若没加载，加载内核模块并配置相关内核参数

[root@node01 ~]# lsmod |grep br_netfilter
[root@node01 ~]# modprobe br_netfilter
[root@node01 ~]# lsmod |grep br_netfilter
br_netfilter           22209  0
bridge                136173  1 br_netfilter
[root@node01 ~]#

配置相关内核参数

[root@node01 ~]# cat /etc/sysctl.conf 
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
#
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1

[root@node01 ~]# sysctl -p
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1

配置L3 agent
L3 agent 为自助虚拟网络提供路由和 NAT 服务。
编辑 /etc/neutron/l3_agent.ini 文件并完成以下操作：
```
[DEFAULT]
# ...
interface_driver = openvswitch
```
配置DHCP agent
编辑/etc/neutron/dhcp_agent.ini配置文件，在[DEFAULT]配置段配置网络接口驱动为linuxbridge,启用元数据隔离，并配置dhcp驱动程序
```
[DEFAULT]

interface_driver = openvswitch
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
enable_isolated_metadata = true
```

6.1.4 网络后续配置

配置metadata agent
编辑/etc/neutron/metadata_agent.ini配置文件
在[DEFAULT]配置段配置metadata server地址和共享密钥
```
[DEFAULT]
# ...
nova_metadata_host = openstack-vip.net
metadata_proxy_shared_secret = 123456
```
提示：metadata_proxy_shared_secret 这个是配置共享密钥的参数，后面的密钥可以随机生成，也可以设定任意字符串；

在compute服务的配置文件中使用neutron服务

编辑/etc/nova/nova.conf配置文件 , 在[neutron]配置段配置neutron相关信息

[neutron]
# ...
auth_url = http://openstack-vip.net:5000
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = service
username = neutron
password = neutron
service_metadata_proxy = true
metadata_proxy_shared_secret = 123456

提示：这里的metadata_proxy_shared_secret要和上面配置的metadata agent中配置的密钥保持一致即可；

将ml2的配置文件软连接到/etc/neutron/plugin.ini

[root@node01 ~]# ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini

初始化neutron数据库

[root@node01 ~]# su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf  --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron
INFO  [alembic.runtime.migration] Context impl MySQLImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
  Running upgrade for neutron ...
INFO  [alembic.runtime.migration] Context impl MySQLImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> kilo
INFO  [alembic.runtime.migration] Running upgrade kilo -> 354db87e3225
INFO  [alembic.runtime.migration] Running upgrade 354db87e3225 -> 599c6a226151
INFO  [alembic.runtime.migration] Running upgrade 599c6a226151 -> 52c5312f6baf
INFO  [alembic.runtime.migration] Running upgrade 52c5312f6baf -> 313373c0ffee
INFO  [alembic.runtime.migration] Running upgrade 313373c0ffee -> 8675309a5c4f
INFO  [alembic.runtime.migration] Running upgrade 8675309a5c4f -> 45f955889773
INFO  [alembic.runtime.migration] Running upgrade 45f955889773 -> 26c371498592
INFO  [alembic.runtime.migration] Running upgrade 26c371498592 -> 1c844d1677f7
INFO  [alembic.runtime.migration] Running upgrade 1c844d1677f7 -> 1b4c6e320f79
INFO  [alembic.runtime.migration] Running upgrade 1b4c6e320f79 -> 48153cb5f051
INFO  [alembic.runtime.migration] Running upgrade 48153cb5f051 -> 9859ac9c136
INFO  [alembic.runtime.migration] Running upgrade 9859ac9c136 -> 34af2b5c5a59
INFO  [alembic.runtime.migration] Running upgrade 34af2b5c5a59 -> 59cb5b6cf4d
INFO  [alembic.runtime.migration] Running upgrade 59cb5b6cf4d -> 13cfb89f881a
INFO  [alembic.runtime.migration] Running upgrade 13cfb89f881a -> 32e5974ada25
INFO  [alembic.runtime.migration] Running upgrade 32e5974ada25 -> ec7fcfbf72ee
INFO  [alembic.runtime.migration] Running upgrade ec7fcfbf72ee -> dce3ec7a25c9
INFO  [alembic.runtime.migration] Running upgrade dce3ec7a25c9 -> c3a73f615e4
INFO  [alembic.runtime.migration] Running upgrade c3a73f615e4 -> 659bf3d90664
INFO  [alembic.runtime.migration] Running upgrade 659bf3d90664 -> 1df244e556f5
INFO  [alembic.runtime.migration] Running upgrade 1df244e556f5 -> 19f26505c74f
INFO  [alembic.runtime.migration] Running upgrade 19f26505c74f -> 15be73214821
INFO  [alembic.runtime.migration] Running upgrade 15be73214821 -> b4caf27aae4
INFO  [alembic.runtime.migration] Running upgrade b4caf27aae4 -> 15e43b934f81
INFO  [alembic.runtime.migration] Running upgrade 15e43b934f81 -> 31ed664953e6
INFO  [alembic.runtime.migration] Running upgrade 31ed664953e6 -> 2f9e956e7532
INFO  [alembic.runtime.migration] Running upgrade 2f9e956e7532 -> 3894bccad37f
INFO  [alembic.runtime.migration] Running upgrade 3894bccad37f -> 0e66c5227a8a
INFO  [alembic.runtime.migration] Running upgrade 0e66c5227a8a -> 45f8dd33480b
INFO  [alembic.runtime.migration] Running upgrade 45f8dd33480b -> 5abc0278ca73
INFO  [alembic.runtime.migration] Running upgrade 5abc0278ca73 -> d3435b514502
INFO  [alembic.runtime.migration] Running upgrade d3435b514502 -> 30107ab6a3ee
INFO  [alembic.runtime.migration] Running upgrade 30107ab6a3ee -> c415aab1c048
INFO  [alembic.runtime.migration] Running upgrade c415aab1c048 -> a963b38d82f4
INFO  [alembic.runtime.migration] Running upgrade kilo -> 30018084ec99
INFO  [alembic.runtime.migration] Running upgrade 30018084ec99 -> 4ffceebfada
INFO  [alembic.runtime.migration] Running upgrade 4ffceebfada -> 5498d17be016
INFO  [alembic.runtime.migration] Running upgrade 5498d17be016 -> 2a16083502f3
INFO  [alembic.runtime.migration] Running upgrade 2a16083502f3 -> 2e5352a0ad4d
INFO  [alembic.runtime.migration] Running upgrade 2e5352a0ad4d -> 11926bcfe72d
INFO  [alembic.runtime.migration] Running upgrade 11926bcfe72d -> 4af11ca47297
INFO  [alembic.runtime.migration] Running upgrade 4af11ca47297 -> 1b294093239c
INFO  [alembic.runtime.migration] Running upgrade 1b294093239c -> 8a6d8bdae39
INFO  [alembic.runtime.migration] Running upgrade 8a6d8bdae39 -> 2b4c2465d44b
INFO  [alembic.runtime.migration] Running upgrade 2b4c2465d44b -> e3278ee65050
INFO  [alembic.runtime.migration] Running upgrade e3278ee65050 -> c6c112992c9
INFO  [alembic.runtime.migration] Running upgrade c6c112992c9 -> 5ffceebfada
INFO  [alembic.runtime.migration] Running upgrade 5ffceebfada -> 4ffceebfcdc
INFO  [alembic.runtime.migration] Running upgrade 4ffceebfcdc -> 7bbb25278f53
INFO  [alembic.runtime.migration] Running upgrade 7bbb25278f53 -> 89ab9a816d70
INFO  [alembic.runtime.migration] Running upgrade 89ab9a816d70 -> c879c5e1ee90
INFO  [alembic.runtime.migration] Running upgrade c879c5e1ee90 -> 8fd3918ef6f4
INFO  [alembic.runtime.migration] Running upgrade 8fd3918ef6f4 -> 4bcd4df1f426
INFO  [alembic.runtime.migration] Running upgrade 4bcd4df1f426 -> b67e765a3524
INFO  [alembic.runtime.migration] Running upgrade a963b38d82f4 -> 3d0e74aa7d37
INFO  [alembic.runtime.migration] Running upgrade 3d0e74aa7d37 -> 030a959ceafa
INFO  [alembic.runtime.migration] Running upgrade 030a959ceafa -> a5648cfeeadf
INFO  [alembic.runtime.migration] Running upgrade a5648cfeeadf -> 0f5bef0f87d4
INFO  [alembic.runtime.migration] Running upgrade 0f5bef0f87d4 -> 67daae611b6e
INFO  [alembic.runtime.migration] Running upgrade b67e765a3524 -> a84ccf28f06a
INFO  [alembic.runtime.migration] Running upgrade a84ccf28f06a -> 7d9d8eeec6ad
INFO  [alembic.runtime.migration] Running upgrade 67daae611b6e -> 6b461a21bcfc
INFO  [alembic.runtime.migration] Running upgrade 6b461a21bcfc -> 5cd92597d11d
INFO  [alembic.runtime.migration] Running upgrade 5cd92597d11d -> 929c968efe70
INFO  [alembic.runtime.migration] Running upgrade 929c968efe70 -> a9c43481023c
INFO  [alembic.runtime.migration] Running upgrade a9c43481023c -> 804a3c76314c
INFO  [alembic.runtime.migration] Running upgrade 804a3c76314c -> 2b42d90729da
INFO  [alembic.runtime.migration] Running upgrade 2b42d90729da -> 62c781cb6192
INFO  [alembic.runtime.migration] Running upgrade 62c781cb6192 -> c8c222d42aa9
INFO  [alembic.runtime.migration] Running upgrade c8c222d42aa9 -> 349b6fd605a6
INFO  [alembic.runtime.migration] Running upgrade 349b6fd605a6 -> 7d32f979895f
INFO  [alembic.runtime.migration] Running upgrade 7d32f979895f -> 594422d373ee
INFO  [alembic.runtime.migration] Running upgrade 594422d373ee -> 61663558142c
INFO  [alembic.runtime.migration] Running upgrade 61663558142c -> 867d39095bf4, port forwarding
INFO  [alembic.runtime.migration] Running upgrade 867d39095bf4 -> d72db3e25539, modify uniq port forwarding
INFO  [alembic.runtime.migration] Running upgrade d72db3e25539 -> cada2437bf41
INFO  [alembic.runtime.migration] Running upgrade cada2437bf41 -> 195176fb410d, router gateway IP QoS
INFO  [alembic.runtime.migration] Running upgrade 195176fb410d -> fb0167bd9639
INFO  [alembic.runtime.migration] Running upgrade fb0167bd9639 -> 0ff9e3881597
INFO  [alembic.runtime.migration] Running upgrade 0ff9e3881597 -> 9bfad3f1e780
INFO  [alembic.runtime.migration] Running upgrade 9bfad3f1e780 -> 63fd95af7dcd
INFO  [alembic.runtime.migration] Running upgrade 63fd95af7dcd -> c613d0b82681
INFO  [alembic.runtime.migration] Running upgrade c613d0b82681 -> c3e9d13c4367
INFO  [alembic.runtime.migration] Running upgrade c3e9d13c4367 -> 86274d77933e
INFO  [alembic.runtime.migration] Running upgrade 86274d77933e -> f4b9654dd40c
INFO  [alembic.runtime.migration] Running upgrade f4b9654dd40c -> a010322604bc
INFO  [alembic.runtime.migration] Running upgrade a010322604bc -> 263d454a9655
INFO  [alembic.runtime.migration] Running upgrade 263d454a9655 -> Ibac91d24da2
INFO  [alembic.runtime.migration] Running upgrade Ibac91d24da2 -> 2217c4222de6
INFO  [alembic.runtime.migration] Running upgrade 2217c4222de6 -> 18a7e90ae768
INFO  [alembic.runtime.migration] Running upgrade 18a7e90ae768 -> e4e236b0e1ff
INFO  [alembic.runtime.migration] Running upgrade e4e236b0e1ff -> e88badaa9591
INFO  [alembic.runtime.migration] Running upgrade e88badaa9591 -> d8bdf05313f4
INFO  [alembic.runtime.migration] Running upgrade d8bdf05313f4 -> dfe425060830
INFO  [alembic.runtime.migration] Running upgrade dfe425060830 -> fd6107509ccd
INFO  [alembic.runtime.migration] Running upgrade fd6107509ccd -> 1ea5dab0897a
INFO  [alembic.runtime.migration] Running upgrade 1ea5dab0897a -> 49d8622c5221
INFO  [alembic.runtime.migration] Running upgrade 49d8622c5221 -> I38991de2b4
INFO  [alembic.runtime.migration] Running upgrade I38991de2b4 -> 532aa95457e2
INFO  [alembic.runtime.migration] Running upgrade 532aa95457e2 -> f010820fc498
INFO  [alembic.runtime.migration] Running upgrade f010820fc498 -> a964d94b4677
INFO  [alembic.runtime.migration] Running upgrade a964d94b4677 -> 26d1e9f5c766
INFO  [alembic.runtime.migration] Running upgrade 26d1e9f5c766 -> 1e0744e4ffea
INFO  [alembic.runtime.migration] Running upgrade 1e0744e4ffea -> 6135a7bd4425
INFO  [alembic.runtime.migration] Running upgrade 6135a7bd4425 -> 8df53b0d2c0e
INFO  [alembic.runtime.migration] Running upgrade 8df53b0d2c0e -> 1bb3393de75d, add qos policy rule Packet Rate Limit
INFO  [alembic.runtime.migration] Running upgrade 1bb3393de75d -> c181bb1d89e4
INFO  [alembic.runtime.migration] Running upgrade c181bb1d89e4 -> ba859d649675
INFO  [alembic.runtime.migration] Running upgrade ba859d649675 -> e981acd076d3
INFO  [alembic.runtime.migration] Running upgrade e981acd076d3 -> 76df7844a8c6, add Local IP tables
INFO  [alembic.runtime.migration] Running upgrade 76df7844a8c6 -> 1ffef8d6f371, migrate RBAC registers from "target_tenant" to "target_project"
INFO  [alembic.runtime.migration] Running upgrade 1ffef8d6f371 -> 8160f7a9cebb, drop portbindingports table
INFO  [alembic.runtime.migration] Running upgrade 8160f7a9cebb -> cd9ef14ccf87
INFO  [alembic.runtime.migration] Running upgrade cd9ef14ccf87 -> 34cf8b009713
INFO  [alembic.runtime.migration] Running upgrade 34cf8b009713 -> I43e0b669096
INFO  [alembic.runtime.migration] Running upgrade I43e0b669096 -> 4e6e655746f6
INFO  [alembic.runtime.migration] Running upgrade 4e6e655746f6 -> 659cbedf30a1
INFO  [alembic.runtime.migration] Running upgrade 659cbedf30a1 -> 21ff98fabab1
INFO  [alembic.runtime.migration] Running upgrade 21ff98fabab1 -> 5881373af7f5
INFO  [alembic.runtime.migration] Running upgrade 5881373af7f5 -> fc153938cdc1
INFO  [alembic.runtime.migration] Running upgrade fc153938cdc1 -> 93f394357a27
INFO  [alembic.runtime.migration] Running upgrade 93f394357a27 -> 6f1145bff34c
INFO  [alembic.runtime.migration] Running upgrade 6f1145bff34c -> 682c319773d7
INFO  [alembic.runtime.migration] Running upgrade 682c319773d7 -> 0aefee21cd87
INFO  [alembic.runtime.migration] Running upgrade 0aefee21cd87 -> b1199a3adbef
INFO  [alembic.runtime.migration] Running upgrade b1199a3adbef -> c33da356b165
INFO  [alembic.runtime.migration] Running upgrade c33da356b165 -> 89c58a70ceba
INFO  [alembic.runtime.migration] Running upgrade 7d9d8eeec6ad -> a8b517cff8ab
INFO  [alembic.runtime.migration] Running upgrade a8b517cff8ab -> 3b935b28e7a0
INFO  [alembic.runtime.migration] Running upgrade 3b935b28e7a0 -> b12a3ef66e62
INFO  [alembic.runtime.migration] Running upgrade b12a3ef66e62 -> 97c25b0d2353
INFO  [alembic.runtime.migration] Running upgrade 97c25b0d2353 -> 2e0d7a8a1586
INFO  [alembic.runtime.migration] Running upgrade 2e0d7a8a1586 -> 5c85685d616d
  OK

验证：连接neutron数据库中是否有表生成

[root@node02 ~]# mysql -uroot -p
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 10583
Server version: 10.5.22-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.


MariaDB [(none)]> use neutron
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MariaDB [neutron]> show tables;
+-----------------------------------------+
| Tables_in_neutron                       |
+-----------------------------------------+
| address_associations                    |
| address_groups                          |
| address_scopes                          |
| addressgrouprbacs                       |
| addressscoperbacs                       |
| agents                                  |
| alembic_version                         |
| allowedaddresspairs                     |
| arista_provisioned_nets                 |
| arista_provisioned_tenants              |
| arista_provisioned_vms                  |
| auto_allocated_topologies               |
| bgp_peers                               |
| bgp_speaker_dragent_bindings            |
| bgp_speaker_network_bindings            |
| bgp_speaker_peer_bindings               |
| bgp_speakers                            |
| brocadenetworks                         |
| brocadeports                            |
| cisco_csr_identifier_map                |
| cisco_hosting_devices                   |
| cisco_ml2_apic_contracts                |
| cisco_ml2_apic_host_links               |
| cisco_ml2_apic_names                    |
| cisco_ml2_n1kv_network_bindings         |
| cisco_ml2_n1kv_network_profiles         |
| cisco_ml2_n1kv_policy_profiles          |
| cisco_ml2_n1kv_port_bindings            |
| cisco_ml2_n1kv_profile_bindings         |
| cisco_ml2_n1kv_vlan_allocations         |
| cisco_ml2_n1kv_vxlan_allocations        |
| cisco_ml2_nexus_nve                     |
| cisco_ml2_nexusport_bindings            |
| cisco_port_mappings                     |
| cisco_router_mappings                   |
| conntrack_helpers                       |
| consistencyhashes                       |
| default_security_group                  |
| dnsnameservers                          |
| dvr_fip_gateway_port_network            |
| dvr_host_macs                           |
| externalnetworks                        |
| extradhcpopts                           |
| firewall_policies                       |
| firewall_rules                          |
| firewalls                               |
| flavors                                 |
| flavorserviceprofilebindings            |
| floatingipdnses                         |
| floatingips                             |
| ha_router_agent_port_bindings           |
| ha_router_networks                      |
| ha_router_vrid_allocations              |
| healthmonitors                          |
| ikepolicies                             |
| ipallocationpools                       |
| ipallocations                           |
| ipamallocationpools                     |
| ipamallocations                         |
| ipamsubnets                             |
| ipsec_site_connections                  |
| ipsecpeercidrs                          |
| ipsecpolicies                           |
| local_ip_associations                   |
| local_ips                               |
| logs                                    |
| lsn                                     |
| lsn_port                                |
| maclearningstates                       |
| members                                 |
| meteringlabelrules                      |
| meteringlabels                          |
| ml2_brocadenetworks                     |
| ml2_brocadeports                        |
| ml2_distributed_port_bindings           |
| ml2_flat_allocations                    |
| ml2_geneve_allocations                  |
| ml2_geneve_endpoints                    |
| ml2_gre_allocations                     |
| ml2_gre_endpoints                       |
| ml2_nexus_vxlan_allocations             |
| ml2_nexus_vxlan_mcast_groups            |
| ml2_port_binding_levels                 |
| ml2_port_bindings                       |
| ml2_ucsm_port_profiles                  |
| ml2_vlan_allocations                    |
| ml2_vxlan_allocations                   |
| ml2_vxlan_endpoints                     |
| multi_provider_networks                 |
| ndp_proxies                             |
| network_segment_ranges                  |
| network_subnet_lock                     |
| networkconnections                      |
| networkdhcpagentbindings                |
| networkdnsdomains                       |
| networkgatewaydevicereferences          |
| networkgatewaydevices                   |
| networkgateways                         |
| networkqueuemappings                    |
| networkrbacs                            |
| networks                                |
| networksecuritybindings                 |
| networksegments                         |
| neutron_nsx_network_mappings            |
| neutron_nsx_port_mappings               |
| neutron_nsx_router_mappings             |
| neutron_nsx_security_group_mappings     |
| nexthops                                |
| nsxv_edge_dhcp_static_bindings          |
| nsxv_edge_vnic_bindings                 |
| nsxv_firewall_rule_bindings             |
| nsxv_internal_edges                     |
| nsxv_internal_networks                  |
| nsxv_port_index_mappings                |
| nsxv_port_vnic_mappings                 |
| nsxv_router_bindings                    |
| nsxv_router_ext_attributes              |
| nsxv_rule_mappings                      |
| nsxv_security_group_section_mappings    |
| nsxv_spoofguard_policy_network_mappings |
| nsxv_tz_network_bindings                |
| nsxv_vdr_dhcp_bindings                  |
| nuage_net_partition_router_mapping      |
| nuage_net_partitions                    |
| nuage_provider_net_bindings             |
| nuage_subnet_l2dom_mapping              |
| ovn_hash_ring                           |
| ovn_revision_numbers                    |
| poolloadbalanceragentbindings           |
| poolmonitorassociations                 |
| pools                                   |
| poolstatisticss                         |
| portdataplanestatuses                   |
| portdeviceprofiles                      |
| portdnses                               |
| portforwardings                         |
| porthints                               |
| portnumaaffinitypolicies                |
| portqueuemappings                       |
| ports                                   |
| portsecuritybindings                    |
| portuplinkstatuspropagation             |
| providerresourceassociations            |
| provisioningblocks                      |
| qos_bandwidth_limit_rules               |
| qos_dscp_marking_rules                  |
| qos_fip_policy_bindings                 |
| qos_minimum_bandwidth_rules             |
| qos_minimum_packet_rate_rules           |
| qos_network_policy_bindings             |
| qos_packet_rate_limit_rules             |
| qos_policies                            |
| qos_policies_default                    |
| qos_port_policy_bindings                |
| qos_router_gw_policy_bindings           |
| qospolicyrbacs                          |
| qosqueues                               |
| quotas                                  |
| quotausages                             |
| reservations                            |
| resourcedeltas                          |
| router_extra_attributes                 |
| router_ndp_proxy_state                  |
| routerl3agentbindings                   |
| routerports                             |
| routerroutes                            |
| routerrules                             |
| routers                                 |
| securitygroupdefaultrules               |
| securitygroupportbindings               |
| securitygrouprbacs                      |
| securitygrouprules                      |
| securitygroups                          |
| segmenthostmappings                     |
| serviceprofiles                         |
| sessionpersistences                     |
| standardattributes                      |
| subnet_dns_publish_fixed_ips            |
| subnet_service_types                    |
| subnetpoolprefixes                      |
| subnetpoolrbacs                         |
| subnetpools                             |
| subnetroutes                            |
| subnets                                 |
| subports                                |
| tags                                    |
| trunks                                  |
| tz_network_bindings                     |
| vcns_router_bindings                    |
| vips                                    |
| vpnservices                             |
+-----------------------------------------+
191 rows in set (0.002 sec)

为neutron准备policy.json配置文件

[root@node01 ~]# cat /etc/neutron/policy.json 
{
    "context_is_admin":  "role:admin or role:leader",
    "owner": "tenant_id:%(tenant_id)s",
    "admin_or_owner": "rule:context_is_admin or rule:owner",
    "context_is_advsvc":  "role:advsvc",
    "admin_or_network_owner": "rule:context_is_admin or tenant_id:%(network:tenant_id)s",
    "admin_owner_or_network_owner": "rule:owner or rule:admin_or_network_owner",
    "admin_only": "rule:context_is_admin",
    "regular_user": "",
    "shared": "field:networks:shared=True",
    "shared_subnetpools": "field:subnetpools:shared=True",
    "shared_address_scopes": "field:address_scopes:shared=True",
    "external": "field:networks:router:external=True",
    "default": "rule:admin_or_owner",

    "create_subnet": "rule:admin_or_network_owner",
    "create_subnet:segment_id": "rule:admin_only",
    "create_subnet:service_types": "rule:admin_only",
    "get_subnet": "rule:admin_or_owner or rule:shared",
    "get_subnet:segment_id": "rule:admin_only",
    "update_subnet": "rule:admin_or_network_owner",
    "update_subnet:service_types": "rule:admin_only",
    "delete_subnet": "rule:admin_or_network_owner",

    "create_subnetpool": "",
    "create_subnetpool:shared": "rule:admin_only",
    "create_subnetpool:is_default": "rule:admin_only",
    "get_subnetpool": "rule:admin_or_owner or rule:shared_subnetpools",
    "update_subnetpool": "rule:admin_or_owner",
    "update_subnetpool:is_default": "rule:admin_only",
    "delete_subnetpool": "rule:admin_or_owner",

    "create_address_scope": "",
    "create_address_scope:shared": "rule:admin_only",
    "get_address_scope": "rule:admin_or_owner or rule:shared_address_scopes",
    "update_address_scope": "rule:admin_or_owner",
    "update_address_scope:shared": "rule:admin_only",
    "delete_address_scope": "rule:admin_or_owner",

    "create_network": "",
    "get_network": "rule:admin_or_owner or rule:shared or rule:external or rule:context_is_advsvc",
    "get_network:router:external": "rule:regular_user",
    "get_network:segments": "rule:admin_only",
    "get_network:provider:network_type": "rule:admin_only",
    "get_network:provider:physical_network": "rule:admin_only",
    "get_network:provider:segmentation_id": "rule:admin_only",
    "get_network:queue_id": "rule:admin_only",
    "get_network_ip_availabilities": "rule:admin_only",
    "get_network_ip_availability": "rule:admin_only",
    "create_network:shared": "rule:admin_only",
    "create_network:router:external": "rule:admin_only",
    "create_network:is_default": "rule:admin_only",
    "create_network:segments": "rule:admin_only",
    "create_network:provider:network_type": "rule:admin_only",
    "create_network:provider:physical_network": "rule:admin_only",
    "create_network:provider:segmentation_id": "rule:admin_only",
    "update_network": "rule:admin_or_owner",
    "update_network:segments": "rule:admin_only",
    "update_network:shared": "rule:admin_only",
    "update_network:provider:network_type": "rule:admin_only",
    "update_network:provider:physical_network": "rule:admin_only",
    "update_network:provider:segmentation_id": "rule:admin_only",
    "update_network:router:external": "rule:admin_only",
    "delete_network": "rule:admin_or_owner",

    "create_segment": "rule:admin_only",
    "get_segment": "rule:admin_only",
    "update_segment": "rule:admin_only",
    "delete_segment": "rule:admin_only",

    "network_device": "field:port:device_owner=~^network:",
    "create_port": "",
    "create_port:device_owner": "not rule:network_device or rule:context_is_advsvc or rule:admin_or_network_owner",
    "create_port:mac_address": "rule:context_is_advsvc or rule:admin_or_network_owner",
    "create_port:fixed_ips": "rule:context_is_advsvc or rule:admin_or_network_owner",
    "create_port:port_security_enabled": "rule:context_is_advsvc or rule:admin_or_network_owner",
    "create_port:binding:host_id": "rule:admin_only",
    "create_port:binding:profile": "rule:admin_only",
    "create_port:mac_learning_enabled": "rule:context_is_advsvc or rule:admin_or_network_owner",
    "create_port:allowed_address_pairs": "rule:admin_or_network_owner",
    "get_port": "rule:context_is_advsvc or rule:admin_owner_or_network_owner",
    "get_port:queue_id": "rule:admin_only",
    "get_port:binding:vif_type": "rule:admin_only",
    "get_port:binding:vif_details": "rule:admin_only",
    "get_port:binding:host_id": "rule:admin_only",
    "get_port:binding:profile": "rule:admin_only",
    "update_port": "rule:admin_or_owner or rule:context_is_advsvc",
    "update_port:device_owner": "not rule:network_device or rule:context_is_advsvc or rule:admin_or_network_owner",
    "update_port:mac_address": "rule:admin_only or rule:context_is_advsvc",
    "update_port:fixed_ips": "rule:context_is_advsvc or rule:admin_or_network_owner",
    "update_port:port_security_enabled": "rule:context_is_advsvc or rule:admin_or_network_owner",
    "update_port:binding:host_id": "rule:admin_only",
    "update_port:binding:profile": "rule:admin_only",
    "update_port:mac_learning_enabled": "rule:context_is_advsvc or rule:admin_or_network_owner",
    "update_port:allowed_address_pairs": "rule:admin_or_network_owner",
    "delete_port": "rule:context_is_advsvc or rule:admin_owner_or_network_owner",

    "get_router:ha": "rule:admin_only",
    "create_router": "rule:regular_user",
    "create_router:external_gateway_info:enable_snat": "rule:admin_only",
    "create_router:distributed": "rule:admin_only",
    "create_router:ha": "rule:admin_only",
    "get_router": "rule:admin_or_owner",
    "get_router:distributed": "rule:admin_only",
    "update_router:external_gateway_info:enable_snat": "rule:admin_only",
    "update_router:distributed": "rule:admin_only",
    "update_router:ha": "rule:admin_only",
    "delete_router": "rule:admin_or_owner",

    "add_router_interface": "rule:admin_or_owner",
    "remove_router_interface": "rule:admin_or_owner",

    "create_router:external_gateway_info:external_fixed_ips": "rule:admin_only",
    "update_router:external_gateway_info:external_fixed_ips": "rule:admin_only",

    "insert_rule": "rule:admin_or_owner",
    "remove_rule": "rule:admin_or_owner",

    "create_qos_queue": "rule:admin_only",
    "get_qos_queue": "rule:admin_only",

    "update_agent": "rule:admin_only",
    "delete_agent": "rule:admin_only",
    "get_agent": "rule:admin_only",

    "create_dhcp-network": "rule:admin_only",
    "delete_dhcp-network": "rule:admin_only",
    "get_dhcp-networks": "rule:admin_only",
    "create_l3-router": "rule:admin_only",
    "delete_l3-router": "rule:admin_only",
    "get_l3-routers": "rule:admin_only",
    "get_dhcp-agents": "rule:admin_only",
    "get_l3-agents": "rule:admin_only",
    "get_loadbalancer-agent": "rule:admin_only",
    "get_loadbalancer-pools": "rule:admin_only",
    "get_agent-loadbalancers": "rule:admin_only",
    "get_loadbalancer-hosting-agent": "rule:admin_only",

    "create_floatingip": "rule:regular_user",
    "create_floatingip:floating_ip_address": "rule:admin_only",
    "update_floatingip": "rule:admin_or_owner",
    "delete_floatingip": "rule:admin_or_owner",
    "get_floatingip": "rule:admin_or_owner",

    "create_network_profile": "rule:admin_only",
    "update_network_profile": "rule:admin_only",
    "delete_network_profile": "rule:admin_only",
    "get_network_profiles": "",
    "get_network_profile": "",
    "update_policy_profiles": "rule:admin_only",
    "get_policy_profiles": "",
    "get_policy_profile": "",

    "create_metering_label": "rule:admin_only",
    "delete_metering_label": "rule:admin_only",
    "get_metering_label": "rule:admin_only",

    "create_metering_label_rule": "rule:admin_only",
    "delete_metering_label_rule": "rule:admin_only",
    "get_metering_label_rule": "rule:admin_only",

    "get_service_provider": "rule:regular_user",
    "get_lsn": "rule:admin_only",
    "create_lsn": "rule:admin_only",

    "create_flavor": "rule:admin_only",
    "update_flavor": "rule:admin_only",
    "delete_flavor": "rule:admin_only",
    "get_flavors": "rule:regular_user",
    "get_flavor": "rule:regular_user",
    "create_service_profile": "rule:admin_only",
    "update_service_profile": "rule:admin_only",
    "delete_service_profile": "rule:admin_only",
    "get_service_profiles": "rule:admin_only",
    "get_service_profile": "rule:admin_only",

    "get_policy": "rule:regular_user",
    "create_policy": "rule:admin_only",
    "update_policy": "rule:admin_only",
    "delete_policy": "rule:admin_only",
    "get_policy_bandwidth_limit_rule": "rule:regular_user",
    "create_policy_bandwidth_limit_rule": "rule:admin_only",
    "delete_policy_bandwidth_limit_rule": "rule:admin_only",
    "update_policy_bandwidth_limit_rule": "rule:admin_only",
    "get_policy_dscp_marking_rule": "rule:regular_user",
    "create_policy_dscp_marking_rule": "rule:admin_only",
    "delete_policy_dscp_marking_rule": "rule:admin_only",
    "update_policy_dscp_marking_rule": "rule:admin_only",
    "get_rule_type": "rule:regular_user",
    "get_policy_minimum_bandwidth_rule": "rule:regular_user",
    "create_policy_minimum_bandwidth_rule": "rule:admin_only",
    "delete_policy_minimum_bandwidth_rule": "rule:admin_only",
    "update_policy_minimum_bandwidth_rule": "rule:admin_only",

    "restrict_wildcard": "(not field:rbac_policy:target_tenant=*) or rule:admin_only",
    "create_rbac_policy": "",
    "create_rbac_policy:target_tenant": "rule:restrict_wildcard",
    "update_rbac_policy": "rule:admin_or_owner",
    "update_rbac_policy:target_tenant": "rule:restrict_wildcard and rule:admin_or_owner",
    "get_rbac_policy": "rule:admin_or_owner",
    "delete_rbac_policy": "rule:admin_or_owner",

    "create_flavor_service_profile": "rule:admin_only",
    "delete_flavor_service_profile": "rule:admin_only",
    "get_flavor_service_profile": "rule:regular_user",
    "get_auto_allocated_topology": "rule:admin_or_owner",

    "create_trunk": "rule:regular_user",
    "get_trunk": "rule:admin_or_owner",
    "delete_trunk": "rule:admin_or_owner",
    "get_subports": "",
    "add_subports": "rule:admin_or_owner",
    "remove_subports": "rule:admin_or_owner",
    "create_security_group": "rule:admin_or_owner"
}

[root@node01 ~]# chown root:neutron /etc/neutron/policy.json  
[root@node01 ~]# systemctl restart  neutron-server

重启nova-api服务

[root@node01 ~]# systemctl restart openstack-nova-api.service

启动neutron相关服务，并将其设置为开机启动

[root@node01 ~]# systemctl enable neutron-server.service   neutron-openvswitch-agent.service neutron-dhcp-agent.service   neutron-metadata-agent.service
[root@node01 ~]# systemctl start neutron-server.service   neutron-openvswitch-agent.service neutron-dhcp-agent.service   neutron-metadata-agent.service

提示：请确保9696端口正常监听；

如果我们选用的是self-service network 我们还需要启动L3 agent 服务，并将其设置为开机启动

[root@node01 ~]# systemctl enable neutron-l3-agent.service
Created symlink /etc/systemd/system/multi-user.target.wants/neutron-l3-agent.service → /usr/lib/systemd/system/neutron-l3-agent.service.
[root@node01 ~]# systemctl start neutron-l3-agent.service

到此控制节点的neutron服务就配置好了

6.2 计算节点配置

6.2.1 基础环境配置

安装neutron相关服务包

[root@node03 ~]# dnf install openstack-neutron-openvswitch

配置组件

网络通用组件配置包括认证机制、消息队列和插件。

编辑 /etc/neutron/neutron.conf 文件并完成以下操作

在 [database] 部分，注释掉任何连接选项，因为计算节点不直接访问数据库。

在 [DEFAULT] 部分，配置 RabbitMQ 消息队列访问权限：

[DEFAULT]
# ...
transport_url = rabbit://openstack:openstack123@openstack-vip.net

在 [DEFAULT] 和 [keystone_authtoken] 部分，配置身份服务访问权限：

[DEFAULT]
# ...
auth_strategy = keystone

[keystone_authtoken]
# ...
www_authenticate_uri = http://openstack-vip.net:5000
auth_url = http://openstack-vip.net:5000
memcached_servers = openstack-vip.net:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = neutron
password = neutron

在 [oslo_concurrency] 部分，配置lock路径：

[oslo_concurrency]
# ...
lock_path = /var/lib/neutron/tmp

6.2.2 provider网络环境配置

添加br-ex网桥

ovs-vsctl add-br br-ex
ovs-vsctl add-port br-ex ens4

配置网络选项, 配置Open vSwitch agent
Open vSwitch代理构建了用于实例的l2（桥接和交换）虚拟网络基础设施，并处理安全组。
编辑 /etc/neutron/plugins/ml2/openvswitch_agent.ini 文件并完成以下操作：
在 [ovs] 部分，将提供商虚拟网络映射到提供商物理网络接口：
```
[ovs]
bridge_mappings = external:br-ex
```
在 [securitygroup] 部分，启用安全组并配置 Open vSwitch 原生或混合 iptables 防火墙驱动程序：
```
[securitygroup]
# ...
enable_security_group = true
firewall_driver = openvswitch
#firewall_driver = iptables_hybrid
```

确定br_netfilter内核模块是加载启用，若没加载，加载内核模块并配置相关内核参数，这一步，之后启动openvswitch服务之后会自动配置

[root@node01 ~]# lsmod |grep br_netfilter
[root@node01 ~]# modprobe br_netfilter
[root@node01 ~]# lsmod |grep br_netfilter
br_netfilter           22209  0
bridge                136173  1 br_netfilter
[root@node01 ~]#

配置相关内核参数

[root@node01 ~]# cat /etc/sysctl.conf 
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
#
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1

[root@node01 ~]# sysctl -p
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1

6.2.3 self-service网络环境配置

配置网络选项, 配置Open vSwitch agent
Open vSwitch代理构建了用于实例的l2（桥接和交换）虚拟网络基础设施，并处理安全组。
编辑 /etc/neutron/plugins/ml2/openvswitch_agent.ini 文件并完成以下操作：
在 [ovs] 部分，将提供商虚拟网络映射到提供商物理网络接口：
```
[ovs]
bridge_mappings = external:br-ex
local_ip = 192.168.71.30
```
在配置段配置启用vxlan，并配置本地管理ip地址和开启l2_population
```
[agent]
tunnel_types = vxlan
l2_population = true
```
在 [securitygroup] 部分，启用安全组并配置 Open vSwitch 原生或混合 iptables 防火墙驱动程序：
```
[securitygroup]
# ...
enable_security_group = true
firewall_driver = openvswitch
#firewall_driver = iptables_hybrid
```

配置虚拟机的流量桥接口

[root@node03 ~]#  ovs-vsctl add-br br-ex
[root@node03 ~]# ovs-vsctl add-port br-ex ens4

确定br_netfilter内核模块是加载启用，若没加载，加载内核模块并配置相关内核参数，这一步，之后启动openvswitch服务之后会自动配置

[root@node01 ~]# lsmod |grep br_netfilter
[root@node01 ~]# modprobe br_netfilter
[root@node01 ~]# lsmod |grep br_netfilter
br_netfilter           22209  0
bridge                136173  1 br_netfilter
[root@node01 ~]#

配置相关内核参数

[root@node01 ~]# cat /etc/sysctl.conf 
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
#
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1

[root@node01 ~]# sysctl -p
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1

6.2.4 计算节点网络配置

修改compute配置文件, 以保证compute服务可以与neutron对接

编辑 /etc/nova/nova.conf 文件并完成以下操作

在 [neutron] 部分，配置访问参数：

[neutron]
# ...
auth_url = http://openstack-vip.net:5000
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = service
username = neutron
password = neutron

重启nova服务

[root@node03 ~]# systemctl restart openstack-nova-compute.service

启动linux 桥接服务

[root@node03 ~]# systemctl enable neutron-openvswitch-agent.service
Created symlink /etc/systemd/system/multi-user.target.wants/neutron-openvswitch-agent.service → /usr/lib/systemd/system/neutron-openvswitch-agent.service.
[root@node03 ~]# systemctl start neutron-openvswitch-agent.service

到此，计算节点的网络服务就安装配置完成；

6.3 验证

验证：在控制节点上导出admin环境变量，列出加载的扩展，以验证成功启动neutron服务器进程

[root@node01 ~]# openstack extension list --network
+----------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Name                                                                                                                                                           | Alias                                 | Description                                                                                                                                              |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Address group                                                                                                                                                  | address-group                         | Support address group                                                                                                                                    |
| Address scope                                                                                                                                                  | address-scope                         | Address scopes extension.                                                                                                                                |
......

如果能够看上面显示的内容，则表示neutron服务启动了很多进程，服务没有问题；

provider网络中查看各种服务

[root@node01 ~]# openstack network agent list
+--------------------------------------+--------------------+--------+-------------------+-------+-------+---------------------------+
| ID                                   | Agent Type         | Host   | Availability Zone | Alive | State | Binary                    |
+--------------------------------------+--------------------+--------+-------------------+-------+-------+---------------------------+
| 023934fe-ce17-48cd-8a0a-45e62e000aba | Open vSwitch agent | node01 | None              | :-)   | UP    | neutron-openvswitch-agent |
| 799ea930-5732-4204-ba37-8c421e0b5843 | Metadata agent     | node01 | None              | :-)   | UP    | neutron-metadata-agent    |
| 92a041f4-1a06-44e1-a1ea-1d850998cde4 | DHCP agent         | node01 | nova              | :-)   | UP    | neutron-dhcp-agent        |
| df7fa2e5-1256-487d-8282-c55f3afca05b | Open vSwitch agent | node03 | None              | :-)   | UP    | neutron-openvswitch-agent |
+--------------------------------------+--------------------+--------+-------------------+-------+-------+---------------------------+

列出网络agent列表，看看各agent是否都是up状态

[root@node01 ~]# openstack network agent list
+--------------------------------------+--------------------+--------+-------------------+-------+-------+---------------------------+
| ID                                   | Agent Type         | Host   | Availability Zone | Alive | State | Binary                    |
+--------------------------------------+--------------------+--------+-------------------+-------+-------+---------------------------+
| 1ffab965-6296-44e8-80d7-5a37a2c1db68 | DHCP agent         | node01 | nova              | :-)   | UP    | neutron-dhcp-agent        |
| 8ff66c87-2b93-4d67-ba90-0ab728af12eb | L3 agent           | node01 | nova              | :-)   | UP    | neutron-l3-agent          |
| a4310c8d-6d64-40f1-ab37-84aaa8ccca5e | Open vSwitch agent | node01 | None              | :-)   | UP    | neutron-openvswitch-agent |
| c18ebea3-c688-4552-bff5-12c2352b230f | Open vSwitch agent | node03 | None              | :-)   | UP    | neutron-openvswitch-agent |
| f917c681-0d5e-4adb-b3ec-d0076ea96f5d | Metadata agent     | node01 | None              | :-)   | UP    | neutron-metadata-agent    |
+--------------------------------------+--------------------+--------+-------------------+-------+-------+---------------------------+

提示：能够看到node01上有4个agent，node03有一个agent都处于up状态,说明我们配置neutron agent没有问题，都正常运行着；

ok，到此neutron网络服务在控制节点和计算节点的安装配置验证就完成了；

7. 基于桥接的内外网络

前面的每个计算节点里面有两张网卡, 一张作为管理口, 直接管理节点本身, 另外一张网卡作为单独的桥接口, 用于虚拟机的流量口。本章主要是为了在每个节点中, 再单独添加一张网卡作为虚拟机流量的桥接口, 作为内部沟通的网络通信。以此保证虚拟机同时可以拥有两张网卡, 一张与互联网通信。另外一张网卡用于网络中的其他服务器通信。本次实验中，给集群中的六个节点均再添加一张192.168.10.0/24网段的网卡

环境准备, 这里需要给所有节点新添加一个网卡。这里新添加的网卡名为ens5
为所有添加网口的节点创建新的网桥, 并将新网桥桥接到新添加的网口中
```
[root@node01 ~]# ovs-vsctl add-br br-ex2
[root@node01 ~]# ovs-vsctl add-port br-ex2 ens5
```
修改控制节点的配置文件
修改配置文件/etc/neutron/plugins/ml2/ml2_conf.ini ,后面添加,internal即可。定义新添加的物理网络名称
```
[root@node01 ~]# vi /etc/neutron/plugins/ml2/ml2_conf.ini
flat_networks = external,internal
```
修改配置文件/etc/neutron/plugins/ml2/openvswitch_agent.ini ,后面添加,internal:br-ex2。定义新添加的物理网络名称桥接的具体网口
```
[root@node01 ~]# vi /etc/neutron/plugins/ml2/openvswitch_agent.ini 
bridge_mappings = external:br-ex,internal:br-ex2
```

重启控制节点neutron服务

[root@node01 ~]# systemctl restart neutron-server neutron-openvswitch-agent

修改计算节点的配置文件

[root@node03 ~]# cat /etc/neutron/plugins/ml2/openvswitch_agent.ini
bridge_mappings = external:br-ex,internal:br-ex2

重启计算节点网络服务

[root@node03 ~]# systemctl restart  neutron-openvswitch-agent

控制端创建网络

[root@node01 ~]# openstack network create  --share --external   --provider-physical-network internal   --provider-network-type flat internal-net
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | UP                                   |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2024-10-02T18:14:22Z                 |
| description               |                                      |
| dns_domain                | None                                 |
| id                        | abc5cb2f-8c3f-446c-93f5-abca2c025455 |
| ipv4_address_scope        | None                                 |
| ipv6_address_scope        | None                                 |
| is_default                | None                                 |
| is_vlan_transparent       | None                                 |
| mtu                       | 1500                                 |
| name                      | internal-net                         |
| port_security_enabled     | True                                 |
| project_id                | 26d018f3ebb94774be3e6d3a65e0f750     |
| provider:network_type     | flat                                 |
| provider:physical_network | internal                             |
| provider:segmentation_id  | None                                 |
| qos_policy_id             | None                                 |
| revision_number           | 1                                    |
| router:external           | External                             |
| segments                  | None                                 |
| shared                    | True                                 |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      |                                      |
| tenant_id                 | 26d018f3ebb94774be3e6d3a65e0f750     |
| updated_at                | 2024-10-02T18:14:22Z                 |
+---------------------------+--------------------------------------+
[root@node01 ~]#

通过刚创建的网络, 创建需要的子网

[root@node01 ~]# openstack subnet create --network internal-net --allocation-pool start=192.168.10.100,end=192.168.10.200 --dns-nameserver 223.5.5.5 --subnet-range 192.168.10.0/24 internal-net-sub
+----------------------+--------------------------------------+
| Field                | Value                                |
+----------------------+--------------------------------------+
| allocation_pools     | 192.168.10.100-192.168.10.200        |
| cidr                 | 192.168.10.0/24                      |
| created_at           | 2024-10-02T18:17:29Z                 |
| description          |                                      |
| dns_nameservers      | 223.5.5.5                            |
| dns_publish_fixed_ip | None                                 |
| enable_dhcp          | True                                 |
| gateway_ip           | 192.168.10.1                         |
| host_routes          |                                      |
| id                   | fef662c5-ad1d-42ff-adc5-09583a911885 |
| ip_version           | 4                                    |
| ipv6_address_mode    | None                                 |
| ipv6_ra_mode         | None                                 |
| name                 | internal-net-sub                     |
| network_id           | abc5cb2f-8c3f-446c-93f5-abca2c025455 |
| project_id           | 26d018f3ebb94774be3e6d3a65e0f750     |
| revision_number      | 0                                    |
| segment_id           | None                                 |
| service_types        |                                      |
| subnetpool_id        | None                                 |
| tags                 |                                      |
| updated_at           | 2024-10-02T18:17:29Z                 |
+----------------------+--------------------------------------+

查看新建子网

[root@node01 ~]# openstack subnet list 
+--------------------------------------+------------------+--------------------------------------+-----------------+
| ID                                   | Name             | Network                              | Subnet          |
+--------------------------------------+------------------+--------------------------------------+-----------------+
| c4485771-22be-4bed-8514-b87c14bbf1c9 | provider-net-sub | 84d97504-ffd7-4c4e-b5b6-1066a7b1b97b | 192.168.71.0/24 |
| fef662c5-ad1d-42ff-adc5-09583a911885 | internal-net-sub | abc5cb2f-8c3f-446c-93f5-abca2c025455 | 192.168.10.0/24 |
+--------------------------------------+------------------+--------------------------------------+-----------------+

通过页面创建虚拟机
页面创建虚拟机的过程中, 选择网络时,选择两个网口对应的网络配置。其他配置均正常操作
创建虚拟机如下。需要注意, 页面添加的网口顺序, 与虚拟机内部的识别顺序相关。页面中先添加的网口在虚拟机里面排在前面
在openstack外部ping临时创建的两台虚拟机192.168.10.0/24网段。测试网络均正常

参考链接

云计算管理平台之OpenStack网络服务neutron - Linux-1874 - 博客园 (cnblogs.com)

玩转企业云计算平台系列（六）：Openstack 网络服务 Neutron (qq.com)