0%

Linux虚拟网络学习笔记

Linux虚拟网络是近几年虚拟化及容器等技术的基础,掌握了这些基础,可以更深入的理解openstack、docker的网络功能,以及测试过程中的问题定位。

本文通过实验的方式学习这些虚拟网络功能,包括tap设备,veth pair, bridge及router

环境准备

注:本实验基于Ubuntu16.04

1
2
$ apt install uml-utilities
$ apt install bridge-utils

tap设备

Linux的tun/tap驱动实现了虚拟网卡的功能,tun表示虚拟的是点对点设备,tap表示虚拟的是以太网设备

tap位于网络OSI模型的二层(数据链路层),tun位于网络的三层。

1. 创建tap

1
2
$ tunctl -t tap_test
$ ifconfig tap_test 192.168.100.1/24

2. 创建namespace

1
$ ip netns add ns_test

3. 迁移网口到namespace

1
$ ip link set tap_test netns ns_test

迁移后,对应的ip会没有

4. 进入namespace并查看接口信息

1
2
3
4
5
6
7
8
9
10
$ ip netns exec ns_test /bin/bash
$ ifconfig tap_test 192.168.50.1/24
$ ifconfig
tap_test Link encap:Ethernet HWaddr 76:71:70:f2:ac:f6
inet addr:192.168.50.1 Bcast:192.168.50.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

veth pair

Veth pair是一对虚拟网卡,从一个veth网卡发出的数据包可以直接到达它的peer veth。相当于两个接口之间接着一根网线

1. 创建两个ns

1
2
$ ip netns add ns1
$ ip netns add ns2

2. 创建veth pair

1
$ ip link add tap1 type veth peer name tap2

3. 迁移网口到namespace

1
2
$ ip link set tap1 netns ns1
$ ip link set tap2 netns ns2

4. 分别绑定IP地址

1
2
$ ip netns exec ns1 ifconfig tap1 192.168.40.1/24
$ ip netns exec ns2 ifconfig tap2 192.168.40.2/24

5. 测试连通性

1
2
3
4
$ ip netns exec ns1 ping 192.168.40.2
PING 192.168.40.2 (192.168.40.2) 56(84) bytes of data.
64 bytes from 192.168.40.2: icmp_seq=1 ttl=64 time=0.096 ms
64 bytes from 192.168.40.2: icmp_seq=2 ttl=64 time=0.049 ms

bridge

两个namespace中的网络可以通过veth pair访问,但3个之间甚至多个互通,veth pair就无法胜任,此时需要用到bridge/switch

下面的实验模拟4个namespace中的接口通过bridge互通

1. 创建veth pair

1
2
3
4
$ ip link add tap1 type veth peer name tap1_peer
$ ip link add tap2 type veth peer name tap2_peer
$ ip link add tap3 type veth peer name tap3_peer
$ ip link add tap4 type veth peer name tap4_peer

2. 创建namespace并迁移tap接口

1
2
3
4
5
6
7
8
9
$ ip netns add ns1
$ ip netns add ns2
$ ip netns add ns3
$ ip netns add ns4
# 迁移tap接口
$ ip link set tap1 netns ns1
$ ip link set tap2 netns ns2
$ ip link set tap3 netns ns3
$ ip link set tap4 netns ns4

3. 创建bridge

1
$ brctl addbr br1

4. 将对应的tap添加到bridge中

1
2
3
4
$ brctl addif br1 tap1_peer
$ brctl addif br1 tap2_peer
$ brctl addif br1 tap3_peer
$ brctl addif br1 tap4_peer

5. 配置IP地址

1
2
3
4
$ ip netns exec ns1 ifconfig tap1 192.168.50.1/24
$ ip netns exec ns2 ifconfig tap2 192.168.50.2/24
$ ip netns exec ns3 ifconfig tap3 192.168.50.3/24
$ ip netns exec ns4 ifconfig tap4 192.168.50.4/24

注: 此时是无法相互访问

6. 设置网桥及对应接口状态为up

1
2
3
4
5
$ ifconfig br1 up
$ ifconfig tap1_peer up
$ ifconfig tap2_peer up
$ ifconfig tap3_peer up
$ ifconfig tap4_peer up

7. 测试连通性

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ ip netns exec ns4 ping 192.168.50.1 -c 1
PING 192.168.50.1 (192.168.50.1) 56(84) bytes of data.
64 bytes from 192.168.50.1: icmp_seq=1 ttl=64 time=0.095 ms

--- 192.168.50.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.095/0.095/0.095/0.000 ms
$ ip netns exec ns4 ping 192.168.50.2 -c 1
PING 192.168.50.2 (192.168.50.2) 56(84) bytes of data.
64 bytes from 192.168.50.2: icmp_seq=1 ttl=64 time=0.106 ms

--- 192.168.50.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.106/0.106/0.106/0.000 ms

...

router

注:Linux默认不开启转发功能,实验前需要先打开

修改/etc/sysctl.conf文件,设置net.ipv4.ip_forward=1,重启生效

临时修改: echo "1" > /proc/sys/net/ipv4/ip_forward

1. 创建veth pair

1
2
$ ip link add tap5 type veth peer name tap5_peer
$ ip link add tap6 type veth peer name tap6_peer

2. 创建namespace并迁移tap接口

1
2
3
4
$ ip netns add ns5
$ ip netns add ns6
$ ip link set tap5 netns ns5
$ ip link set tap6 netns ns6

3. 配置IP地址

1
2
3
4
$ ifconfig tap5_peer 192.168.100.1/24
$ ifconfig tap6_peer 192.168.200.1/24
$ ip netns exec ns5 ifconfig tap5 192.168.100.2/24
$ ip netns exec ns6 ifconfig tap6 192.168.200.2/24

4. 为namespace配置路由

1
2
$ ip netns exec ns5 route add -net 192.168.200.0/24 gw 192.168.100.1
$ ip netns exec ns6 route add -net 192.168.100.0/24 gw 192.168.200.1

5. 测试

1
2
3
4
$ ip netns exec ns5 ping 192.168.200.2
PING 192.168.200.2 (192.168.200.2) 56(84) bytes of data.
64 bytes from 192.168.200.2: icmp_seq=1 ttl=63 time=0.077 ms
64 bytes from 192.168.200.2: icmp_seq=2 ttl=63 time=0.087 ms

tun

tun应该是tunnel的缩写,启用了IP层隧道功能

Linux原生支持5种隧道{ ipip | gre | sit | isatap | vti }

忽略上图中的tun1和tun2后,整个topo与router中的一样。那么先按照router章节创建两个ns及配置

1. 创建veth pair并分别迁移到对应的namespace

1
2
3
4
5
6
$ ip link add tap1 type veth peer name tap1_peer
$ ip link add tap2 type veth peer name tap2_peer
$ ip netns add ns1
$ ip netns add ns2
$ ip link set tap1 netns ns1
$ ip link set tap2 netns ns2

2. 配置IP和路由

1
2
3
4
5
6
$ ifconfig tap1_peer 192.168.100.1/24
$ ifconfig tap2_peer 192.168.200.1/24
$ ip netns exec ns1 ifconfig tap1 192.168.100.2/24
$ ip netns exec ns2 ifconfig tap2 192.168.200.2/24
$ ip netns exec ns1 route add -net 192.168.200.0/24 gw 192.168.100.1
$ ip netns exec ns2 route add -net 192.168.100.0/24 gw 192.168.200.1

3. 在ns1中创建tun1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ ip netns exec ns1 ip tunnel add tun1 mode ipip remote 192.168.200.2 local 192.168.100.2 ttl 255
$ ip netns exec ns1 ip addr add 192.168.50.10 peer 192.168.60.10 dev tun1
$ ip netns exec ns1 ifconfig tun1 up
$ ip netns exec ns1 ifconfig
tap1 Link encap:Ethernet HWaddr 46:a0:97:02:8c:07
inet addr:192.168.100.2 Bcast:192.168.100.255 Mask:255.255.255.0
inet6 addr: fe80::44a0:97ff:fe02:8c07/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:928 (928.0 B) TX bytes:928 (928.0 B)

tun1 Link encap:IPIP Tunnel HWaddr
inet addr:192.168.50.10 P-t-P:192.168.60.10 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

4. 在ns2中创建tun2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ ip netns exec ns2 ip tunnel add tun2 mode ipip remote 192.168.100.2 local 192.168.200.2 ttl 255
$ ip netns exec ns2 ip addr add 192.168.60.10 peer 192.168.50.10 dev tun2
$ ip netns exec ns2 ifconfig tun2 up
$ ip netns exec ns2 ifconfig
tap2 Link encap:Ethernet HWaddr aa:2e:e9:18:94:95
inet addr:192.168.200.2 Bcast:192.168.200.255 Mask:255.255.255.0
inet6 addr: fe80::a82e:e9ff:fe18:9495/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:928 (928.0 B) TX bytes:928 (928.0 B)

tun2 Link encap:IPIP Tunnel HWaddr
inet addr:192.168.60.10 P-t-P:192.168.50.10 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

5. 测试tun的连通性

1
2
3
4
$ ip netns exec ns1 ping 192.168.60.10
PING 192.168.60.10 (192.168.60.10) 56(84) bytes of data.
64 bytes from 192.168.60.10: icmp_seq=1 ttl=64 time=0.333 ms
...

6. 抓包看看

1
2
3
4
5
6
7
8
9
10
11
12
$ ip netns exec ns2 tcpdump -i tap2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap2, link-type EN10MB (Ethernet), capture size 262144 bytes
09:55:19.399964 IP 192.168.100.2 > 192.168.200.2: IP 192.168.50.10 > 192.168.60.10: ICMP echo request, id 17197, seq 1, length 64 (ipip-proto-4)
09:55:19.400004 IP 192.168.200.2 > 192.168.100.2: IP 192.168.60.10 > 192.168.50.10: ICMP echo reply, id 17197, seq 1, length 64 (ipip-proto-4)
...
$ ip netns exec ns2 tcpdump -i tun2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun2, link-type RAW (Raw IP), capture size 262144 bytes
09:55:57.735663 IP 192.168.50.10 > 192.168.60.10: ICMP echo request, id 17199, seq 1, length 64
09:55:57.735685 IP 192.168.60.10 > 192.168.50.10: ICMP echo reply, id 17199, seq 1, length 64
...

注:namespace中抓包可能不会立即打印在屏幕上

7. 查看路由表项

1
2
3
4
5
6
$ ip netns exec ns1 route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.60.10 0.0.0.0 255.255.255.255 UH 0 0 0 tun1
192.168.100.0 0.0.0.0 255.255.255.0 U 0 0 0 tap1
192.168.200.0 192.168.100.1 255.255.255.0 UG 0 0 0 tap1

可以看到,ns1中自动创建了一条指向192.168.60.10的路由,下一跳是tun1

小结

Openstack的neutron组件正是基于这些Linux虚拟网络功能实现了虚拟机之间的网络通道。

其中,tap、tun、veth pair被用于bridge之间的连接、bridge与vm的连接、bridge与router之间的连接。

而bridge提供二层转发功能,router提供三层转发功能。