溫馨提示×
您好，登錄后才能下訂單哦！
點擊 登錄注冊 即表示同意《億速云用戶服務條款》
您好，登錄后才能下訂單哦！
這篇文章主要介紹Kubernetes網絡分析中如何實現Container間通信，文中介紹的非常詳細，具有一定的參考價值，感興趣的小伙伴們一定要看完！
上圖顯示了Kubernetes的基本結構圖。
Master管理多個Slave節點
Slave節點上面可以運行多個Pod
Pod可以部署多個副本，多個副本可以運行在不同的Node上
一個Pod可以包含多個Container，一個Pod內的Container共享同樣的網絡地址空間
最重要的是最后一句話：一個Pod內的Container共享同樣的網絡地址空間。這是通過Mapped Container做到的。
基本描述為下：
容器 A 的網絡模式為正常docker的網絡模式
容器 B 的網絡模式為應用容器A的網絡模式
###2.1 共享網絡模式###
下面是一個例子來驗證，我這里創建了一個busybox的Pod。
[root@centos7-node-221 ~]$ kubectl get po
NAME READY STATUS RESTARTS AGE
busybox 1/1 Running 224 9d
[root@centos7-node-221 ~]$ kubectl describe po busybox
Name: busybox
Namespace: default
Image(s): busybox
Node: centos7-node-226/192.168.1.226
Labels: <none>
Status: Running
Reason:
Message:
IP: 172.16.58.6
Replication Controllers: <none>
Containers:
busybox:
Image: busybox
State: Running
Started: Thu, 08 Oct 2015 08:20:30 -0400
Last Termination State: Terminated
Exit Code: 0
Started: Thu, 08 Oct 2015 07:20:26 -0400
Finished: Thu, 08 Oct 2015 08:20:26 -0400
Ready: True
Restart Count: 224
Variables:
Conditions:
Type Status
Ready True
Volumes:
default-token-lv94w:
Type: Secret (a secret that should populate this volume)
SecretName: default-token-lv94w
Events:
FirstSeen LastSeen Count From SubobjectPath Reason Message
9d 37m 225 {kubelet centos7-node-226} spec.containers{busybox} pulled Container image "busybox" already present on machine
37m 37m 1 {kubelet centos7-node-226} spec.containers{busybox} Created Created with docker id fc8580292210
37m 37m 1 {kubelet centos7-node-226} spec.containers{busybox} Started Started with docker id fc8580292210我們去192.168.1.226看下這個Pod和其Container.
[root@centos7-node-226 ~]$ docker ps | grep busybox fc8580292210 busybox "sleep 3600" 37 minutes ago Up 37 minutes k8s_busybox.62fa0587_busybox_default_86e98e8c-665f-11e5-af98-525400d7abb6_7f734c4d 02d259dc8ab5 gcr.io/google_containers/pause:0.8.0 "/pause" 9 days ago Up 9 days k8s_POD.7be6d81d_busybox_default_86e98e8c-665f-11e5-af98-525400d7abb6_ff9224f5
發現有兩個容器，一個是pause容器，一個是busybox容器。其中pause容器為主網絡容器，其他容器都共享pause容器的網絡模式。我們分別看下其網絡模式。下面是兩個容器的網絡模式。
[root@centos7-node-226 ~]$ docker inspect 02d259dc8ab5 | grep NetworkMode "NetworkMode": "bridge", [root@centos7-node-226 ~]$ docker inspect fc8580292210 | grep NetworkMode "NetworkMode": "container:02d259dc8ab59c1746d54d2df24d8733b2b9379a9fdfbfdc2066429b4a934a04", # 這個container的id號碼就是上一個container的id的long形式
所以可以看到fc8580292210(busybox)使用的是pause容器的網絡空間。
讓我們進一步驗證。
下面我在 192.168.1.224 搭建了一個dns的pod，里面有4個容器，共享一個網絡空間，我們采用查看其ip地址、hostname和網絡IO的方式來鑒定。 下面是容器的id號
[root@centos7-node-224 ~]$ docker ps | grep dns b00a08d078d6 dockerimages.yinnut.com:15043/skydns:2015-03-11-001 "/skydns -machines=h 8 hours ago Up 8 hours k8s_skydns.c878079e_kube-dns-v9-y05vd_kube-system_12725077-64c0-11e5-9309-525400d7abb6_46f95e60 4e843585b938 dockerimages.yinnut.com:15043/exechealthz:1.0 "/exechealthz '-cmd= 11 days ago Up 11 days k8s_healthz.8ab20f84_kube-dns-v9-y05vd_kube-system_12725077-64c0-11e5-9309-525400d7abb6_f7c469e5 296ff779abb2 dockerimages.yinnut.com:15043/kube2sky:1.11 "/kube2sky -domain=c 11 days ago Up 11 days k8s_kube2sky.2a46d768_kube-dns-v9-y05vd_kube-system_12725077-64c0-11e5-9309-525400d7abb6_349c7246 f0118fac6952 dockerimages.yinnut.com:15043/etcd:2.0.9 "/usr/local/bin/etcd 11 days ago Up 11 days k8s_etcd.64e02c2f_kube-dns-v9-y05vd_kube-system_12725077-64c0-11e5-9309-525400d7abb6_9235054b f281dbf1ec41 gcr.io/google_containers/pause:0.8.0 "/pause" 11 days ago Up 11 days k8s_POD.6e934112_kube-dns-v9-y05vd_kube-system_12725077-64c0-11e5-9309-525400d7abb6_a8ea96d0
我們查看前三個 b00a08d078d6 4e843585b938 296ff779abb2 的上述屬性。
[root@centos7-node-224 ~]$ for id in b00a08d078d6 4e843585b938 296ff779abb2 ; do echo $id; docker exec $id cat /etc/hosts ; docker exec $id cat /etc/resolv.conf ; echo "" ; done b00a08d078d6 172.16.60.4 kube-dns-v9-y05vd 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters nameserver 192.168.1.208 search 8.8.8.8 options ndots:5 4e843585b938 172.16.60.4 kube-dns-v9-y05vd 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters nameserver 192.168.1.208 search 8.8.8.8 options ndots:5 296ff779abb2 172.16.60.4 kube-dns-v9-y05vd 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters nameserver 192.168.1.208 search 8.8.8.8 options ndots:5
[root@centos7-node-224 ~]$ for id in b00a08d078d6 4e843585b938 296ff779abb2 ; do echo $id; docker exec $id ip a ; echo "" ; done b00a08d078d6 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 4e843585b938 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 296ff779abb2 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever
[root@centos7-node-224 ~]$ for id in b00a08d078d6 4e843585b938 296ff779abb2 ; do echo $id; docker exec $id netstat -lan ; echo "" ; done b00a08d078d6 Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 127.0.0.1:4001 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:2379 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:2380 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:7001 0.0.0.0:* LISTEN tcp 0 0 172.16.60.4:48582 10.254.0.1:443 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:35394 ESTABLISHED tcp 0 0 172.16.60.4:48584 10.254.0.1:443 ESTABLISHED tcp 0 0 127.0.0.1:60161 127.0.0.1:2379 ESTABLISHED tcp 0 0 127.0.0.1:51445 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:51445 ESTABLISHED tcp 0 0 127.0.0.1:35550 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:35394 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:2379 127.0.0.1:60161 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:51433 ESTABLISHED tcp 0 0 127.0.0.1:51433 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:35550 ESTABLISHED Active UNIX domain sockets (servers and established) Proto RefCnt Flags Type State I-Node Path 4e843585b938 Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 127.0.0.1:4001 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:2379 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:2380 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:7001 0.0.0.0:* LISTEN tcp 0 0 172.16.60.4:48582 10.254.0.1:443 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:35394 ESTABLISHED tcp 0 0 172.16.60.4:48584 10.254.0.1:443 ESTABLISHED tcp 0 0 127.0.0.1:60161 127.0.0.1:2379 ESTABLISHED tcp 0 0 127.0.0.1:51445 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:51445 ESTABLISHED tcp 0 0 127.0.0.1:35550 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:35394 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:2379 127.0.0.1:60161 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:51433 ESTABLISHED tcp 0 0 127.0.0.1:51433 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:35550 ESTABLISHED Active UNIX domain sockets (servers and established) Proto RefCnt Flags Type State I-Node Path 296ff779abb2 Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 127.0.0.1:4001 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:2379 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:2380 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:7001 0.0.0.0:* LISTEN tcp 0 0 172.16.60.4:48582 10.254.0.1:443 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:35394 ESTABLISHED tcp 0 0 172.16.60.4:48584 10.254.0.1:443 ESTABLISHED tcp 0 0 127.0.0.1:60161 127.0.0.1:2379 ESTABLISHED tcp 0 0 127.0.0.1:51445 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:51445 ESTABLISHED tcp 0 0 127.0.0.1:35550 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:35394 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:2379 127.0.0.1:60161 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:51433 ESTABLISHED tcp 0 0 127.0.0.1:51433 127.0.0.1:4001 ESTABLISHED tcp 0 0 127.0.0.1:4001 127.0.0.1:35550 ESTABLISHED Active UNIX domain sockets (servers and established) Proto RefCnt Flags Type State I-Node Path
下面分析下最復雜的Container之間的通信。
先說最簡單的, Pod內的Container通信，由于共享網絡地址空間，直接訪問127.0.0.1即可。
例子： 192.168.1.224的fluentd-elasticsearch容器要連接192.168.1.223的elasticsearch-logging容器。
192.168.1.223的elasticsearch-logging容器及其IP地址:
[root@centos7-node-223 ~]$ docker ps |grep elasticsearch-logging 667cfd84c979 dockerimages.yinnut.com:15043/elasticsearch:1.7 "/run.sh" 12 days ago Up 12 days k8s_elasticsearch-logging.89fda9f_elasticsearch-logging-v1-i8x6q_kube-system_8b558d2c-62a3-11e5-9d7b-525400d7abb6_2a02a2c8 5201c8cbdebd gcr.io/google_containers/pause:0.8.0 "/pause" 12 days ago Up 12 days k8s_POD.8ecd2043_elasticsearch-logging-v1-i8x6q_kube-system_8b558d2c-62a3-11e5-9d7b-525400d7abb6_4022db35 [root@centos7-node-223 ~]$ docker inspect 5201c8cbdebd |grep IPAddress "IPAddress": "172.16.77.4", #IP地址 "SecondaryIPAddresses": null,
可以看到elasticsearch-logging的容器的Pod的IP地址為172.16.77.4
192.168.1.223的fluentd-elasticsearch容器及其IP地址:
[root@centos7-node-224 ~]$ docker ps |grep fluentd-elasticsearch d326d81468b5 gcr.io/google_containers/fluentd-elasticsearch:1.11 "td-agent -q" 12 days ago Up 12 days k8s_fluentd-elasticsearch.27a08aa3_fluentd-elasticsearch-centos7-node-224_kube-system_7dcc6ce562f3742190a876fda85e2359_58c54ef3 f9b76639d241 gcr.io/google_containers/pause:0.8.0 "/pause" 12 days ago Up 12 days k8s_POD.7be6d81d_fluentd-elasticsearch-centos7-node-224_kube-system_7dcc6ce562f3742190a876fda85e2359_333e52c0 [root@centos7-node-224 ~]$ docker inspect f9b76639d241 | grep IPAddress "IPAddress": "172.16.60.2", "SecondaryIPAddresses": null,
可以看到fluentd-elasticsearch的容器的Pod的IP地址為172.16.60.2
我們看下 fluentd-elasticsearch 的網絡連接情況
[root@centos7-node-224 ~]$ docker exec d326d81468b5 netstat -nla | grep 172.16.77.4 tcp 0 0 172.16.60.2:56354 172.16.77.4:9200 TIME_WAIT tcp 0 0 172.16.60.2:56350 172.16.77.4:9200 TIME_WAIT tcp 0 0 172.16.60.2:56347 172.16.77.4:9200 TIME_WAIT tcp 0 0 172.16.60.2:56357 172.16.77.4:9200 TIME_WAIT tcp 0 0 172.16.60.2:56344 172.16.77.4:9200 TIME_WAIT tcp 0 0 172.16.60.2:56352 172.16.77.4:9200 TIME_WAIT
可以看到其的確是連接了 172.16.77.4 的9200端口。 而對方 elasticsearch-logging 容器的確開啟了9200端口
[root@centos7-node-223 ~]$ docker exec 667cfd84c979 ss -l|grep LISTEN tcp LISTEN 0 50 :::9200 :::* tcp LISTEN 0 50 :::9300 :::*
那么這個過程是如何完成的呢？？？
192.168.1.224/fluentd-elasticsearch -> 192.168.1.223/elasticsearch-logging
192.168.1.224/fluentd-elasticsearch 需要連接到elasticsearch-logging容器.
域名到IP對應。 elasticsearch-logging -> 解析為10.254.24.205
root@fluentd-elasticsearch-centos7-node-224:/$ dig elasticsearch-logging ; <<>> DiG 9.9.5-3ubuntu0.5-Ubuntu <<>> elasticsearch-logging ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 39181 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;elasticsearch-logging. IN A ;; Query time: 1 msec ;; SERVER: 10.254.0.10#53(10.254.0.10) ;; WHEN: Fri Oct 09 06:04:35 UTC 2015 ;; MSG SIZE rcvd: 39
訪問該IP地址10.254.24.205:9200端口。根據路由，請求將會到達網關172.16.60.1,也就是這個docker的宿主機的docker0網卡地址。
# 容器內 root@fluentd-elasticsearch-centos7-node-224:/$ ip route default via 172.16.60.1 dev eth0 172.16.60.0/24 dev eth0 proto kernel scope link src 172.16.60.2 # 物理機 [root@centos7-node-224 ~]$ ifconfig docker0 docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet 172.16.60.1 netmask 255.255.255.0 broadcast 0.0.0.0 inet6 fe80::5484:7aff:fefe:9799 prefixlen 64 scopeid 0x20<link> ether 56:84:7a:fe:97:99 txqueuelen 0 (Ethernet) RX packets 10182154 bytes 1777103288 (1.6 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 11195534 bytes 2271907616 (2.1 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
iptables負責轉發請求到192.168.1.224:36967，而這個端口上kube-proxy進程在監聽。
[root@centos7-node-224 ~]$ iptables-save | grep 10.254.24.205 |grep 9200 -A KUBE-PORTALS-CONTAINER -d 10.254.24.205/32 -p tcp -m comment --comment "kube-system/elasticsearch-logging:" -m tcp --dport 9200 -j REDIRECT --to-ports 36967 -A KUBE-PORTALS-HOST -d 10.254.24.205/32 -p tcp -m comment --comment "kube-system/elasticsearch-logging:" -m tcp --dport 9200 -j DNAT --to-destination 192.168.1.224:36967 [root@centos7-node-224 ~]$ netstat -nlp|grep 36967 tcp6 0 0 :::36967 :::* LISTEN 930/kube-proxy
誰負責響應10.254.24.205:9200的請求？由上述分析，看起來是kube-proxy，那么kube-proxy進程看起來是個proxy，那么被轉發給誰處理？當然給Pod啦。可以到這個服務的Selector是k8s-app=elasticsearch-logging
[root@centos7-node-224 ~]$ kubectl get svc --all-namespaces | grep 10.254.24.205 kube-system elasticsearch-logging 10.254.24.205 <none> 9200/TCP k8s-app=elasticsearch-logging 14d
找到其對應的Pod為elasticsearch-logging-v1-gph5i和elasticsearch-logging-v1-i8x6q
[root@centos7-node-224 ~]$ kubectl get po -l k8s-app=elasticsearch-logging --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system elasticsearch-logging-v1-gph5i 1/1 Running 6 14d kube-system elasticsearch-logging-v1-i8x6q 1/1 Running 5 14d
我們查看其中的elasticsearch-logging-v1-i8x6q容器的IP地址，發現為172.16.77.4
[root@centos7-node-224 ~]$ kubectl describe po elasticsearch-logging-v1-i8x6q --namespace=kube-system | grep IP IP: 172.16.77.4
故而很明顯kube-proxy 會把部分請求轉發給 其中的一個Pod來處理，而這個Pod的IP地址是172.16.77.4 . 而 172.16.77.4 這個Pod 在 192.168.1.223 機器上.
[root@centos7-node-223 ~]$ docker inspect 5201c8cbdebd | grep IPAddress "IPAddress": "172.16.77.4", "SecondaryIPAddresses": null,
那如何與172.16.77.4進行通信呢？跨機器之間通信則采用flannel等諸如此類的overlay網絡或者ovs等L2網絡。
以上是“Kubernetes網絡分析中如何實現Container間通信”這篇文章的所有內容，感謝各位的閱讀！希望分享的內容對大家有幫助，更多相關知識，歡迎關注億速云行業資訊頻道！
免責聲明：本站發布的內容（圖片、視頻和文字）以原創、轉載和分享為主，文章觀點不代表本網站立場，如果涉及侵權請聯系站長郵箱：is@yisu.com進行舉報，并提供相關證據，一經查實，將立刻刪除涉嫌侵權內容。
