搭建 LVS＋HA網站服務群集

搭建keepalived+DR/NAT模式的高可用web群集，這篇博文以keepalived+DR的環境來搭建一個高可用的web服務群集。

相關技術文檔可以去本人主頁查看：https://blog.51cto.com/14227204?，https://blog.51cto.com/14227204/2438901

環境如下：

一、環境分析：

1、2個調度器和2個web節點使用同一個網段地址，可以直接和外網通信。為了共享存儲的

安全性，一般將web節點和存儲服務器規劃到內網環境，所以web節點必須有兩個及以上

網卡的接口。

2、我這里資源有限，也為了配置方便，所以調度器和web節點分別只有兩個，在web訪問請

求量不大的情況下，足夠了，但是若訪問請求比較大，那么最少要分別配置三個調度器和

web節點，如果只有兩個web節點的話，訪問量又比較大，那么一旦有一個宕機了，那剩下

一個獨苗必定會因為扛不住激增的訪問請求，而被打死。

3、準備系統映像，以便安裝相關服務。

4、自行配置防火墻策略和除了VIP之外的IP地址（我這里直接關閉了防火墻）。

5、keepalived會自動調用IP_vs模塊，所以無需手動加載。

二、開始搭建：

配置主調度器:

[root@lvs1?/]#?yum?-y?install?ipvsadm?keepalived????????#?安裝keepalived?和?ipvsadm管理工具
[root@lvs1?keepalived]#?vim?/etc/sysctl.conf????????#?調整內核參數,關閉ICMP重定向
...........
net.ipv4.conf.all.send_redirects?=?0
net.ipv4.conf.default.send_redirects?=?0
net.ipv4.conf.ens33.send_redirects?=?0
[root@lvs1?/]#?sysctl?-p???????????????????????#?刷新使配置生效
net.ipv4.conf.all.send_redirects?=?0
net.ipv4.conf.default.send_redirects?=?0
net.ipv4.conf.ens33.send_redirects?=?0
[root@lvs1?/]#?cd?/etc/keepalived/
[root@lvs1?keepalived]#?cp?keepalived.conf?keepalived.conf.bak????????#?復制一份keepalived?主配文件作為備份,以免修改時出錯
[root@lvs1?/]#?vim?/etc/keepalived/keepalived.conf???????????#?編輯主配文件
!?Configuration?File?for?keepalived

global_defs?{
???notification_email?{
?????acassen@firewall.loc
?????failover@firewall.loc?????????????#?當出錯時,將報錯信息發送到的收件人地址,可根據需要填寫
?????sysadmin@firewall.loc
???}
???notification_email_from?Alexandre.Cassen@firewall.loc????#發件人姓名、地址（可不做修改）???
???smtp_server?192.168.200.1
???smtp_connect_timeout?30
???router_id?LVS1????????????????#本服務器的名稱改一下，在群集中所有調度器名稱里必須唯一
}??

vrrp_instance?VI_1?{
????state?MASTER?????????????#?設為主調度器
????interface?ens33??????????#承載VIP地址的物理網卡接口根據實際情況改一下
????virtual_router_id?51???????
????priority?100??????????#?主調度器的優先級
????advert_int?1
????authentication?{?????????#?主?從熱備認證信息
????????auth_type?PASS
????????auth_pass?1111
????}
????virtual_ipaddress?{????????#?指定群集?VIP地址
????????200.0.0.100
????}???
}

virtual_server?200.0.0.100?80?{??????#?虛擬服務器地址(VIP)?端口
????delay_loop?15????????????????#?健康檢查的間隔時間
????lb_algo?rr????????????????#?輪詢調度算法
????lb_kind?DR??????????????#?指定工作模式,這里為DR,也可改為NAT
????!?persistence_timeout?50??????????#為了一會測試看到效果，將連接保持這行前加“?！”將該行注釋掉
????protocol?TCP
????
????real_server?200.0.0.3?80?{??????#?web節點的地址及端口
????????weight?1
????????TCP_CHECK?{
????????????connect_port?80
????????????connect_timeout?3
????????????nb_get_retry?3
????????????????????????delay_before_retry?3
????????}
????}
????????real_server?200.0.0.4?80?{?????#?另一?web節點地址及端口
????????????????weight?1
????????????????TCP_CHECK?{
????????????????????????connect_port?80??????????#?配置連接端口
????????????????????????connect_timeout?3
????????????????????????nb_get_retry?3
????????????????????????delay_before_retry?3
????????}
????}
}
[root@lvs1?/]#?systemctl?restart?keepalived?
[root@lvs1?/]#?systemctl?enable?keepalived

主調度器到這就告一段落配置完成了:

配置從調度器:

[root@localhost?/]#?yum?-y?install?keepalived?ipvsadm
[root@localhost?/]#?scp?root@200.0.0.1:/etc/sysctl.conf?/etc/???????#?可通過scp命令將配置較繁雜的復制過來
root@200.0.0.1's?password:?
sysctl.conf?????????????
??????
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????100%??566?????0.6KB/s???00:00????[root@localhost?/]#?sysctl?-p
[root@localhost?/]#?sysctl?-p???????????#?刷新使配置生效
net.ipv4.conf.all.send_redirects?=?0
net.ipv4.conf.default.send_redirects?=?0
net.ipv4.conf.ens33.send_redirects?=?0
[root@localhost?/]#?vim?/etc/keepalived/keepalived.conf?
......................
router_id?LVS2???????????????????????#?route-id?要不一樣
vrrp_instance?VI_1?{
????state?BACKUP???????????????#?狀態改為?BACKUP?最好大寫
????interface?ens33??????????????#?網卡如果一樣的話可不更改
????virtual_router_id?51
????priority?90????????????????#?優先級要比主調度器小
????advert_int?1
????authentication?{
????????auth_type?PASS
????????auth_pass?1111
????}
????virtual_ipaddress?{???????????????#?就需要改這些其他配置均與主調度器相同
????????200.0.0.100
????}
}
[root@localhost?/]#?systemctl?enable?keepalived
[root@localhost?/]#?systemctl?restart??keepalived?????????#?重啟服務使配置生效

若需要部署多個從調度器，按照以上這個從（備份）調度器配置即可

web1節點配置:

[root@web1?/]#?cd?/etc/sysconfig/network-scripts/
[root@web1?network-scripts]#?cp?ifcfg-lo?ifcfg-lo:0
[root@web1?network-scripts]#?vim?ifcfg-lo:0
DEVICE=lo:0
IPADDR=200.0.0.100???????????????#?VIP??地址
NETMASK=255.255.255.255??????????#?掩碼為1
ONBOOT=yes
[root@web1?network-scripts]#?ifup?lo:0??????#?啟動虛接口
[root@web1?network-scripts]#?ifconfig?lo:0??????#?查看配置有無生效
lo:0:?flags=73<UP,LOOPBACK,RUNNING>??mtu?65536
????????inet?200.0.0.100??netmask?255.255.255.255
????????loop??txqueuelen?1??(Local?Loopback)
[root@web1?/]#?route?add?-host?200.0.0.100?dev?lo:0????????????#?添加本地路由
[root@web1?/]#?vim?/etc/rc.local???????????????#設置開機自動，添加這條路由記錄??????????????
????????????????................................
/sbin/route?add?-host?200.0.0.100?dev?lo:0
[root@web1?/]#?vim?/etc/sysctl.conf?????#?調整/proc參數,關閉?ARP響應
net.ipv4.conf.all.arp_ignore?=?1
net.ipv4.conf.all.arp_announce?=?2
net.ipv4.conf.default.arp_ignore?=?1
net.ipv4.conf.default.arp_announce?=?2
net.ipv4.conf.lo.arp_ignore?=?1
net.ipv4.conf.lo.arp_announce?=?2
[root@web1?/]#?sysctl?-p?????????????#?刷新使配置生效
net.ipv4.conf.all.arp_ignore?=?1
net.ipv4.conf.all.arp_announce?=?2
net.ipv4.conf.default.arp_ignore?=?1
net.ipv4.conf.default.arp_announce?=?2
net.ipv4.conf.lo.arp_ignore?=?1
net.ipv4.conf.lo.arp_announce?=?2
[root@web1?/]#?yum?-y?install?httpd
[root@web1?/]#?echo?test1.com?>?/var/www/html/index.html
[root@web1?/]#?systemctl?start?httpd
[root@web1?/]#?systemctl?enable?httpd

web2節點和web1節點配置相同，這里我就省略了，但是這里我為了方便看出驗證效果，將web2的測試文件寫為test2.com

若訪問到的是同一頁面，在排除配置上錯誤的情況下，可以打開多個網頁，或者稍等一會再刷新，因為它可能有一個保持連接的時間，所以會存在延遲。

三、搭建 NFS 共享存儲服務：

[root@nfs?/]#?mkdir?opt/wwwroot
[root@nfs?/]#?vim?/etc/exports???????#?編寫配置文件
/opt/wwwroot????192.168.1.0/24(rw,sync,no_root_squash)
[root@nfs?/]#?systemctl?restart?nfs?????????#?重啟服務使配置生效
[root@nfs?/]#?systemctl?restart?rpcbind
[root@nfs?/]#?showmount?-e??????????????????#?查看本機發布的掛載目錄
Export?list?for?nfs:
/opt/wwwroot?192.168.1.0/24
[root@nfs?/]#?echo?nfs.test.com?>?/opt/wwwroot/index.html

所有節點掛載共享存儲目錄：

[root@web1?/]#?showmount?-e?192.168.1.5???????#?查看共享服務器共享的所有目錄
Export?list?for?192.168.1.5:
/opt/wwwroot?192.168.1.0/24
[root@web1?/]#?mount?192.168.1.5:/opt/wwwroot/?/var/www/html/?????????#?掛載到本地
[root@web1?/]#?vim?/etc/fstab???????????????#設置自動掛載?
???????????????????.........................
192.168.1.5:/opt/wwwroot??/var/www/html???nfs???defaults,_netdev?0?0

web1和web2都需要掛載

1）VIP在哪個調度器上，查詢該調度器承載VIP地址的物理接口，即可看到VIP地址（VIP地址在備份調度器上查不到的）：

[root@LVS1?~]#?ip?a?show?dev?ens33??????????????#查詢承載VIP地址的物理網卡ens332:?ens33:?<BROADCAST,MULTICAST,UP,LOWER_UP>?ate?UP?groupn?1000
???link/ether?00:0c:29:77:2c:03?brd?ff:ff:ff:ff:ff:ff
???inet?200.0.0.1/24?brd?200.0.0.255?scope?global?noprefixroute?ens33
??????valid_lft?forever?preferred_lft?forever
???inet?200.0.0.100/32?scope?global?ens33???????????????????#VIP地址。
??????valid_lft?forever?preferred_lft?forever
???inet6?fe80::95f8:eeb7:2ed2:d13c/64?scope?link?noprefixroute?
??????valid_lft?forever?preferred_lft?forever

2）查詢有哪些web節點：

[root@LVS1?~]#?ipvsadm?-ln??????????????????#查詢web節點池及VIPIP?Virtual?Server?version?1.2.1?(size=4096)
Prot?LocalAddress:Port?Scheduler?Flags
???RemoteAddress:Port???????????Forward?Weight?ActiveConn?InActConn
TCP??200.0.0.100:80?rr??
???200.0.0.3:80?????????????????Route???1??????0??????????0?????????
??200.0.0.4:80?????????????????Route???1??????0??????????0

3）模擬Web2節點和主調度器宕機，并在備份調度器上再次查詢VIP以及web節點：

[root@LVS2?~]#?ip?a?show?dev?ens33???????#可以看到VIP地址已經轉移到了備份調度器上2:?ens33:?<BROADCAST,MULTICAST,UP,LOWER_UP>?
????link/ether?00:0c:29:9a:09:98?brd?ff:ff:ff:ff:ff:ff
????inet?200.0.0.2/24?brd?200.0.0.255?scope?global?noprefixroute?ens33
???????valid_lft?forever?preferred_lft?forever
????inet?200.0.0.100/32?scope?global?ens33??????????????????????#VIP地址。
???????valid_lft?forever?preferred_lft?forever
????inet6?fe80::3050:1a9b:5956:5297/64?scope?link?noprefixroute?
???????valid_lft?forever?preferred_lft?forever
[root@LVS2?~]#?ipvsadm?-ln???????????????????#Web2節點宕機后，就查不到了。IP?Virtual?Server?version?1.2.1?(size=4096)
Prot?LocalAddress:Port?Scheduler?Flags
??->?RemoteAddress:Port???????????Forward?Weight?ActiveConn?InActConn
TCP??200.0.0.100:80?rr
??->?200.0.0.3:80?????????????????Route???1??????0??????????0?????????
????#當主調度器或Web2節點恢復正常后，將會自動添加到群集中，并且正常運行。

4）查看調度器故障切換時的日志消息：

[root@LVS2?~]#?tail?-30?/var/log/messages