kubernetes實踐之二十六：GlusterFS

發布時間：2020-08-07 18:36:35 來源：ITPUB博客閱讀：177 作者：百聯達欄目：云計算

一：前言
GlusterFS是Scale-Out存儲解決方案Gluster的核心，它是一個開源的分布式文件系統，具有強大的橫向擴展能力，通過擴展能夠支持數PB存儲容量和處理數千客戶端。GlusterFS借助TCP/IP或InfiniBand RDMA網絡將物理分布的存儲資源聚集在一起，使用單一全局命名空間來管理數據。GlusterFS基于可堆疊的用戶空間設計，可為各種不同的數據負載提供優異的性能。

服務器：
10.116.137.196 k8s_master
10.116.82.28 k8s_node1
10.116.36.57 k8s_node2

二：安裝glusterfs
我們直接在物理機上使用yum安裝，在三臺服務器上分別執行以下安裝命令。

1.先安裝 gluster 源
yum install centos-release-gluster -y

2.安裝 glusterfs 組件
yum install -y glusterfs glusterfs-server glusterfs-fuse glusterfs-rdma glusterfs-geo-replication glusterfs-devel

3.創建 glusterfs 目錄
mkdir /mnt/glusterd

4.修改 glusterd 目錄

點擊(此處)折疊或打開

volume management
type mgmt/glusterd
option working-directory /mnt/glusterd
option transport-type socket,rdma
option transport.socket.keepalive-time 10
option transport.socket.keepalive-interval 2
option transport.socket.read-fail-log off
option ping-timeout 0
option event-threads 1
# option lock-timer 180
# option transport.address-family inet6
# option base-port 49152
end-volume

5.啟動glusterfs

systemctl enable glusterd.service
systemctl start glusterd.service
systemctl status glusterd.service

三：配置 glusterfs

1.開放端口
iptables -I INPUT -p tcp --dport 24007 -j ACCEPT

2.創建存儲目錄
mkdir /opt/gfs_data

3.添加節點到集群
在10.116.137.196上執行以下操作：
gluster peer probe 10.116.82.28
gluster peer probe 10.116.36.57

4.查看集群狀態
gluster peer status

點擊(此處)折疊或打開

Number of Peers: 2
Hostname: 10.116.82.28
Uuid: f73138ca-e32e-4d87-a99d-cf842fc29447
State: Peer in Cluster (Connected)
Hostname: 10.116.36.57
Uuid: 18e22d2c-049b-4b0c-8cc7-2560319e6c05
State: Peer in Cluster (Connected)
[root@iZwz95trb3stk6afg8oozuZ kubernetes]# clear
[root@iZwz95trb3stk6afg8oozuZ kubernetes]# gluster volume create k8s-volume transport tcp 10.116.137.196:/mnt/gfs_data 10.116.82.28:/mnt/gfs_data 10.116.36.57:/mnt/gfs_data force
volume create: k8s-volume: success: please start the volume to access data
[root@iZwz95trb3stk6afg8oozuZ kubernetes]# gluster volume info
Volume Name: k8s-volume
Type: Distribute
Volume ID: 62900029-02c9-4870-951c-38fafd5f5d9b
Status: Created
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 10.116.137.196:/mnt/gfs_data
Brick2: 10.116.82.28:/mnt/gfs_data
Brick3: 10.116.36.57:/mnt/gfs_data
Options Reconfigured:
transport.address-family: inet
nfs.disable: on

四：配置 volume

1.volume的模式
a.分布卷（默認模式）：即DHT, 也叫分布卷: 將文件已hash算法隨機分布到一臺服務器節點中存儲。
b.復制模式：即AFR, 創建volume 時帶 replica x 數量: 將文件復制到 replica x 個節點中。
c.條帶模式：即Striped, 創建volume 時帶 stripe x 數量：將文件切割成數據塊，分別存儲到 stripe x 個節點中 ( 類似raid 0 )。
d.分布式條帶模式：最少需要4臺服務器才能創建。創建volume 時 stripe 2 server = 4 個節點：是DHT 與 Striped 的組合型。
e.分布式復制模式：最少需要4臺服務器才能創建。創建volume 時 replica 2 server = 4 個節點：是DHT 與 AFR 的組合型。
f.條帶復制卷模式：最少需要4臺服務器才能創建。創建volume 時 stripe 2 replica 2 server = 4 個節點：是 Striped 與 AFR 的組合型。
g.三種模式混合：至少需要8臺服務器才能創建。 stripe 2 replica 2 , 每4個節點組成一個組。

2.創建分布卷
gluster volume create k8s-volume transport tcp 10.116.137.196:/mnt/gfs_data 10.116.82.28:/mnt/gfs_data 10.116.36.57:/mnt/gfs_data force

3.查看volume狀態
gluster volume info

點擊(此處)折疊或打開

Volume Name: k8s-volume
Type: Distribute
Volume ID: 62900029-02c9-4870-951c-38fafd5f5d9b
Status: Created
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 10.116.137.196:/mnt/gfs_data
Brick2: 10.116.82.28:/mnt/gfs_data
Brick3: 10.116.36.57:/mnt/gfs_data
Options Reconfigured:
transport.address-family: inet
nfs.disable: on

4.啟動分布卷
gluster volume start k8s-volume

五：Glusterfs調優

1.開啟指定volume的配額
gluster volume quota k8s-volume enable
2.限制指定volume的配額
gluster volume quota k8s-volume limit-usage / 5GB
3.設置cache大小, 默認32MB
gluster volume set k8s-volume performance.cache-size 64MB
4.設置io線程, 太大會導致進程崩潰
gluster volume set k8s-volume performance.io-thread-count 16
5.設置網絡檢測時間, 默認42s
gluster volume set k8s-volume network.ping-timeout 10
6.設置寫緩沖區的大小, 默認1M
gluster volume set k8s-volume performance.write-behind-window-size 512MB

六：Kubernetes中配置glusterfs

1.配置 endpoints(glusterfs-endpoints.json)

點擊(此處)折疊或打開

{
"kind": "Endpoints",
"apiVersion": "v1",
"metadata": {
"name": "glusterfs-cluster"
},
"subsets": [
{
"addresses": [
{
"ip": "10.116.137.196"
}
],
"ports": [
{
"port": 1990
}
]
}
]
}

kubectl apply -f glusterfs-endpoints.json

2.配置 service(glusterfs-service.json)

點擊(此處)折疊或打開

{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "glusterfs-cluster"
},
"spec": {
"ports": [
{"port": 1990}
]
}
}

kubectl apply -f glusterfs-service.json

3.創建測試 pod

點擊(此處)折疊或打開

{
"apiVersion": "v1",
"kind": "Pod",
"metadata": {
"name": "glusterfs"
},
"spec": {
"containers": [
{
"name": "glusterfs",
"image": "registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0",
"volumeMounts": [
{
"mountPath": "/mnt/glusterfs",
"name": "glusterfsvol"
}
]
}
],
"volumes": [
{
"name": "glusterfsvol",
"glusterfs": {
"endpoints": "glusterfs-cluster",
"path": "k8s-volume",
"readOnly": true
}
}
]
}
}

kubectl apply -f glusterfs-pod.json 在Pod所在的Node節點上執行 df -h 可以查看k8s-volume掛載目錄

4.配置PersistentVolume(glusterfs-pv.yaml)

點擊(此處)折疊或打開

apiVersion: v1
kind: PersistentVolume
metadata:
name: gluster-dev-volume
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteMany
glusterfs:
endpoints: "glusterfs-cluster"
path: "k8s-volume"
readOnly: false

kubectl apply -f glusterfs-pv.yaml

5.配置PVC(glusterfs-pvc.yaml)

點擊(此處)折疊或打開

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: glusterfs-nginx
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi

kubectl apply -f glusterfs-pvc.yaml

6.創建nginx deployment 掛載 volume(nginx-deployment.yaml)

點擊(此處)折疊或打開

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-dm
spec:
replicas: 2
template:
metadata:
labels:
name: nginx
spec:
containers:
- name: nginx
image: nginx:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
volumeMounts:
- name: gluster-dev-volume
mountPath: "/usr/share/nginx/html"
volumes:
- name: gluster-dev-volume
persistentVolumeClaim:
claimName: glusterfs-nginx

kubectl apply -f nginx-deployment.yaml

a.查看 deployment
kubectl get pods |grep nginx-dm

點擊(此處)折疊或打開

nginx-dm-64dcbb8d55-pp9vn 1/1 Running 0 1h
nginx-dm-64dcbb8d55-sclj6 1/1 Running 0 1h

b.查看掛載
kubectl exec -it nginx-dm-64dcbb8d55-pp9vn -- df -h

點擊(此處)折疊或打開

Filesystem Size Used Avail Use% Mounted on
overlay 40G 23G 16G 60% /
tmpfs 3.7G 0 3.7G 0% /dev
tmpfs 3.7G 0 3.7G 0% /sys/fs/cgroup
/dev/xvda1 40G 23G 16G 60% /etc/hosts
shm 64M 0 64M 0% /dev/shm
10.116.137.196:k8s-volume 5.0G 0 5.0G 0% /usr/share/nginx/html
tmpfs 3.7G 12K 3.7G 1% /run/secrets/kubernetes.io/serviceaccount

c.創建文件測試
kubectl exec -it nginx-dm-64dcbb8d55-pp9vn -- touch /usr/share/nginx/html/index.html
kubectl exec -it nginx-dm-64dcbb8d55-sclj6 -- ls -lt /usr/share/nginx/html/index.html

d.驗證 glusterfs
因為我們使用分布卷，所以可以看到某個節點中有文件
ls /mnt/gfs_data/
index.html

向AI問一下細節

亚洲激情专区-91九色丨porny丨老师-久久久久久久女国产乱让韩-国产精品午夜小视频观看

kubernetes實踐之二十六：GlusterFS

猜你喜歡

亚洲激情专区-91九色丨porny丨老师-久久久久久久女国产乱让韩-国产精品午夜小视频观看

kubernetes實踐之二十六：GlusterFS

猜你喜歡

最新資訊

相關推薦

相關標簽