Ceph k8s

k8s对接ceph

Posted on 2021-11-08,2 min read

配置StorageClass

所有的k8s节点的node节点要能访问到ceph的服务端,所以所有的node节点要安装客户端(ceph-common),我上面是直接安装ceph,也是可以的。

生成key文件

$ grep key /etc/ceph/ceph.client.admin.keyring |awk '{printf "%s", $NF}'|base64
QVFCWXB0RmIzK2dqTEJBQUtsYm4vaHU2NWZ2eHlaaGRnM2hwc1E9PQ==

配置访问ceph的secret

下面的key默认是default的Namespace,所有只能在default下使用,要想其他namespace下使用,需要在指定namespace下创建key,修改namespace即可。

$ vim ceph-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: ceph-secret
  namespace: default
type: "kubernetes.io/rbd"
data:
  key: QVFCWXB0RmIzK2dqTEJBQUtsYm4vaHU2NWZ2eHlaaGRnM2hwc1E9PQ==
$ kubectl apply -f ceph-secret.yaml 
secret/ceph-secret created
$ kubectl get secret
NAME                  TYPE                                  DATA      AGE
ceph-secret           kubernetes.io/rbd                     1         4s
default-token-lplp6   kubernetes.io/service-account-token   3         50d
mysql-root-password   Opaque                                1         2d

配置ceph的存储类

$ vim ceph-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
   name: jax-ceph
provisioner: kubernetes.io/rbd
parameters:
  monitors: 10.10.3.150:6789,10.10.3.151:6789,10.10.3.152:6789
  adminId: admin
  adminSecretName: ceph-secret
  adminSecretNamespace: default
  pool: rbd
  userId: admin
  userSecretName: ceph-secret
$ kubectl apply -f ceph-storageclass.yaml 
storageclass.storage.k8s.io/jax-ceph created
$ kubectl get storageclass
NAME              PROVISIONER          AGE
jax-ceph          kubernetes.io/rbd    1

到此动态存储创建完成

statefulset示例

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: myapp
spec:
  serviceName: myapp-sts-svc
  replicas: 2
  selector:
    matchLabels:
      app: myapp-pod
  template:
    metadata:
      labels:
        app: myapp-pod
    spec:
      containers:
      - name: myapp
        image: ikubernetes/myapp:v1
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: myappdata
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: myappdata
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: "jax-ceph"
      resources:
        requests:
          storage: 5Gi

下一篇: kubernetes 部署 rook+ceph 存储系统→