配置StorageClass
所有的k8s节点的node节点要能访问到ceph的服务端，所以所有的node节点要安装客户端（ceph-common）,我上面是直接安装ceph，也是可以的。
生成key文件
$ grep key /etc/ceph/ceph.client.admin.keyring |awk '{printf "%s", $NF}'|base64
QVFCWXB0RmIzK2dqTEJBQUtsYm4vaHU2NWZ2eHlaaGRnM2hwc1E9PQ==
配置访问ceph的secret
下面的key默认是default的Namespace，所有只能在default下使用，要想其他namespace下使用，需要在指定namespace下创建key，修改namespace即可。
$ vim ceph-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: ceph-secret
namespace: default
type: "kubernetes.io/rbd"
data:
key: QVFCWXB0RmIzK2dqTEJBQUtsYm4vaHU2NWZ2eHlaaGRnM2hwc1E9PQ==
$ kubectl apply -f ceph-secret.yaml
secret/ceph-secret created
$ kubectl get secret
NAME TYPE DATA AGE
ceph-secret kubernetes.io/rbd 1 4s
default-token-lplp6 kubernetes.io/service-account-token 3 50d
mysql-root-password Opaque 1 2d
配置ceph的存储类
$ vim ceph-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: jax-ceph
provisioner: kubernetes.io/rbd
parameters:
monitors: 10.10.3.150:6789,10.10.3.151:6789,10.10.3.152:6789
adminId: admin
adminSecretName: ceph-secret
adminSecretNamespace: default
pool: rbd
userId: admin
userSecretName: ceph-secret
$ kubectl apply -f ceph-storageclass.yaml
storageclass.storage.k8s.io/jax-ceph created
$ kubectl get storageclass
NAME PROVISIONER AGE
jax-ceph kubernetes.io/rbd 1
到此动态存储创建完成
statefulset示例
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: myapp
spec:
serviceName: myapp-sts-svc
replicas: 2
selector:
matchLabels:
app: myapp-pod
template:
metadata:
labels:
app: myapp-pod
spec:
containers:
- name: myapp
image: ikubernetes/myapp:v1
ports:
- containerPort: 80
name: web
volumeMounts:
- name: myappdata
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: myappdata
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: "jax-ceph"
resources:
requests:
storage: 5Gi