Create NFS dynamic provisioner for Openshift. Use nfs for dynamic provisioning with OCP 3.11

-


I will explain how to install NFS server on Centos 7.X and and configure openshift to use this NFS share as Persistent Volume

nfs-client from kubernetes-incubator project is an automatic provisioner that use your existing and already configured NFS server to support dynamic provisioning of Kubernetes/OpenShift Persistent Volumes via Persistent Volume Claims.
Install NFS Server

Create a Centos/RHEL 7 based instance and run the following commands as root

yum install -y nfs-utils
systemctl enable rpcbind
systemctl enable nfs-server
systemctl start rpcbind
systemctl start nfs-servermkdir /var/nfsshare
chmod -R 755 /var/nfssharefirewall-cmd --permanent --zone=public --add-service=nfs
firewall-cmd --permanent --zone=public --add-service=rpc-bind
firewall-cmd --reload

To allow all clients put *

# vi /etc/exports
/var/nfsshare *(rw,sync,no_root_squash)

or you can limit to a subnet/IP

/var/nfsshare 172.16.2.0/24(rw,sync,no_root_squash)

# systemctl restart nfs-server
Install nfs-utils package on all openshift nodes

yum install -y nfs-utils

Download kubernetes-incubator

Login to openshift master instance as origin user (or other user you used to deploy openhift)

$ curl -L -o kubernetes-incubator.zip https://github.com/kubernetes-incubator/external-storage/archive/master.zip
unzip kubernetes-incubator.zip
$ cd external-storage-master/nfs-client/

Change default namespace with current project/namespace. If you are in different project right now. Please switch to target project before running the commands below:

$ NAMESPACE=`oc project -q`
$ sed -i'' "s/namespace:.*/namespace: $NAMESPACE/g" ./deploy/rbac.yaml

Create the service account and role bindings.

$ oc create -f deploy/rbac.yaml

Allow run as root and mount permission to nfs-client-provisioner. To grant this permission, you must be logged in to openshift with admin privileges.

$ oc adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:$NAMESPACE:nfs-client-provisioner
edit deploy/deployment.yaml

Change the following parameters with yours <YOUR NFS SERVER HOSTNAME> It is 172.16.2.5 in my case and change /var/nfs with /var/nfsshare
which we configured above. Also You don’t have to change PROVISIONER_NAME value fuseim.pri/ifs but it is good to change with your enviroment. for example myproject/nfs
— name: PROVISIONER_NAME
value: fuseim.pri/ifs
— name: NFS_SERVER
value: 172.16.2.5
— name: NFS_PATH
value: /var/nfsshare
volumes:
— name: nfs-client-root
nfs:
server: 172.16.2.5
path: /var/nfsshare
Edit deploy/class.yaml file

set provisioner: fuseim.pri/ifs value as you defined PROVISIONER_NAME in deploy/deployment.yaml (myproject/nfs). It must match PROVISIONER_NAME otherwise it will fail!

class.yaml defines the NFS-Client’s Kubernetes Storage Class with name: managed-nfs-storage

We will define this storage-class name in claim yaml files later.
Deploy and create storage class

$ oc create -f deploy/class.yaml
$ oc create -f deploy/deployment.yaml

Check nfs-client-provisioner pod. Ensure that nfs-client-provisioner is running.

$ oc get pods
NAME                                    READY STATUS RESTARTS AGE
nfs-client-provisioner-6f59b7b5f4-sd7r2    1/1 Running 2      1h

Check the logs if there is a failure. If you don’t add hostmount-anyuid above, it will never work! Please double check it if you see an issue.

$ oc logs nfs-client-provisioner-6f59b7b5f4-sd7r2

Now we can create claim and test pod. You don’t have to mount NFS share on openshift master or nodes. nfs-client-provisioner will do it automatically and on demand.
Create a test claim

$ oc create -f deploy/test-claim.yaml

Check the claim status

$ oc get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE

test-claim Bound pvc-88cb0c46–2617–11e9–87e4–000d3a444ea0 1Mi RWX managed-nfs-storage 2h

If you see Pending, it means there is something wrong with NFS sharing or nfs-client-provisioner deployment!

$ cat test-claim.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
 name: test-claim
 annotations:
 volume.beta.kubernetes.io/storage-class: “managed-nfs-storage”
spec:
 accessModes:
 — ReadWriteMany
 resources:
 requests:
 storage: 1Mi

Whenever you create a new claim, you must add
volume.beta.kubernetes.io/storage-class: “managed-nfs-storage” annotation into metadata of the claim yaml to reference the NFS share.
Create the pod with uses test-claim

$ oc create -f test-pod.yaml
pod/test-pod created

Lets check NFS server sharing for SUCCESS file created by test pod.

nfsserver $ ls -l /var/nfsshare/
total 0

drwxrwxrwx. 2 root root 21 Feb 1 14:13 myproject-test-claim-pvc-88cb0c46–2617–11e9–87e4–000d3a444ea0# ls  -l /var/nfsshare/myproject-test-claim-pvc-88cb0c46-2617-11e9-87e4-000d3a444ea0/
total 0
-rw-r--r--. 1 root root 0 Feb  1 14:13 SUCCESS
[root@dev-reverse-proxy-1 ~]#

All done!

Comments

Post a Comment

Popular posts from this blog

using libguestfs virt-customize tool to modify qcow2 image and reset root password.

-
ibguestfs is a C library and a set of tools for accessing and modifying virtual disk images used in platform virtualization. The tools can be used for viewing and editing virtual machines managed by libvirt and files inside VMs, scripting changes to VMs, creating VMs, and much else beside. virt-customize is one of them as example. If you forgot root user password here is how you can modify . # virsh shutdown debian9-vm1 # virt-customize -a /var/lib/libvirt/images/debian9-vm1.qcow2 --root-password password:NewRootUserPasswordHere  not only this but you can edit many more. The –run-command option is used to execute any command inside a virtual image file. 1) register with rhel subscription virt-customize -a rhel-server-7.6.qcow2 --run-command 'subscription-manager attach --pool [subscription-pool]'     Install software or packages- virt-customize -a rhel-server-7.6.qcow2 --install net-tools  Upload a file into image virt-customize -a rhel-se...
Read more

Running cockpit behind nginx reverse proxy with nginx ssl and cockpit non ssl

-
Domain for web access to cockpit : cockpit.mylab.local cockpit server ip : 192.168.1.61 nginx server ip   : 192.168.1.65 Linux distro using is Centos 7.6 Login to cockpit server Create a file [root@repo ~]# vim /etc/cockpit/cockpit.conf content will be like [WebService] Origins = https://cockpit.mylab.local wss://cockpit.mylab.local ProtocolHeader = X-Forwarded-Proto AllowUnencrypted = true save and restart cockpit # systemctl restart cockpit now  login to your nginx webserver and install nginx # yum install nginx -y create a new file vim /etc/nginx/conf.d/cockpit.conf paste content as bellow server {     listen         80;     listen         443 ssl http2;     server_name    cockpit.mylab.local;     ssl_certificate /etc/ssl/certs/cockpit-selfsigned.crt;     ssl_certifica...
Read more

setting up openshift alert manager mail alerting based on critical and warning

-
Configure alert manager to send mail. This will send mail to 2 diffrent groups one is managemnet group for ciritical alerts only and second is default group for all alerts. $  oc project openshift-monitoring The secret file where alertmanager.yaml content ( encoded in base64 ) will remain is alertmanager-main convert the yaml content into base64 first $  cat alertmanager.yaml | base64 -w0 copy value now edit secret $ oc edit secret alertmanager-main find value starts with alertmanager.yaml: replace all base64 content from your own . save and exit now in order to take this secret in effect we must bounce the pod $  oc delete po -l alertmanager=main check if all pod came online $  oc get po -l alertmanager=main done! alertmanager.yaml content resolve_timeout : 5m route : group_wait : 30s group_interval : 5m repeat_interval : 12h ...
Read more