Create NFS dynamic provisioner for Openshift. Use nfs for dynamic provisioning with OCP 3.11

-


I will explain how to install NFS server on Centos 7.X and and configure openshift to use this NFS share as Persistent Volume

nfs-client from kubernetes-incubator project is an automatic provisioner that use your existing and already configured NFS server to support dynamic provisioning of Kubernetes/OpenShift Persistent Volumes via Persistent Volume Claims.
Install NFS Server

Create a Centos/RHEL 7 based instance and run the following commands as root

yum install -y nfs-utils
systemctl enable rpcbind
systemctl enable nfs-server
systemctl start rpcbind
systemctl start nfs-servermkdir /var/nfsshare
chmod -R 755 /var/nfssharefirewall-cmd --permanent --zone=public --add-service=nfs
firewall-cmd --permanent --zone=public --add-service=rpc-bind
firewall-cmd --reload

To allow all clients put *

# vi /etc/exports
/var/nfsshare *(rw,sync,no_root_squash)

or you can limit to a subnet/IP

/var/nfsshare 172.16.2.0/24(rw,sync,no_root_squash)

# systemctl restart nfs-server
Install nfs-utils package on all openshift nodes

yum install -y nfs-utils

Download kubernetes-incubator

Login to openshift master instance as origin user (or other user you used to deploy openhift)

$ curl -L -o kubernetes-incubator.zip https://github.com/kubernetes-incubator/external-storage/archive/master.zip
unzip kubernetes-incubator.zip
$ cd external-storage-master/nfs-client/

Change default namespace with current project/namespace. If you are in different project right now. Please switch to target project before running the commands below:

$ NAMESPACE=`oc project -q`
$ sed -i'' "s/namespace:.*/namespace: $NAMESPACE/g" ./deploy/rbac.yaml

Create the service account and role bindings.

$ oc create -f deploy/rbac.yaml

Allow run as root and mount permission to nfs-client-provisioner. To grant this permission, you must be logged in to openshift with admin privileges.

$ oc adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:$NAMESPACE:nfs-client-provisioner
edit deploy/deployment.yaml

Change the following parameters with yours <YOUR NFS SERVER HOSTNAME> It is 172.16.2.5 in my case and change /var/nfs with /var/nfsshare
which we configured above. Also You don’t have to change PROVISIONER_NAME value fuseim.pri/ifs but it is good to change with your enviroment. for example myproject/nfs
— name: PROVISIONER_NAME
value: fuseim.pri/ifs
— name: NFS_SERVER
value: 172.16.2.5
— name: NFS_PATH
value: /var/nfsshare
volumes:
— name: nfs-client-root
nfs:
server: 172.16.2.5
path: /var/nfsshare
Edit deploy/class.yaml file

set provisioner: fuseim.pri/ifs value as you defined PROVISIONER_NAME in deploy/deployment.yaml (myproject/nfs). It must match PROVISIONER_NAME otherwise it will fail!

class.yaml defines the NFS-Client’s Kubernetes Storage Class with name: managed-nfs-storage

We will define this storage-class name in claim yaml files later.
Deploy and create storage class

$ oc create -f deploy/class.yaml
$ oc create -f deploy/deployment.yaml

Check nfs-client-provisioner pod. Ensure that nfs-client-provisioner is running.

$ oc get pods
NAME                                    READY STATUS RESTARTS AGE
nfs-client-provisioner-6f59b7b5f4-sd7r2    1/1 Running 2      1h

Check the logs if there is a failure. If you don’t add hostmount-anyuid above, it will never work! Please double check it if you see an issue.

$ oc logs nfs-client-provisioner-6f59b7b5f4-sd7r2

Now we can create claim and test pod. You don’t have to mount NFS share on openshift master or nodes. nfs-client-provisioner will do it automatically and on demand.
Create a test claim

$ oc create -f deploy/test-claim.yaml

Check the claim status

$ oc get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE

test-claim Bound pvc-88cb0c46–2617–11e9–87e4–000d3a444ea0 1Mi RWX managed-nfs-storage 2h

If you see Pending, it means there is something wrong with NFS sharing or nfs-client-provisioner deployment!

$ cat test-claim.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
 name: test-claim
 annotations:
 volume.beta.kubernetes.io/storage-class: “managed-nfs-storage”
spec:
 accessModes:
 — ReadWriteMany
 resources:
 requests:
 storage: 1Mi

Whenever you create a new claim, you must add
volume.beta.kubernetes.io/storage-class: “managed-nfs-storage” annotation into metadata of the claim yaml to reference the NFS share.
Create the pod with uses test-claim

$ oc create -f test-pod.yaml
pod/test-pod created

Lets check NFS server sharing for SUCCESS file created by test pod.

nfsserver $ ls -l /var/nfsshare/
total 0

drwxrwxrwx. 2 root root 21 Feb 1 14:13 myproject-test-claim-pvc-88cb0c46–2617–11e9–87e4–000d3a444ea0# ls  -l /var/nfsshare/myproject-test-claim-pvc-88cb0c46-2617-11e9-87e4-000d3a444ea0/
total 0
-rw-r--r--. 1 root root 0 Feb  1 14:13 SUCCESS
[root@dev-reverse-proxy-1 ~]#

All done!

Comments

Post a Comment

Popular posts from this blog

Running web ssh client on port 443 /80 with nginx as reverse proxy

-
Package used         : shellinabox , nginx Repository used        : epel-release Linux distro used    : Centos 7.6 nginx reverse proxy server ip : 192.168.1.65 shellinabox web ssh server ip : 192.168.1.111 login to web ssh server first install epel-repo # yum install epel-release -y install shellinabox package # yum -y install shellinabox start and enable shellinaboxd service # systemctl enable shellinaboxd && systemctl start shellinaboxd By default shellinaboxd service starts with ssl port 4200 for us to run this in a reverse proxy environment need to make this as insecure hence the encryption will be done by nginx to end user. edit file /etc/sysconfig/shellinabox  like bellow- USER=shellinabox GROUP=shellinabox CERTDIR=/var/lib/shellinabox PORT=4200 OPTS="--disable-ssl-menu -s /:LOGIN" OPTS="-t -s /:SSH:192.168.1.111" save file exit and restart service # systemctl restart shellinaboxd Now open firewall for listening port 4200 from nginx proxy server only #
Read more

Running cockpit behind nginx reverse proxy with nginx ssl and cockpit non ssl

-
Domain for web access to cockpit : cockpit.mylab.local cockpit server ip : 192.168.1.61 nginx server ip   : 192.168.1.65 Linux distro using is Centos 7.6 Login to cockpit server Create a file [root@repo ~]# vim /etc/cockpit/cockpit.conf content will be like [WebService] Origins = https://cockpit.mylab.local wss://cockpit.mylab.local ProtocolHeader = X-Forwarded-Proto AllowUnencrypted = true save and restart cockpit # systemctl restart cockpit now  login to your nginx webserver and install nginx # yum install nginx -y create a new file vim /etc/nginx/conf.d/cockpit.conf paste content as bellow server {     listen         80;     listen         443 ssl http2;     server_name    cockpit.mylab.local;     ssl_certificate /etc/ssl/certs/cockpit-selfsigned.crt;     ssl_certificate_key  /etc/ssl/private/cockpit-selfsigned.key;     location / {         proxy_pass http://192.168.1.61:9090;         proxy_set_header Host $host;         proxy_http_version 1.1;
Read more

Setup VOD streaming server with nginx using RTMP on Ubuntu 18.04

-
Install required packages for source code nginx build # apt-get install -y build-essential ca-certificates wget curl libpcre3 libpcre3-dev autoconf unzip automake libtool tar git libssl-dev zlib1g-dev uuid-dev lsb-release vim htop sysstat ufw fail2ban makepasswd libgd-dev libgeoip-dev -y create a directory # mkdir /usr/local/src/nginx/ # cd /usr/local/src/nginx/ download and extract nginx tar # wget -qO- http://nginx.org/download/nginx-1.14.2.tar.gz | tar zxf - clone nginx rtmp module # git clone https://github.com/arut/nginx-rtmp-module Download other modules - PCRE version 8.42 # wget https://ftp.pcre.org/pub/pcre/pcre-8.42.tar.gz && tar xzvf pcre-8.42.tar.gz - zlib version 1.2.11 # wget https://www.zlib.net/zlib-1.2.11.tar.gz && tar xzvf zlib-1.2.11.tar.gz - OpenSSL version 1.1.0h # wget https://www.openssl.org/source/openssl-1.1.0h.tar.gz && tar xzvf openssl-1.1.0h.tar.gz # cd nginx-1.14.2/ configure  nginx with modules and options # ./configure --add-modu
Read more