Clustered NFS ganesha Configuration Steps:

1 - NFS Gateway(s)
2 - Ganesha Grace Database -- Active-Active Only
3 - Rados Namespace to store Grace db objects
4 - Rados Namespace to store Exports objects
5 - Ceph Dashboard Integration

------------
1 - NFS Gateway(s)

Required Packages:
	nfs-ganesha-2.7.3-0.1.el7.x86_64
	nfs-ganesha-ceph-2.7.3-0.1.el7.x86_64

Configuration files:
/etc/ganesha/ganesha.conf

	Cluster Recovery backend:
	Use rados_cluster when using active active NFS
	Use rados_ng when using single NFS (or active passive w. k8s or Pacemaker)
		NFSv4 {
			RecoveryBackend = 'rados_cluster';
			#RecoveryBackend = 'rados_ng';
		}
	
	RADOS_URL Block:
	Specify the userid reponsible of reading the export config objects from the cluster
	Specify the watch_url to watch the export index file of the gateway.
		Whenever a export is created, modified or removed, NFS-ganesha will reload its configuration
		This allows dynamic exports when configuring with the Ceph Dashboard
		Note that the watch url object must be unique to each NFS gateway and should always be the same object used for exports
	
		RADOS_URLS {
			ceph_conf = '/etc/ceph/ceph.conf';
			userid = "admin";
			watch_url = "rados://data/ganesha-export-index/conf-nfs1";
		}
	
	RADOS_KV Block
	Specify the RADOS pool and namespace where the Grace db is stored
	If using single NFS (or active passive w. k8s or Pacemaker) nodeid is not required.
		RADOS_KV {
			pool = "metadata";
			namespace = "ganesha-grace";
			nodeid = "nfs1";
		}
	
	Exports
	Exports are managed from the Ceph UI (Ceph => v14). Previous Ceph releases must be managed from cli.
	Specify NFS server to read exports by giving the export-index RADOS URL
	Note that each NFS gateway will have an unique export-index. Where required object name syntax is conf-<HOSTNAME>
	
		%url rados://data/ganesha-export-index/conf-nfs1

	
2/3 - Ganesha Grace Database/Rados Namespace to store Grace db objects

NOT REQUIRED IF USING SINGLE NFS GATEWAY (or active passive w. k8s or Pacemaker)

Required Packages:
	nfs-ganesha-rados-grace-2.7.3-0.1.el7.x86_64

Create Grace DB:

	Be sure to use the same pool and namespace as defined in the ganesha.conf on each NFS gateway. 
	Specify every NFS gateway in the clusters nodeid
	
		ganesha-rados-grace -p metadata -n ganesha-grace add nfs1 nfs2 nfs3
		
	Check grace db status with:
	
		ganesha-rados-grace -p metadata -n ganesha-grace
		
	This will create the objects that it needs to function.
	There will be an object called "grace".
	Each NFS gateway also has its on set of recovery objects. One for each epoch transition 

4 - Rados Namespace to store Exports objects	
	
Create empty export-index for each NFS gateway:

	Be sure to use the same pool and namespace as defined in the ganesha.conf
	
		rados -p data -N ganesha-export-index put conf-nfs1 /dev/null
		rados -p data -N ganesha-export-index put conf-nfs2 /dev/null
		rados -p data -N ganesha-export-index put conf-nfs3 /dev/null
				
5 - Ceph Dashboard Integration

Enable NFS-ganesha Management:
	Tell the Dashboard, in which RADOS pool and namespace the configuration objects are stored.
	
		ceph dashboard set-ganesha-clusters-rados-pool-namespace data/ganesha-export-index
		
	If using multiple NFS-Ganesha clusters, specify cluster_id before pool and namespace. If not specified defaults to "_default_"
	
		ceph dashboard set-ganesha-clusters-rados-pool-namespace ganesha1:data/ganesha-export-index
		
	Navigate to NFS Management Screen and add/edit/remove exports
	
	
