NFSv4 and UIDs on OpenSolaris and Ubuntu

NFS clients and servers negotiate to use the highest NFS version they both support. NFSv4 usually performs much better than NFSv3, but requires a little more setup. Here I get NFSv4 working between an OpenSolaris file server and a diskless Ubuntu client. In theory, a plain mount(8) gives us a NFSv4 mount.

# mount server:/data1/opennebula/on22 /mnt/
#

Use nfsstat -m to see what kind of mount they negotiated

# nfsstat -m
...
/mnt from server:/data1/opennebula/on22
Flags: rw,relatime,vers=4,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.0.2.2,minorversion=0,addr=192.0.2.1


We have NFSv4, huzzah! Go look at the files.

# ls -lai /mnt/
total 12K
5 drwxr-xr-x 8 4294967294 4294967294 8 2011-04-19 11:50 .
3 drwxr-xr-x 21 root root 26 2011-03-17 10:22 ..
6 drwxr-xr-x 2 4294967294 4294967294 24 2011-04-19 11:50 bin
29 drwxr-xr-x 16 4294967294 4294967294 21 2011-04-19 11:50 etc
74 drwxr-xr-x 2 4294967294 4294967294 2 2011-04-19 11:50 include
75 drwxr-xr-x 7 4294967294 4294967294 7 2011-04-19 11:50 lib
296 drwxr-xr-x 5 4294967294 4294967294 5 2011-04-19 11:50 share
332 drwxr-xr-x 5 4294967294 4294967294 10 2011-04-19 11:50 var


A UID of 4294967294? That’s awesome. Wrong, but awesome. 4294967294 is -1 on a 64-bit system. Many modern Linuxish systems assign nobody and nogroup (the standard unprivileged NFS accounts) a UID and GID of -1. While my files are owned by uid 1003 on the server, and the client’s mount point is owned by uid 1003, NFSv4 defaults to mapping all UIDs to nobody. Use rpc.idmapd to map UIDs between systems. Go to /etc/default/nfs-common and enable idmapd.

NEED_IDMAPD=yes

Lower case seems to be required: I originally set this to YES and the process didn’t start.

Reboot the client, and the files are now owned by nobody. Well, at least that’s a legitimate system user, one originally created for NFS. The files are owned by UID 1003 on the server, however.

Here’s where NFSv4 gets interesting. In NFSv3 and earlier, file ownership over NFS is controlled by UID. Systems administrators worked hard to keep UIDs synchronized across their systems so that NFS permissions would be consistent. You can remap UIDs over NFS, of course, but maintaining those maps is vastly annoying.

NFSv4 maps file permissions by UIDs, but uses usernames for ACLs and ownership. Both must be correct, or common operations won’t work as expected. I have an OpenSolaris NFS server that contains lots of files for lots of diskless systems with lots of different usernames. Some of those usernames do not exist on the fileserver. While I keep user accounts in LDAP, I (mostly) don’t bother with system or program accounts. To share files over NFSv4, though, the accounts must exist on both client and server.

NFSv4 uses helper programs to map usernames and UIDs: nfsmapid on OpenSolaris, rpc.idmapd on Ubuntu, and nfsuserd on FreeBSD. (Please insert a screaming rant here: these are all basically the same program. Why, why, why change the name? We don’t give ping(8) different names even though it has completely different under-the-hood implementations on each program, do we? Sheesh.)

NFSv4 maps usernames within a domain, generally (but not necessarily) the machine’s domain name. If the NFSv4 client and server domain names doesn’t match, all the usernames will show up as “nobody.” OpenSolaris’ nfsmapid pulls the domain name from the machine’s domain name. I had to set the domain name on Ubuntu 10.10 in /etc/idmapd.conf.

NFSv4 now works in my environment.

Note that NFSv4 also has a variety of other changes. All exports are part of a single unified namespace. OpenSolaris handles that for you. If you use a different NFSv4 server, you might need to manage that namespace yourself. But that’ll be a topic for another post, when I get my FreeBSD/ZFS/iSCSI/NFSv4 server working.