Re: [HPDD-discuss] exporting a lustre FS as NFS

Eli, In my experience testing and fixing this for Cray, NFS export is highly kernel and Lustre version dependent. I've never tested it with a kernel that recent, but I'm not surprised to see issues. We do it all the time from SLES11SP2/3 and CentOS (Lustre 2.4/2.5) without issue, but nothing with a kernel version that recent. There are probably VFS/NFS changes in your kernel that Lustre has not been set up to accomodate. Perhaps someone else will have more useful experience, but I suspect Lustre patches would be necessary to get that working. - Patrick ________________________________ From: HPDD-discuss [hpdd-discuss-bounces(a)lists.01.org] on behalf of E.S. Rosenberg [esr+hpdd-discuss(a)mail.hebrew.edu] Sent: Monday, June 16, 2014 1:49 PM To: hpdd-discuss(a)lists.01.org Subject: [HPDD-discuss] exporting a lustre FS as NFS In the interest of easy data access from computers that are not part of the cluster we would like to export the lustre filesystem as NFS from one of the nodes. So: - Has anyone done it? - What are the pitfalls? - Any other useful tips? Tech details: Lustre: 2.4.3 Kernel: 3.14.3 + aufs Distro: Debian testing/sid We will probably be upgrading to lustre 2.5.x in the near future. Thanks, Eli Trace from the test subject: Jun 16 18:07:17 kernel:LustreError: 3795:0:(llite_internal.h:1141:ll_inode2fid()) ASSERTION( inode != ((void *)0) ) failed: Jun 16 18:07:17 kernel:LustreError: 3795:0:(llite_internal.h:1141:ll_inode2fid()) LBUG Jun 16 18:07:17 kernel:CPU: 0 PID: 3795 Comm: nfsd Tainted: G WC 3.14.3-aufs-mos-1 #1 Jun 16 18:07:17 kernel:Hardware name: Dell Inc. PowerEdge C6220/03C9JJ, BIOS 1.2.1 05/27/2013 Jun 16 18:07:17 kernel: 0000000000000000 ffff881047dd5ba0 ffffffff8175c9e4 ffffffffa1842970 Jun 16 18:07:17 kernel: ffff881047dd5bc0 ffffffffa006954c 0000000000000000 ffff880845cd5148 Jun 16 18:07:17 kernel: ffff881047dd5c00 ffffffffa18054fd 0cb158b46edf5345 0000000000000013 Jun 16 18:07:17 kernel:Call Trace: Jun 16 18:07:17 kernel: [<ffffffff8175c9e4>] dump_stack+0x45/0x56 Jun 16 18:07:17 kernel: [<ffffffffa006954c>] lbug_with_loc+0x3c/0x90 [libcfs] Jun 16 18:07:17 kernel: [<ffffffffa18054fd>] ll_encode_fh+0x109/0x13e [lustre] Jun 16 18:07:17 kernel: [<ffffffff81203f79>] exportfs_encode_inode_fh+0x1b/0x86 Jun 16 18:07:17 kernel: [<ffffffff8120402f>] exportfs_encode_fh+0x4b/0x60 Jun 16 18:07:17 kernel: [<ffffffff810f420f>] ? lookup_real+0x27/0x42 Jun 16 18:07:17 kernel: [<ffffffff81207689>] _fh_update.part.7+0x39/0x48 Jun 16 18:07:17 kernel: [<ffffffff81207c2a>] fh_compose+0x3d1/0x3fa Jun 16 18:07:17 kernel: [<ffffffff81210fe4>] encode_entryplus_baggage+0xd3/0x125 Jun 16 18:07:17 kernel: [<ffffffff8121121f>] encode_entry.isra.14+0x150/0x2cb Jun 16 18:07:17 kernel: [<ffffffff8121247d>] nfs3svc_encode_entry_plus+0xf/0x11 Jun 16 18:07:17 kernel: [<ffffffff81209e7e>] nfsd_readdir+0x160/0x1f8 Jun 16 18:07:17 kernel: [<ffffffff8121246e>] ? nfs3svc_encode_entry+0xe/0xe Jun 16 18:07:17 kernel: [<ffffffff8120831b>] ? nfsd_splice_actor+0xe8/0xe8 Jun 16 18:07:17 kernel: [<ffffffff81056154>] ? groups_free+0x22/0x44 Jun 16 18:07:17 kernel: [<ffffffff8120fa3d>] nfsd3_proc_readdirplus+0xe3/0x1df Jun 16 18:07:17 kernel: [<ffffffff81205269>] nfsd_dispatch+0xca/0x1ad Jun 16 18:07:17 kernel: [<ffffffff8173579b>] svc_process+0x469/0x768 Jun 16 18:07:17 kernel: [<ffffffff81204d39>] nfsd+0xc5/0x117 Jun 16 18:07:17 kernel: [<ffffffff81204c74>] ? nfsd_destroy+0x6b/0x6b Jun 16 18:07:17 kernel: [<ffffffff81051764>] kthread+0xd6/0xde Jun 16 18:07:17 kernel: [<ffffffff8105168e>] ? kthread_create_on_node+0x15d/0x15d Jun 16 18:07:17 kernel: [<ffffffff8176400c>] ret_from_fork+0x7c/0xb0 Jun 16 18:07:17 kernel: [<ffffffff8105168e>] ? kthread_create_on_node+0x15d/0x15d