lustre 2.5.0 + zfs
by luka leskovec
Hello all,
i got a running lustre 2.5.0 + zfs setup on top of centos 6.4 (the kernels
available on the public whamcloud site), my clients are on centos 6.5
(minor version difference, i recompiled the client sources with the options
specified on the whamcloud site)
but now i have some problems. I cannot judge how serious it is, as the only
problems i observe are slow responses on ls, rm and tar and apart from that
it works great. i also export it over nfs, which sometimes hangs the client
on which it is exported, but i expect this is an issue related to how many
service threads i have running on my servers (old machines).
but my osses (i got two) keep spitting out these messages into the system
log:
xxxxxxxxxxxxxxxxxxxxxx kernel: SPL: Showing stack for process 3264
xxxxxxxxxxxxxxxxxxxxxx kernel: Pid: 3264, comm: txg_sync Tainted:
P --------------- 2.6.32-358.18.1.el6_lustre.x86_64 #1
xxxxxxxxxxxxxxxxxxxxxx kernel: Call Trace:
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa01595a7>] ?
spl_debug_dumpstack+0x27/0x40 [spl]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa0161337>] ?
kmem_alloc_debug+0x437/0x4c0 [spl]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa0163b13>] ?
task_alloc+0x1d3/0x380 [spl]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa0160f8f>] ?
kmem_alloc_debug+0x8f/0x4c0 [spl]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa02926f0>] ? spa_deadman+0x0/0x120
[zfs]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa016432b>] ?
taskq_dispatch_delay+0x19b/0x2a0 [spl]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa0164612>] ?
taskq_cancel_id+0x102/0x1e0 [spl]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa028259a>] ? spa_sync+0x1fa/0xa80
[zfs]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffff810a2431>] ? ktime_get_ts+0xb1/0xf0
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa0295707>] ?
txg_sync_thread+0x307/0x590 [zfs]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffff810560a9>] ?
set_user_nice+0xc9/0x130
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa0295400>] ?
txg_sync_thread+0x0/0x590 [zfs]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa0162478>] ?
thread_generic_wrapper+0x68/0x80 [spl]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffffa0162410>] ?
thread_generic_wrapper+0x0/0x80 [spl]
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffff81096a36>] ? kthread+0x96/0xa0
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffff8100c0ca>] ? child_rip+0xa/0x20
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffff810969a0>] ? kthread+0x0/0xa0
xxxxxxxxxxxxxxxxxxxxxx kernel: [<ffffffff8100c0c0>] ? child_rip+0x0/0x20
does anyone know, is this a serious problem, or just aesthetics? any way to
solve this? any hints?
best regards,
Luka Leskovec
7 years, 8 months
Re: [HPDD-discuss] iozone and lustre
by Dilger, Andreas
On 2014/03/10, 6:01 AM, "E.S. Rosenberg" <esr(a)cs.huji.ac.il<mailto:esr@cs.huji.ac.il>> wrote:
On Tue, Mar 4, 2014 at 4:04 PM, Simmons, James A.
>>If yes, you most likely stumbled on https://jira.hpdd.intel.com/browse/LU-4209
>We are running the in-kernel client, kernel 3.13.x on Debian machines
>The lfs utility in use is from the lustre 2.4.2 branch
>I see that a patch was submitted to the linux-kernel team, does anyone have a link to that patch? I'd like to or patch our kernel or see what version of the kernel we should be building, if it made >it into 3.14-rcX then we'll build that...
http://www.spinics.net/lists/linux-fsdevel/msg72386.html
Correct me if I'm wrong but it looks like that patch hit a dead end...
Andreas did any fsdevs ever voice an opinion on generic/lustre flags?
The patch was resubmitted, and Al Viro had relatively little to say about it (i.e. he didn't completely hate it, which is good). I've since sent an updated patch to Greg again.
Any chance it'll be in 3.14?
The patch is currently in the staging tree, I'm not sure which release it will be in.
Cheers, Andreas
--
Andreas Dilger
Lustre Software Architect
Intel High Performance Data Division
7 years, 8 months
Lustre client on SLES11sp2
by Daguman, Brainard R
Hi All,
I've got Lustre file system ver2.4.2 built up on CentOS6.4 MDS/OSSs/Client(test only) systems using the same version of OS and kernel 2.6.32-358.23.2.
Need to load Lustre client on several compute systems with SLES11sp2 build; matrix show this is doable. How do I enable SLES11sp2 client which has different kernel builds? I believe this require to recompile Lustre on client systems with correct source; what are the steps?
--
Thanks,
Brainard
7 years, 9 months
Lustre Error on Lustre client machines
by Venkat Reddy
Hi,
I am getting the following error messages in Lustre Clients continuously,
I am unable to find any clu on this, Please advoice.
I am posting one client messages.
Mar 20 00:05:20 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.43@o2ib. The ost_write operation failed with -5
Mar 20 00:05:20 node22 kernel: LustreError: Skipped 373 previous similar
messages
Mar 20 00:05:20 node22 kernel: LustreError:
3248:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff881069aeb000 x1462271803650175/t0(0)
o4->lustre-OST0001-osc-ffff880819776400@192.168.1.43@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395254164 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 00:05:20 node22 kernel: LustreError:
3248:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 374 previous
similar messages
Mar 20 00:15:24 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.45@o2ib. The ost_write operation failed with -5
Mar 20 00:15:24 node22 kernel: LustreError: Skipped 367 previous similar
messages
Mar 20 00:15:24 node22 kernel: LustreError:
3251:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff881069f61c00 x1462271803659320/t0(0)
o4->lustre-OST0006-osc-ffff880819776400@192.168.1.45@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395254732 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 00:15:24 node22 kernel: LustreError:
3251:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 368 previous
similar messages
Mar 20 00:28:04 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.45@o2ib. The ost_write operation failed with -5
Mar 20 00:28:04 node22 kernel: LustreError: Skipped 13 previous similar
messages
Mar 20 00:28:04 node22 kernel: LustreError:
3260:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff88106a0db400 x1462271803668389/t0(0)
o4->lustre-OST0006-osc-ffff880819776400@192.168.1.45@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395255492 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 00:28:04 node22 kernel: LustreError:
3260:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 13 previous
similar messages
Mar 20 00:38:34 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.45@o2ib. The ost_write operation failed with -5
Mar 20 00:38:34 node22 kernel: LustreError: Skipped 13 previous similar
messages
Mar 20 00:38:34 node22 kernel: LustreError:
3250:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff88106b499000 x1462271803676412/t0(0)
o4->lustre-OST0006-osc-ffff880819776400@192.168.1.45@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395256121 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 00:38:34 node22 kernel: LustreError:
3250:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 13 previous
similar messages
Mar 20 00:48:37 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.43@o2ib. The ost_write operation failed with -5
Mar 20 00:48:37 node22 kernel: LustreError: Skipped 16 previous similar
messages
Mar 20 00:48:37 node22 kernel: LustreError:
3254:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff881060f94800 x1462271803685186/t0(0)
o4->lustre-OST0000-osc-ffff880819776400@192.168.1.43@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395256724 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 00:48:37 node22 kernel: LustreError:
3254:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 16 previous
similar messages
Mar 20 00:58:45 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.45@o2ib. The ost_write operation failed with -5
Mar 20 00:58:45 node22 kernel: LustreError: Skipped 21 previous similar
messages
Mar 20 00:58:45 node22 kernel: LustreError:
3253:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff88106b499c00 x1462271803693596/t0(0)
o4->lustre-OST0006-osc-ffff880819776400@192.168.1.45@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395257371 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 00:58:45 node22 kernel: LustreError:
3253:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 21 previous
similar messages
Mar 20 01:09:09 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.45@o2ib. The ost_write operation failed with -5
Mar 20 01:09:09 node22 kernel: LustreError: Skipped 13 previous similar
messages
Mar 20 01:09:09 node22 kernel: LustreError:
3259:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff88106b499000 x1462271803701825/t0(0)
o4->lustre-OST0006-osc-ffff880819776400@192.168.1.45@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395257993 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 01:09:09 node22 kernel: LustreError:
3259:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 13 previous
similar messages
Mar 20 01:19:39 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.43@o2ib. The ost_write operation failed with -5
Mar 20 01:19:39 node22 kernel: LustreError: Skipped 18 previous similar
messages
Mar 20 01:19:39 node22 kernel: LustreError:
3251:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff881060e57800 x1462271803711034/t0(0)
o4->lustre-OST0000-osc-ffff880819776400@192.168.1.43@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395258587 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 01:19:39 node22 kernel: LustreError:
3251:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 18 previous
similar messages
Mar 20 01:29:40 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.45@o2ib. The ost_write operation failed with -5
Mar 20 01:29:40 node22 kernel: LustreError: Skipped 21 previous similar
messages
Mar 20 01:29:40 node22 kernel: LustreError:
3256:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff88106a0db000 x1462271803719047/t0(0)
o4->lustre-OST0006-osc-ffff880819776400@192.168.1.45@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395259225 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 01:29:40 node22 kernel: LustreError:
3256:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 21 previous
similar messages
Mar 20 01:40:09 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.43@o2ib. The ost_write operation failed with -5
Mar 20 01:40:09 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.43@o2ib. The ost_write operation failed with -5
Mar 20 01:40:09 node22 kernel: LustreError: Skipped 83 previous similar
messages
Mar 20 01:40:09 node22 kernel: LustreError: Skipped 83 previous similar
messages
Mar 20 01:40:09 node22 kernel: LustreError:
3255:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff88106b272400 x1462271803727400/t0(0)
o4->lustre-OST0001-osc-ffff880819776400@192.168.1.43@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395259855 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 01:40:09 node22 kernel: LustreError:
3256:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff881062400800 x1462271803727404/t0(0)
o4->lustre-OST0001-osc-ffff880819776400@192.168.1.43@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395259855 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 01:40:09 node22 kernel: LustreError:
3256:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 82 previous
similar messages
Mar 20 01:50:10 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.43@o2ib. The ost_write operation failed with -5
Mar 20 01:50:10 node22 kernel: LustreError: Skipped 190 previous similar
messages
Mar 20 01:50:10 node22 kernel: LustreError:
3254:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff881069aeb000 x1462271803735098/t0(0)
o4->lustre-OST0001-osc-ffff880819776400@192.168.1.43@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395260417 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 01:50:10 node22 kernel: LustreError:
3254:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 189 previous
similar messages
Mar 20 02:00:17 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.43@o2ib. The ost_write operation failed with -5
Mar 20 02:00:17 node22 kernel: LustreError: Skipped 199 previous similar
messages
Mar 20 02:00:17 node22 kernel: LustreError:
3260:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff881069aeb000 x1462271803743031/t0(0)
o4->lustre-OST0001-osc-ffff880819776400@192.168.1.43@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395261061 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 02:00:17 node22 kernel: LustreError:
3260:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 198 previous
similar messages
Mar 20 02:10:20 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.44@o2ib. The ost_write operation failed with -5
Mar 20 02:10:20 node22 kernel: LustreError: Skipped 210 previous similar
messages
Mar 20 02:10:20 node22 kernel: LustreError:
3249:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff88106b5c1800 x1462271803750283/t0(0)
o4->lustre-OST0005-osc-ffff880819776400@192.168.1.44@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395261665 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 02:10:20 node22 kernel: LustreError:
3249:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 213 previous
similar messages
Mar 20 02:20:23 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.43@o2ib. The ost_write operation failed with -5
Mar 20 02:20:23 node22 kernel: LustreError: Skipped 191 previous similar
messages
Mar 20 02:20:23 node22 kernel: LustreError:
3251:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff88106a962400 x1462271803758875/t0(0)
o4->lustre-OST0000-osc-ffff880819776400@192.168.1.43@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395262269 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 02:20:23 node22 kernel: LustreError:
3251:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 191 previous
similar messages
Mar 20 02:30:25 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.43@o2ib. The ost_write operation failed with -5
Mar 20 02:30:25 node22 kernel: LustreError: Skipped 149 previous similar
messages
Mar 20 02:30:25 node22 kernel: LustreError:
3254:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff88106a962400 x1462271803766929/t0(0)
o4->lustre-OST0000-osc-ffff880819776400@192.168.1.43@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395262833 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 02:30:25 node22 kernel: LustreError:
3254:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 148 previous
similar messages
Mar 20 02:40:26 node22 kernel: LustreError: 11-0: an error occurred while
communicating with 192.168.1.43@o2ib. The ost_write operation failed with -5
Mar 20 02:40:26 node22 kernel: LustreError: Skipped 225 previous similar
messages
Mar 20 02:40:26 node22 kernel: LustreError:
3254:0:(osc_request.c:1689:osc_brw_redo_request()) @@@ redo for recoverable
error -5 req@ffff881069aeb800 x1462271803774144/t0(0)
o4->lustre-OST0001-osc-ffff880819776400@192.168.1.43@o2ib:6/4 lens 488/416
e 0 to 0 dl 1395263434 ref 2 fl Interpret:R/0/0 rc -5/-5
Mar 20 02:40:26 node22 kernel: LustreError:
3254:0:(osc_request.c:1689:osc_brw_redo_request()) Skipped 225 previous
similar messages
[root@node22 ~]#
Thank you
Venkat Reddy
7 years, 9 months
Lustre Servers - Disable SELinux in build
by Patrick Farrell
We at Cray recently noticed that on CentOS 6.4/5 servers even with
SELinux disabled there is still a noticeable amount of memory in use:
cat /proc/slabinfo | grep selinux
selinux_inode_security 7578 7579 72 53 1 : tunables 120
60 0 : slabdata 143 143 0
That's from an idle server, and it's not much usage (7579 objects, 72
bytes per object, ~ 0.5 MB of memory), but on an active server, there
can be millions of objects, leading to a few hundred MB of memory usage.
As far as I can tell, disabling selinux is required for Lustre servers
to function. When the kernel is built with SELinux disabled in the
.config, this memory usage goes away.
I'd like to propose changing the Lustre provided RHEL kernel config file
to not build SELINUX in to the kernel.
This won't affect clients unless they're running the patched kernel -
only the patched Lustre server kernels are changed by this, and as far
as I know, servers always have SELinux disabled.
I wanted to float this to a broad audience and get any objections before
creating an LU and pushing a patch to Gerrit. Does anyone have a reason
not to do this?
Thanks,
- Patrick Farrell
Patch, in essence:
--- kernel-2.6.32-2.6-rhel6-x86_64.config
+++ kernel-2.6.32-2.6-rhel6-x86_64.config
@@ -4411,15 +4411,7 @@
CONFIG_SECURITY_FILE_CAPABILITIES=y
# CONFIG_SECURITY_ROOTPLUG is not set
CONFIG_INTEL_TXT=y
-CONFIG_LSM_MMAP_MIN_ADDR=65535
-CONFIG_SECURITY_SELINUX=y
-CONFIG_SECURITY_SELINUX_BOOTPARAM=y
-CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=1
-CONFIG_SECURITY_SELINUX_DISABLE=y
-CONFIG_SECURITY_SELINUX_DEVELOP=y
-CONFIG_SECURITY_SELINUX_AVC_STATS=y
-CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1
-# CONFIG_SECURITY_SELINUX_POLICYDB_VERSION_MAX is not set
+# CONFIG_SECURITY_SELINUX is not set
# CONFIG_SECURITY_SMACK is not set
# CONFIG_SECURITY_TOMOYO is not set
CONFIG_IMA=y
7 years, 9 months
LUG 2014: Last Day to Register
by OpenSFS Administration
<http://www.opensfs.org/events/lug14/>
Lustre User Group 2014
Online Registration Closes Today
Today, March 31, 2014 is the last day to register online for the 12th Annual
Lustre <http://www.opensfs.org/lug14/> R User Group (LUG) Conference! Don't
miss the opportunity to participate in this exclusive event on April 8-10 in
Miami, FL.
With over 180 people already registered from over 50 different companies
worldwide, LUG offers you the unique opportunity to network with industry
leaders, end-users, developers, and vendors of Lustre.
The event will feature over 30+ Lustre focused sessions, including the
first-annual State of the Lustre Community, as well as a number of
opportunities for attendees to actively participate in industry dialogue on
best practices and emerging technologies, including:
* Networking Reception: Join us for an evening reception to network
with fellow attendees.
* Poster Exhibition: Participate in the first LUG poster exhibition,
and learn about the OpenSFS Benchmarking Working Group, Cooperative work
with SAMBA on Lustre, Correlation of File and Batch System Activities in the
HRSK-II Project, and more!
* OpenSFS Working Group Meetings: Attendees are invited to join the
OpenSFS Working Group meetings following LUG on April 10, regardless of
their OpenSFS membership status.
<https://opensfs.wufoo.com/forms/lug-conference-2014/> Register Today!
We encourage you to review the <http://www.opensfs.org/lug-2014-agenda/>
LUG 2014 Agenda to learn more about the networking opportunities, poster
exhibition, and sessions at this year's conference.
We look forward to seeing you at LUG! If you have any questions, please feel
free to contact <mailto:admin@opensfs.org> admin(a)opensfs.org.
Best regards,
OpenSFS LUG Planning Committee
_________________________
OpenSFS Administration
3855 SW 153rd Drive Beaverton, OR 97006 USA
Phone: +1 503-619-0561 | Fax: +1 503-644-6708
Twitter: <https://twitter.com/opensfs> @OpenSFS
Email: <mailto:admin@opensfs.org> admin(a)opensfs.org | Website:
<http://www.opensfs.org> www.opensfs.org
<http://www.opensfs.org/lug-2014-sponsorship/>
<http://www.opensfs.org/lug-2014-sponsorship/> Click here to learn how to
become a LUG Sponsor
Open Scalable File Systems, Inc. was founded in 2010 to advance Lustre
development, ensuring it remains vendor-neutral, open, and free. Since its
inception, OpenSFS has been responsible for advancing the Lustre file system
and delivering new releases on behalf of the open source community. Through
working groups, events, and ongoing funding initiatives, OpenSFS harnesses
the power of collaborative development to fuel innovation and growth of the
Lustre file system worldwide.
7 years, 9 months
WAS: lustre client 2.1.6 for sles 11.2
by Matt Bettinger
We are going to 2.4.2 now and just built source/rpms ok on sles 11.2.
Still would like to know of preference of mellanox ofed vs Open Fabrics.
Thanks!
7 years, 9 months
lustre client 2.1.6 for sles 11.2
by Matt Bettinger
Hello,
Is 2.1.6 lustre client build supported on sles 11.2?
I have installed OFED-3.5-2 and installed the RPMs Then get an error
during the config process of lustre client
hous0162:/usr/src/lustre-2.1.6 # ./configure
--with-linux=/usr/src/linux-3.0.13-0.27
--with-linux-obj=/usr/src/linux-3.0.13-0.27-obj/x86_64/default/
....
....
checking for /boot/kernel.h... no
checking for /var/adm/running-kernel.h... no
checking for /usr/src/linux-3.0.13-0.27-obj/x86_64/default/.config... yes
checking for /usr/src/linux-3.0.13-0.27-obj/x86_64/default/include/linux/autoconf.h...
no
configure: error: Run make config in /usr/src/linux-3.0.13-0.27.
hous0162:/usr/src/lustre-2.1.6 #
I have already make config in the linux src and the .config file exists
hous0162:/usr/src/linux-3.0.13-0.27 # ls -la .config
-rw-r--r-- 1 root root 76743 Mar 31 10:29 .config
Any suggestions in getting this to work would be appreciated!
Also, is it suggested to use mellanox ofed or are we ok with the ofed-3.5-2?
Thanks!
MB
7 years, 9 months
understanding OST counters
by Brock Palen
I am building some monitoring scripts of our lustre setup and I am having trouble wrapping my head around:
/proc/fs/lustre/obdfilter/scratch-OST0000/stats
read_bytes 100195530 samples [bytes] 0 1048576 54085118202030
write_bytes 178494565 samples [bytes] 1 1048576 166914498969357
What do each of the each of the numbers mean?
Brock Palen
www.umich.edu/~brockp
CAEN Advanced Computing
XSEDE Campus Champion
brockp(a)umich.edu
(734)936-1985
7 years, 9 months
