Hello Tao,
Yes, I rebooted the machine after the change.
Thanks,
-Upanshu
-----Original Message-----
From: Peng Tao [mailto:bergwolf@gmail.com]
Sent: Monday, July 22, 2013 9:59 PM
To: Singhal, Upanshu
Cc: malcolm.j.cowe(a)intel.com; hpdd-discuss(a)ml01.01.org
Subject: Re: [HPDD-discuss] HPDD-discuss Digest, Vol 8, Issue 14
On Mon, Jul 22, 2013 at 5:22 PM, Singhal, Upanshu <upanshu.singhal(a)emc.com> wrote:
Hello Malcolm,
Thanks for your email, I have selinux=disabled, but still having the same issue.
Did you reboot after disabling selinux? You need to reboot to have the
configuration take effect.
Thanks,
Tao
Thanks,
-Upanshu
-----Original Message-----
From: hpdd-discuss-bounces(a)ml01.01.org
[mailto:hpdd-discuss-bounces@ml01.01.org] On Behalf Of
hpdd-discuss-request(a)ml01.01.org
Sent: Monday, July 22, 2013 2:31 PM
To: hpdd-discuss(a)ml01.01.org
Subject: HPDD-discuss Digest, Vol 8, Issue 14
Send HPDD-discuss mailing list submissions to
hpdd-discuss(a)lists.01.org
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.01.org/mailman/listinfo/hpdd-discuss
or, via email, send a message with subject or body 'help' to
hpdd-discuss-request(a)lists.01.org
You can reach the person managing the list at
hpdd-discuss-owner(a)lists.01.org
When replying, please edit your Subject line so it is more specific than "Re:
Contents of HPDD-discuss digest..."
Today's Topics:
1. Re: Issue installing Lustre 1.87 (Cowe, Malcolm J)
2. Re: Issue installing Lustre 1.87 (Singhal, Upanshu)
----------------------------------------------------------------------
Message: 1
Date: Mon, 22 Jul 2013 07:42:39 +0000
From: "Cowe, Malcolm J" <malcolm.j.cowe(a)intel.com>
To: "hpdd-discuss(a)lists.01.org" <hpdd-discuss(a)lists.01.org>
Subject: Re: [HPDD-discuss] Issue installing Lustre 1.87
Message-ID:
<5B2521AFFB62E743BE362149134E9A890D88B436(a)PGSMSX102.gar.corp.intel.com
>
Content-Type: text/plain; charset="us-ascii"
+1: you almost certainly have SELinux enabled. The formatting of the file system will
fail with this error when SELinux is configured. For example, on a test system I have set
up a quick experiment to replicate your problem by setting SELinux to
"enforcing":
[root@m64-1 ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
[root@m64-1 ~]# mkfs.lustre --reformat --mgs /dev/sdb
Permanent disk data:
Target: MGS
Index: unassigned
Lustre FS:
Mount type: ldiskfs
Flags: 0x64
(MGS first_time update ) Persistent mount opts:
user_xattr,errors=remount-ro
Parameters:
device size = 1024MB
formatting backing filesystem ldiskfs on /dev/sdb
target name MGS
4k blocks 262144
options -q -O uninit_bg,dir_nlink,quota,huge_file,flex_bg -E
lazy_journal_init -F
mkfs_cmd = mke2fs -j -b 4096 -L MGS -q -O
uninit_bg,dir_nlink,quota,huge_file,flex_bg -E lazy_journal_init -F
/dev/sdb 262144
mkfs.lustre: Can't make configs dir /tmp/mntRmd6cD/CONFIGS (Permission
denied)
mkfs.lustre FATAL: failed to write local files
mkfs.lustre: exiting with -1 (Unknown error 18446744073709551615)
When SELinux is disabled, the file system will format without issue. You will have to
edit the appropriate configuration (on RHEL/CentOS: /etc/selinux/config) and reboot the
system in order for the change to take effect.
Regards,
Malcolm.
From: hpdd-discuss-bounces(a)lists.01.org
[mailto:hpdd-discuss-bounces@lists.01.org] On Behalf Of Patrick
Farrell
Sent: Friday, July 19, 2013 11:56 PM
To: hpdd-discuss(a)lists.01.org
Subject: Re: [HPDD-discuss] Issue installing Lustre 1.87
Upanshu,
The error message you're getting is what's expected if you still have SELinux
enabled on that system. I don't think you need to worry about lctl or the LNET
settings right now - The mkfs.lustre is failing because it can't make the temporary
directory.
I'd make sure selinux is set to disabled in /etc/selinux/config.
The line which starts 'selinux=' should read 'disabled'.
You can check whether or not that setting has taken effect with:
getenforce
I'd suggest making sure of that config file change and restarting the system, then
try your format command again.
- Patrick Farrell
-------------- next part -------------- An HTML attachment was
scrubbed...
URL:
<
http://lists.01.org/pipermail/hpdd-discuss/attachments/20130722/06207
22e/attachment-0001.html>
------------------------------
Message: 2
Date: Mon, 22 Jul 2013 09:02:50 +0000
From: "Singhal, Upanshu" <upanshu.singhal(a)emc.com>
To: Parinay Kondekar <parinay_kondekar(a)xyratex.com>
Cc: "hpdd-discuss(a)lists.01.org" <hpdd-discuss(a)ml01.01.org>
Subject: Re: [HPDD-discuss] Issue installing Lustre 1.87
Message-ID:
<6A5D14D0DFEABC43BE832A0E492963F12613F74A(a)MX101CL01.corp.emc.com>
Content-Type: text/plain; charset="us-ascii"
Hello Parinay,
Below is the output for "modprobe -v lustre", depmod does not show any output.
Attached is log file as well. I check for Patricks suggestion, I am trying to uninstall
e2fsprogs followed by upgrade, facing some issues.
insmod
/lib/modules/2.6.18-194.17.1.el5_lustre.1.8.7/updates/kernel/net/lustr
e/libcfs.ko insmod
/lib/modules/2.6.18-194.17.1.el5_lustre.1.8.7/updates/kernel/fs/lustre
/lvfs.ko insmod
/lib/modules/2.6.18-194.17.1.el5_lustre.1.8.7/updates/kernel/net/lustr
e/lnet.ko networks=tcp0(eth0) insmod
/lib/modules/2.6.18-194.17.1.el5_lustre.1.8.7/updates/kernel/fs/lustre
/obdclass.ko insmod
/lib/modules/2.6.18-194.17.1.el5_lustre.1.8.7/updates/kernel/fs/lustre
/ptlrpc.ko insmod
/lib/modules/2.6.18-194.17.1.el5_lustre.1.8.7/updates/kernel/fs/lustre
/osc.ko insmod
/lib/modules/2.6.18-194.17.1.el5_lustre.1.8.7/updates/kernel/fs/lustre
/mdc.ko insmod
/lib/modules/2.6.18-194.17.1.el5_lustre.1.8.7/updates/kernel/fs/lustre
/lov.ko insmod
/lib/modules/2.6.18-194.17.1.el5_lustre.1.8.7/updates/kernel/fs/lustre
/lustre.ko
Thanks,
-Upanshu
From: Parinay Kondekar [mailto:parinay_kondekar@xyratex.com]
Sent: Monday, July 22, 2013 1:03 PM
To: Singhal, Upanshu
Cc: hpdd-discuss(a)lists.01.org
Subject: Re: [HPDD-discuss] Issue installing Lustre 1.87
Hello Upanshu,
- Are the lustre modules loaded ? What does `modprobe -v lustre` say ? depmod ?
- logs - /var/log/messages . dmesg . If the size is huge, it would be good to attach a
trimmed version.
I hope you also cross-checked for what Patrick suggested.
HTH
On 22 July 2013 12:32, Singhal, Upanshu
<upanshu.singhal@emc.com<mailto:upanshu.singhal@emc.com>> wrote:
Hello Parinay,
Thanks for your email, much appreciated. Multiple interfaces do you mean multiple network
interfaces? If so, then the answer is no, I am using Ethernet only.
Output for "lctl list_nids" is : opening /dev/lnet failed: No such
device. hint: the kernel modules may not be loaded. IOC_LIBCFS_GET_NI
error 19: No such device Output for "lctl ping" is: opening /dev/lnet
failed: No such device. hint: the kernel modules may not be loaded.
failed to ping 10.243.109.63@tcp<mailto:10.243.109.63@tcp>: No such
device
It seems that I am missing some module on my machines or do need to configure anything
for the above errors?
Log messages from /var/log/messages are below, is there any other location to get the log
messages?
LDISKFS FS on sdb, internal journal
LDISKFS-fs: mounted filesystem with ordered data mode.
LDISKFS-fs: file extents enabled
LDISKFS-fs: mballoc enabled
LDISKFS-fs error (device sdb): ldiskfs_ext_find_extent: :463: bad
header in inode #3629057: invalid magic - magic 0, entries 0, max
0(0), depth 0(0) Remounting filesystem read-only LDISKFS-fs error
(device sdb) in start_transaction: Readonly filesystem
LDISKFS-fs: mballoc: 0 blocks 0 reqs (0 success)
LDISKFS-fs: mballoc: 0 extents scanned, 0 goal hits, 0 2^N hits, 0
breaks, 0 lost
LDISKFS-fs: mballoc: 0 generated and it took 0
LDISKFS-fs: mballoc: 0 preallocated, 0 discarded
Thanks,
-Upanshu
From: Parinay Kondekar
[mailto:parinay_kondekar@xyratex.com<mailto:parinay_kondekar@xyratex.c
om>]
Sent: Friday, July 19, 2013 7:16 PM
To: Singhal, Upanshu
Cc: hpdd-discuss@lists.01.org<mailto:hpdd-discuss@lists.01.org>
Subject: Re: [HPDD-discuss] Issue installing Lustre 1.87
More logs would help. like messages.
- Do you have multiple interfaces ? It would be good to check "lctl list_nids"
and "lctl ping "
- For 2.3 and other you can refer to -
http://downloads.whamcloud.com/public/lustre/
latest manual -
http://build.whamcloud.com/job/lustre-manual/lastSuccessfulBuild/artif
act/lustre_manual.xhtml (2.x) 1.8.x (
http://wiki.lustre.org/manual/LustreManual18_HTML/index.html )
HTH
On 19 July 2013 18:26, Singhal, Upanshu
<upanshu.singhal@emc.com<mailto:upanshu.singhal@emc.com>> wrote:
Hello,
I am installing Lustre 1.87 on RHEL 5.8 and having issue while configuring OST on one of
the node. These are the steps I am performing reading through manual, please reply how to
proceed further. Please let me know if you find any issue with these steps:
1. I have 2 RHEL 5.8 hosts, one for MGS and another one for OSS
2. Configured Static IP address on both the hosts
3. Disabled SELINUX in /etc/selinux/config file
4. Provision 1 raw disk each to both the systems for MDT and OST, did not created
any partition on them
5. Installed lustre 1.87 RPM packages on both the machines in the given order
o kernel-2.6.18-194.17.1.el5_lustre.1.8.7.x86_64.rpm
o lustre-ldiskfs-3.1.6-2.6.18_194.17.1.el5_lustre.1.8.7.x86_64.rpm
o lustre-modules-1.8.7-2.6.18_194.17.1.el5_lustre.1.8.7.x86_64.rpm
o lustre-1.8.7-2.6.18_194.17.1.el5_lustre.1.8.7.x86_64.rpm
o e2fsprogs-1.42.ora1-0redhat.rhel5.x86_64.rpm
o e2fsprogs-devel-1.42.ora1-0redhat.rhel5.x86_64.rpm
o Am I missing any package or order?
6. Modified /etc/modprobe.conf to have : options lnet networks=tcp0(eth0)
7. Rebooted both the machines
8. Disabled iptables and ip6tables
9. On machine 1 for MDS / MGS Server, successfully executed following command to
create MGS/MDT file system
a. mkfs.lustre -fsname=lustre -mgs -mdt /dev/sdb
b. mount -t lustre /dev/sdb /mnt
10. On machine 2 executed following command, but it fails with the error as I mentioned
before:
a. mkfs.lustre -fsname=lustre -ost
-mgsnode=10.243.107.39@tcp0<mailto:-mgsnode=10.243.107.39@tcp0> /dev/sdb
i. Fails with the
error :
1. mkfs.lustre: Can't make configs dir /tmp/mntRiWwg6/CONFIGS: Input/output
error
2. mkfs.lustre FATAL: failed to write local files
3. mkfs.lustre: exiting with -1 (Unknown error 18446744073709551615)
b. I tried the same command with the machine 1 as well to have MGS and OSS on the
same machine, but still have the same issue.
Please let me know are these the similar approach you take or something else I need to
do. Next thing I am planning is to download and compile the sources of Lustre 2.3 and
perform the same exercise on RHEL 6.3. Can you please let me know what all source rpms I
need to download and build?
Thanks.
Upanshu Singhal
_______________________________________________
HPDD-discuss mailing list
HPDD-discuss@lists.01.org<mailto:HPDD-discuss@lists.01.org>
https://lists.01.org/mailman/listinfo/hpdd-discuss
-------------- next part -------------- An HTML attachment was
scrubbed...
URL:
<
http://lists.01.org/pipermail/hpdd-discuss/attachments/20130722/80479
2db/attachment.html>
-------------- next part -------------- A non-text attachment was
scrubbed...
Name: messages
Type: application/octet-stream
Size: 1941 bytes
Desc: messages
URL:
<
http://lists.01.org/pipermail/hpdd-discuss/attachments/20130722/80479
2db/attachment.obj>
------------------------------
_______________________________________________
HPDD-discuss mailing list
HPDD-discuss(a)lists.01.org
https://lists.01.org/mailman/listinfo/hpdd-discuss
End of HPDD-discuss Digest, Vol 8, Issue 14
*******************************************
_______________________________________________
HPDD-discuss mailing list
HPDD-discuss(a)lists.01.org
https://lists.01.org/mailman/listinfo/hpdd-discuss