IMO you should try out strace to see if anything is noticed.
"Write failed: Broken pipe" is quite common message and difficult to
conclude anything with.
On Tue, Sep 30, 2014 at 8:16 PM, aayush agrawal <
Yes, I see ib0 in output of ifconfig -a.
I also tried with options lnet networks=*o2ib0*(ib0) but no luck.
While loading lnet I do see error in var/log/messages:
kernel: LNet: HW CPU cores: 32, npartitions: 4
alg: No test for crc32 (crc32-table)
kernel: alg: No test for adler32 (adler32-zlib)
kernel: alg: No test for crc32 (crc32-pclmul)
kernel: padlock: VIA PadLock Hash Engine not detected.
modprobe: FATAL: Error inserting padlock_sha
(/lib/modules/2.6.32_358/kernel/drivers/crypto/padlock-sha.ko): No such
But as per below link this should not be a problem?
modprobe lnet completes successfully and I see "Write failed: Broken pipe"
after running "lctl network up" and after this session gets logout from the
On 9/30/2014 7:21 PM, Parinay Kondekar wrote:
- what is the output of 'ifconfig -a' , do you see ib0 there ?
mentioning 'options lnet networks=*o2ib0*(ib0)' should be enough.
- anything in syslog ?
On Tue, Sep 30, 2014 at 6:03 PM, aayush agrawal <
> I am trying to build lustre 2.5.0 against
> MLNX_OFED_LINUX-2.2-1.0.1-rhel6.4-x86_64 on CentOS6.4 with kernel version
> But I am not able to set lnet config settings properly. I used settings
> suggested in lustre 2.x manual. But then not able to get network up using
> I have two server machines, one for mgs+mdt and second for oss and one
> client machine. I want to setup Infiniband on all these machines.
> I could run below steps successfully for all the three machines:
> 1. Run script mlnxofedinstall
> # ./mlnxofedinstall -vvv --add-kernel-support --without-32bit
> --without-fw-update --hpc
> 2. Restart openibd service
> # /etc/init.d/openibd restart
> 3. configure ib0 interface.
> 4. configure lustre with o2ib
> # ./configure --with-linux=Path_to_linux-2.6.32-358.18.1.el6
> 5. make lustre rpms:
> # make rpms
> This gave me below compilation error
> I looked online for this error and found bug registered on the same:
> Below patch from above link solved the problem and hence I could build
> lustre rpms:
> Now first I want to do the Infiniband setup for mgs and mdt on single
> machine which also has Ethernet IP. Then I want to format and mount mgs and
> So I installed above created lustre rpms and then added below line in
> options lnet networks=o2ib(ib0)
> Then I rebooted the machine to remove all lustre related modules
> including lnet and then ran modprobe lnet command to add above
> parameters and the ran lctl network up which is giving me below error:
> LNET configure error 100: Network is down
> I looked online and found below discussion on same error:
> As per suggestion in above mail I tried with below line in
> /etc/modprobe.d/lustre.conf. In below command for IB_IP, I have given
> infiniband IP.
> options lnet *networks=o2ib(ib0)* routes="tcp0 IB_IP@o2ib"
> This command hangs for around 2 to 3 minutes and then gives error: Write
> failed: Broken pipe. Same is the case for "options lnet
> But if I set: options lnet *networks=tcp0(eth0),o2ib(ib0)* routes="tcp1
> IB_IP@o2ib" then it gives LNET configure error 100: Network is down.
> It seems that for network=o2ib(ibo) I am getting error Write failed:
> Broken pipe.
> Am I missing anything while following above steps? Or how do I resolve
> above error?
> HPDD-discuss mailing list