Lustre and kernel buffer interaction
by John Bauer
I have been trying to understand a behavior I am observing in an IOR
benchmark on Lustre. I have pared it down to a simple example.
The IOR benchmark is running in MPI mode. There are 2 ranks, each
running on its own node. Each rank does the following:
Note : Test was run on the "swan" cluster at Cray Inc., using /lus/scratch
write a file. ( 10GB )
fsync the file
close the file
MPI_barrier
open the file that was written by the other rank.
read the file that was written by the other rank.
close the file that was written by the other rank.
The writing of each file goes as expected.
The fsync takes very little time ( about .05 seconds).
The first reads of the file( written by the other rank ) start out *very
*slowly. While theses first reads are proceeding slowly, the
kernel's cached memory ( the Cached: line in /proc/meminfo) decreases
from the size of the file just written to nearly zero.
Once the cached memory has reached nearly zero, the file reading
proceeds as expected.
I have attached a jpg of the instrumentation of the processes that
illustrates this behavior.
My questions are:
Why does the reading of the file, written by the other rank, wait until
the cached data drains to nearly zero before proceeding normally.
Shouldn't the fsync ensure that the file's data is written to the
backing storage so this draining of the cached memory should be simply
releasing pages with no further I/O?
For this case the "dead" time is only about 4 seconds, but this "dead"
time scales directly with the size of the files.
John
--
John Bauer
I/O Doctors LLC
507-766-0378
bauerj(a)iodoctors.com
7 years, 2 months
support for user extended file attributes
by Chris Hunter
Anyone know which version of lustre started supporting user extended
file attributes ?
These are the file attributes set with the "setfattr" command
eg) setfattr -n user.comment -v "this is a comment" file.test
thanks,
chris hunter
yale hpc group
7 years, 3 months
[PATCH 00/27] Convert seq_<foo> output calls to return void
by Joe Perches
As Al Viro said:
we are getting well-meaning folks who try to check that return value,
again and again, getting it wrong every time. Typical idiocies:
* return some kind of error out of ->show() on overflows. Pointless
*and* wrong - only hard errors (== fail read(2) with that) should be
reported that way; the caller does detect overflow and retires with bigger
buffer just fine.
* keep checking it after every sodding call of seq_...(), screwing
the cleanups up more often than not. Pointless, unless you are doing some
seriously expensive calculations to produce something you are going to print.
seq_...() are no-ops in case when overflow has already happened.
seq_has_overflowed() is only for situations when you really want to skip
a serious amount of work generating the data that would end up being
discarded and recalculated again when the caller grabs a bigger buffer and
calls you again. And more often than not it's an indication of ->show()
trying to do the work of iterator - e.g. when you have single_open() with
->show() printing the entire hash table of some sort all in one record.
Most of the time checking return value of seq_...() is better replaced with
not doing that. And "must check return value and Do Something(tm)" is too
strong habit for enough people to cause recurring trouble.
Joe Perches (27):
staging: lustre: Convert "return seq_printf(...)" uses
staging: lustre: Convert seq_ hash functions to return void
staging: lustre: Convert uses of "int rc = seq_printf(...)"
staging: lustre: Convert remaining uses of "= seq_printf(...)"
x86: mtrr: if: Remove use of seq_printf return value
power: wakeup: Remove use of seq_printf return value
ipmi: Remove use of seq_printf return value
rtc: Remove use of seq_printf return value
ipc: Remove use of seq_printf return value
pxa27x_udc: Remove use of seq_printf return value
microblaze: mb: Remove use of seq_printf return value
nios2: cpuinfo: Remove use of seq_printf return value
ARM: plat-pxa: Remove use of seq_printf return value
openrisc: Remove use of seq_printf return value
cris: Remove use of seq_printf return value
mfd: ab8500-debugfs: Remove use of seq_printf return value
staging: i2o: Remove use of seq_printf return value
staging: rtl8192x: Remove use of seq_printf return value
s390: Remove use of seq_printf return value
i8k: Remove use of seq_printf return value
watchdog: bcm281xx: Remove use of seq_printf return value
proc: Remove use of seq_printf return value
cgroup: Remove use of seq_printf return value
tracing: Remove use of seq_printf return value
lru_cache: Remove use of seq_printf return value
parisc: Remove use of seq_printf return value
regulator: dbx500: Remove use of seq_puts/seq_printf return value
arch/arm/plat-pxa/dma.c | 111 ++++++------
arch/cris/arch-v10/kernel/setup.c | 58 +++---
arch/cris/arch-v32/kernel/setup.c | 62 +++----
arch/microblaze/kernel/cpu/mb.c | 149 ++++++++--------
arch/nios2/kernel/cpuinfo.c | 77 ++++----
arch/openrisc/kernel/setup.c | 50 +++---
arch/s390/pci/pci_debug.c | 6 +-
arch/x86/kernel/cpu/mtrr/if.c | 12 +-
drivers/base/power/wakeup.c | 16 +-
drivers/char/i8k.c | 16 +-
drivers/char/ipmi/ipmi_msghandler.c | 12 +-
drivers/char/ipmi/ipmi_si_intf.c | 26 +--
drivers/char/ipmi/ipmi_ssif.c | 4 +-
drivers/mfd/ab8500-debugfs.c | 196 +++++++++++++--------
drivers/parisc/ccio-dma.c | 54 +++---
drivers/parisc/sba_iommu.c | 86 +++++----
drivers/regulator/dbx500-prcmu.c | 32 +---
drivers/rtc/rtc-cmos.c | 36 ++--
drivers/rtc/rtc-ds1305.c | 6 +-
drivers/rtc/rtc-mrst.c | 16 +-
drivers/rtc/rtc-tegra.c | 4 +-
drivers/s390/cio/blacklist.c | 12 +-
drivers/staging/i2o/i2o_proc.c | 18 +-
.../lustre/include/linux/libcfs/libcfs_hash.h | 4 +-
drivers/staging/lustre/lustre/fid/lproc_fid.c | 23 ++-
drivers/staging/lustre/lustre/libcfs/hash.c | 13 +-
drivers/staging/lustre/lustre/llite/lproc_llite.c | 117 ++++++------
drivers/staging/lustre/lustre/lmv/lproc_lmv.c | 18 +-
drivers/staging/lustre/lustre/lov/lproc_lov.c | 30 ++--
drivers/staging/lustre/lustre/mdc/lproc_mdc.c | 6 +-
.../lustre/lustre/obdclass/linux/linux-module.c | 38 ++--
.../lustre/lustre/obdclass/lprocfs_status.c | 108 ++++++------
drivers/staging/lustre/lustre/obdclass/lu_object.c | 25 +--
drivers/staging/lustre/lustre/osc/lproc_osc.c | 67 +++----
.../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c | 25 +--
drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c | 82 +++++----
drivers/staging/rtl8192e/rtllib_module.c | 4 +-
.../staging/rtl8192u/ieee80211/ieee80211_module.c | 4 +-
drivers/usb/gadget/udc/pxa27x_udc.c | 132 +++++++-------
drivers/watchdog/bcm_kona_wdt.c | 27 +--
fs/proc/array.c | 4 +-
fs/proc/base.c | 82 +++++----
ipc/msg.c | 34 ++--
ipc/sem.c | 26 +--
ipc/shm.c | 42 ++---
ipc/util.c | 6 +-
kernel/cgroup.c | 4 +-
kernel/trace/trace_stack.c | 4 +-
lib/lru_cache.c | 9 +-
49 files changed, 1050 insertions(+), 943 deletions(-)
--
2.1.2
7 years, 3 months
[PATCH 1/1] Lustre: single statement macros should not use do {} while (0)
by Mario J. Rugiero
This fixes all appearances of the warning but one, as that one seems to
be intentional:
WARNING: Single statement macros should not use a do {} while (0) loop
#1221: FILE: lustre/include/lustre_dlm.h:1221:
+#define LDLM_LOCK_RELEASE(lock) \
+do { \
+ /*LDLM_DEBUG((lock), "put");*/ \
+ ldlm_lock_put(lock); \
+} while (0)
I didn't know if I should touch this debug statement, and if uncommented
it would certainly be incorrect to erase the do-while construct.
Regards,
Mario.
7 years, 3 months
Memory lost forever for app running on lustre mount
by Brock Palen
We think we have narrowed down a problem related to our lustre system.
We see the following problem on our clients but only for some codes. We slowly watch systems, compute and login, run out of memory. There are not any processes using significant amount of memory.
For one user who's code (starx) can in one run run the entire system out of memory only does this when reading it's data from our lustre mount. When using NFS the issue doesn't happen.
Client: 2.5.2
Server: 2.5.19 (ddn exascaler 2.1)
RHEL6 kernel 2.6.32-431.40.2.el6.x86_64
We see all the memory disappear, and this is NOT cache memory this is real memory, it never comes back, they all show up as anonymous pages, we start swapping and eventually OOM nodes with no large memory use process.
Has anyone seen an issue like this? We have nodes all over our cluster where the memory footprint (ignoring cache) gets smaller and smaller.
Eg here is one of our login nodes:
Mem: 49416180k total, 48890424k used, 525756k free, 18300k buffers
Swap: 12582908k total, 12582908k used, 0k free, 727252k cached
Here is the result of ps -eo pid,args,pmem --sort pmem
http://pastebin.com/tDcBq1HF
Here is the result of
slabtop -o and cat /proc/meminfo
http://pastebin.com/RAGrHNzd
We can reproduce the problem on demand with the specific users code, but as noted we think it effects other nodes.
Thanks!
Brock Palen
www.umich.edu/~brockp
Assoc. Director Advanced Research Computing - TS
XSEDE Campus Champion
brockp(a)umich.edu
(734)936-1985
7 years, 3 months
LUG Abstract Submissions Closing Feb 27th
by OpenSFS Administration
<http://opensfs.org/events/lug-2015/>
Last Chance to Submit LUG Abstracts!
Denver Marriott City Center, April 13-15
Friday, February 27th is the last day to submit abstracts for the
<http://opensfs.org/events/lug-2015/> 13th annual LustreR User Group (LUG)
conference. If you or someone you know has undertaken specific Lustre
development challenges, new development features, or edge-case
implementations, then we invite you to share your experience with over 200+
Lustre-focused attendees at LUG 2015. To submit an abstract visit the
<http://opensfs.org/lug-call-for-presentations/> LUG 2015 Call for
Presentations Submission web page
OpenSFS is also accepting panel discussion topics and poster session
submissions:
* Panel discussions offer a forum for leading developers, vendors,
and users of Lustre to debate future requirements, explore upcoming
enhancements, and share real world best practices. If you'd like to suggest
a topic for a panel discussion - or be part of a panel, please contact
<mailto:panelist@opensfs.org> admin(a)opensfs.org.
* <http://opensfs.org/lug-call-for-presentations/> Poster sessions
provide an opportunity for informal, interactive presentation and discussion
of diverse topics related to Lustre technology.
Register for LUG:
By attending this year's LUG conference you and your associates will have
the exclusive opportunity to collaborate with industry leaders to advance
the Lustre file system and contribute to new releases on behalf of the open
source community.
<http://opensfs.org/events/lug-2015/> Register now to take advantage of the
special early bird rate before March 20th. The early bird registration price
is $525.
Visit <http://opensfs.org/events/lug-2015/> the LUG Event Page for more
information about lodging and networking opportunities. Agenda details will
be coming soon!
Register for the Lustre Developer Day:
In conjunction with LUG, OpenSFS will be hosting a
<http://opensfs.org/lustre-developer-day/> Lustre Developer Day in Denver,
Colorado on April 16, 2015.
The Lustre Developer Day is a separate event for Lustre software developers.
All topics concerning Lustre software development are welcome. The agenda
will be finalized with input from registered attendees. Please send any
agenda ideas (information you would like to present, topics you'd like to
cover, projects you'd like to discuss, etc.) to OpenSFS Administration.
* Who: Lustre software developers and those interested in Lustre
development
* What: A day of sessions and discussions dedicated to Lustre
development
* Where: Denver, Colorado
* When: Thursday, April 16th
* How:
<http://events.r20.constantcontact.com/register/event?oeidk=a07eaipyxajbb17a
22c&llr=8irdhctab> Register online. Registration is limited to 50 attendees.
More information including registration fees and hotel details are available
on the <http://opensfs.org/lustre-developer-day/> Lustre Developer Day web
page.
If you have any questions, please feel free to contact
<mailto:admin@opensfs.org> admin(a)opensfs.org. We look forward to seeing you
in Denver!
Best regards,
OpenSFS LUG Planning Committee
_________________________
OpenSFS Administration
3855 SW 153rd Drive Beaverton, OR 97003 USA
Phone: +1 503-619-0561 | Fax: +1 503-644-6708
Twitter: <https://twitter.com/opensfs> @OpenSFS
Email: <mailto:admin@opensfs.org> admin(a)opensfs.org | Website:
<http://www.opensfs.org> www.opensfs.org
Silver Sponsors:
<http://cdn.opensfs.org/wp-content/uploads/2015/01/LUG_2015_Sponsorship_Oppo
rtunities.pdf> Click here to learn how to become a 2015 LUG Sponsor
Member Benefits at LUG:
We are happy to offer one complimentary event pass to each member company of
OpenSFS as a benefit of membership. The registration discount code has been
sent to the primary contact, but it may be transferred to another
representative at the same company if the primary contact will not be
attending.
Open Scalable File Systems, Inc. was founded in 2010 to advance Lustre
development, ensuring it remains vendor-neutral, open, and free. Since its
inception, OpenSFS has been responsible for advancing the Lustre file system
and delivering new releases on behalf of the open source community. Through
working groups, events, and ongoing funding initiatives, OpenSFS harnesses
the power of collaborative development to fuel innovation and growth of the
Lustre file system worldwide.
7 years, 3 months
Reminder: Lustre Usage Survey Closes Feb 28!
by OpenSFS Administration
Hello Lustre Community members,
As a reminder, the annual Lustre usage survey is available until February
28th. This fourth annual survey will be utilized to identify trends in
Lustre usage and assist with future planning on releases. The results of the
survey will be presented at LUG 2015 <http://opensfs.org/events/lug-2015/> .
We would appreciate your input and participation in the survey, available at
https://www.surveymonkey.com/r/63R8N7X. Note that all questions are optional
so it is ok to submit a partially completed survey if you prefer not to
disclose some information.
Best regards,
OpenSFS Administration
__________________________
OpenSFS Administration
3855 SW 153rd Drive Beaverton, OR 97003 USA
Phone: +1 503-619-0561 | Fax: +1 503-644-6708
Twitter: <https://twitter.com/opensfs> @OpenSFS
Email: <mailto:admin@opensfs.org> admin(a)opensfs.org | Website:
<http://www.opensfs.org/> www.opensfs.org
7 years, 4 months
[PATCH] drivers: staging: lustre: Fix "space prohibited between function name and open parenthesis" errors
by Adrien Descamps
Fix checkpatch.ph errors "space prohibited between function name and
open parenthesis" in socklnd.h
Signed-off-by: Adrien Descamps <adrien.descamps(a)gmail.com>
---
.../staging/lustre/lnet/klnds/socklnd/socklnd.h | 130 ++++++++++-----------
1 file changed, 65 insertions(+), 65 deletions(-)
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
index 03488d2..c54c995 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
@@ -406,7 +406,7 @@ ksocknal_route_mask(void)
}
static inline struct list_head *
-ksocknal_nid2peerlist (lnet_nid_t nid)
+ksocknal_nid2peerlist(lnet_nid_t nid)
{
unsigned int hash = ((unsigned int)nid) % ksocknal_data.ksnd_peer_hash_size;
@@ -414,25 +414,25 @@ ksocknal_nid2peerlist (lnet_nid_t nid)
}
static inline void
-ksocknal_conn_addref (ksock_conn_t *conn)
+ksocknal_conn_addref(ksock_conn_t *conn)
{
- LASSERT (atomic_read(&conn->ksnc_conn_refcount) > 0);
+ LASSERT(atomic_read(&conn->ksnc_conn_refcount) > 0);
atomic_inc(&conn->ksnc_conn_refcount);
}
-extern void ksocknal_queue_zombie_conn (ksock_conn_t *conn);
+extern void ksocknal_queue_zombie_conn(ksock_conn_t *conn);
extern void ksocknal_finalize_zcreq(ksock_conn_t *conn);
static inline void
-ksocknal_conn_decref (ksock_conn_t *conn)
+ksocknal_conn_decref(ksock_conn_t *conn)
{
- LASSERT (atomic_read(&conn->ksnc_conn_refcount) > 0);
+ LASSERT(atomic_read(&conn->ksnc_conn_refcount) > 0);
if (atomic_dec_and_test(&conn->ksnc_conn_refcount))
ksocknal_queue_zombie_conn(conn);
}
static inline int
-ksocknal_connsock_addref (ksock_conn_t *conn)
+ksocknal_connsock_addref(ksock_conn_t *conn)
{
int rc = -ESHUTDOWN;
@@ -448,11 +448,11 @@ ksocknal_connsock_addref (ksock_conn_t *conn)
}
static inline void
-ksocknal_connsock_decref (ksock_conn_t *conn)
+ksocknal_connsock_decref(ksock_conn_t *conn)
{
- LASSERT (atomic_read(&conn->ksnc_sock_refcount) > 0);
+ LASSERT(atomic_read(&conn->ksnc_sock_refcount) > 0);
if (atomic_dec_and_test(&conn->ksnc_sock_refcount)) {
- LASSERT (conn->ksnc_closing);
+ LASSERT(conn->ksnc_closing);
libcfs_sock_release(conn->ksnc_sock);
conn->ksnc_sock = NULL;
ksocknal_finalize_zcreq(conn);
@@ -460,61 +460,61 @@ ksocknal_connsock_decref (ksock_conn_t *conn)
}
static inline void
-ksocknal_tx_addref (ksock_tx_t *tx)
+ksocknal_tx_addref(ksock_tx_t *tx)
{
- LASSERT (atomic_read(&tx->tx_refcount) > 0);
+ LASSERT(atomic_read(&tx->tx_refcount) > 0);
atomic_inc(&tx->tx_refcount);
}
-extern void ksocknal_tx_prep (ksock_conn_t *, ksock_tx_t *tx);
-extern void ksocknal_tx_done (lnet_ni_t *ni, ksock_tx_t *tx);
+extern void ksocknal_tx_prep(ksock_conn_t *, ksock_tx_t *tx);
+extern void ksocknal_tx_done(lnet_ni_t *ni, ksock_tx_t *tx);
static inline void
-ksocknal_tx_decref (ksock_tx_t *tx)
+ksocknal_tx_decref(ksock_tx_t *tx)
{
- LASSERT (atomic_read(&tx->tx_refcount) > 0);
+ LASSERT(atomic_read(&tx->tx_refcount) > 0);
if (atomic_dec_and_test(&tx->tx_refcount))
ksocknal_tx_done(NULL, tx);
}
static inline void
-ksocknal_route_addref (ksock_route_t *route)
+ksocknal_route_addref(ksock_route_t *route)
{
- LASSERT (atomic_read(&route->ksnr_refcount) > 0);
+ LASSERT(atomic_read(&route->ksnr_refcount) > 0);
atomic_inc(&route->ksnr_refcount);
}
-extern void ksocknal_destroy_route (ksock_route_t *route);
+extern void ksocknal_destroy_route(ksock_route_t *route);
static inline void
-ksocknal_route_decref (ksock_route_t *route)
+ksocknal_route_decref(ksock_route_t *route)
{
- LASSERT (atomic_read (&route->ksnr_refcount) > 0);
+ LASSERT(atomic_read(&route->ksnr_refcount) > 0);
if (atomic_dec_and_test(&route->ksnr_refcount))
- ksocknal_destroy_route (route);
+ ksocknal_destroy_route(route);
}
static inline void
-ksocknal_peer_addref (ksock_peer_t *peer)
+ksocknal_peer_addref(ksock_peer_t *peer)
{
- LASSERT (atomic_read (&peer->ksnp_refcount) > 0);
+ LASSERT(atomic_read(&peer->ksnp_refcount) > 0);
atomic_inc(&peer->ksnp_refcount);
}
-extern void ksocknal_destroy_peer (ksock_peer_t *peer);
+extern void ksocknal_destroy_peer(ksock_peer_t *peer);
static inline void
-ksocknal_peer_decref (ksock_peer_t *peer)
+ksocknal_peer_decref(ksock_peer_t *peer)
{
- LASSERT (atomic_read (&peer->ksnp_refcount) > 0);
+ LASSERT(atomic_read(&peer->ksnp_refcount) > 0);
if (atomic_dec_and_test(&peer->ksnp_refcount))
- ksocknal_destroy_peer (peer);
+ ksocknal_destroy_peer(peer);
}
-int ksocknal_startup (lnet_ni_t *ni);
-void ksocknal_shutdown (lnet_ni_t *ni);
+int ksocknal_startup(lnet_ni_t *ni);
+void ksocknal_shutdown(lnet_ni_t *ni);
int ksocknal_ctl(lnet_ni_t *ni, unsigned int cmd, void *arg);
-int ksocknal_send (lnet_ni_t *ni, void *private, lnet_msg_t *lntmsg);
+int ksocknal_send(lnet_ni_t *ni, void *private, lnet_msg_t *lntmsg);
int ksocknal_recv(lnet_ni_t *ni, void *private, lnet_msg_t *lntmsg,
int delayed, unsigned int niov,
struct kvec *iov, lnet_kiov_t *kiov,
@@ -522,44 +522,44 @@ int ksocknal_recv(lnet_ni_t *ni, void *private, lnet_msg_t *lntmsg,
int ksocknal_accept(lnet_ni_t *ni, struct socket *sock);
extern int ksocknal_add_peer(lnet_ni_t *ni, lnet_process_id_t id, __u32 ip, int port);
-extern ksock_peer_t *ksocknal_find_peer_locked (lnet_ni_t *ni, lnet_process_id_t id);
-extern ksock_peer_t *ksocknal_find_peer (lnet_ni_t *ni, lnet_process_id_t id);
-extern void ksocknal_peer_failed (ksock_peer_t *peer);
-extern int ksocknal_create_conn (lnet_ni_t *ni, ksock_route_t *route,
+extern ksock_peer_t *ksocknal_find_peer_locked(lnet_ni_t *ni, lnet_process_id_t id);
+extern ksock_peer_t *ksocknal_find_peer(lnet_ni_t *ni, lnet_process_id_t id);
+extern void ksocknal_peer_failed(ksock_peer_t *peer);
+extern int ksocknal_create_conn(lnet_ni_t *ni, ksock_route_t *route,
struct socket *sock, int type);
-extern void ksocknal_close_conn_locked (ksock_conn_t *conn, int why);
-extern void ksocknal_terminate_conn (ksock_conn_t *conn);
-extern void ksocknal_destroy_conn (ksock_conn_t *conn);
-extern int ksocknal_close_peer_conns_locked (ksock_peer_t *peer,
+extern void ksocknal_close_conn_locked(ksock_conn_t *conn, int why);
+extern void ksocknal_terminate_conn(ksock_conn_t *conn);
+extern void ksocknal_destroy_conn(ksock_conn_t *conn);
+extern int ksocknal_close_peer_conns_locked(ksock_peer_t *peer,
__u32 ipaddr, int why);
-extern int ksocknal_close_conn_and_siblings (ksock_conn_t *conn, int why);
-extern int ksocknal_close_matching_conns (lnet_process_id_t id, __u32 ipaddr);
+extern int ksocknal_close_conn_and_siblings(ksock_conn_t *conn, int why);
+extern int ksocknal_close_matching_conns(lnet_process_id_t id, __u32 ipaddr);
extern ksock_conn_t *ksocknal_find_conn_locked(ksock_peer_t *peer,
ksock_tx_t *tx, int nonblk);
extern int ksocknal_launch_packet(lnet_ni_t *ni, ksock_tx_t *tx,
lnet_process_id_t id);
extern ksock_tx_t *ksocknal_alloc_tx(int type, int size);
-extern void ksocknal_free_tx (ksock_tx_t *tx);
+extern void ksocknal_free_tx(ksock_tx_t *tx);
extern ksock_tx_t *ksocknal_alloc_tx_noop(__u64 cookie, int nonblk);
extern void ksocknal_next_tx_carrier(ksock_conn_t *conn);
-extern void ksocknal_queue_tx_locked (ksock_tx_t *tx, ksock_conn_t *conn);
-extern void ksocknal_txlist_done (lnet_ni_t *ni, struct list_head *txlist,
+extern void ksocknal_queue_tx_locked(ksock_tx_t *tx, ksock_conn_t *conn);
+extern void ksocknal_txlist_done(lnet_ni_t *ni, struct list_head *txlist,
int error);
-extern void ksocknal_notify (lnet_ni_t *ni, lnet_nid_t gw_nid, int alive);
-extern void ksocknal_query (struct lnet_ni *ni, lnet_nid_t nid, unsigned long *when);
+extern void ksocknal_notify(lnet_ni_t *ni, lnet_nid_t gw_nid, int alive);
+extern void ksocknal_query(struct lnet_ni *ni, lnet_nid_t nid, unsigned long *when);
extern int ksocknal_thread_start(int (*fn)(void *arg), void *arg, char *name);
-extern void ksocknal_thread_fini (void);
-extern void ksocknal_launch_all_connections_locked (ksock_peer_t *peer);
-extern ksock_route_t *ksocknal_find_connectable_route_locked (ksock_peer_t *peer);
-extern ksock_route_t *ksocknal_find_connecting_route_locked (ksock_peer_t *peer);
-extern int ksocknal_new_packet (ksock_conn_t *conn, int skip);
-extern int ksocknal_scheduler (void *arg);
-extern int ksocknal_connd (void *arg);
-extern int ksocknal_reaper (void *arg);
-extern int ksocknal_send_hello (lnet_ni_t *ni, ksock_conn_t *conn,
+extern void ksocknal_thread_fini(void);
+extern void ksocknal_launch_all_connections_locked(ksock_peer_t *peer);
+extern ksock_route_t *ksocknal_find_connectable_route_locked(ksock_peer_t *peer);
+extern ksock_route_t *ksocknal_find_connecting_route_locked(ksock_peer_t *peer);
+extern int ksocknal_new_packet(ksock_conn_t *conn, int skip);
+extern int ksocknal_scheduler(void *arg);
+extern int ksocknal_connd(void *arg);
+extern int ksocknal_reaper(void *arg);
+extern int ksocknal_send_hello(lnet_ni_t *ni, ksock_conn_t *conn,
lnet_nid_t peer_nid, ksock_hello_msg_t *hello);
-extern int ksocknal_recv_hello (lnet_ni_t *ni, ksock_conn_t *conn,
+extern int ksocknal_recv_hello(lnet_ni_t *ni, ksock_conn_t *conn,
ksock_hello_msg_t *hello, lnet_process_id_t *id,
__u64 *incarnation);
extern void ksocknal_read_callback(ksock_conn_t *conn);
@@ -569,15 +569,15 @@ extern int ksocknal_lib_zc_capable(ksock_conn_t *conn);
extern void ksocknal_lib_save_callback(struct socket *sock, ksock_conn_t *conn);
extern void ksocknal_lib_set_callback(struct socket *sock, ksock_conn_t *conn);
extern void ksocknal_lib_reset_callback(struct socket *sock, ksock_conn_t *conn);
-extern void ksocknal_lib_push_conn (ksock_conn_t *conn);
-extern int ksocknal_lib_get_conn_addrs (ksock_conn_t *conn);
-extern int ksocknal_lib_setup_sock (struct socket *so);
-extern int ksocknal_lib_send_iov (ksock_conn_t *conn, ksock_tx_t *tx);
-extern int ksocknal_lib_send_kiov (ksock_conn_t *conn, ksock_tx_t *tx);
-extern void ksocknal_lib_eager_ack (ksock_conn_t *conn);
-extern int ksocknal_lib_recv_iov (ksock_conn_t *conn);
-extern int ksocknal_lib_recv_kiov (ksock_conn_t *conn);
-extern int ksocknal_lib_get_conn_tunables (ksock_conn_t *conn, int *txmem,
+extern void ksocknal_lib_push_conn(ksock_conn_t *conn);
+extern int ksocknal_lib_get_conn_addrs(ksock_conn_t *conn);
+extern int ksocknal_lib_setup_sock(struct socket *so);
+extern int ksocknal_lib_send_iov(ksock_conn_t *conn, ksock_tx_t *tx);
+extern int ksocknal_lib_send_kiov(ksock_conn_t *conn, ksock_tx_t *tx);
+extern void ksocknal_lib_eager_ack(ksock_conn_t *conn);
+extern int ksocknal_lib_recv_iov(ksock_conn_t *conn);
+extern int ksocknal_lib_recv_kiov(ksock_conn_t *conn);
+extern int ksocknal_lib_get_conn_tunables(ksock_conn_t *conn, int *txmem,
int *rxmem, int *nagle);
extern int ksocknal_tunables_init(void);
--
2.1.4
7 years, 4 months
client version to interop with 2.1 & 2.5
by Chris Hunter
Can anyone recommend lustre client version to use with 2.1 & 2.5
filesystems ?
I suppose 2.4 would split the difference.
regards,
chris hunter
yale hpc group
7 years, 4 months
