[PATCH] staging: lustre: lustre: mdc: mdc_lib.c: Removed an unnecessary NULL check
by Rickard Strandqvist
Removed an unnecessary NULL check.
I have checked the only place this function is called from.
Signed-off-by: Rickard Strandqvist <rickard_strandqvist(a)spectrumdigital.se>
---
drivers/staging/lustre/lustre/mdc/mdc_lib.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_lib.c b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
index e8732cc..a798c2b 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_lib.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
@@ -224,10 +224,9 @@ void mdc_open_pack(struct ptlrpc_request *req, struct md_op_data *op_data,
rec->cr_fsuid = from_kuid(&init_user_ns, current_fsuid());
rec->cr_fsgid = from_kgid(&init_user_ns, current_fsgid());
rec->cr_cap = cfs_curproc_cap_pack();
- if (op_data != NULL) {
- rec->cr_fid1 = op_data->op_fid1;
- rec->cr_fid2 = op_data->op_fid2;
- }
+ rec->cr_fid1 = op_data->op_fid1;
+ rec->cr_fid2 = op_data->op_fid2;
+
rec->cr_mode = mode;
cr_flags = mds_pack_open_flags(flags, mode);
rec->cr_rdev = rdev;
--
1.7.10.4
6 years, 11 months
Re: [HPDD-discuss] Long req_waittime
by Dilger, Andreas
You're cutting off all of the useful values from these stats. All this
tells us is that you sent 239189 RPCs from this client, but you chopped
off the actual stats that report how long the RPCs took (in microseconds:
min, max, sum, sum_squared for stddev). You can use something like:
llstat -i 10 fsname-OST0000-*
to get a nicely formatted output (condensed to 80 columns for this email):
/proc/fs/lustre/osc/myth-OST0000-osc-ffff880079252800/stats @
1418771188.848400
Name Count Rate Events Unit last min avg max
stddev
req_waittime 0 0 38104 [usec] 0 511 29747 859246
38441.08
req_active 0 0 38104 [reqs] 0 1 1.15 17
0.64
read_bytes 0 0 10967 [bytes] 0 0 644180 1048576
470678.11
write_bytes 0 0 8831 [bytes] 0 188 314025 1048576
154912.70
ost_read 0 0 10967 [usec] 0 1311 43729 859246
55680.31
ost_write 0 0 8831 [usec] 0 4003 21733 578621
22564.67
ost_connect 0 0 1 [usec] 0 1581 1581 1581
0.00
ost_punch 0 0 70 [usec] 0 981 10499 51744
10787.99
ost_statfs 0 0 8060 [usec] 0 511 2217 110251
2467.72
ost_sync 0 0 8240 [usec] 0 660 51698 265859
20041.93
ldlm_cancel 0 0 638 [usec] 0 688 16629 202345
25984.31
obd_ping 0 0 444 [usec] 0 840 3848 52907
2766.05
These are aggregate stats since the client/server was first mounted, or
last cleared (by writing into the /proc file or using "lctl set_param
NNNN=clear").
This will at least give you some more useful info about the stats to start
looking at what is going slowly.
Cheers, Andreas
--
Andreas Dilger
Lustre Software Architect
Intel High Performance Data Division
On 2014/12/16, 3:51 PM, "Kumar, Amit" <ahkumar(a)mail.smu.edu> wrote:
>Dear All,
>
>I am trying figure out what is causing long req_waittime for our OST¹s.
>This looks really bad. Any tips on digging into this is greatly
>appreciated.
>
>I am checking the health of OST and all look okay. I will collect RPC
>stats and see if I see something there.
>
>
>Here is a sample output from client
>req_waittime 239189
>req_active 239189
>ost_connect 1
>ost_statfs 254
>ldlm_cancel 13
>obd_ping 238907
>
>sample from one OST:
>snapshot_time 1418769865.299088
>req_waittime 1142788387
>req_qdepth 1142788387
>req_active 1142788387
>req_timeout 1142788387
>reqbuf_avail 2389588796
>ldlm_glimpse_enqueue 7125998
>ldlm_extent_enqueue 86631679
>ost_setattr 136096
>ost_create 33258
>ost_destroy 757369
>ost_connect 13145
>ost_disconnect 4302
>obd_ping 1048086540
>
>are these cumulative stats, don¹t believe so???
>
>Best Regards,
>Amit
>
6 years, 11 months
[PATCH v2 0/2] lprocfs_write_frac_u64_helper cleanup
by Chris Rorvick
Added a second patch to address Dan Carpenter's concern with the
complexity of passing the sign through `mult'. Compile tested only.
Chris Rorvick (2):
drivers: staging: lustre: Use mult if units not specified
drivers: staging: lustre: Track sign separately
drivers/staging/lustre/lustre/obdclass/lprocfs_status.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
--
2.1.0
6 years, 11 months
Long req_waittime
by Kumar, Amit
Dear All,
I am trying figure out what is causing long req_waittime for our OST’s. This looks really bad. Any tips on digging into this is greatly appreciated.
I am checking the health of OST and all look okay. I will collect RPC stats and see if I see something there.
Here is a sample output from client
req_waittime 239189
req_active 239189
ost_connect 1
ost_statfs 254
ldlm_cancel 13
obd_ping 238907
sample from one OST:
snapshot_time 1418769865.299088
req_waittime 1142788387
req_qdepth 1142788387
req_active 1142788387
req_timeout 1142788387
reqbuf_avail 2389588796
ldlm_glimpse_enqueue 7125998
ldlm_extent_enqueue 86631679
ost_setattr 136096
ost_create 33258
ost_destroy 757369
ost_connect 13145
ost_disconnect 4302
obd_ping 1048086540
are these cumulative stats, don’t believe so???
Best Regards,
Amit
6 years, 11 months
[PATCH] drivers: staging: lustre: Use mult if units not specified
by Chris Rorvick
Units can be passed to lprocfs_write_frac_u64_helper() via a suffix
(e.g., "...K", "...M", etc.) tacked onto the value. A comment states
that "specified units override the multiplier," though the multiplier is
overridden regardless. Update the conditional logic so that it only
applies when units are specified.
Signed-off-by: Chris Rorvick <chris(a)rorvick.com>
---
drivers/staging/lustre/lustre/obdclass/lprocfs_status.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
index 61e04af..92ed0a0 100644
--- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
+++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
@@ -1910,7 +1910,7 @@ int lprocfs_write_frac_u64_helper(const char *buffer, unsigned long count,
units <<= 10;
}
/* Specified units override the multiplier */
- if (units)
+ if (units > 1)
mult = mult < 0 ? -units : units;
frac *= mult;
--
2.1.0
6 years, 11 months
[PATCH] staging: lustre: lustre: obdclass: lprocfs_status.c: Fix for possible null pointer dereference
by Rickard Strandqvist
There is otherwise a risk of a possible null pointer dereference.
Was largely found by using a static code analysis program called cppcheck.
Signed-off-by: Rickard Strandqvist <rickard_strandqvist(a)spectrumdigital.se>
---
.../lustre/lustre/obdclass/lprocfs_status.c | 24 +++++++++++---------
1 file changed, 13 insertions(+), 11 deletions(-)
diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
index 61e04af..5227324 100644
--- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
+++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
@@ -1897,17 +1897,19 @@ int lprocfs_write_frac_u64_helper(const char *buffer, unsigned long count,
}
units = 1;
- switch (*end) {
- case 'p': case 'P':
- units <<= 10;
- case 't': case 'T':
- units <<= 10;
- case 'g': case 'G':
- units <<= 10;
- case 'm': case 'M':
- units <<= 10;
- case 'k': case 'K':
- units <<= 10;
+ if (end) {
+ switch (*end) {
+ case 'p': case 'P':
+ units <<= 10;
+ case 't': case 'T':
+ units <<= 10;
+ case 'g': case 'G':
+ units <<= 10;
+ case 'm': case 'M':
+ units <<= 10;
+ case 'k': case 'K':
+ units <<= 10;
+ }
}
/* Specified units override the multiplier */
if (units)
--
1.7.10.4
6 years, 11 months
[PATCH] staging: lustre: lustre: osc: lproc_osc.c: Fix for possible null pointer dereference
by Rickard Strandqvist
There is otherwise a risk of a possible null pointer dereference.
Was largely found by using a static code analysis program called cppcheck.
Signed-off-by: Rickard Strandqvist <rickard_strandqvist(a)spectrumdigital.se>
---
drivers/staging/lustre/lustre/osc/lproc_osc.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/staging/lustre/lustre/osc/lproc_osc.c b/drivers/staging/lustre/lustre/osc/lproc_osc.c
index 9f719bc..9ba6293 100644
--- a/drivers/staging/lustre/lustre/osc/lproc_osc.c
+++ b/drivers/staging/lustre/lustre/osc/lproc_osc.c
@@ -237,13 +237,15 @@ static ssize_t osc_cur_grant_bytes_seq_write(struct file *file, const char *buff
size_t count, loff_t *off)
{
struct obd_device *obd = ((struct seq_file *)file->private_data)->private;
- struct client_obd *cli = &obd->u.cli;
+ struct client_obd *cli;
int rc;
__u64 val;
if (obd == NULL)
return 0;
+ cli = &obd->u.cli;
+
rc = lprocfs_write_u64_helper(buffer, count, &val);
if (rc)
return rc;
--
1.7.10.4
6 years, 11 months
[PATCH] staging: lustre: lustre: obdclass: lprocfs_status.c: Fix for possible null pointer dereference
by Rickard Strandqvist
There is otherwise a risk of a possible null pointer dereference.
Was largely found by using a static code analysis program called cppcheck.
Signed-off-by: Rickard Strandqvist <rickard_strandqvist(a)spectrumdigital.se>
---
.../lustre/lustre/obdclass/lprocfs_status.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
index 61e04af..4a7891a 100644
--- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
+++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
@@ -1897,17 +1897,15 @@ int lprocfs_write_frac_u64_helper(const char *buffer, unsigned long count,
}
units = 1;
- switch (*end) {
- case 'p': case 'P':
- units <<= 10;
- case 't': case 'T':
- units <<= 10;
- case 'g': case 'G':
- units <<= 10;
- case 'm': case 'M':
- units <<= 10;
- case 'k': case 'K':
- units <<= 10;
+ if (end) {
+ switch (*end) {
+ case 'p': case 'P':
+ case 't': case 'T':
+ case 'g': case 'G':
+ case 'm': case 'M':
+ case 'k': case 'K':
+ units <<= 10;
+ }
}
/* Specified units override the multiplier */
if (units)
--
1.7.10.4
6 years, 11 months
CFP: International Workshop on The Lustre Ecosystem
by Michael J. Brim
[Apologies if you receive multiple copies of this call.]
**** CALL FOR EXTENDED ABSTRACTS ****
International Workshop on The Lustre Ecosystem: Challenges and Opportunities
http://lustre.ornl.gov/ecosystem
March 3-4, 2015
Annapolis, Maryland, USA
**** IMPORTANT DATES ****
Note: all submission deadlines are midnight anywhere-on-earth
Extended abstracts due: January 23, 2015
Acceptance notification: February 11, 2015
Technical presentations: March 4, 2015
Full papers due: March 13, 2015
**** SCOPE and TOPICS ****
Large-scale storage systems are often difficult to manage with
complicated interactions between expensive storage hardware, high
performance interconnection networks, and client computer systems. The
Lustre parallel file system has been widely adopted by high-performance
computing (HPC) centers as an effective system for managing large-scale
storage resources. Lustre achieves unprecedented aggregate performance
by parallelizing I/O over file system clients and storage targets at
extreme scales. Today, 7 out of 10 fastest supercomputers in the world
use Lustre for high-performance storage.
Lustre development has focused on improving the performance and
scalability of large-scale scientific workloads. In particular,
large-scale checkpoint storage and retrieval, which is characterized by
bursty I/O from coordinated parallel clients, has been the primary
driver of Lustre development over the last decade. With the advent of
extreme scale computing and Big Data computing, many HPC centers are
seeing increased user interest in running diverse workloads that place
new demands on Lustre.
In this workshop, we seek contributions that explore improvements in the
performance and flexibility of the Lustre file system for supporting
diverse workloads. This will be a great opportunity for the Lustre
community to discuss the challenges associated with enhancing Lustre for
diverse applications, the technological advances necessary, and the
associated ecosystem.
Specific topics of interest include, but are not limited to:
• Workload Characterization
• Adaptability and Scalability of Lustre for Diverse Workloads
• Resilience and Serviceability of Lustre
• Knowledge Provenance in Lustre
• Application-driven Lustre Benchmarking
• Integrating Big Data Technologies with Lustre
• Performance Monitoring Tools for Lustre
**** SUBMISSION GUIDELINES ****
Authors should electronically submit extended abstracts of previously
unpublished work in PDF format. Abstracts may consist of up to 5 US
letter-size (8.5 by 11in.) pages, including figures, tables, and
references. Submissions should be formatted in IEEE Computer Society
conference proceedings style.
Accepted abstracts are expected to be expanded into technical papers of
at most 10 pages in length, all-inclusive. Full papers will be due the
week following the workshop, and will undergo shepherding in preparation
for publication in the workshop proceedings.
**** ORGANIZING COMMITTEE ****
==PROGRAM CO-CHAIRS ==
Neena Imam, Oak Ridge National Laboratory, USA
Michael Brim, Oak Ridge National Laboratory, USA
Sarp Oral, Oak Ridge National Laboratory, USA
== TUTORIALS CHAIR ==
Jason Hill, Oak Ridge National Laboratory, USA
**** TECHNICAL PROGRAM COMMITTEE ****
Jeremy Archuleta, Oak Ridge National Laboratory, USA
Michael Brim, Oak Ridge National Laboratory, USA
Jason Hill, Oak Ridge National Laboratory, USA
Josh Lothian, Oak Ridge National Laboratory, USA
Sarp Oral, Oak Ridge National Laboratory, USA
Joel Reed, Oak Ridge National Laboratory, USA
Rob Ross, Argonne National Laboratory, USA
Nathan Rutman, Seagate, USA
Stephen Simms, Indiana University, USA
Sudharshan Vazhkudai, Oak Ridge National Laboratory, USA
Weikuan Yu, Auburn University, USA
6 years, 11 months
