Problem with building the new kernel as RPM after patching with lustre
by Akhilesh Gadde
Hi,
I tried all the possible ways to resolve the build creation problem that I
am facing. I am following the steps provided in this wiki link by Intel -
Link <https://wiki.hpdd.intel.com/pages/viewpage.action?pageId=8126821>.
The main difference is that this document was created for centos 6.4
version. In the step - "Build the new kernel as an RPM.", when I am doing
'make rpm' to build the rpm package of the kernel, the build is failing.
Has anyone tried to install Lustre 2.7 on CentOS 6.6 by following the steps
given in Intel wiki link that I mentioned above?? If yes, please let me
know.
P.S: I tried with EPEL version 5 as mentioned in documentation and also
with the latest 6 version.
Also, I changed the kernel version to match my kernel while downloading the
kernel source. I am not sure what is going wrong here. :(
Regards,
Akhilesh.
=========Last few lines of 'make rpm'==============
CC [M] drivers/isdn/mISDN/dsp_pipeline.o
CC [M] net/wireless/wext-sme.o
CC [M] drivers/md/persistent-data/dm-space-map-common.o
CC [M] drivers/isdn/mISDN/dsp_hwec.o
CC [M] drivers/md/persistent-data/dm-space-map-disk.o
CC [M] net/wireless/genetlink-compat.o
LD [M] drivers/isdn/mISDN/mISDN_core.o
LD [M] drivers/isdn/mISDN/mISDN_dsp.o
LD drivers/md/md-mod.o
LD [M] drivers/isdn/mISDN/l1oip.o
LD drivers/isdn/built-in.o
CC [M] drivers/md/persistent-data/dm-space-map-metadata.o
CC [M] net/wireless/lib80211.o
LD [M] drivers/md/raid456.o
LD drivers/md/built-in.o
CC [M] drivers/md/persistent-data/dm-transaction-manager.o
CC [M] net/wireless/lib80211_crypt_wep.o
CC [M] drivers/md/persistent-data/dm-btree.o
CC [M] net/wireless/lib80211_crypt_ccmp.o
CC [M] net/wireless/lib80211_crypt_tkip.o
CC [M] drivers/md/persistent-data/dm-btree-remove.o
LD net/wireless/built-in.o
LD [M] net/wireless/cfg80211.o
CC [M] drivers/md/persistent-data/dm-btree-spine.o
LD net/built-in.o
LD [M] drivers/md/persistent-data/dm-persistent-data.o
make[2]: *** [drivers] Error 2
make[2]: *** Waiting for unfinished jobs....
error: Bad exit status from /var/tmp/rpm-tmp.IJLJy7 (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.IJLJy7 (%build)
make[1]: *** [rpm] Error 1
make: *** [rpm] Error 2
[build@centvm2 linux-2.6.32-504.1.3.el6.x86_64]$
==================================================
On Wed, Apr 15, 2015 at 1:23 AM, Akhilesh Gadde <
akhilesh.gadde(a)stonybrook.edu> wrote:
> Thanks to Patrick for providing me a way to get this work. It worked and
> the build was created.
>
> Now, I am struck with another problem when building the new kernel as RPM.
> :(
>
> # make rpm (problem when build is being created in this step)
>
> LD [M] drivers/isdn/hisax/hisax_st5481.o
>
> LD drivers/isdn/built-in.o
>
> make[2]: *** [drivers] Error 2
>
> error: Bad exit status from /var/tmp/rpm-tmp.lWkmKN (%build)
>
> RPM build errors:
>
> Bad exit status from /var/tmp/rpm-tmp.lWkmKN (%build)
>
> make[1]: *** [rpm] Error 1
>
> make: *** [rpm] Error 2
>
> I googled and someone mentioned to add "%define __debug_package %{nil}" to
> beginning of kernel.spec file.
>
> Any idea why this is happening. I am clueless. Tried 5 times atleast now.
> Same error everytime. :(
> On Apr 14, 2015 1:55 AM, "Akhilesh Gadde" <akhilesh.gadde(a)stonybrook.edu>
> wrote:
>
>> Hi,
>>
>> I am trying to build and install Lustre on CenOS 6.6. I am following the
>> steps detailed in Intel wiki -
>> https://wiki.hpdd.intel.com/pages/viewpage.action?pageId=8126821
>>
>> My kernel:
>> [centos@centvm1 ~]$ uname -a
>> Linux centvm1 *2.6.32-504.1.3*.el6.x86_64 #1 SMP Tue Nov 11 17:57:25 UTC
>> 2014 x86_64 x86_64 x86_64 GNU/Linux
>>
>> I used EPEL 5 as mentioned in step 3 and to install the kernel source, I
>> modified the build number (from the one given in wiki page) to my current
>> build.
>> When I'm trying to prepare the kernel source using rpmbuild, it is always
>> getting struck and not getting completed. Last few messages:
>> ###
>> ### rngd -r /dev/hwrandom
>> ###
>> ### If one isn't available, the pseudo-random number generator can be
>> used:
>> ###
>> ### rngd -r /dev/urandom
>> ###
>> + gpg --homedir . --batch --gen-key
>> /home/build/kernel/rpmbuild/SOURCES/genkey
>> gpg: WARNING: unsafe permissions on homedir `.'
>> gpg: keyring `./secring.gpg' created
>> gpg: keyring `./pubring.gpg' created
>> gpg: WARNING: some OpenPGP programs can't handle a DSA key with this
>> digest size
>> <<<<<<<<<<<<<<struck at this point>>>>>>>>>>>>>>>>
>>
>> Has anyone ever faced this issue while installing Lustre on CentOS 6.6??
>>
>> Regards,
>> Akhilesh Gadde.
>>
>
6 years, 2 months
Preparing source using rpmbuild struck without completion (CentOS 6.6)
by Akhilesh Gadde
Hi,
I am trying to build and install Lustre on CenOS 6.6. I am following the
steps detailed in Intel wiki -
https://wiki.hpdd.intel.com/pages/viewpage.action?pageId=8126821
My kernel:
[centos@centvm1 ~]$ uname -a
Linux centvm1 *2.6.32-504.1.3*.el6.x86_64 #1 SMP Tue Nov 11 17:57:25 UTC
2014 x86_64 x86_64 x86_64 GNU/Linux
I used EPEL 5 as mentioned in step 3 and to install the kernel source, I
modified the build number (from the one given in wiki page) to my current
build.
When I'm trying to prepare the kernel source using rpmbuild, it is always
getting struck and not getting completed. Last few messages:
###
### rngd -r /dev/hwrandom
###
### If one isn't available, the pseudo-random number generator can be used:
###
### rngd -r /dev/urandom
###
+ gpg --homedir . --batch --gen-key
/home/build/kernel/rpmbuild/SOURCES/genkey
gpg: WARNING: unsafe permissions on homedir `.'
gpg: keyring `./secring.gpg' created
gpg: keyring `./pubring.gpg' created
gpg: WARNING: some OpenPGP programs can't handle a DSA key with this digest
size
<<<<<<<<<<<<<<struck at this point>>>>>>>>>>>>>>>>
Has anyone ever faced this issue while installing Lustre on CentOS 6.6??
Regards,
Akhilesh Gadde.
6 years, 2 months
Installing Lustre on Ubuntu 14.04
by Akhilesh Gadde
Hi,
I am planning to set up a cluster running Lustre FS with all nodes running
on Ubuntu 14.04 in our lab and looking for references of any such
deployments and possibly pointers in that direction.
I am aware that officially Lustre supports only Cent OS or RHEL for servers
(MDS/OSS). But I wanted to try with Ubuntu as the OS on all clients, OSS
and MDS nodes.
Though there are enough references on the net showing that clients in the
cluster were running Ubuntu, I found it difficult to find much information
about servers running on Ubuntu.
The links I found like the Debian wiki for installing show that the kernel
need to be recomplied for installing lustre servers. (Their explanation was
with kernel version 2.6.32). The links presented there looks outdated and
were not functional. (Wiki was made for Lustre 2.2 version)
The current Ubuntu 14.04 LTS runs on 3.13.0-49-generic kernel version. I
looked up GitHub of the latest kernel version and it shows that the lustre
client's code has been incorporated into the kernel. I am unable to find
any information wrt to the servers.
So, should I downgrade my kernel to 2.6.X version, recomplie it with the
patch and use it for lustre server(s) installation?
Hope to hear back with some help regarding this.
Thanks,
Akhilesh Gadde.
6 years, 2 months
[PATCH v2 0/1] drop uses of some OBD alloc and free functions
by Julia Lawall
Replace OBD_ALLOC, OBD_ALLOC_WAIT, OBD_ALLOC_PTR, and OBD_ALLOC_PTR_WAIT by
kzalloc or calloc, as appropriate.
Replace OBD_FREE and OBD_FREE_PTR by kfree.
The complete semantic patch that makes these changes is as follows:
(http://coccinelle.lip6.fr/)
// <smpl>
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC(ptr,sizeof e1 * e2)
+ ptr = kcalloc(e2, sizeof e1, GFP_NOFS)
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC_WAIT(ptr,sizeof e1 * e2)
+ ptr = kcalloc(sizeof e1, e2, GFP_KERNEL)
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC(ptr,e2 * sizeof e1)
+ ptr = kcalloc(e2, sizeof e1, GFP_NOFS)
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC_WAIT(ptr,e2 * sizeof e1)
+ ptr = kcalloc(e2, sizeof e1, GFP_KERNEL)
@@
expression ptr,e2;
type t;
@@
- OBD_ALLOC(ptr,sizeof (t) * e2)
+ ptr = kcalloc(e2, sizeof (t), GFP_NOFS)
@@
expression ptr,e2;
type t;
@@
- OBD_ALLOC_WAIT(ptr,sizeof (t) * e2)
+ ptr = kcalloc(e2, sizeof (t), GFP_KERNEL)
@@
expression ptr,e2;
type t;
@@
- OBD_ALLOC(ptr,e2 * sizeof (t))
+ ptr = kcalloc(e2, sizeof (t), GFP_NOFS)
@@
expression ptr,e2;
type t;
@@
- OBD_ALLOC_WAIT(ptr,e2 * sizeof (t))
+ ptr = kcalloc(e2, sizeof (t), GFP_KERNEL)
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC(ptr,e1 * e2)
+ ptr = kcalloc(e1, e2, GFP_NOFS)
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC_WAIT(ptr,e1 * e2)
+ ptr = kcalloc(e1, e2, GFP_KERNEL)
// -----------------------------------------------------------------------
@@
expression ptr,size;
@@
- OBD_ALLOC(ptr,size)
+ ptr = kzalloc(size, GFP_NOFS)
@@
expression ptr,size;
@@
- OBD_ALLOC_WAIT(ptr,size)
+ ptr = kzalloc(size, GFP_KERNEL)
@@
expression ptr;
@@
- OBD_ALLOC_PTR(ptr)
+ ptr = kzalloc(sizeof(*ptr), GFP_NOFS)
@@
expression ptr;
@@
- OBD_ALLOC_PTR_WAIT(ptr)
+ ptr = kzalloc(sizeof(*ptr), GFP_KERNEL)
// ----------------------------------------------------------------------
@@
expression ptr, size;
@@
- OBD_FREE(ptr, size);
+ kfree(ptr);
@@
expression ptr;
@@
- OBD_FREE_PTR(ptr);
+ kfree(ptr);
// </smpl>
---
v2: put sequence number in [PATCH] subject
6 years, 2 months
[PATCH] drop uses of some OBD alloc and free functions
by Julia Lawall
Replace OBD_ALLOC, OBD_ALLOC_WAIT, OBD_ALLOC_PTR, and OBD_ALLOC_PTR_WAIT by
kzalloc or calloc, as appropriate.
Replace OBD_FREE and OBD_FREE_PTR by kfree.
The complete semantic patch that makes these changes is as follows:
(http://coccinelle.lip6.fr/)
// <smpl>
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC(ptr,sizeof e1 * e2)
+ ptr = kcalloc(e2, sizeof e1, GFP_NOFS)
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC_WAIT(ptr,sizeof e1 * e2)
+ ptr = kcalloc(sizeof e1, e2, GFP_KERNEL)
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC(ptr,e2 * sizeof e1)
+ ptr = kcalloc(e2, sizeof e1, GFP_NOFS)
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC_WAIT(ptr,e2 * sizeof e1)
+ ptr = kcalloc(e2, sizeof e1, GFP_KERNEL)
@@
expression ptr,e2;
type t;
@@
- OBD_ALLOC(ptr,sizeof (t) * e2)
+ ptr = kcalloc(e2, sizeof (t), GFP_NOFS)
@@
expression ptr,e2;
type t;
@@
- OBD_ALLOC_WAIT(ptr,sizeof (t) * e2)
+ ptr = kcalloc(e2, sizeof (t), GFP_KERNEL)
@@
expression ptr,e2;
type t;
@@
- OBD_ALLOC(ptr,e2 * sizeof (t))
+ ptr = kcalloc(e2, sizeof (t), GFP_NOFS)
@@
expression ptr,e2;
type t;
@@
- OBD_ALLOC_WAIT(ptr,e2 * sizeof (t))
+ ptr = kcalloc(e2, sizeof (t), GFP_KERNEL)
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC(ptr,e1 * e2)
+ ptr = kcalloc(e1, e2, GFP_NOFS)
@@
expression ptr,e1,e2;
@@
- OBD_ALLOC_WAIT(ptr,e1 * e2)
+ ptr = kcalloc(e1, e2, GFP_KERNEL)
// -----------------------------------------------------------------------
@@
expression ptr,size;
@@
- OBD_ALLOC(ptr,size)
+ ptr = kzalloc(size, GFP_NOFS)
@@
expression ptr,size;
@@
- OBD_ALLOC_WAIT(ptr,size)
+ ptr = kzalloc(size, GFP_KERNEL)
@@
expression ptr;
@@
- OBD_ALLOC_PTR(ptr)
+ ptr = kzalloc(sizeof(*ptr), GFP_NOFS)
@@
expression ptr;
@@
- OBD_ALLOC_PTR_WAIT(ptr)
+ ptr = kzalloc(sizeof(*ptr), GFP_KERNEL)
// ----------------------------------------------------------------------
@@
expression ptr, size;
@@
- OBD_FREE(ptr, size);
+ kfree(ptr);
@@
expression ptr;
@@
- OBD_FREE_PTR(ptr);
+ kfree(ptr);
// </smpl>
6 years, 2 months
[PATCH] staging: lustre: llite: drop uses of OBD free functions
by Julia Lawall
From: Julia Lawall <Julia.Lawall(a)lip6.fr>
Replace OBD_FREE and OBD_FREE_PTR by kfree.
The semantic patch that makes these changes is as follows:
(http://coccinelle.lip6.fr/)
// <smpl>
@@
expression ptr, size;
@@
- OBD_FREE(ptr, size);
+ kfree(ptr);
@@
expression ptr;
@@
- OBD_FREE_PTR(ptr);
+ kfree(ptr);
// </smpl>
Signed-off-by: Julia Lawall <Julia.Lawall(a)lip6.fr>
---
The corresponding ALLOC functions were already replaced by kalloc in this
code.
drivers/staging/lustre/lustre/llite/dcache.c | 6 +--
drivers/staging/lustre/lustre/llite/dir.c | 34 +++++++++----------
drivers/staging/lustre/lustre/llite/file.c | 36 ++++++++++-----------
drivers/staging/lustre/lustre/llite/llite_close.c | 6 +--
drivers/staging/lustre/lustre/llite/llite_lib.c | 31 ++++++++----------
drivers/staging/lustre/lustre/llite/llite_nfs.c | 2 -
drivers/staging/lustre/lustre/llite/llite_rmtacl.c | 4 +-
drivers/staging/lustre/lustre/llite/lloop.c | 8 ++--
drivers/staging/lustre/lustre/llite/namei.c | 2 -
drivers/staging/lustre/lustre/llite/statahead.c | 20 +++++------
drivers/staging/lustre/lustre/llite/xattr_cache.c | 6 +--
11 files changed, 77 insertions(+), 78 deletions(-)
diff --git a/drivers/staging/lustre/lustre/llite/dcache.c b/drivers/staging/lustre/lustre/llite/dcache.c
index fe1fd05..87b6a80 100644
--- a/drivers/staging/lustre/lustre/llite/dcache.c
+++ b/drivers/staging/lustre/lustre/llite/dcache.c
@@ -52,7 +52,7 @@ static void free_dentry_data(struct rcu_head *head)
struct ll_dentry_data *lld;
lld = container_of(head, struct ll_dentry_data, lld_rcu_head);
- OBD_FREE_PTR(lld);
+ kfree(lld);
}
/* should NOT be called with the dcache lock, see fs/dcache.c */
@@ -67,7 +67,7 @@ static void ll_release(struct dentry *de)
if (lld->lld_it) {
ll_intent_release(lld->lld_it);
- OBD_FREE(lld->lld_it, sizeof(*lld->lld_it));
+ kfree(lld->lld_it);
}
de->d_fsdata = NULL;
@@ -194,7 +194,7 @@ int ll_d_init(struct dentry *de)
de->d_fsdata = lld;
__d_lustre_invalidate(de);
} else {
- OBD_FREE_PTR(lld);
+ kfree(lld);
}
spin_unlock(&de->d_lock);
} else {
diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
index a5bc694..0f5d57c 100644
--- a/drivers/staging/lustre/lustre/llite/dir.c
+++ b/drivers/staging/lustre/lustre/llite/dir.c
@@ -239,7 +239,7 @@ static int ll_dir_filler(void *_hash, struct page *page0)
ll_pagevec_lru_add_file(&lru_pvec);
if (page_pool != &page0)
- OBD_FREE(page_pool, sizeof(struct page *) * max_pages);
+ kfree(page_pool);
return rc;
}
@@ -650,7 +650,7 @@ static int ll_send_mgc_param(struct obd_export *mgc, char *string)
sizeof(struct mgs_send_param), msp, NULL);
if (rc)
CERROR("Failed to set parameter: %d\n", rc);
- OBD_FREE_PTR(msp);
+ kfree(msp);
return rc;
}
@@ -787,7 +787,7 @@ int ll_dir_setstripe(struct inode *inode, struct lov_user_md *lump,
end:
if (param != NULL)
- OBD_FREE(param, MGS_PARAM_MAXLEN);
+ kfree(param);
}
return rc;
}
@@ -1072,7 +1072,7 @@ static int copy_and_ioctl(int cmd, struct obd_export *exp,
rc = obd_iocontrol(cmd, exp, size, copy, NULL);
out:
- OBD_FREE(copy, size);
+ kfree(copy);
return rc;
}
@@ -1163,7 +1163,7 @@ static int quotactl_ioctl(struct ll_sb_info *sbi, struct if_quotactl *qctl)
oqctl->qc_cmd = Q_QUOTAOFF;
obd_quotactl(sbi->ll_md_exp, oqctl);
}
- OBD_FREE_PTR(oqctl);
+ kfree(oqctl);
return rc;
}
/* If QIF_SPACE is not set, client should collect the
@@ -1206,11 +1206,11 @@ static int quotactl_ioctl(struct ll_sb_info *sbi, struct if_quotactl *qctl)
oqctl->qc_dqblk.dqb_valid &= ~QIF_SPACE;
}
- OBD_FREE_PTR(oqctl_tmp);
+ kfree(oqctl_tmp);
}
out:
QCTL_COPY(qctl, oqctl);
- OBD_FREE_PTR(oqctl);
+ kfree(oqctl);
}
return rc;
@@ -1437,7 +1437,7 @@ lmv_out_free:
}
free_lmv:
if (tmp)
- OBD_FREE(tmp, lum_size);
+ kfree(tmp);
return rc;
}
case LL_IOC_REMOVE_ENTRY: {
@@ -1657,7 +1657,7 @@ free_lmm:
if (rc < 0)
CDEBUG(D_INFO, "obd_quotacheck failed: rc %d\n", rc);
- OBD_FREE_PTR(oqctl);
+ kfree(oqctl);
return error ?: rc;
}
case OBD_IOC_POLL_QUOTACHECK: {
@@ -1691,7 +1691,7 @@ free_lmm:
goto out_poll;
}
out_poll:
- OBD_FREE_PTR(check);
+ kfree(check);
return rc;
}
case LL_IOC_QUOTACTL: {
@@ -1712,7 +1712,7 @@ out_poll:
rc = -EFAULT;
out_quotactl:
- OBD_FREE_PTR(qctl);
+ kfree(qctl);
return rc;
}
case OBD_IOC_GETDTNAME:
@@ -1781,13 +1781,13 @@ out_quotactl:
/* We don't know the true size yet; copy the fixed-size part */
if (copy_from_user(hur, (void *)arg, sizeof(*hur))) {
- OBD_FREE_PTR(hur);
+ kfree(hur);
return -EFAULT;
}
/* Compute the whole struct size */
totalsize = hur_len(hur);
- OBD_FREE_PTR(hur);
+ kfree(hur);
if (totalsize < 0)
return -E2BIG;
@@ -1865,7 +1865,7 @@ out_quotactl:
if (!copy)
return -ENOMEM;
if (copy_from_user(copy, (char *)arg, sizeof(*copy))) {
- OBD_FREE_PTR(copy);
+ kfree(copy);
return -EFAULT;
}
@@ -1873,7 +1873,7 @@ out_quotactl:
if (copy_to_user((char *)arg, copy, sizeof(*copy)))
rc = -EFAULT;
- OBD_FREE_PTR(copy);
+ kfree(copy);
return rc;
}
case LL_IOC_HSM_COPY_END: {
@@ -1884,7 +1884,7 @@ out_quotactl:
if (!copy)
return -ENOMEM;
if (copy_from_user(copy, (char *)arg, sizeof(*copy))) {
- OBD_FREE_PTR(copy);
+ kfree(copy);
return -EFAULT;
}
@@ -1892,7 +1892,7 @@ out_quotactl:
if (copy_to_user((char *)arg, copy, sizeof(*copy)))
rc = -EFAULT;
- OBD_FREE_PTR(copy);
+ kfree(copy);
return rc;
}
default:
diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index 85e74d1..b53e4c7 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -213,7 +213,7 @@ out:
md_clear_open_replay_data(md_exp, och);
/* Free @och if it is not waiting for DONE_WRITING. */
och->och_fh.cookie = DEAD_HANDLE_MAGIC;
- OBD_FREE_PTR(och);
+ kfree(och);
}
if (req) /* This is close request */
ptlrpc_req_finished(req);
@@ -693,7 +693,7 @@ restart:
out_och_free:
if (rc) {
if (och_p && *och_p) {
- OBD_FREE(*och_p, sizeof(struct obd_client_handle));
+ kfree(*och_p);
*och_p = NULL; /* OBD_FREE writes some magic there */
(*och_usecount)--;
}
@@ -875,7 +875,7 @@ out_close:
out_release_it:
ll_intent_release(&it);
out:
- OBD_FREE_PTR(och);
+ kfree(och);
return ERR_PTR(rc);
}
@@ -1779,7 +1779,7 @@ int ll_fid2path(struct inode *inode, void __user *arg)
rc = -EFAULT;
gf_free:
- OBD_FREE(gfout, outsize);
+ kfree(gfout);
return rc;
}
@@ -1883,7 +1883,7 @@ int ll_data_version(struct inode *inode, __u64 *data_version,
*data_version = obdo->o_data_version;
}
- OBD_FREE_PTR(obdo);
+ kfree(obdo);
out:
ccc_inode_lsm_put(inode, lsm);
return rc;
@@ -2110,7 +2110,7 @@ putgl:
free:
if (llss != NULL)
- OBD_FREE_PTR(llss);
+ kfree(llss);
return rc;
}
@@ -2195,10 +2195,10 @@ static int ll_hsm_import(struct inode *inode, struct file *file,
out:
if (hss != NULL)
- OBD_FREE_PTR(hss);
+ kfree(hss);
if (attr != NULL)
- OBD_FREE_PTR(attr);
+ kfree(attr);
return rc;
}
@@ -2350,7 +2350,7 @@ ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
op_data = ll_prep_md_op_data(NULL, inode, NULL, NULL, 0, 0,
LUSTRE_OPC_ANY, hus);
if (IS_ERR(op_data)) {
- OBD_FREE_PTR(hus);
+ kfree(hus);
return PTR_ERR(op_data);
}
@@ -2361,7 +2361,7 @@ ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
rc = -EFAULT;
ll_finish_md_op_data(op_data);
- OBD_FREE_PTR(hus);
+ kfree(hus);
return rc;
}
case LL_IOC_HSM_STATE_SET: {
@@ -2373,13 +2373,13 @@ ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
return -ENOMEM;
if (copy_from_user(hss, (char *)arg, sizeof(*hss))) {
- OBD_FREE_PTR(hss);
+ kfree(hss);
return -EFAULT;
}
rc = ll_hsm_state_set(inode, hss);
- OBD_FREE_PTR(hss);
+ kfree(hss);
return rc;
}
case LL_IOC_HSM_ACTION: {
@@ -2394,7 +2394,7 @@ ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
op_data = ll_prep_md_op_data(NULL, inode, NULL, NULL, 0, 0,
LUSTRE_OPC_ANY, hca);
if (IS_ERR(op_data)) {
- OBD_FREE_PTR(hca);
+ kfree(hca);
return PTR_ERR(op_data);
}
@@ -2405,7 +2405,7 @@ ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
rc = -EFAULT;
ll_finish_md_op_data(op_data);
- OBD_FREE_PTR(hca);
+ kfree(hca);
return rc;
}
case LL_IOC_SET_LEASE: {
@@ -2500,13 +2500,13 @@ ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
return -ENOMEM;
if (copy_from_user(hui, (void *)arg, sizeof(*hui))) {
- OBD_FREE_PTR(hui);
+ kfree(hui);
return -EFAULT;
}
rc = ll_hsm_import(inode, file, hui);
- OBD_FREE_PTR(hui);
+ kfree(hui);
return rc;
}
default: {
@@ -3257,7 +3257,7 @@ void ll_iocontrol_unregister(void *magic)
list_del(&tmp->iocd_list);
up_write(&llioc.ioc_sem);
- OBD_FREE(tmp, size);
+ kfree(tmp);
return;
}
}
@@ -3625,6 +3625,6 @@ int ll_layout_restore(struct inode *inode)
hur->hur_request.hr_itemcount = 1;
rc = obd_iocontrol(LL_IOC_HSM_REQUEST, cl_i2sbi(inode)->ll_md_exp,
len, hur, NULL);
- OBD_FREE(hur, len);
+ kfree(hur);
return rc;
}
diff --git a/drivers/staging/lustre/lustre/llite/llite_close.c b/drivers/staging/lustre/lustre/llite/llite_close.c
index a94ba02..7bdae72 100644
--- a/drivers/staging/lustre/lustre/llite/llite_close.c
+++ b/drivers/staging/lustre/lustre/llite/llite_close.c
@@ -305,7 +305,7 @@ out:
ll_finish_md_op_data(op_data);
if (och) {
md_clear_open_replay_data(ll_i2sbi(inode)->ll_md_exp, och);
- OBD_FREE_PTR(och);
+ kfree(och);
}
}
@@ -374,7 +374,7 @@ int ll_close_thread_start(struct ll_close_queue **lcq_ret)
task = kthread_run(ll_close_thread, lcq, "ll_close");
if (IS_ERR(task)) {
- OBD_FREE(lcq, sizeof(*lcq));
+ kfree(lcq);
return PTR_ERR(task);
}
@@ -389,5 +389,5 @@ void ll_close_thread_shutdown(struct ll_close_queue *lcq)
atomic_inc(&lcq->lcq_stop);
wake_up(&lcq->lcq_waitq);
wait_for_completion(&lcq->lcq_comp);
- OBD_FREE(lcq, sizeof(*lcq));
+ kfree(lcq);
}
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index bf1ec27..c2bed83 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -145,7 +145,7 @@ static void ll_free_sbi(struct super_block *sb)
spin_lock(&ll_sb_lock);
list_del(&sbi->ll_list);
spin_unlock(&ll_sb_lock);
- OBD_FREE(sbi, sizeof(*sbi));
+ kfree(sbi);
}
}
@@ -177,7 +177,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt,
osfs = kzalloc(sizeof(*osfs), GFP_NOFS);
if (!osfs) {
- OBD_FREE_PTR(data);
+ kfree(data);
return -ENOMEM;
}
@@ -296,7 +296,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt,
valid ^ CLIENT_CONNECT_MDT_REQD, ",");
LCONSOLE_ERROR_MSG(0x170, "Server %s does not support feature(s) needed for correct operation of this client (%s). Please upgrade server or downgrade client.\n",
sbi->ll_md_exp->exp_obd->obd_name, buf);
- OBD_FREE(buf, PAGE_CACHE_SIZE);
+ kfree(buf);
err = -EPROTO;
goto out_md_fid;
}
@@ -501,7 +501,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt,
err = md_getattr(sbi->ll_md_exp, op_data, &request);
if (oc)
capa_put(oc);
- OBD_FREE_PTR(op_data);
+ kfree(op_data);
if (err) {
CERROR("%s: md_getattr failed for root: rc = %d\n",
sbi->ll_md_exp->exp_obd->obd_name, err);
@@ -583,9 +583,9 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt,
}
if (data != NULL)
- OBD_FREE_PTR(data);
+ kfree(data);
if (osfs != NULL)
- OBD_FREE_PTR(osfs);
+ kfree(osfs);
return err;
out_root:
@@ -604,9 +604,9 @@ out_md:
sbi->ll_md_exp = NULL;
out:
if (data != NULL)
- OBD_FREE_PTR(data);
+ kfree(data);
if (osfs != NULL)
- OBD_FREE_PTR(osfs);
+ kfree(osfs);
lprocfs_unregister_mountpoint(sbi);
return err;
}
@@ -932,7 +932,7 @@ int ll_fill_super(struct super_block *sb, struct vfsmount *mnt)
lsi->lsi_llsbi = sbi = ll_init_sbi();
if (!sbi) {
module_put(THIS_MODULE);
- OBD_FREE_PTR(cfg);
+ kfree(cfg);
return -ENOMEM;
}
@@ -994,15 +994,15 @@ int ll_fill_super(struct super_block *sb, struct vfsmount *mnt)
out_free:
if (md)
- OBD_FREE(md, strlen(lprof->lp_md) + instlen + 2);
+ kfree(md);
if (dt)
- OBD_FREE(dt, strlen(lprof->lp_dt) + instlen + 2);
+ kfree(dt);
if (err)
ll_put_super(sb);
else if (sbi->ll_flags & LL_SBI_VERBOSE)
LCONSOLE_WARN("Mounted %s\n", profilenm);
- OBD_FREE_PTR(cfg);
+ kfree(cfg);
return err;
} /* ll_fill_super */
@@ -1126,8 +1126,7 @@ void ll_clear_inode(struct inode *inode)
ll_md_real_close(inode, FMODE_READ);
if (S_ISLNK(inode->i_mode) && lli->lli_symlink_name) {
- OBD_FREE(lli->lli_symlink_name,
- strlen(lli->lli_symlink_name) + 1);
+ kfree(lli->lli_symlink_name);
lli->lli_symlink_name = NULL;
}
@@ -1957,7 +1956,7 @@ void ll_umount_begin(struct super_block *sb)
obd_iocontrol(IOC_OSC_SET_ACTIVE, sbi->ll_dt_exp,
sizeof(*ioc_data), ioc_data, NULL);
- OBD_FREE_PTR(ioc_data);
+ kfree(ioc_data);
}
/* Really, we'd like to wait until there are no requests outstanding,
@@ -2236,7 +2235,7 @@ void ll_finish_md_op_data(struct md_op_data *op_data)
{
capa_put(op_data->op_capa1);
capa_put(op_data->op_capa2);
- OBD_FREE_PTR(op_data);
+ kfree(op_data);
}
int ll_show_options(struct seq_file *seq, struct dentry *dentry)
diff --git a/drivers/staging/lustre/lustre/llite/llite_nfs.c b/drivers/staging/lustre/lustre/llite/llite_nfs.c
index 243a784..a4c59ef 100644
--- a/drivers/staging/lustre/lustre/llite/llite_nfs.c
+++ b/drivers/staging/lustre/lustre/llite/llite_nfs.c
@@ -116,7 +116,7 @@ struct inode *search_inode_for_lustre(struct super_block *sb,
/* mds_fid2dentry ignores f_type */
rc = md_getattr(sbi->ll_md_exp, op_data, &req);
- OBD_FREE_PTR(op_data);
+ kfree(op_data);
if (rc) {
CERROR("can't get object attrs, fid "DFID", rc %d\n",
PFID(fid), rc);
diff --git a/drivers/staging/lustre/lustre/llite/llite_rmtacl.c b/drivers/staging/lustre/lustre/llite/llite_rmtacl.c
index f4da156..c8a450b 100644
--- a/drivers/staging/lustre/lustre/llite/llite_rmtacl.c
+++ b/drivers/staging/lustre/lustre/llite/llite_rmtacl.c
@@ -94,7 +94,7 @@ static void rce_free(struct rmtacl_ctl_entry *rce)
if (!list_empty(&rce->rce_list))
list_del(&rce->rce_list);
- OBD_FREE_PTR(rce);
+ kfree(rce);
}
static struct rmtacl_ctl_entry *__rct_search(struct rmtacl_ctl_table *rct,
@@ -205,7 +205,7 @@ void ee_free(struct eacl_entry *ee)
if (ee->ee_acl)
lustre_ext_acl_xattr_free(ee->ee_acl);
- OBD_FREE_PTR(ee);
+ kfree(ee);
}
static struct eacl_entry *__et_search_del(struct eacl_table *et, pid_t key,
diff --git a/drivers/staging/lustre/lustre/llite/lloop.c b/drivers/staging/lustre/lustre/llite/lloop.c
index 413a840..cc00fd1 100644
--- a/drivers/staging/lustre/lustre/llite/lloop.c
+++ b/drivers/staging/lustre/lustre/llite/lloop.c
@@ -840,9 +840,9 @@ out_mem4:
out_mem3:
while (i--)
put_disk(disks[i]);
- OBD_FREE(disks, max_loop * sizeof(*disks));
+ kfree(disks);
out_mem2:
- OBD_FREE(loop_dev, max_loop * sizeof(*loop_dev));
+ kfree(loop_dev);
out_mem1:
unregister_blkdev(lloop_major, "lloop");
ll_iocontrol_unregister(ll_iocontrol_magic);
@@ -863,8 +863,8 @@ static void lloop_exit(void)
unregister_blkdev(lloop_major, "lloop");
- OBD_FREE(disks, max_loop * sizeof(*disks));
- OBD_FREE(loop_dev, max_loop * sizeof(*loop_dev));
+ kfree(disks);
+ kfree(loop_dev);
}
module_init(lloop_init);
diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c
index 49f1cb0..3724f8a 100644
--- a/drivers/staging/lustre/lustre/llite/namei.c
+++ b/drivers/staging/lustre/lustre/llite/namei.c
@@ -665,7 +665,7 @@ static int ll_atomic_open(struct inode *dir, struct dentry *dentry,
out_release:
ll_intent_release(it);
- OBD_FREE(it, sizeof(*it));
+ kfree(it);
return rc;
}
diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c
index b75562c..2c10ab7 100644
--- a/drivers/staging/lustre/lustre/llite/statahead.c
+++ b/drivers/staging/lustre/lustre/llite/statahead.c
@@ -312,7 +312,7 @@ static void ll_sa_entry_cleanup(struct ll_statahead_info *sai,
entry->se_minfo = NULL;
ll_intent_release(&minfo->mi_it);
iput(minfo->mi_dir);
- OBD_FREE_PTR(minfo);
+ kfree(minfo);
}
if (req) {
@@ -336,7 +336,7 @@ static void ll_sa_entry_put(struct ll_statahead_info *sai,
ll_sa_entry_cleanup(sai, entry);
iput(entry->se_inode);
- OBD_FREE(entry, entry->se_size);
+ kfree(entry);
atomic_dec(&sai->sai_cache_count);
}
}
@@ -544,7 +544,7 @@ static void ll_sai_put(struct ll_statahead_info *sai)
LASSERT(agl_list_empty(sai));
iput(inode);
- OBD_FREE_PTR(sai);
+ kfree(sai);
}
}
@@ -772,7 +772,7 @@ out:
if (rc != 0) {
ll_intent_release(it);
iput(dir);
- OBD_FREE_PTR(minfo);
+ kfree(minfo);
}
if (sai != NULL)
ll_sai_put(sai);
@@ -786,8 +786,8 @@ static void sa_args_fini(struct md_enqueue_info *minfo,
iput(minfo->mi_dir);
capa_put(minfo->mi_data.op_capa1);
capa_put(minfo->mi_data.op_capa2);
- OBD_FREE_PTR(minfo);
- OBD_FREE_PTR(einfo);
+ kfree(minfo);
+ kfree(einfo);
}
/**
@@ -816,15 +816,15 @@ static int sa_args_init(struct inode *dir, struct inode *child,
minfo = kzalloc(sizeof(*minfo), GFP_NOFS);
if (!minfo) {
- OBD_FREE_PTR(einfo);
+ kfree(einfo);
return -ENOMEM;
}
op_data = ll_prep_md_op_data(&minfo->mi_data, dir, child, qstr->name,
qstr->len, 0, LUSTRE_OPC_ANY, NULL);
if (IS_ERR(op_data)) {
- OBD_FREE_PTR(einfo);
- OBD_FREE_PTR(minfo);
+ kfree(einfo);
+ kfree(minfo);
return PTR_ERR(op_data);
}
@@ -1720,7 +1720,7 @@ int do_statahead_enter(struct inode *dir, struct dentry **dentryp,
out:
if (sai != NULL)
- OBD_FREE_PTR(sai);
+ kfree(sai);
spin_lock(&lli->lli_sa_lock);
lli->lli_opendir_key = NULL;
lli->lli_opendir_pid = 0;
diff --git a/drivers/staging/lustre/lustre/llite/xattr_cache.c b/drivers/staging/lustre/lustre/llite/xattr_cache.c
index 69ea92a..6956dec 100644
--- a/drivers/staging/lustre/lustre/llite/xattr_cache.c
+++ b/drivers/staging/lustre/lustre/llite/xattr_cache.c
@@ -144,7 +144,7 @@ static int ll_xattr_cache_add(struct list_head *cache,
return 0;
err_value:
- OBD_FREE(xattr->xe_name, xattr->xe_namelen);
+ kfree(xattr->xe_name);
err_name:
OBD_SLAB_FREE_PTR(xattr, xattr_kmem);
@@ -170,8 +170,8 @@ static int ll_xattr_cache_del(struct list_head *cache,
if (ll_xattr_cache_find(cache, xattr_name, &xattr) == 0) {
list_del(&xattr->xe_list);
- OBD_FREE(xattr->xe_name, xattr->xe_namelen);
- OBD_FREE(xattr->xe_value, xattr->xe_vallen);
+ kfree(xattr->xe_name);
+ kfree(xattr->xe_value);
OBD_SLAB_FREE_PTR(xattr, xattr_kmem);
return 0;
6 years, 2 months
MDS crash
by Marc Boisis
Hi,
My lustre servers are running Lustre 2.5.2 and when I’m trying to walk trough the entire filesystem (24To) with a simple find, the mds crash with this kernel error.
Have you an idea ?
<4>R13: ffff882018f2db58 R14: ffff8840214c7800 R15: 0000000000007000
<4>FS: 0000000000000000(0000) GS:ffff88011cc40000(0000) knlGS:0000000000000000
<4>CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
<4>CR2: 0000003200e73e90 CR3: 000000404fa2d000 CR4: 00000000001407e0
<4>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<4>Process mdt_rdpg00_002 (pid: 12981, threadinfo ffff882033eba000, task ffff882036d79500)
<4>Stack:
<4> ffff88203267c438 ffffffffa0d55830 ffff882018f2db58 0000000000000000
<4><d> ffff882033ebb610 ffffffffa0d130eb ffff882033ebb600 ffffffff8109aeef
<4><d> ffff88401541ec40 ffff88203267c438 0000000000000400 ffff882018f2db58
<4>Call Trace:
<4> [<ffffffffa0d130eb>] __ldiskfs_handle_dirty_metadata+0x7b/0x100 [ldiskfs]
<4> [<ffffffff8109aeef>] ? wake_up_bit+0x2f/0x40
<4> [<ffffffffa0d48995>] ldiskfs_quota_write+0x165/0x210 [ldiskfs]
<4> [<ffffffff811ee86e>] write_blk+0x2e/0x30
<4> [<ffffffff811eee1a>] remove_free_dqentry+0x8a/0x140
<4> [<ffffffff811ef7c7>] do_insert_tree+0x317/0x3d0
<4> [<ffffffff811ef735>] do_insert_tree+0x285/0x3d0
<4> [<ffffffff811ef735>] do_insert_tree+0x285/0x3d0
<4> [<ffffffff811ef735>] do_insert_tree+0x285/0x3d0
<4> [<ffffffff811ef978>] qtree_write_dquot+0xf8/0x150
<4> [<ffffffff811eebee>] ? qtree_read_dquot+0x5e/0x200
<4> [<ffffffff811ee0c0>] v2_write_dquot+0x30/0x40
<4> [<ffffffff811ea270>] dquot_acquire+0xc0/0x140
<4> [<ffffffffa0d47b26>] ldiskfs_acquire_dquot+0x66/0xb0 [ldiskfs]
<4> [<ffffffff811ec25c>] dqget+0x2ac/0x390
<4> [<ffffffff811ec808>] dquot_initialize+0x98/0x240
<4> [<ffffffffa0d47d83>] ldiskfs_dquot_initialize+0x83/0xd0 [ldiskfs]
<4> [<ffffffffa0e12b7f>] osd_attr_set+0x12f/0x540 [osd_ldiskfs]
<4> [<ffffffffa099196b>] lod_attr_set+0x12b/0x450 [lod]
<4> [<ffffffffa0b5c9b1>] mdd_attr_set_internal+0x151/0x230 [mdd]
<4> [<ffffffffa0b5f30a>] mdd_attr_set+0x117a/0x1470 [mdd]
<4> [<ffffffffa0ad644c>] mdt_mfd_close+0x7ac/0x1bc0 [mdt]
<4> [<ffffffffa082ed65>] ? lustre_msg_buf+0x55/0x60 [ptlrpc]
<4> [<ffffffffa0855d26>] ? __req_capsule_get+0x166/0x710 [ptlrpc]
<4> [<ffffffffa0682105>] ? class_handle2object+0x95/0x190 [obdclass]
<4> [<ffffffffa0ad8bf2>] mdt_close+0x642/0xa80 [mdt]
<4> [<ffffffffa0aae58a>] mdt_handle_common+0x52a/0x1470 [mdt]
<4> [<ffffffffa0aea735>] mds_readpage_handle+0x15/0x20 [mdt]
<4> [<ffffffffa083fbc5>] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc]
<4> [<ffffffffa05364ce>] ? cfs_timer_arm+0xe/0x10 [libcfs]
<4> [<ffffffffa05473cf>] ? lc_watchdog_touch+0x6f/0x170 [libcfs]
<4> [<ffffffffa08372a9>] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc]
<4> [<ffffffff810546b9>] ? __wake_up_common+0x59/0x90
<4> [<ffffffffa0840f2d>] ptlrpc_main+0xaed/0x1740 [ptlrpc]
<4> [<ffffffffa0840440>] ? ptlrpc_main+0x0/0x1740 [ptlrpc]
<4> [<ffffffff8109ab56>] kthread+0x96/0xa0
<4> [<ffffffff8100c20a>] child_rip+0xa/0x20
<4> [<ffffffff8109aac0>] ? kthread+0x0/0xa0
<4> [<ffffffff8100c200>] ? child_rip+0x0/0x20
<4>Code: c6 9c 03 00 00 4c 89 f7 e8 b1 f1 3b e1 48 8b 33 ba 01 00 00 00 4c 89 e7 e8 11 ec ff ff 4c 89 f0 66 ff 00 66 66 90 e9 73 ff ff ff <0f> 0b eb fe 0f 0b eb fe 0f 0b 66 0f 1f 84 00 00 00 00 00 eb f5
<1>RIP [<ffffffffa016b8ad>] jbd2_journal_dirty_metadata+0x10d/0x150 [jbd2]
<4> RSP <ffff882033ebb5b0>
quotas are disabled:
[root@mds1 ~]# lctl get_param osd-*.*.quota_slave.info
osd-ldiskfs.led-MDT0000.quota_slave.info=
target name: led-MDT0000
pool ID: 0
type: md
quota enabled: none
conn to master: setup
space acct: ug
user uptodate: glb[0],slv[0],reint[0]
group uptodate: glb[0],slv[0],reint[0]
6 years, 2 months
Lustre and kernel buffer interaction
by John Bauer
I have been trying to understand a behavior I am observing in an IOR
benchmark on Lustre. I have pared it down to a simple example.
The IOR benchmark is running in MPI mode. There are 2 ranks, each
running on its own node. Each rank does the following:
Note : Test was run on the "swan" cluster at Cray Inc., using /lus/scratch
write a file. ( 10GB )
fsync the file
close the file
MPI_barrier
open the file that was written by the other rank.
read the file that was written by the other rank.
close the file that was written by the other rank.
The writing of each file goes as expected.
The fsync takes very little time ( about .05 seconds).
The first reads of the file( written by the other rank ) start out *very
*slowly. While theses first reads are proceeding slowly, the
kernel's cached memory ( the Cached: line in /proc/meminfo) decreases
from the size of the file just written to nearly zero.
Once the cached memory has reached nearly zero, the file reading
proceeds as expected.
I have attached a jpg of the instrumentation of the processes that
illustrates this behavior.
My questions are:
Why does the reading of the file, written by the other rank, wait until
the cached data drains to nearly zero before proceeding normally.
Shouldn't the fsync ensure that the file's data is written to the
backing storage so this draining of the cached memory should be simply
releasing pages with no further I/O?
For this case the "dead" time is only about 4 seconds, but this "dead"
time scales directly with the size of the files.
John
--
John Bauer
I/O Doctors LLC
507-766-0378
bauerj(a)iodoctors.com
6 years, 2 months
How to read the I/O in flight statistic in obdfilter.*.brw_stats
by Michael Kluge
Hi all,
can anyone please explain how this individual statistic is
calculated/updated? From what I understand it shows how many read or
write requests have been in flight at certain points in time in the
past. What I don't know is: when is this statistic updated. If I had to
guess I would say it must be any of these:
1) whenever a new I/O request is issued, it puts the number of open
requests in the statistic field
2) there is a service thread that reads the I/O queue size at a fixed
interval
Any help is appreciated!
Regards, Michael
--
Dr.-Ing. Michael Kluge
Technische Universität Dresden
Center for Information Services and
High Performance Computing (ZIH)
D-01062 Dresden
Germany
Contact:
Willersbau, Room A 208
Phone: (+49) 351 463-34217
Fax: (+49) 351 463-37773
e-mail: michael.kluge(a)tu-dresden.de
WWW: http://www.tu-dresden.de/zih
6 years, 2 months
