Hi all,

I read on some email threads about the issue, and there seems to be 2 ways:

1) e2fsck

     e2fsck -f -y $DEVICE

2) tune2fs

    tune2fs -O ^quota $DEVICE
    tune2fs -O quota $DEVICE

Those commands should be run on the MDT and every OST having e2fsck errors.

Which one do you advice me to apply?

Best regards

 

On 2016-09-30 12:01, Ramiro Alba wrote:

Hi all,

I am using Lustre 2.5.3 under CentOS 6.5 both servers and clients and
the lustre file system is using 6 OSTs of 30TB size with 2 OSSs.

Last days I've been having problems with OSS server rebooting without a
known reason after running fine for several hours (betwen 3 to 24), and
I also noticed that quotas were not showing the correct usage.

At the last OSS server reboot issue, I deciced to run e2fsck both on MDT and
OSTs, and those bellow were the errors at command output:

------------------
MDT:
------------------
e2fsck -f /dev/mapper/vglustre-MDT

e2fsck 1.42.9.wc1 (24-Feb-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
[ERROR] quotaio_tree.c:590:check_reference:: Illegal reference (1045 >= 10) in user quota file. Quota file is probably corrupted.
Please run e2fsck (8) to fix it.
[ERROR] quotaio_tree.c:590:check_reference:: Illegal reference (16322 >= 10) in user quota file. Quota file is probably corrupted.
Please run e2fsck (8) to fix it.
[ERROR] quotaio_tree.c:590:check_reference:: Illegal reference (2940928 >= 10) in user quota file. Quota file is probably corrupted.
Please run e2fsck (8) to fix it.
jffstg-MDT0000: 1218855/47185920 files (0.2% non-contiguous), 6191259/23592960 blocks

------------------
OSSs:
------------------
[root@jffoss1 system]# e2fsck -fp /dev/mapper/ost0; e2fsck -fp /dev/mapper/ost1; e2fsck -fp /dev/mapper/ost2

MMP interval is 10 seconds and total wait time is 42 seconds. Please wait...
jffstg-OST0000: recovering journal
[QUOTA WARNING] Usage inconsistent for ID 0:actual (1766737936384, 3488) != expected (0, 0)
[QUOTA WARNING] Usage inconsistent for ID 1042:actual (507613184, 3613) != expected (40960, 1)
[QUOTA WARNING] Usage inconsistent for ID 1031:actual (1041013137408, 5278) != expected (42952433664, 119)
[QUOTA WARNING] Usage inconsistent for ID 1304:actual (562630971392, 5127) != expected (0, 0)
[QUOTA WARNING] Usage inconsistent for ID 1338:actual (1634377928704, 7847) != expected (2098733056, 12)
[QUOTA WARNING] Usage inconsistent for ID 1037:actual (948502528, 1224) != expected (2150400, 0)
[QUOTA WARNING] Usage inconsistent for ID 1367:actual (1078501474304, 18130) != expected (20640858112, 55)
[QUOTA WARNING] Usage inconsistent for ID 1030:actual (84762128384, 3973) != expected (26271744, 2)
[QUOTA WARNING] Usage inconsistent for ID 1041:actual (258749169664, 2678) != expected (109670400, 11)
[QUOTA WARNING] Usage inconsistent for ID 1056:actual (389121245184, 1333) != expected (9640079360, 21)
[QUOTA WARNING] Usage inconsistent for ID 1363:actual (1562884812800, 28993) != expected (41607168, 44)
[QUOTA WARNING] Usage inconsistent for ID 1015:actual (263251644416, 1778) != expected (1230139392, 8)
[QUOTA WARNING] Usage inconsistent for ID 1027:actual (121355857920, 4042) != expected (918818816, 5)
[QUOTA WARNING] Usage inconsistent for ID 1000:actual (44268187648, 165) != expected (0, 0)
jffstg-OST0000: Update quota info for quota type 0.
[ERROR] quotaio_tree.c:241:find_free_dqentry:: find_free_dqentry(): Data block full unexpectedly.
[ERROR] quotaio_tree.c:241:find_free_dqentry:: find_free_dqentry(): Data block full unexpectedly.
[QUOTA WARNING] Usage inconsistent for ID 0:actual (1766738993152, 3492) != expected (0, 0)
[QUOTA WARNING] Usage inconsistent for ID 100:actual (14970901614592, 187889) != expected (0, 0)
jffstg-OST0000: Update quota info for quota type 1.
jffstg-OST0000: 191438/30519552 files (22.0% non-contiguous), 4088922050/7812984832 blocks

The other OSTs showed similar errors.


Are there any e2fsck options to solve this issue? (May be with -y?)
How can I reset quotas and start clean?

Please, any suggestions will be very welcomed

Thanks in advance
Best regards

-- 
Ramiro Alba

Centre Tecnològic de Tranferència de Calor
http://www.cttc.upc.edu

Escola Tècnica Superior d'Enginyeries
Industrial i Aeronàutica de Terrassa
Colom 11, E-08222, Terrassa, Barcelona, Spain
Tel: (+34) 93 739 8928

_______________________________________________
HPDD-discuss mailing list
HPDD-discuss@lists.01.org
https://lists.01.org/mailman/listinfo/hpdd-discuss

 

--
Ramiro Alba

Centre Tecnològic de Tranferència de Calor
http://www.cttc.upc.edu

Escola Tècnica Superior d'Enginyeries
Industrial i Aeronàutica de Terrassa
Colom 11, E-08222, Terrassa, Barcelona, Spain
Tel: (+34) 93 739 8928

--
Aquest missatge ha estat analitzat per MailScanner
a la cerca de virus i d'altres continguts perillosos,
i es considera que está net.