4:06 p.m.
Are there any potential issues with using 4k sector (aka "advanced
format") disk drives with Lustre ?
Most Lustre implementations would use RAID arrays. I recognize there may
be changes necessary in RAID chunk/stripe settings with 4k drives. Also
since lustre does not recommend using drive parition tables, there
should not be a problem with partition alignment.
Are there changes required at lfdiskfs or zfs level ?
regards,
chris hunter
chuntera(a)gmail.com
10:35 p.m.
Both ldiskfs and ZFS work well with 4KB sector drives, since this is already very common
for deployment. Ldiskfs is always using 4KB blocksize anyway.
One caveat for ZFS is that if such drives misrepresent themselves as 512-byte drives it is
necessary to specify "ashift=12" at pool creation time to set the minimum block
size to 2^12 = 4096 bytes, otherwise the 512-byte emulation in the drive will hurt ZFS
performance significantly.
Cheers, Andreas
On Apr 5, 2016, at 08:07, Chris Hunter <chuntera(a)gmail.com>
wrote:
Are there any potential issues with using 4k sector (aka "advanced
format") disk drives with Lustre ?
Most Lustre implementations would use RAID arrays. I recognize there may
be changes necessary in RAID chunk/stripe settings with 4k drives. Also
since lustre does not recommend using drive parition tables, there
should not be a problem with partition alignment.
Are there changes required at lfdiskfs or zfs level ?
regards,
chris hunter
chuntera(a)gmail.com
_______________________________________________
HPDD-discuss mailing list
HPDD-discuss(a)lists.01.org
https://lists.01.org/mailman/listinfo/hpdd-discuss
4:36 p.m.
Hi,
To follow-up about 4k sector drives.
How much on-disk space would ldiskfs MDT inodes use on 4k sector
drives ?
For example the LOM limits has 2TB ldiskfs MDT volume = 1 billion
inodes, which is based on 2k inode sizes.
Would each 2k entry in the MDT inode table occupy a 4k drive sector ?
To answer my own question, from the ext4 wiki, I believe inode table is
a "packed" data structure to make efficient use of file blocks.
https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Inode_Table
regards,
chris hunter
chuntera(a)gmail.com
On Tue, 2016-04-05 at 21:35 +0000, Dilger, Andreas wrote:
Both ldiskfs and ZFS work well with 4KB sector drives, since this is
already very common for deployment. Ldiskfs is always using 4KB blocksize anyway.
One caveat for ZFS is that if such drives misrepresent themselves as 512-byte drives it
is necessary to specify "ashift=12" at pool creation time to set the minimum
block size to 2^12 = 4096 bytes, otherwise the 512-byte emulation in the drive will hurt
ZFS performance significantly.
Cheers, Andreas
> On Apr 5, 2016, at 08:07, Chris Hunter <chuntera(a)gmail.com> wrote:
>
> Are there any potential issues with using 4k sector (aka "advanced
> format") disk drives with Lustre ?
>
> Most Lustre implementations would use RAID arrays. I recognize there may
> be changes necessary in RAID chunk/stripe settings with 4k drives. Also
> since lustre does not recommend using drive parition tables, there
> should not be a problem with partition alignment.
>
> Are there changes required at lfdiskfs or zfs level ?
>
> regards,
> chris hunter
> chuntera(a)gmail.com
>
>
> _______________________________________________
> HPDD-discuss mailing list
> HPDD-discuss(a)lists.01.org
> https://lists.01.org/mailman/listinfo/hpdd-discuss
8:06 a.m.
On 2016/04/25, 09:36, "Chris Hunter" <chuntera(a)gmail.com> wrote:
Hi,
To follow-up about 4k sector drives.
How much on-disk space would ldiskfs MDT inodes use on 4k sector
drives ?
The same as they always do. The 4KB sector size is the minimum IO size,
and matches the default blocksize used for all ldiskfs filesystems.
ldiskfs will pack multiple inodes into one block as you write below.
For example the LOM limits has 2TB ldiskfs MDT volume = 1 billion
inodes, which is based on 2k inode sizes.
Would each 2k entry in the MDT inode table occupy a 4k drive sector?
To clarify this further, the "2KB per inode" is not the actual size of
the inode, but rather the ratio of space per inode used at format time.
The inode size is 512 bytes per inode, so 8 inodes are packed into a 4KB
block. The remaining 1.5KB per inode is for things like directory
blocks, xattr blocks for files with more stripes than can fit into the
inode (7 or more stripes in most cases), and Lustre internal metadata
like ChangeLogs, Object Index (OI) files, recovery logs, etc.
I added more details in my patch http://review.whamcloud.com/19786
"LUDOC-331 setup: explain ldiskfs inode size/ratio better"
Cheers, Andreas
To answer my own question, from the ext4 wiki, I believe inode table
is
a "packed" data structure to make efficient use of file blocks.
https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Inode_Table
regards,
chris hunter
chuntera(a)gmail.com
On Tue, 2016-04-05 at 21:35 +0000, Dilger, Andreas wrote:
> Both ldiskfs and ZFS work well with 4KB sector drives, since this is
>already very common for deployment. Ldiskfs is always using 4KB
>blocksize anyway.
>
> One caveat for ZFS is that if such drives misrepresent themselves as
>512-byte drives it is necessary to specify "ashift=12" at pool creation
>time to set the minimum block size to 2^12 = 4096 bytes, otherwise the
>512-byte emulation in the drive will hurt ZFS performance significantly.
>
> Cheers, Andreas
>
> > On Apr 5, 2016, at 08:07, Chris Hunter <chuntera(a)gmail.com> wrote:
> >
> > Are there any potential issues with using 4k sector (aka "advanced
> > format") disk drives with Lustre ?
> >
> > Most Lustre implementations would use RAID arrays. I recognize there
>may
> > be changes necessary in RAID chunk/stripe settings with 4k drives.
>Also
> > since lustre does not recommend using drive parition tables, there
> > should not be a problem with partition alignment.
> >
> > Are there changes required at lfdiskfs or zfs level ?
> >
> > regards,
> > chris hunter
> > chuntera(a)gmail.com
> >
> >
> > _______________________________________________
> > HPDD-discuss mailing list
> > HPDD-discuss(a)lists.01.org
> > https://lists.01.org/mailman/listinfo/hpdd-discuss
Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel High Performance Data Division
2213
days inactive
2234
days old
3 comments
2 participants
participants (2)
-
Chris Hunter -
Dilger, Andreas