The Solaris 11 sources are still available at Illumos.org
what Opensolaris was once, minus Suns bug database).
Also, if you keep the performance in mind, remember that the world is
moving towards many cores with many hardware threads per core, so
optimising for the "two core with low latency" use case is wrong like
a sin. More likely is the "8 core with 4 threads per core" use case
for benchmarking because that's what we will end up in even low end
hardware soon, maybe with variable bandwidth between cores if
something like ScaleMP is used.
On 11 February 2016 at 11:55, Jan Kara <jack(a)suse.cz> wrote:
On Wed 10-02-16 23:39:43, Cedric Blancher wrote:
> AFAIK Solaris 11 uses a sparse tree instead of a array. Solves the
> scalability problem AND deals with variable page size.
Well, but then you have to have this locking tree for every inode so the
memory overhead is relatively large, no? I've played with range locking of
mapping in the past but its performance was not stellar. Do you have any
reference for what Solaris does?
> On 10 February 2016 at 23:09, Dave Chinner <david(a)fromorbit.com> wrote:
> > On Wed, Feb 10, 2016 at 11:32:49AM +0100, Jan Kara wrote:
> >> On Tue 09-02-16 10:18:53, Dan Williams wrote:
> >> > On Tue, Feb 9, 2016 at 9:24 AM, Jan Kara <jack(a)suse.cz> wrote:
> >> > > Hello,
> >> > >
> >> > > I was thinking about current issues with DAX fault locking 
> >> > > corruption due to racing faults allocating blocks) and also races
> >> > > currently don't allow us to clear dirty tags in the radix tree
due to races
> >> > > between faults and cache flushing . Both of these exist because
> >> > > have an equivalent of page lock available for DAX. While we have
> >> > > reasonable solution available for problem , so far I'm not
aware of a
> >> > > decent solution for . After briefly discussing the issue with
Mel he had
> >> > > a bright idea that we could used hashed locks to deal with 
(and I think
> >> > > we can solve  with them as well). So my proposal looks as
> >> > >
> >> > > DAX will have an array of mutexes (the array can be made per
> >> > > initially a global one should be OK). We will use mutexes in the
array as a
> >> > > replacement for page lock - we will use hashfn(mapping, index) to
> >> > > particular mutex protecting our offset in the mapping. On fault /
> >> > > mkwrite, we'll grab the mutex similarly to page lock and
release it once we
> >> > > are done updating page tables. This deals with races in . When
> >> > > caches we grab the mutex before clearing writeable bit in page
> >> > > and clearing dirty bit in the radix tree and drop it after we have
> >> > > caches for the pfn. This deals with races in .
> >> > >
> >> > > Thoughts?
> >> > >
> >> >
> >> > I like the fact that this makes the locking explicit and
> >> > straightforward rather than something more tricky. Can we make the
> >> > hashfn pfn based? I'm thinking we could later reuse this as part
> >> > the solution for eliminating the need to allocate struct page, and we
> >> > don't have the 'mapping' available in all paths...
> >> So Mel originally suggested to use pfn for hashing as well. My concern with
> >> using pfn is that e.g. if you want to fill a hole, you don't have a pfn
> >> lock. What you really need to protect is a logical offset in the file to
> >> serialize allocation of underlying blocks, its mapping into page tables,
> >> and flushing the blocks out of caches. So using inode/mapping and offset
> >> for the hashing is easier (it isn't obvious to me we can fix hole
> >> races with pfn-based locking).
> > So how does that file+offset hash work when trying to lock different
> > ranges? file+offset hashing to determine the lock to use only works
> > if we are dealing with fixed size ranges that the locks affect.
> > e.g. offset has 4k granularity for a single page faults, but we also
> > need to handle 2MB granularity for huge page faults, and IIRC 1GB
> > granularity for giant page faults...
> > What's the plan here?
> > Cheers,
> > Dave.
> > --
> > Dave Chinner
> > david(a)fromorbit.com
> > --
> > To unsubscribe from this list: send the line "unsubscribe
> > the body of a message to majordomo(a)vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> Cedric Blancher <cedric.blancher(a)gmail.com>
> Institute Pasteur
Jan Kara <jack(a)suse.com>
SUSE Labs, CR
Cedric Blancher <cedric.blancher(a)gmail.com>