On 5/7/19 10:22 AM, Theodore Ts'o wrote:
On Tue, May 07, 2019 at 10:01:19AM +0200, Greg KH wrote:
very helpful to cut the text here, plus not explicitly indicating that
text was cut (yes, I know the ">>>" will be a clue for the careful
losing the set up for my question.
>> My understanding is that the intent of KUnit is to avoid
booting a kernel on
>> real hardware or in a virtual machine. That seems to be a matter of semantics
>> to me because isn't invoking a UML Linux just running the Linux kernel in
>> a different form of virtualization?
>> So I do not understand why KUnit is an improvement over kselftest.
>> It seems to me that KUnit is just another piece of infrastructure that I
>> am going to have to be familiar with as a kernel developer. More overhead,
>> more information to stuff into my tiny little brain.
>> I would guess that some developers will focus on just one of the two test
>> environments (and some will focus on both), splitting the development
>> resources instead of pooling them on a common infrastructure.
>> What am I missing?
> kselftest provides no in-kernel framework for testing kernel code
> specifically. That should be what kunit provides, an "easy" way to
> write in-kernel tests for things.
> Brendan, did I get it right?
Yes, that's basically right. You don't *have* to use KUnit. It's
If KUnit is added to the kernel, and a subsystem that I am submitting
code for has chosen to use KUnit instead of kselftest, then yes, I do
*have* to use KUnit if my submission needs to contain a test for the
code unless I want to convince the maintainer that somehow my case
is special and I prefer to use kselftest instead of KUnittest.
supposed to be a simple way to run a large number of small tests
for specific small components in a system.
kselftest also supports running a subset of tests. That subset of tests
can also be a large number of small tests. There is nothing inherent
in KUnit vs kselftest in this regard, as far as I am aware.
For example, I currently use xfstests using KVM and GCE to test all
ext4. These tests require using multiple 5 GB and 20GB virtual disks,
and it works by mounting ext4 file systems and exercising ext4 through
the system call interfaces, using userspace tools such as fsstress,
fsx, fio, etc. It requires time overhead to start the VM, create and
allocate virtual disks, etc. For example, to run a single 3 seconds
xfstest (generic/001), it requires full 10 seconds to run it via
KUnit is something else; it's specifically intended to allow you
create lightweight tests quickly and easily, and by reducing the
effort needed to write and run unit tests, hopefully we'll have a lot
more of them and thus improve kernel quality.
The same is true of kselftest. You can create lightweight tests in
As an example, I have a volunteer working on developing KUinit tests
for ext4. We're going to start by testing the ext4 extent status
tree. The source code is at fs/ext4/extent_status.c; it's
approximately 1800 LOC. The Kunit tests for the extent status tree
will exercise all of the corner cases for the various extent status
tree functions --- e.g., ext4_es_insert_delayed_block(),
ext4_es_remove_extent(), ext4_es_cache_extent(), etc. And it will do
this in isolation without our needing to create a test file system or
using a test block device.
Next we'll test the ext4 block allocator, again in isolation. To
the block allocator we will have to write "mock functions" which
simulate reading allocation bitmaps from disk. Again, this will allow
the test writer to explicitly construct corner cases and validate that
the block allocator works as expected without having to reverese
engineer file system data structures which will force a particular
code path to be executed.
This would be a difference, but mock functions do not exist in KUnit.
The KUnit test will call the real kernel function in the UML kernel.
I think Brendan has indicated a desire to have mock functions in the
Brendan, do I understand that correctly?
So this is why it's largely irrelevant to me that KUinit uses
fact, it's a feature. We're not testing device drivers, or the
scheduler, or anything else architecture-specific. UML is not about
virtualization. What it's about in this context is allowing us to
start running test code as quickly as possible. Booting KVM takes
about 3-4 seconds, and this includes initializing virtio_scsi and
other device drivers. If by using UML we can hold the amount of
unnecessary kernel subsystem initialization down to the absolute
minimum, and if it means that we can communicating to the test
framework via a userspace "printf" from UML/KUnit code, as opposed to
via a virtual serial port to KVM's virtual console, it all makes for
lighter weight testing.
Why did I go looking for a volunteer to write KUnit tests for ext4?
Well, I have a plan to make some changes in restructing how ext4's
write path works, in order to support things like copy-on-write, a
more efficient delayed allocation system, etc. This will require
making changes to the extent status tree, and by having unit tests for
the extent status tree, we'll be able to detect any bugs that we might
accidentally introduce in the es tree far more quickly than if we
didn't have those tests available. Google has long found that having
these sorts of unit tests is a real win for developer velocity for any
non-trivial code module (or C++ class), even when you take into
account the time it takes to create the unit tests.
P.S. Many thanks to Brendan for finding such a volunteer for me; the
person in question is a SRE from Switzerland who is interested in
getting involved with kernel testing, and this is going to be their
20% project. :-)