Re: [iGVT-g] [ANNOUNCE][RFC] KVMGT - the implementation of Intel GVT-g(full GPU virtualization) for KVM
by Jike Song
Hi all,
We are pleased to announce another update of Intel GVT-g for KVM.
Intel GVT-g is a full GPU virtualization solution with mediated pass-through, starting from 4th generation Intel Core(TM) processors with Intel Graphics processors. A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature, and sharing capability. KVM is supported by Intel GVT-g(a.k.a. KVMGT).
Repositories
Kernel: https://github.com/01org/igvtg-kernel (2015q3-3.18.0 branch)
Qemu: https://github.com/01org/igvtg-qemu (kvmgt_public2015q3 branch)
This update consists of:
- KVMGT is now merged with XenGT in unified repositories(kernel and qemu), but currently
different branches for qemu. KVMGT and XenGT share same iGVT-g core logic.
- PPGTT supported, hence the Windows guest support
- KVMGT now supports both 4th generation (Haswell) and 5th generation (Broadwell) Intel Core(TM) processors
- 2D/3D/Media decoding have been validated on Ubuntu 14.04 and Windows7/Windows 8.1
Next update will be around early Jan, 2016.
Known issues:
- At least 2GB memory is suggested for VM to run most 3D workloads.
- 3Dmark06 running in Windows VM may have some stability issue.
- Using VLC to play .ogg file may cause mosaic or slow response.
Please subscribe the mailing list to report BUGs, discuss, and/or contribute:
https://lists.01.org/mailman/listinfo/igvt-g
More information about Intel GVT-g background, architecture, etc can be found at(may not be up-to-date):
https://01.org/igvt-g
http://www.linux-kvm.org/images/f/f3/01x08b-KVMGT-a.pdf
https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian
Note:
The KVMGT project should be considered a work in progress. As such it is not a complete product nor should it be considered one. Extra care should be taken when testing and configuring a system to use the KVMGT project.
--
Thanks,
Jike
On 12/04/2014 10:24 AM, Jike Song wrote:
> Hi all,
>
> We are pleased to announce the first release of KVMGT project. KVMGT is the implementation of Intel GVT-g technology, a full GPU virtualization solution. Under Intel GVT-g, a virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance of performance, feature, and sharing capability.
>
>
> KVMGT is still in the early stage:
>
> - Basic functions of full GPU virtualization works, guest can see a full-featured vGPU.
> We ran several 3D workloads such as lightsmark, nexuiz, urbanterror and warsow.
>
> - Only Linux guest supported so far, and PPGTT must be disabled in guest through a
> kernel parameter(see README.kvmgt in QEMU).
>
> - This drop also includes some Xen specific changes, which will be cleaned up later.
>
> - Our end goal is to upstream both XenGT and KVMGT, which shares ~90% logic for vGPU
> device model (will be part of i915 driver), with only difference in hypervisor
> specific services
>
> - insufficient test coverage, so please bear with stability issues :)
>
>
>
> There are things need to be improved, esp. the KVM interfacing part:
>
> 1 a domid was added to each KVMGT guest
>
> An ID is needed for foreground OS switching, e.g.
>
> # echo <domid> > /sys/kernel/vgt/control/foreground_vm
>
> domid 0 is reserved for host OS.
>
>
> 2 SRCU workarounds.
>
> Some KVM functions, such as:
>
> kvm_io_bus_register_dev
> install_new_memslots
>
> must be called *without* &kvm->srcu read-locked. Otherwise it hangs.
>
> In KVMGT, we need to register an iodev only *after* BAR registers are
> written by guest. That means, we already have &kvm->srcu hold -
> trapping/emulating PIO(BAR registers) makes us in such a condition.
> That will make kvm_io_bus_register_dev hangs.
>
> Currently we have to disable rcu_assign_pointer() in such functions.
>
> These were dirty workarounds, your suggestions are high welcome!
>
>
> 3 syscalls were called to access "/dev/mem" from kernel
>
> An in-kernel memslot was added for aperture, but using syscalls like
> open and mmap to open and access the character device "/dev/mem",
> for pass-through.
>
>
>
>
> The source codes(kernel, qemu as well as seabios) are available at github:
>
> git://github.com/01org/KVMGT-kernel
> git://github.com/01org/KVMGT-qemu
> git://github.com/01org/KVMGT-seabios
>
> In the KVMGT-qemu repository, there is a "README.kvmgt" to be referred.
>
>
>
> More information about Intel GVT-g and KVMGT can be found at:
>
> https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian
> http://events.linuxfoundation.org/sites/events/files/slides/KVMGT-a%20Ful...
>
>
> Appreciate your comments, BUG reports, and contributions!
>
>
>
>
> --
> Thanks,
> Jike
>
4 years, 10 months
Re: [iGVT-g] [Intel-gfx] [Announcement] 2015-Q3 release of XenGT - a Mediated Graphics Passthrough Solution from Intel
by Jike Song
Hi all,
We are pleased to announce another update of Intel GVT-g for Xen.
Intel GVT-g is a full GPU virtualization solution with mediated pass-through, starting from 4th generation Intel Core(TM) processors with Intel Graphics processors. A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature, and sharing capability. Xen is currently supported on Intel Processor Graphics (a.k.a. XenGT); and the core logic can be easily ported to other hypervisors.
Repositories
Kernel: https://github.com/01org/igvtg-kernel (2015q3-3.18.0 branch)
Xen: https://github.com/01org/igvtg-xen (2015q3-4.5 branch)
Qemu: https://github.com/01org/igvtg-qemu (xengt_public2015q3 branch)
This update consists of:
- XenGT is now merged with KVMGT in unified repositories(kernel and qemu), but currently
different branches for qemu. XenGT and KVMGT share same iGVT-g core logic.
- fix sysfs/debugfs access seldom crash issue
- fix a BUG in XenGT I/O emulation logic
- improve 3d workload stability
Next update will be around early Jan, 2016.
Known issues:
- At least 2GB memory is suggested for VM to run most 3D workloads.
- Keymap might be incorrect in guest. Config file may need to explicitly specify "keymap='en-us'". Although it looks like the default value, earlier we saw the problem of wrong keymap code if it is not explicitly set.
- When using three monitors, doing hotplug between Guest pause/unpause may not be able to lightup all monitors automatically. Some specific monitor issues.
- Cannot move mouse pointer smoothly in guest by default launched by VNC mode. Configuration file need to explicitly specify "usb=1" to enable a USB bus, and "usbdevice='tablet'" to add pointer device using absolute coordinates.
- Resume dom0 from S3 may cause some error message.
- i915 unload/reload cannot works well with less than 3 vcpu when upowerd service was running
- Unigen Tropics running in multiple guests will cause dom0 and guests tdr.
Please subscribe the mailing list to report BUGs, discuss, and/or contribute:
https://lists.01.org/mailman/listinfo/igvt-g
More information about Intel GVT-g background, architecture, etc can be found at(may not be up-to-date):
https://01.org/igvt-g
https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian
http://events.linuxfoundation.org/sites/events/files/slides/XenGT-Xen%20S...
http://events.linuxfoundation.org/sites/events/files/slides/XenGT-Xen%20S...
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-xengt
Note:
The XenGT project should be considered a work in progress. As such it is not a complete product nor should it be considered one. Extra care should be taken when testing and configuring a system to use the XenGT project.
--
Thanks,
Jike
On 07/07/2015 10:49 AM, Jike Song wrote:
> Hi all,
>
> We're pleased to announce a public update to Intel Graphics Virtualization Technology(Intel GVT-g, formerly known as XenGT).
>
> Intel GVT-g is a full GPU virtualization solution with mediated pass-through, starting from 4th generation Intel Core(TM) processors with Intel Graphics processors. A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature, and sharing capability. Xen is currently supported on Intel Processor Graphics (a.k.a. XenGT); and the core logic can be easily ported to other hypervisors, for example, the experimental code has been released to support GVT-g running on a KVM hypervisor (a.k.a KVMGT).
>
> Tip of repositories
> -------------------------
>
> Kernel: 5b73653d5ca, Branch: master-2015Q2-3.18.0
> Qemu: 2a75bbff62c1, Branch: master
> Xen: 38c36f0f511b1, Branch: master-2015Q2-4.5
>
> This update consists of:
> - Change time based scheduler timer to be configurable to enhance stability
> - Fix stability issues that VM/Dom0 got tdr when hang up at some specific instruction on BDW
> - Optimize the emulation of el_status register to enhance stability
> - 2D/3D performance in linux VMs has been improved about 50% on BDW
> - Fix abnormal idle power consumption issue due to wrong forcewake policy
> - Fix tdr issue when running 2D/3D/Media workloads in Windows VMs simultaneously
> - KVM support is still in a separate branch as prototype work. We plan to integrate KVM/Xen support together in the future releases
> - Next update will be around early Oct, 2015
>
> Notice that this release can support both Intel 4th generation Core CPU(code name: Haswell) and Intel 5th generation Core CPU (code name: Broadwell), while the limitation of the latter include:
> * 3D conformance may have some failure
> * Under high demand 3D workloads, stability issues are detected
> * Multi-monitor scenario is not fully tested, while single monitor of VGA/HDMI/DP/eDP should just work
> * Hotplug DP may cause black screen even on native environment
>
> Where to get
>
> kernel: https://github.com/01org/XenGT-Preview-kernel.git
> xen: https://github.com/01org/XenGT-Preview-xen.git
> qemu: https://github.com/01org/XenGT-Preview-qemu.git
>
>
> We have a mailing list for GVT-g development, bug report and technical discussion:
>
> https://lists.01.org/mailman/listinfo/igvt-g
>
> More information about Intel GVT-g background, architecture, etc can be found at:
>
> https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian
> http://events.linuxfoundation.org/sites/events/files/slides/XenGT-Xen%20S...
> https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-xengt
>
>
> Note: The XenGT project should be considered a work in progress. As such it is not a complete product nor should it be considered one. Extra care should be taken when testing and configuring a system to use the XenGT project.
>
>
> --
> Thanks,
> Jike
>
> On 04/10/2015 09:23 PM, Jike Song wrote:
>> Hi all,
>>
>> We're pleased to announce a public update to Intel Graphics Virtualization Technology (Intel GVT-g, formerly known as XenGT). Intel GVT-g is a complete vGPU solution with mediated pass-through, supported today on 4th generation Intel Core(TM) processors with Intel Graphics processors. A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature, and sharing capability. Though we only support Xen on Intel Processor Graphics so far, the core logic can be easily ported to other hypervisors.
>>
>> Tip of repositories
>> -------------------------
>>
>> Kernel: a011c9f953e, Branch: master-2015Q1-3.18.0
>> Qemu: 2a75bbff62c1, Branch: master
>> Xen: 38c36f0f511b1, Branch: master-2015Q1-4.5
>>
>> Summary this update
>> -------------------------
>> - Preliminary Broadwell support.
>> - kernel update from drm-intel 3.17.0 to drm-intel 3.18.0(tag: drm-intel-next-fixes-2014-12-17, notice that i915 driver code is much newer than kernel stable version).
>> - Next update will be around early July, 2015.
>> - KVM support is still in a separate branch as prototype work. We plan to integrate KVM/Xen support together in future releases.
>>
>> This update consists of:
>> - gvt-g core logic code was moved into i915 driver directory.
>> - Host mediation is used for dom0 i915 driver access, instead of de-privileged dom0.
>> - The Xen-specific code was separated from vgt core logic into a new file "driver/xen/xengt.c".
>> - Broadwell is preliminarily supported in this release. Users could start multiple linux/windows 64-bit virtual machines simultaneously, and perform display switch among them.
>>
>> Notice that it is still preliminary release for BDW, which is not yet in the same level of HSW release. Differences include:
>> * Power/performance tuning on BDW is not yet done.
>> * Stability needs to be improved.
>> * No 32-bit guest OS support.
>> * Multi-monitor scenario is not fully tested, while single monitor of VGA/HDMI/DP/eDP should just work.
>>
>>
>> Where to get:
>> -----------------
>> kerenl: https://github.com/01org/XenGT-Preview-kernel.git
>> Xen: https://github.com/01org/XenGT-Preview-xen.git
>> Qemu: https://github.com/01org/XenGT-Preview-qemu.git
>>
>> Please refer to the new setup guide, which provides step-to-step details about building/configuring/running Intel GVT-g:
>> https://github.com/01org/XenGT-Preview-kernel/blob/master-2015Q1-3.18.0/X...
>>
>> More information about Intel GVT-g background, architecture, etc can be found at:
>> https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian
>> http://events.linuxfoundation.org/sites/events/files/slides/XenGT-Xen%20S...
>> https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-xengt
>>
>> The previous update can be found here:
>> http://lists.xen.org/archives/html/xen-devel/2014-12/msg00474.html
>>
>>
>> Note
>> ---------------
>> The XenGT project should be considered a work in progress, As such it is not a complete product nor should it be considered one., Extra care should be taken when testing and configuring a system to use the XenGT project.
>>
>>
>> --
>> Thanks,
>> Jike
>>
>> On 01/09/2015 04:51 PM, Jike Song wrote:
>>> Hi all,
>>>
>>> We're pleased to announce a public update to Intel Graphics Virtualization Technology (Intel GVT-g, formerly known as XenGT). Intel GVT-g is a complete vGPU solution with mediated pass-through, supported today on 4th generation Intel Core(TM) processors with Intel Graphics processors. A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature, and sharing capability. Though we only support Xen on Intel Processor Graphics so far, the core logic can be easily ported to other hypervisors. The XenGT project should be considered a work in progress, As such it is not a complete product nor should it be considered one., Extra care should be taken when testing and configuring a system to use the XenGT project.
>>>
>>> The news of this update:
>>>
>>> - kernel update from 3.14.1 to drm-intel 3.17.0.
>>> - We plan to integrate Intel GVT-g as a feature in i915 driver. That effort is still under review, not included in this update yet.
>>> - Next update will be around early Apr, 2015.
>>>
>>> This update consists of:
>>>
>>> - Including some bug fixes and stability enhancement.
>>> - Making XenGT device model to be aware of Broadwell. In this version BDW is not yet functioning.
>>> - Available Fence registers number is changed to 32 from 16 to align with HSW hardware.
>>> - New cascade interrupt framework for supporting interrupt virtualization on both Haswell and Broadwell.
>>> - Add back the gem_vgtbuffer. The previous release did not build that module for 3.14 kernel. In this release, the module is back and rebased to 3.17.
>>> - Enable the irq based context switch in vgt driver, which will help reduce the cpu utilization while doing context switch, it is enabled by default, and can be turn off by kernel flag irq_based_ctx_switch.
>>>
>>>
>>> Please refer to the new setup guide, which provides step-to-step details about building/configuring/running Intel GVT-g:
>>>
>>> https://github.com/01org/XenGT-Preview-kernel/blob/master/XenGT_Setup_Gui...
>>>
>>> The new source codes are available at the updated github repos:
>>>
>>> Linux: https://github.com/01org/XenGT-Preview-kernel.git
>>> Xen: https://github.com/01org/XenGT-Preview-xen.git
>>> Qemu: https://github.com/01org/XenGT-Preview-qemu.git
>>>
>>>
>>> More information about Intel GVT-g background, architecture, etc can be found at:
>>>
>>>
>>> https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian
>>> http://events.linuxfoundation.org/sites/events/files/slides/XenGT-Xen%20S...
>>> https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-xengt
>>>
>>>
>>>
>>> The previous update can be found here:
>>>
>>>
>>> http://lists.xen.org/archives/html/xen-devel/2014-12/msg00474.html
>>>
>>>
>>>
>>> Appreciate your comments!
>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Jike
>>>
>>>
>>> On 12/04/2014 10:45 AM, Jike Song wrote:
>>>> Hi all,
>>>>
>>>> We're pleased to announce a public release to Intel Graphics Virtualization Technology (Intel GVT-g, formerly known as XenGT). Intel GVT-g is a complete vGPU solution with mediated pass-through, supported today on 4th generation Intel Core(TM) processors with Intel Graphics processors. A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature, and sharing capability. Though we only support Xen on Intel Processor Graphics so far, the core logic can be easily ported to other hypervisors.
>>>>
>>>>
>>>> The news of this update:
>>>>
>>>>
>>>> - kernel update from 3.11.6 to 3.14.1
>>>>
>>>> - We plan to integrate Intel GVT-g as a feature in i915 driver. That effort is still under review, not included in this update yet
>>>>
>>>> - Next update will be around early Jan, 2015
>>>>
>>>>
>>>> This update consists of:
>>>>
>>>> - Windows HVM support with driver version 15.33.3910
>>>>
>>>> - Stability fixes, e.g. stabilize GPU, the GPU hang occurrence rate becomes rare now
>>>>
>>>> - Hardware Media Acceleration for Decoding/Encoding/Transcoding, VC1, H264 etc. format supporting
>>>>
>>>> - Display enhancements, e.g. DP type is supported for virtual PORT
>>>>
>>>> - Display port capability virtualization: with this feature, dom0 manager could freely assign virtual DDI ports to VM, not necessary to check whether the corresponding physical DDI ports are available
>>>>
>>>>
>>>>
>>>> Please refer to the new setup guide, which provides step-to-step details about building/configuring/running Intel GVT-g:
>>>>
>>>>
>>>> https://github.com/01org/XenGT-Preview-kernel/blob/master/XenGT_Setup_Gui...
>>>>
>>>>
>>>>
>>>> The new source codes are available at the updated github repos:
>>>>
>>>>
>>>> Linux: https://github.com/01org/XenGT-Preview-kernel.git
>>>>
>>>> Xen: https://github.com/01org/XenGT-Preview-xen.git
>>>>
>>>> Qemu: https://github.com/01org/XenGT-Preview-qemu.git
>>>>
>>>>
>>>> More information about Intel GVT-g background, architecture, etc can be found at:
>>>>
>>>>
>>>> https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian
>>>>
>>>> http://events.linuxfoundation.org/sites/events/files/slides/XenGT-Xen%20S...
>>>>
>>>> https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-xengt
>>>>
>>>>
>>>> The previous update can be found here:
>>>>
>>>>
>>>> http://lists.xen.org/archives/html/xen-devel/2014-07/msg03248.html
>>>>
>>>>
>>>> Appreciate your comments!
>>>>
>>>>
>>>> --
>>>> Thanks,
>>>> Jike
>>>>
>>>> On 07/25/2014 04:31 PM, Jike Song wrote:
>>>>> Hi all,
>>>>>
>>>>> We're pleased to announce an update to Intel Graphics Virtualization Technology (Intel GVT-g, formerly known as XenGT). Intel GVT-g is a complete vGPU solution with mediated pass-through, supported today on 4th generation Intel Core(TM) processors with Intel Graphics processors. A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature, and sharing capability. Though we only support Xen on Intel Processor Graphics so far, the core logic can be easily ported to other hypervisors.
>>>>>
>>>>> The news of this update:
>>>>>
>>>>> - Project code name is "XenGT", now official name is Intel Graphics Virtualization Technology (Intel GVT-g)
>>>>> - Currently Intel GVT-g supports Intel Processor Graphics built into 4th generation Intel Core processors - Haswell platform
>>>>> - Moving forward, XenGT will change to quarterly release cadence. Next update will be around early October, 2014.
>>>>>
>>>>> This update consists of:
>>>>>
>>>>> - Stability fixes, e.g. stable DP support
>>>>> - Display enhancements, e.g. virtual monitor support. Users can define a virtual monitor type with customized EDID for virtual machines, not necessarily the same as physical monitors.
>>>>> - Improved support for GPU recovery
>>>>> - Experimental Windows HVM support. To download the experimental version, see setup guide for details
>>>>> - Experimental Hardware Media Acceleration for decoding.
>>>>>
>>>>>
>>>>> Please refer to the new setup guide, which provides step-to-step details about building/configuring/running Intel GVT-g:
>>>>>
>>>>> https://github.com/01org/XenGT-Preview-kernel/blob/master/XenGT_Setup_Gui...
>>>>>
>>>>>
>>>>> The new source codes are available at the updated github repos:
>>>>>
>>>>> Linux: https://github.com/01org/XenGT-Preview-kernel.git
>>>>> Xen: https://github.com/01org/XenGT-Preview-xen.git
>>>>> Qemu: https://github.com/01org/XenGT-Preview-qemu.git
>>>>>
>>>>>
>>>>> More information about Intel GVT-g background, architecture, etc can be found at:
>>>>>
>>>>> https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian
>>>>> http://events.linuxfoundation.org/sites/events/files/slides/XenGT-Xen%20S...
>>>>> https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-xengt
>>>>>
>>>>> The previous update can be found here:
>>>>>
>>>>> http://lists.xen.org/archives/html/xen-devel/2014-02/msg01848.html
>>>>>
>>>>> Appreciate your comments!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Thanks,
>>>>> Jike
>>>>>
4 years, 10 months
[PATCH v3 00/11] igd passthrough chipset tweaks
by Gerd Hoffmann
Hi,
We have some code in our tree to support pci passthrough of intel
graphics devices (igd) on xen, which requires some chipset tweaks
for (a) the host bridge and (b) the lpc/isa-bridge to meat the
expectations of the guest driver.
For kvm we need pretty much the same, also the requirements for vgpu
(xengt/kvmgt) are very simliar. This patch wires up the existing
support for kvm. It also brings a bunch of bugfixes and cleanups.
Unfortunaly the oldish laptop I had planned to use for testing turned
out to have no working iommu support for igd, so this patch series
still has seen very light testing only. Any testing feedback is very
welcome.
Testing with kvm/i440fx:
Add '-M pc,igd-passthru=on' to turn on the chipset tweaks.
Passthrough the igd using vfio.
Testing with kvm/q35:
Add '-M q35,igd-passthru=on' to turn on the the chipset tweaks.
Pick up the linux kernel patch referenced in patch #11, build a
custom kernel with it. Passthrough the igd using vfio.
Testing with xen:
Existing setups should continue working ;)
Changes in v3:
* Handle igd-passthrough-isa-bridge creation in machine init.
* Fix xen build failure.
Changes in v2:
* Added igd-passthrough-isa-bridge support form kvm.
* Added patch to drop has_igd_gfx_passthru.
cheers,
Gerd
Gerd Hoffmann (11):
pc: wire up TYPE_IGD_PASSTHROUGH_I440FX_PCI_DEVICE for !xen
pc: remove has_igd_gfx_passthru global
pc: move igd support code to igd.c
igd: switch TYPE_IGD_PASSTHROUGH_I440FX_PCI_DEVICE to realize
igd: TYPE_IGD_PASSTHROUGH_I440FX_PCI_DEVICE: call parent realize
igd: use defines for standard pci config space offsets
igd: revamp host config read
igd: add q35 support
igd: move igd-passthrough-isa-bridge to igd.c too
igd: handle igd-passthrough-isa-bridge setup in realize()
igd: move igd-passthrough-isa-bridge creation to machine init
hw/i386/pc_piix.c | 130 +++------------------------------
hw/pci-host/Makefile.objs | 3 +
hw/pci-host/igd.c | 181 ++++++++++++++++++++++++++++++++++++++++++++++
hw/pci-host/piix.c | 88 ----------------------
hw/pci-host/q35.c | 6 +-
hw/xen/xen_pt.c | 14 ----
hw/xen/xen_pt.h | 5 +-
include/hw/i386/pc.h | 2 +-
vl.c | 10 ---
9 files changed, 204 insertions(+), 235 deletions(-)
create mode 100644 hw/pci-host/igd.c
--
1.8.3.1
5 years, 7 months
Re: [iGVT-g] VFIO based vGPU(was Re: [Announcement] 2015-Q3 release of XenGT - a Mediated ...)
by Alex Williamson
On Tue, 2016-01-26 at 22:05 +0800, Yang Zhang wrote:
> On 2016/1/26 15:41, Jike Song wrote:
> > On 01/26/2016 05:30 AM, Alex Williamson wrote:
> > > [cc +Neo @Nvidia]
> > >
> > > Hi Jike,
> > >
> > > On Mon, 2016-01-25 at 19:34 +0800, Jike Song wrote:
> > > > On 01/20/2016 05:05 PM, Tian, Kevin wrote:
> > > > > I would expect we can spell out next level tasks toward above
> > > > > direction, upon which Alex can easily judge whether there are
> > > > > some common VFIO framework changes that he can help :-)
> > > >
> > > > Hi Alex,
> > > >
> > > > Here is a draft task list after a short discussion w/ Kevin,
> > > > would you please have a look?
> > > >
> > > > Bus Driver
> > > >
> > > > { in i915/vgt/xxx.c }
> > > >
> > > > - define a subset of vfio_pci interfaces
> > > > - selective pass-through (say aperture)
> > > > - trap MMIO: interface w/ QEMU
> > >
> > > What's included in the subset? Certainly the bus reset ioctls really
> > > don't apply, but you'll need to support the full device interface,
> > > right? That includes the region info ioctl and access through the vfio
> > > device file descriptor as well as the interrupt info and setup ioctls.
> > >
> >
> > [All interfaces I thought are via ioctl:) For other stuff like file
> > descriptor we'll definitely keep it.]
> >
> > The list of ioctl commands provided by vfio_pci:
> >
> > - VFIO_DEVICE_GET_PCI_HOT_RESET_INFO
> > - VFIO_DEVICE_PCI_HOT_RESET
> >
> > As you said, above 2 don't apply. But for this:
> >
> > - VFIO_DEVICE_RESET
> >
> > In my opinion it should be kept, no matter what will be provided in
> > the bus driver.
> >
> > - VFIO_PCI_ROM_REGION_INDEX
> > - VFIO_PCI_VGA_REGION_INDEX
> >
> > I suppose above 2 don't apply neither? For a vgpu we don't provide a
> > ROM BAR or VGA region.
> >
> > - VFIO_DEVICE_GET_INFO
> > - VFIO_DEVICE_GET_REGION_INFO
> > - VFIO_DEVICE_GET_IRQ_INFO
> > - VFIO_DEVICE_SET_IRQS
> >
> > Above 4 are needed of course.
> >
> > We will need to extend:
> >
> > - VFIO_DEVICE_GET_REGION_INFO
> >
> >
> > a) adding a flag: DONT_MAP. For example, the MMIO of vgpu
> > should be trapped instead of being mmap-ed.
>
> I may not in the context, but i am curious how to handle the DONT_MAP in
> vfio driver? Since there are no real MMIO maps into the region and i
> suppose the access to the region should be handled by vgpu in i915
> driver, but currently most of the mmio accesses are handled by Qemu.
VFIO supports the following region attributes:
#define VFIO_REGION_INFO_FLAG_READ (1 << 0) /* Region supports read */
#define VFIO_REGION_INFO_FLAG_WRITE (1 << 1) /* Region supports write */
#define VFIO_REGION_INFO_FLAG_MMAP (1 << 2) /* Region supports mmap */
If MMAP is not set, then the QEMU driver will do pread and/or pwrite to
the specified offsets of the device file descriptor, depending on what
accesses are supported. This is all reported through the REGION_INFO
ioctl for a given index. If mmap is supported, the VM will have direct
access to the area, without faulting to KVM other than to populate the
mapping. Without mmap support, a VM MMIO access traps into KVM, which
returns out to QEMU to service the request, which then finds the
MemoryRegion serviced through vfio, which will then perform a
pread/pwrite through to the kernel vfio bus driver to handle the
access. Thanks,
Alex
5 years, 7 months
Re: [iGVT-g] [RFC 17/29] gvt: Xen hypervisor GVT-g MPT module
by Zhiyuan Lv
Hi Joonas,
On Thu, Jan 28, 2016 at 01:33:33PM +0200, Joonas Lahtinen wrote:
> Hi,
>
> See the file MAINTAINERS and add Cc: lines according to "XEN HYPERVISOR
> INTERFACE". Also I think it'll be useful to split the i915 changes to a
> separate patch next int he series (as the reviewer will be different).
>
> We will have to wait for Xen maintainers to take a position on this. Is
> there KVM counterparts for this stuff incoming?
Yeah, we have KVM part as well. There will be separate mails to discuss
with Xen/KVM community the Mediate-Pass-Through (MPT) interface and its
implementation. This patch can be ignored right now. It is here just to make
the whole patchset functional. Thanks!
Regards,
-Zhiyuan
>
> On to, 2016-01-28 at 18:21 +0800, Zhi Wang wrote:
> > This is the xen hypervisor MPT module which let GVT-g be able to run
> > under
> > Xen hypervisor.
> >
>
> Cc: xen-devel@lists.xenproject.org
> ...and so on...
>
> Regards, Joonas
>
5 years, 7 months
Re: [iGVT-g] VFIO based vGPU(was Re: [Announcement] 2015-Q3 release of XenGT - a Mediated ...)
by Alex Williamson
On Thu, 2016-01-28 at 02:25 +0530, Kirti Wankhede wrote:
>
> On 1/27/2016 9:30 PM, Alex Williamson wrote:
> > On Wed, 2016-01-27 at 13:36 +0530, Kirti Wankhede wrote:
> > >
> > > On 1/27/2016 1:36 AM, Alex Williamson wrote:
> > > > On Tue, 2016-01-26 at 02:20 -0800, Neo Jia wrote:
> > > > > On Mon, Jan 25, 2016 at 09:45:14PM +0000, Tian, Kevin wrote:
> > > > > > > From: Alex Williamson [mailto:alex.williamson@redhat.com]
> > > > >
> > > > > Hi Alex, Kevin and Jike,
> > > > >
> > > > > (Seems I shouldn't use attachment, resend it again to the list, patches are
> > > > > inline at the end)
> > > > >
> > > > > Thanks for adding me to this technical discussion, a great opportunity
> > > > > for us to design together which can bring both Intel and NVIDIA vGPU solution to
> > > > > KVM platform.
> > > > >
> > > > > Instead of directly jumping to the proposal that we have been working on
> > > > > recently for NVIDIA vGPU on KVM, I think it is better for me to put out couple
> > > > > quick comments / thoughts regarding the existing discussions on this thread as
> > > > > fundamentally I think we are solving the same problem, DMA, interrupt and MMIO.
> > > > >
> > > > > Then we can look at what we have, hopefully we can reach some consensus soon.
> > > > >
> > > > > > Yes, and since you're creating and destroying the vgpu here, this is
> > > > > > where I'd expect a struct device to be created and added to an IOMMU
> > > > > > group. The lifecycle management should really include links between
> > > > > > the vGPU and physical GPU, which would be much, much easier to do with
> > > > > > struct devices create here rather than at the point where we start
> > > > > > doing vfio "stuff".
> > > > >
> > > > > Infact to keep vfio-vgpu to be more generic, vgpu device creation and management
> > > > > can be centralized and done in vfio-vgpu. That also include adding to IOMMU
> > > > > group and VFIO group.
> > > > Is this really a good idea? The concept of a vgpu is not unique to
> > > > vfio, we want vfio to be a driver for a vgpu, not an integral part of
> > > > the lifecycle of a vgpu. That certainly doesn't exclude adding
> > > > infrastructure to make lifecycle management of a vgpu more consistent
> > > > between drivers, but it should be done independently of vfio. I'll go
> > > > back to the SR-IOV model, vfio is often used with SR-IOV VFs, but vfio
> > > > does not create the VF, that's done in coordination with the PF making
> > > > use of some PCI infrastructure for consistency between drivers.
> > > >
> > > > It seems like we need to take more advantage of the class and driver
> > > > core support to perhaps setup a vgpu bus and class with vfio-vgpu just
> > > > being a driver for those devices.
> > >
> > > For device passthrough or SR-IOV model, PCI devices are created by PCI
> > > bus driver and from the probe routine each device is added in vfio group.
> >
> > An SR-IOV VF is created by the PF driver using standard interfaces
> > provided by the PCI core. The IOMMU group for a VF is added by the
> > IOMMU driver when the device is created on the pci_bus_type. The probe
> > routine of the vfio bus driver (vfio-pci) is what adds the device into
> > the vfio group.
> >
> > > For vgpu, there should be a common module that create vgpu device, say
> > > vgpu module, add vgpu device to an IOMMU group and then add it to vfio
> > > group. This module can handle management of vgpus. Advantage of keeping
> > > this module a separate module than doing device creation in vendor
> > > modules is to have generic interface for vgpu management, for example,
> > > files /sys/class/vgpu/vgpu_start and /sys/class/vgpu/vgpu_shudown and
> > > vgpu driver registration interface.
> >
> > But you're suggesting something very different from the SR-IOV model.
> > If we wanted to mimic that model, the GPU specific driver should create
> > the vgpu using services provided by a common interface. For instance
> > i915 could call a new vgpu_device_create() which creates the device,
> > adds it to the vgpu class, etc. That vgpu device should not be assumed
> > to be used with vfio though, that should happen via a separate probe
> > using a vfio-vgpu driver. It's that vfio bus driver that will add the
> > device to a vfio group.
> >
>
> In that case vgpu driver should provide a driver registration interface
> to register vfio-vgpu driver.
>
> struct vgpu_driver {
> const char *name;
> int (*probe) (struct vgpu_device *vdev);
> void (*remove) (struct vgpu_device *vdev);
> }
>
> int vgpu_register_driver(struct vgpu_driver *driver)
> {
> ...
> }
> EXPORT_SYMBOL(vgpu_register_driver);
>
> int vgpu_unregister_driver(struct vgpu_driver *driver)
> {
> ...
> }
> EXPORT_SYMBOL(vgpu_unregister_driver);
>
> vfio-vgpu driver registers to vgpu driver. Then from
> vgpu_device_create(), after creating the device it calls
> vgpu_driver->probe(vgpu_device) and vfio-vgpu driver adds the device to
> vfio group.
>
> +--------------+ vgpu_register_driver()+---------------+
> > __init() +------------------------->+ |
> > | | |
> > +<-------------------------+ vgpu.ko |
> > vfio_vgpu.ko | probe()/remove() | |
> > | +---------+ +---------+
> +--------------+ | +-------+-------+ |
> | ^ |
> | callback | |
> | +-------+--------+ |
> | |vgpu_register_device() |
> | | | |
> +---^-----+-----+ +-----+------+-+
> | nvidia.ko | | i915.ko |
> | | | |
> +-----------+ +------------+
>
> Is my understanding correct?
We have an entire driver core subsystem in Linux for the purpose of
matching devices to drivers, I don't think we should be re-inventing
that. That's why I'm suggesting that we should have infrastructure
which facilitates GPU drivers to create vGPU devices in a common way,
perhaps even placing the devices on a virtual vgpu bus, and then allow a
vfio-vgpu driver to register as a driver for devices of that bus/class
and use the existing driver callbacks. Thanks,
Alex
5 years, 7 months
Re: [iGVT-g] VFIO based vGPU(was Re: [Announcement] 2015-Q3 release of XenGT - a Mediated ...)
by Jike Song
On 01/19/2016 03:05 AM, Alex Williamson wrote:
> On Mon, 2016-01-18 at 16:56 +0800, Jike Song wrote:
>>
>> Would you elaborate a bit about 'iommu backends' here? Previously
>> I thought that entire type1 will be duplicated. If not, what is supposed
>> to add, a new vfio_dma_do_map?
>
> I don't know that you necessarily want to re-use any of the
> vfio_iommu_type1.c code as-is, it's just the API that we'll want to
> keep consistent so QEMU doesn't need to learn about a new iommu
> backend. Opportunities for sharing certainly may arise, you may want
> to use a similar red-black tree for storing current mappings, the
> pinning code may be similar, etc. We can evaluate on a case by case
> basis whether it makes sense to pull out common code for each of those.
It would be great if you can help abstracting it :)
>
> As for an iommu backend in general, if you look at the code flow
> example in Documentation/vfio.txt, the user opens a container
> (/dev/vfio/vfio) and a group (/dev/vfio/$GROUPNUM). The group is set
> to associate with a container instance via VFIO_GROUP_SET_CONTAINER and
> then an iommu model is set for the container with VFIO_SET_IOMMU.
> Looking at drivers/vfio/vfio.c:vfio_ioctl_set_iommu(), we look for an
> iommu backend that supports the requested extension (VFIO_TYPE1_IOMMU),
> call the open() callback on it and then attempt to attach the group via
> the attach_group() callback. At this latter callback, the iommu
> backend can compare the device to those that it actually supports. For
> instance the existing vfio_iommu_type1 will attempt to use the IOMMU
> API and should fail if the device cannot be supported with that. The
> current loop in vfio_ioctl_set_iommu() will exit in this case, but as
> you can see in the code, it's easy to make it continue and look for
> another iommu backend that supports the requested extension.
>
Got it, sure type1 API w/ userspace should be kept, while a new backend
being used for vgpu.
>>> The benefit here is that QEMU could work
>>> unmodified, using the type1 vfio-iommu API regardless of whether a
>>> device is directly assigned or virtual.
>>>
>>> Let's look at the type1 interface; we have simple map and unmap
>>> interfaces which map and unmap process virtual address space (vaddr) to
>>> the device address space (iova). The host physical address is obtained
>>> by pinning the vaddr. In the current implementation, a map operation
>>> pins pages and populates the hardware iommu. A vgpu compatible
>>> implementation might simply register the translation into a kernel-
>>> based database to be called upon later. When the host graphics driver
>>> needs to enable dma for the vgpu, it doesn't need to go to QEMU for the
>>> translation, it already possesses the iova to vaddr mapping, which
>>> becomes iova to hpa after a pinning operation.
>>>
>>> So, I would encourage you to look at creating a vgpu vfio iommu
>>> backened that makes use of the type1 api since it will reduce the
>>> changes necessary for userspace.
>>>
>>
>> BTW, that should be done in the 'bus' driver, right?
>
> I think you have some flexibility between the graphics driver and the
> vfio-vgpu driver in where this is done. If we want vfio-vgpu to be
> more generic, then vgpu device creation and management should probably
> be done in the graphics driver and vfio-vgpu should be able to probe
> that device and call back into the graphics driver to handle requests.
> If it turns out there's not much for vfio-vgpu to share, ie. it's just
> a passthrough for device specific emulation, then maybe we want a vfio-
> intel-vgpu instead.
>
Good to know that.
>>
>> Looks that things get more clear overall, with small exceptions.
>> Thanks for the advice:)
>
> Yes, please let me know how I can help. Thanks,
>
> Alex
>
I will start the coding soon, will do :)
--
Thanks,
Jike
5 years, 7 months
Re: [iGVT-g] VFIO based vGPU(was Re: [Announcement] 2015-Q3 release of XenGT - a Mediated ...)
by Alex Williamson
On Wed, 2016-01-27 at 13:36 +0530, Kirti Wankhede wrote:
>
> On 1/27/2016 1:36 AM, Alex Williamson wrote:
> > On Tue, 2016-01-26 at 02:20 -0800, Neo Jia wrote:
> > > On Mon, Jan 25, 2016 at 09:45:14PM +0000, Tian, Kevin wrote:
> > > > > From: Alex Williamson [mailto:alex.williamson@redhat.com]
> > >
> > > Hi Alex, Kevin and Jike,
> > >
> > > (Seems I shouldn't use attachment, resend it again to the list, patches are
> > > inline at the end)
> > >
> > > Thanks for adding me to this technical discussion, a great opportunity
> > > for us to design together which can bring both Intel and NVIDIA vGPU solution to
> > > KVM platform.
> > >
> > > Instead of directly jumping to the proposal that we have been working on
> > > recently for NVIDIA vGPU on KVM, I think it is better for me to put out couple
> > > quick comments / thoughts regarding the existing discussions on this thread as
> > > fundamentally I think we are solving the same problem, DMA, interrupt and MMIO.
> > >
> > > Then we can look at what we have, hopefully we can reach some consensus soon.
> > >
> > > > Yes, and since you're creating and destroying the vgpu here, this is
> > > > where I'd expect a struct device to be created and added to an IOMMU
> > > > group. The lifecycle management should really include links between
> > > > the vGPU and physical GPU, which would be much, much easier to do with
> > > > struct devices create here rather than at the point where we start
> > > > doing vfio "stuff".
> > >
> > > Infact to keep vfio-vgpu to be more generic, vgpu device creation and management
> > > can be centralized and done in vfio-vgpu. That also include adding to IOMMU
> > > group and VFIO group.
> > Is this really a good idea? The concept of a vgpu is not unique to
> > vfio, we want vfio to be a driver for a vgpu, not an integral part of
> > the lifecycle of a vgpu. That certainly doesn't exclude adding
> > infrastructure to make lifecycle management of a vgpu more consistent
> > between drivers, but it should be done independently of vfio. I'll go
> > back to the SR-IOV model, vfio is often used with SR-IOV VFs, but vfio
> > does not create the VF, that's done in coordination with the PF making
> > use of some PCI infrastructure for consistency between drivers.
> >
> > It seems like we need to take more advantage of the class and driver
> > core support to perhaps setup a vgpu bus and class with vfio-vgpu just
> > being a driver for those devices.
>
> For device passthrough or SR-IOV model, PCI devices are created by PCI
> bus driver and from the probe routine each device is added in vfio group.
An SR-IOV VF is created by the PF driver using standard interfaces
provided by the PCI core. The IOMMU group for a VF is added by the
IOMMU driver when the device is created on the pci_bus_type. The probe
routine of the vfio bus driver (vfio-pci) is what adds the device into
the vfio group.
> For vgpu, there should be a common module that create vgpu device, say
> vgpu module, add vgpu device to an IOMMU group and then add it to vfio
> group. This module can handle management of vgpus. Advantage of keeping
> this module a separate module than doing device creation in vendor
> modules is to have generic interface for vgpu management, for example,
> files /sys/class/vgpu/vgpu_start and /sys/class/vgpu/vgpu_shudown and
> vgpu driver registration interface.
But you're suggesting something very different from the SR-IOV model.
If we wanted to mimic that model, the GPU specific driver should create
the vgpu using services provided by a common interface. For instance
i915 could call a new vgpu_device_create() which creates the device,
adds it to the vgpu class, etc. That vgpu device should not be assumed
to be used with vfio though, that should happen via a separate probe
using a vfio-vgpu driver. It's that vfio bus driver that will add the
device to a vfio group.
> In the patch, vgpu_dev.c + vgpu_sysfs.c form such vgpu module and
> vgpu_vfio.c is for VFIO interface. Each vgpu device should be added to
> vfio group, so vgpu_group_init() from vgpu_vfio.c should be called per
> device. In the vgpu module, vgpu devices are created on request, so
> vgpu_group_init() should be called explicitly for per vgpu device.
> That’s why had merged the 2 modules, vgpu + vgpu_vfio to form one vgpu
> module. Vgpu_vfio would remain separate entity but merged with vgpu
> module.
I disagree with this design, creation of a vgpu necessarily involves the
GPU driver and should not be tied to use of the vgpu with vfio. vfio
should be a driver for the device, maybe eventually not the only driver
for the device. Thanks,
Alex
5 years, 8 months
Re: [iGVT-g] [Qemu-devel] VFIO based vGPU(was Re: [Announcement] 2015-Q3 release of XenGT - a Mediated ...)
by Alex Williamson
On Wed, 2016-01-27 at 09:52 +0800, Yang Zhang wrote:
> On 2016/1/27 6:56, Alex Williamson wrote:
> > On Tue, 2016-01-26 at 22:39 +0000, Tian, Kevin wrote:
> > > > From: Alex Williamson [mailto:alex.williamson@redhat.com]
> > > > Sent: Wednesday, January 27, 2016 6:27 AM
> > > >
> > > > On Tue, 2016-01-26 at 22:15 +0000, Tian, Kevin wrote:
> > > > > > From: Alex Williamson [mailto:alex.williamson@redhat.com]
> > > > > > Sent: Wednesday, January 27, 2016 6:08 AM
> > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Today KVMGT (not using VFIO yet) registers I/O emulation callbacks to
> > > > > > > > > KVM, so VM MMIO access will be forwarded to KVMGT directly for
> > > > > > > > > emulation in kernel. If we reuse above R/W flags, the whole emulation
> > > > > > > > > path would be unnecessarily long with obvious performance impact. We
> > > > > > > > > either need a new flag here to indicate in-kernel emulation (bias from
> > > > > > > > > passthrough support), or just hide the region alternatively (let KVMGT
> > > > > > > > > to handle I/O emulation itself like today).
> > > > > > > >
> > > > > > > > That sounds like a future optimization TBH. There's very strict
> > > > > > > > layering between vfio and kvm. Physical device assignment could make
> > > > > > > > use of it as well, avoiding a round trip through userspace when an
> > > > > > > > ioread/write would do. Userspace also needs to orchestrate those kinds
> > > > > > > > of accelerators, there might be cases where userspace wants to see those
> > > > > > > > transactions for debugging or manipulating the device. We can't simply
> > > > > > > > take shortcuts to provide such direct access. Thanks,
> > > > > > > >
> > > > > > >
> > > > > > > But we have to balance such debugging flexibility and acceptable performance.
> > > > > > > To me the latter one is more important otherwise there'd be no real usage
> > > > > > > around this technique, while for debugging there are other alternative (e.g.
> > > > > > > ftrace) Consider some extreme case with 100k traps/second and then see
> > > > > > > how much impact a 2-3x longer emulation path can bring...
> > > > > >
> > > > > > Are you jumping to the conclusion that it cannot be done with proper
> > > > > > layering in place? Performance is important, but it's not an excuse to
> > > > > > abandon designing interfaces between independent components. Thanks,
> > > > > >
> > > > >
> > > > > Two are not controversial. My point is to remove unnecessary long trip
> > > > > as possible. After another thought, yes we can reuse existing read/write
> > > > > flags:
> > > > > - KVMGT will expose a private control variable whether in-kernel
> > > > > delivery is required;
> > > >
> > > > But in-kernel delivery is never *required*. Wouldn't userspace want to
> > > > deliver in-kernel any time it possibly could?
> > > >
> > > > > - when the variable is true, KVMGT will register in-kernel MMIO
> > > > > emulation callbacks then VM MMIO request will be delivered to KVMGT
> > > > > directly;
> > > > > - when the variable is false, KVMGT will not register anything.
> > > > > VM MMIO request will then be delivered to Qemu and then ioread/write
> > > > > will be used to finally reach KVMGT emulation logic;
> > > >
> > > > No, that means the interface is entirely dependent on a backdoor through
> > > > KVM. Why can't userspace (QEMU) do something like register an MMIO
> > > > region with KVM handled via a provided file descriptor and offset,
> > > > couldn't KVM then call the file ops without a kernel exit? Thanks,
> > > >
> > >
> > > Could you elaborate this thought? If it can achieve the purpose w/o
> > > a kernel exit definitely we can adapt to it. :-)
> >
> > I only thought of it when replying to the last email and have been doing
> > some research, but we already do quite a bit of synchronization through
> > file descriptors. The kvm-vfio pseudo device uses a group file
> > descriptor to ensure a user has access to a group, allowing some degree
> > of interaction between modules. Eventfds and irqfds already make use of
> > f_ops on file descriptors to poke data. So, if KVM had information that
> > an MMIO region was backed by a file descriptor for which it already has
> > a reference via fdget() (and verified access rights and whatnot), then
> > it ought to be a simple matter to get to f_ops->read/write knowing the
> > base offset of that MMIO region. Perhaps it could even simply use
> > __vfs_read/write(). Then we've got a proper reference to the file
> > descriptor for ownership purposes and we've transparently jumped across
> > modules without any implicit knowledge of the other end. Could it work?
> > Thanks,
>
> ioeventfd is a good example.
> As i known, all access to the MMIO of IGD is trapped into kernel. Also,
> the pci config space is emulated by Qemu. Same the for VGA, which is
> emulated too. I guest interrupt also is emulated(This means we cannot
> benifit from VT-d pi). The most important is that KVMGT doesn't required
> hardware IOMMU. As we known, VFIO is for the direct device assignment,
> but most of thing for KVMGT are emulated, why we should use VFIO for it?
What is a vGPU? It's a PCI device exposed to QEMU that needs to support
emulated and direct MMIO paths into the kernel driver, PCI config space
emulation, and various interrupt models. What does the VFIO API
provide? Exactly those things.
Yes, vfio is typically used for assigning physical devices, but it has a
very modular infrastructure which allows sub-drivers to be written that
can do much more complicated and device specific passthrough and
emulation in the kernel. vfio typically works with a platform IOMMU,
but any devices that can provide isolation and translation services will
work. In the case of graphics cards, there's effectively already an
IOMMU on the device, in the case of vGPU, this is mediated through the
physical GPU driver.
So what's the benefit? VFIO already has the IOMMU and device access
interfaces, is already supported by QEMU and libvirt, and re-using these
for vGPU avoids a proliferation of new vendor specific devices, each
with their own implementation of these interfaces and each requiring
unique libvirt and upper level management device specific knowledge.
That's why. Thanks,
Alex
5 years, 8 months
Re: [iGVT-g] VFIO based vGPU(was Re: [Announcement] 2015-Q3 release of XenGT - a Mediated ...)
by Jike Song
On 01/27/2016 09:34 AM, Yang Zhang wrote:
> On 2016/1/27 8:06, Jike Song wrote:
>> On 01/26/2016 10:05 PM, Yang Zhang wrote:
>>> On 2016/1/26 15:41, Jike Song wrote:
>>>
>>>> We will need to extend:
>>>>
>>>> - VFIO_DEVICE_GET_REGION_INFO
>>>>
>>>>
>>>> a) adding a flag: DONT_MAP. For example, the MMIO of vgpu
>>>> should be trapped instead of being mmap-ed.
>>>
>>> I may not in the context, but i am curious how to handle the DONT_MAP in
>>> vfio driver? Since there are no real MMIO maps into the region and i
>>> suppose the access to the region should be handled by vgpu in i915
>>> driver, but currently most of the mmio accesses are handled by Qemu.
>>>
>>
>> Hi Yang,
>>
>> MMIO accesses are supposed to be handled in kernel, without vm-exiting
>> to QEMU, similar to in-kernel irqchip :)
>
> The question is current vfio doesn't support it. The long discussion
> between Alex and Kevin is what i am to understand how KVMGT works under
> vfio framework.
>
Yes, good to expose it earlier.
Previously Kevin and I thought KVMGT is free to register an iodev,
responsible for a MMIO range r/w, to KVM hypervisor directly. If this is
not acceptable then we will have to figure out an alternative.
--
Thanks,
Jike
5 years, 8 months