Linux guest on Broadwell crashes with "invalid address range"
by Joseph Kogut
I'm experimenting with the 2015-BdwPV branch of the XenGT project, and
when I start a Linux guest, the domain crashes shortly after the i915
driver attempts to load.
Dmesg on my dom0 shows this:
> [ 103.536696] vGT error:(g2h_gm_range:1994) VM(2): invalid address range: g_addr(0x35000), size(0x4)
> [ 103.536812] Assert at drivers/gpu/drm/i915/vgt/aperture_gm.c line 70
> [ 103.536895] vGT warning:(mmio_g2h_gmadr:70) Killing VM2
> [ 105.562578] prepare to destroy vgt (1)
> [ 105.562856] check render ownership...
> [ 105.562948] vgt instance has been removed from run queue
> [ 105.563005] check display ownership...
> [ 105.563090] release display/render ownership... done
> [ 106.294376] vGT: vgt_release_instance done
I'm wondering if there's a bug in my machine's IOMMU implementation.
Prior to this message, there's a bunch of warnings about untracked
MMIO read/write(s).
> [ 87.981879] vGT warning:(fpga_dbg_mmio_write:2317) VM 2 writes FPGA_DBG register: 80000000.
> [ 87.982227] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x4094,len=4, val=0x0!!!
> [ 87.990274] VM2 write register RC_STATE_CTRL_1 with 0x0
> [ 87.991648] vGT (1) , write fence register 100020, 4 out of assignment 4.
> [ 87.991771] vGT(1) , read fence register 100020, 4 out of assignment 4.
> [ 87.991878] vGT (1) , write fence register 100024, 4 out of assignment 4.
> [ 87.992001] vGT(1) , read fence register 100024, 4 out of assignment 4.
> [ 87.992116] vGT (1) , write fence register 100028, 5 out of assignment 4.
> [ 87.992241] vGT(1) , read fence register 100028, 5 out of assignment 4.
> [ 87.992346] vGT (1) , write fence register 10002c, 5 out of assignment 4.
> [ 87.992467] vGT(1) , read fence register 10002c, 5 out of assignment 4.
> [ 87.992575] vGT (1) , write fence register 100030, 6 out of assignment 4.
> [ 87.992697] vGT(1) , read fence register 100030, 6 out of assignment 4.
> [ 87.992802] vGT (1) , write fence register 100034, 6 out of assignment 4.
> [ 87.992922] vGT(1) , read fence register 100034, 6 out of assignment 4.
> [ 87.993031] vGT (1) , write fence register 100038, 7 out of assignment 4.
> [ 87.993159] vGT(1) , read fence register 100038, 7 out of assignment 4.
> [ 87.993265] vGT (1) , write fence register 10003c, 7 out of assignment 4.
> [ 87.993390] vGT(1) , read fence register 10003c, 7 out of assignment 4.
> [ 87.993497] vGT (1) , write fence register 100040, 8 out of assignment 4.
> [ 87.993617] vGT(1) , read fence register 100040, 8 out of assignment 4.
> [ 87.993725] vGT (1) , write fence register 100044, 8 out of assignment 4.
> [ 87.993852] vGT(1) , read fence register 100044, 8 out of assignment 4.
> [ 87.993957] vGT (1) , write fence register 100048, 9 out of assignment 4.
> [ 87.994090] vGT(1) , read fence register 100048, 9 out of assignment 4.
> [ 87.994273] vGT (1) , write fence register 10004c, 9 out of assignment 4.
> [ 87.994430] vGT(1) , read fence register 10004c, 9 out of assignment 4.
> [ 87.994571] vGT (1) , write fence register 100050, a out of assignment 4.
> [ 87.994725] vGT(1) , read fence register 100050, a out of assignment 4.
> [ 87.994864] vGT (1) , write fence register 100054, a out of assignment 4.
> [ 87.995019] vGT(1) , read fence register 100054, a out of assignment 4.
> [ 87.995174] vGT (1) , write fence register 100058, b out of assignment 4.
> [ 87.995330] vGT(1) , read fence register 100058, b out of assignment 4.
> [ 87.995471] vGT (1) , write fence register 10005c, b out of assignment 4.
> [ 87.995623] vGT(1) , read fence register 10005c, b out of assignment 4.
> [ 87.995760] vGT (1) , write fence register 100060, c out of assignment 4.
> [ 87.995915] vGT(1) , read fence register 100060, c out of assignment 4.
> [ 87.996051] vGT (1) , write fence register 100064, c out of assignment 4.
> [ 87.996207] vGT(1) , read fence register 100064, c out of assignment 4.
> [ 87.996345] vGT (1) , write fence register 100068, d out of assignment 4.
> [ 87.996504] vGT(1) , read fence register 100068, d out of assignment 4.
> [ 87.996639] vGT (1) , write fence register 10006c, d out of assignment 4.
> [ 87.996794] vGT(1) , read fence register 10006c, d out of assignment 4.
> [ 87.996932] vGT (1) , write fence register 100070, e out of assignment 4.
> [ 87.997091] vGT(1) , read fence register 100070, e out of assignment 4.
> [ 87.997232] vGT (1) , write fence register 100074, e out of assignment 4.
> [ 87.997386] vGT(1) , read fence register 100074, e out of assignment 4.
> [ 87.997543] vGT (1) , write fence register 100078, f out of assignment 4.
> [ 87.997717] vGT(1) , read fence register 100078, f out of assignment 4.
> [ 87.997859] vGT (1) , write fence register 10007c, f out of assignment 4.
> [ 87.998012] vGT(1) , read fence register 10007c, f out of assignment 4.
> [ 87.998247] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x100080,len=4, val=0x0!!!
> [ 87.998494] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x100080,len=4, val=0x0!!!
> [ 87.998723] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x100084,len=4, val=0x0!!!
> [ 87.998969] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x100084,len=4, val=0x0!!!
> [ 87.999204] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x100088,len=4, val=0x0!!!
> [ 87.999453] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x100088,len=4, val=0x0!!!
> [ 87.999692] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x10008c,len=4, val=0x0!!!
> [ 87.999950] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x10008c,len=4, val=0x0!!!
> [ 88.000186] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x100090,len=4, val=0x0!!!
> [ 88.000447] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x100090,len=4, val=0x0!!!
> [ 88.000684] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x100094,len=4, val=0x0!!!
> [ 88.002447] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x100094,len=4, val=0x0!!!
> [ 88.002618] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x100098,len=4, val=0x0!!!
> [ 88.002792] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x100098,len=4, val=0x0!!!
> [ 88.002953] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x10009c,len=4, val=0x0!!!
> [ 88.003139] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x10009c,len=4, val=0x0!!!
> [ 88.003301] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000a0,len=4, val=0x0!!!
> [ 88.003474] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000a0,len=4, val=0x0!!!
> [ 88.003639] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000a4,len=4, val=0x0!!!
> [ 88.003813] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000a4,len=4, val=0x0!!!
> [ 88.003973] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000a8,len=4, val=0x0!!!
> [ 88.004152] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000a8,len=4, val=0x0!!!
> [ 88.004410] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000ac,len=4, val=0x0!!!
> [ 88.004652] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000ac,len=4, val=0x0!!!
> [ 88.004882] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000b0,len=4, val=0x0!!!
> [ 88.005121] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000b0,len=4, val=0x0!!!
> [ 88.005351] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000b4,len=4, val=0x0!!!
> [ 88.005596] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000b4,len=4, val=0x0!!!
> [ 88.005821] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000b8,len=4, val=0x0!!!
> [ 88.006064] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000b8,len=4, val=0x0!!!
> [ 88.006297] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000bc,len=4, val=0x0!!!
> [ 88.006540] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000bc,len=4, val=0x0!!!
> [ 88.006766] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000c0,len=4, val=0x0!!!
> [ 88.007009] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000c0,len=4, val=0x0!!!
> [ 88.007241] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000c4,len=4, val=0x0!!!
> [ 88.007484] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000c4,len=4, val=0x0!!!
> [ 88.007787] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000c8,len=4, val=0x0!!!
> [ 88.008043] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000c8,len=4, val=0x0!!!
> [ 88.008273] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000cc,len=4, val=0x0!!!
> [ 88.008521] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000cc,len=4, val=0x0!!!
> [ 88.008754] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000d0,len=4, val=0x0!!!
> [ 88.009001] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000d0,len=4, val=0x0!!!
> [ 88.009212] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000d4,len=4, val=0x0!!!
> [ 88.009503] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000d4,len=4, val=0x0!!!
> [ 88.009746] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000d8,len=4, val=0x0!!!
> [ 88.009988] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000d8,len=4, val=0x0!!!
> [ 88.010232] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000dc,len=4, val=0x0!!!
> [ 88.010518] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000dc,len=4, val=0x0!!!
> [ 88.010766] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000e0,len=4, val=0x0!!!
> [ 88.011048] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000e0,len=4, val=0x0!!!
> [ 88.011302] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000e4,len=4, val=0x0!!!
> [ 88.011560] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000e4,len=4, val=0x0!!!
> [ 88.011791] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000e8,len=4, val=0x0!!!
> [ 88.012047] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000e8,len=4, val=0x0!!!
> [ 88.012284] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000ec,len=4, val=0x0!!!
> [ 88.012540] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000ec,len=4, val=0x0!!!
> [ 88.012778] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000f0,len=4, val=0x0!!!
> [ 88.013031] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000f0,len=4, val=0x0!!!
> [ 88.013265] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000f4,len=4, val=0x0!!!
> [ 88.013519] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000f4,len=4, val=0x0!!!
> [ 88.013757] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000f8,len=4, val=0x0!!!
> [ 88.014011] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000f8,len=4, val=0x0!!!
> [ 88.014246] vGT warning:(vgt_emulate_write:452) vGT: untracked MMIO write: vm_id(2), offset=0x1000fc,len=4, val=0x0!!!
> [ 88.014520] vGT warning:(vgt_emulate_read:350) vGT: untracked MMIO read: vm_id(2), offset=0x1000fc,len=4, val=0x0!!!
> [ 88.014742] vGT: write to MSI capa(92) with val (0)
> [ 88.015006] vGT: write to MSI capa(92) with val (0)
> [ 88.015112] vGT: write to MSI capa(94) with val (fee47000)
> [ 88.015210] vGT: write to MSI capa(98) with val (4300)
> [ 88.015442] vGT: write to MSI capa(92) with val (1)
> [ 88.019758] vGT info:(vga_control_w:1790) VM(1): Disable VGA mode 80000000
> [ 88.020988] vGT info:(vgt_handle_default_event_virt:1008) IRQ: VM(2) receive event 58 (AUX Channel A)
> [ 88.102950] vGT warning:(get_sbi_reg_cached_value:1830) vGT(1): SBI reading did not find the cached value for offset 0x1f00. 0 will be returned!
> [ 88.103390] vGT warning:(get_sbi_reg_cached_value:1830) vGT(1): SBI reading did not find the cached value for offset 0x20c. 0 will be returned!
5 years, 10 months
- 1
- 0
Re: [iGVT-g] [Intel-gfx] About the iGVT-g's requirement to pin guest contexts in VM
by Zhiyuan Lv
Hi Daniel,
Thanks for the comments! And my reply in line:
On Wed, Sep 02, 2015 at 10:19:03AM +0200, Daniel Vetter wrote:
> > >
> > > Also you obviously have to complete the copying from shadow->guest ctx
> > > before you send the irq to the guest to signal ctx completion. Which means
> > > there's really no overall problem here from a design pov, the only thing
> >
> > Right. We cannot control when guest driver sees seqno change, but we
> > can control when guest sees context interrupts. The guest CSB update
> > and interrupt injection will be after we finish writing guest
> > contexts.
> >
> > So right now we have two options of context shadowing: one is to track
> > the whole life-cycle of guest context, and another is to do the shadow
> > work in context schedule-in/schedule-out time. Zhi draws a nice
> > picture of them.
> >
> > Currently we do not have concrete performance comparison of the two
> > approaches. We will have a try and see. And about this patchset, I
> > will remove the "context notification" part and send out an updated
> > version. Thanks!
> >
> > > you have to do is fix up bugs in the host code (probably you should just
> > > write through the ggtt).
> >
> > Sorry could you elaborate a little more about this? Guest context may
> > not always be in aperture right?
>
> Yeah the high-level problem is that global gtt is contended (we already
> have trouble with that on xengt and there's the ongoing but unfished
> partial mmap support for that). And permanently pinning execlist contexts
> will cause lots of troubles.
>
> Windows can do this because it segments the global gtt into different
> parts (at least last time I looked at their memory manager), which means
> execlist will never sit in the middle of the range used for mmaps. But
> linux has a unified memory manager, which means execlist can sit anywhere,
> and therefore badly fragment the global gtt. If we pin them then that will
> cause trouble after long system uptime. And afaiui xengt is mostly aimed
> at servers, where the uptime assumption should be "runs forever".
In server side, we do not expect host to run much graphics workloads
(all should be in virtual machines with shorter uptime). But yeah, gm
fragmentation is still an issue. Thanks for the explanation!
>
> Compounding factor is that despite that I raised this in the original
> review execlist is still not yet using the active list in upstream and
> instead does short-time pinning. It's better than pinning forever but
> still breaks the eviction logic.
>
> What Chris Wilson and I talked about forever is adding an object-specific
> global_gtt_unbind hook. The idea is that this would be called when
> unbinding/evicting a special object (e.g. hw context), and you could use
> that to do the host signalling. That would be the perfect combination of
> both approaches:
Thanks for the suggestion! That sounds great! Right now in XenGT part
we still plan to try the shadow context work in context
schedule-in/out time. With this way, we do not need to pin context as
well as the host notification. We will collect some performance data
to see how it works.
I sent out v2 patch set which has removed the pin/unpin of execlist
contexts. The patchset addressed review comments from you, Chris and
Joonas(Sorry I forgot to add CC to related people). Is that patch set
OK to be merged? Thanks!
Regards,
-Zhiyuan
>
> - Fast: host signalling (and therefore shadow context recreation) would
> only be done when the execlist context has actually moved around. That
> almost never happens, and hence per-execbuf overhead would be as low as
> with your pinning solution.
>
> - Flexible: The i915 memory manager is still in full control since we
> don't pin any objects unecessarily.
>
> Cheers, Daniel
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx(a)lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/intel-gfx
5 years, 10 months
- 1
- 0
- ← Newer
- 1
- Older →