I think there is confusion here, Alex properly explained the
PCIE-device do a ATS request to the IOMMU which returns a valid
translation for a virtual address. Device can then use that address
directly without going through IOMMU for translation.
This makes sense and to be honest I now understand ATS and its interaction with ACS a lot
better than I did 24 hours ago ;-).
ATS is implemented by the IOMMU not by the device (well device
the client side of it). Also ATS is meaningless without something like
PASID as far as i know.
I think it's the client side that is what is important to us. Not many EPs support ATS
today and it's not clear if many will in the future. So assuming we want to do p2pdma
between devices (some of) which do NOT support ATS how best do we handle the ACS issue?
Disabling the IOMMU seems a bit strong to me given this impacts all the PCI domains in the
system and not just the domain we wish to do P2P on.