Mailing List Archive

Xen IOMMU disabled due to IVRS table... Blah blah blah
This issue has been discussed many times before, but I haven't found
an answer in my specific problem.

Motherboard: Asus Sabertooth 990fx R2.0
Linux 3.2.0-35 through 3.8.x (tested 5 kernels to be ridiculously thorough)

Affected: Xen-hypervisor >4.1.2
Not Affected: Xen-hypervisor <4.1.2


Xen 4.1.2 and earlier works fine regardless of the BIOS version of the
Mobo. After upgrading Xen hypervisor beyond 4.1.2, we get the
following:
(XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
(XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
(XEN) Table is not found!
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 3110.540 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
(XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
(XEN) IVHD Error: Invalid IO-APIC 0xff
(XEN) AMD-Vi: Error initialization
(XEN) I/O virtualisation disabled


From what I understand, Xen used to just care less about the state of
your ACPI/IVRS but that this was potentially a security concern and
checks were implemented to disable IOMMU whenever it would be unsafe
(potentially) to have it enabled.
Now, I understand the logic here, but I disagree with the
implementation. There are a fairly large number of people who are
affected and we know Asus isn't going to fix the IVRS table. I
recognize the importance of implementing these checks and hell, even
automatically disabling IOMMU without notice (though on a consumer
board, this is a bit paranoid).
I understand there are kernel command line options you can pass to
forcibly bypass the checks and leave IOMMU enabled in most cases (ie:
iommu=no-amd-iommu-perdev-intremap) but thus far none of them seem to
be enough to force IOMMU back on.
I read somewhere in one of the many discussions on the subject, that
there are cases where you simply aren't allowed to disable the checks
(I believe this was when the IVRS table indicies don't line up or
something, which seems to be my issue).

As I said, I know other people have precisely this problem with a
whole line of mid range Asus boards, but haven't seen a lot of
discussion on the list. Anyone reading this run into the same
problem? I bought this board specifically because of it's IOMMU
implementation, which worked fine... at the time... Ultimately I know
for a fact that the boards ACPI/IVRS can't be TOO wrong as it's been
working for 6 months without an issue. I'm just sorta in a situation
where it's time to upgrade my base OS and hacking Xen 4.1.2 in seems
like a silly amount of work when all I really need is an option to
disable the checks.


xm dmesg from a working vs. borked system
Working:
(XEN) Xen version 4.1.2 (Ubuntu 4.1.2-2ubuntu2)
(stefan.bader@canonical.com) (gcc version 4.6.3 (Ubuntu/Linaro
4.6.3-1ubuntu2) ) Thu Mar 8 14:31:11 UTC 2012
(XEN) Bootloader: GRUB 1.99-21ubuntu3.9
(XEN) Command line: pci_msitranslate=0 xen-pciback=passthrough
xen-pciback.hide=(06:00.0)(06.00.1)
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 6 MBR signatures
(XEN) Found 6 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009e800 (usable)
(XEN) 000000000009e800 - 00000000000a0000 (reserved)
(XEN) 00000000000e0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000bc348000 (usable)
(XEN) 00000000bc348000 - 00000000bc78c000 (reserved)
(XEN) 00000000bc78c000 - 00000000bc797000 (ACPI data)
(XEN) 00000000bc797000 - 00000000bd4f8000 (ACPI NVS)
(XEN) 00000000bd4f8000 - 00000000bd8d5000 (reserved)
(XEN) 00000000bd8d5000 - 00000000bd8d6000 (usable)
(XEN) 00000000bd8d6000 - 00000000bdad9000 (ACPI NVS)
(XEN) 00000000bdad9000 - 00000000bdf00000 (usable)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec01000 (reserved)
(XEN) 00000000fec10000 - 00000000fec11000 (reserved)
(XEN) 00000000fec20000 - 00000000fec21000 (reserved)
(XEN) 00000000fed00000 - 00000000fed01000 (reserved)
(XEN) 00000000fed61000 - 00000000fed71000 (reserved)
(XEN) 00000000fed80000 - 00000000fed90000 (reserved)
(XEN) 00000000fef00000 - 0000000100000000 (reserved)
(XEN) 0000000100001000 - 000000043f000000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT BC78E078, 0064 (r1 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: FACP BC795BA8, 00F4 (r4 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has
zero address or length: 0000000000000000/1 [20070126]
(XEN) ACPI: DSDT BC78E170, 7A38 (r2 ALASKA A M I 0 INTL 20051117)
(XEN) ACPI: FACS BD4F2F80, 0040
(XEN) ACPI: APIC BC795CA0, 009E (r3 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: FPDT BC795D40, 0044 (r1 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: MCFG BC795D88, 003C (r1 ALASKA A M I 1072009 MSFT 10013)
(XEN) ACPI: HPET BC795DC8, 0038 (r1 ALASKA A M I 1072009 AMI 5)
(XEN) ACPI: BGRT BC796260, 0038 (r0 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: IVRS BC795E58, 0100 (r1 AMD RD890S 202031 AMD 0)
(XEN) ACPI: SSDT BC795F58, 0304 (r1 AMD POWERNOW 1 AMD 1)
(XEN) System RAM: 16311MB (16702516kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
bd4f2f80/0000000000000000, using 32
(XEN) Processor #16 5:1 APIC version 16
(XEN) Processor #17 5:1 APIC version 16
(XEN) Processor #18 5:1 APIC version 16
(XEN) Processor #19 5:1 APIC version 16
(XEN) Processor #20 5:1 APIC version 16
(XEN) Processor #21 5:1 APIC version 16
(XEN) Processor #22 5:1 APIC version 16
(XEN) Processor #23 5:1 APIC version 16
(XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
(XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
(XEN) Table is not found!
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 3110.553 MHz processor.
(XEN) Initing memory sharing.
(XEN) AMD-Vi: IOMMU 0 Enabled.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) HVM: ASIDs enabled.
(XEN) SVM: Supported advanced features:
(XEN) - Nested Page Tables (NPT)
(XEN) - Last Branch Record (LBR) Virtualisation
(XEN) - Next-RIP Saved on #VMEXIT
(XEN) - VMCB Clean Bits
(XEN) - Pause-Intercept Filter
(XEN) HVM: SVM enabled
(XEN) HVM: Hardware Assisted Paging detected.
(XEN) Brought up 8 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x205f000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4044532
pages to be allocated)
(XEN) Init. ramdisk: 000000043c7dd000->000000043efffa00
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff8205f000
(XEN) Init. ramdisk: ffffffff8205f000->ffffffff84881a00
(XEN) Phys-Mach map: ffffffff84882000->ffffffff867b18b8
(XEN) Start info: ffffffff867b2000->ffffffff867b24b4
(XEN) Page tables: ffffffff867b3000->ffffffff867ec000
(XEN) Boot stack: ffffffff867ec000->ffffffff867ed000
(XEN) TOTAL: ffffffff80000000->ffffffff86c00000
(XEN) ENTRY ADDRESS: ffffffff81cfb200
(XEN) Dom0 has maximum 8 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
switch input to Xen)
(XEN) Freed 220kB init memory.
(XEN) traps.c:2432:d0 Domain attempted WRMSR 00000000c0010201 from
0x0000000000000000 to 0x000000000000abcd.
(XEN) physdev.c:155: dom0: wrong map_pirq type 3


BORKED:
(XEN) Xen version 4.2.1 (Ubuntu 4.2.1-0ubuntu3.1)
(stefan.bader@canonical.com) (gcc (Ubuntu/Linaro 4.7.3-1ubuntu1)
4.7.3) Mon Apr 29 19:35:31 UTC 2013
(XEN) Bootloader: GRUB 2.00-13ubuntu3
(XEN) Command line: iommu=no-amd-iommu-perdev-intremap x2apic=off
xen-pciback.hide=(06:00.0)(06.00.1)
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 7 MBR signatures
(XEN) Found 6 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009e800 (usable)
(XEN) 000000000009e800 - 00000000000a0000 (reserved)
(XEN) 00000000000e0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000ba7ac000 (usable)
(XEN) 00000000ba7ac000 - 00000000babe0000 (reserved)
(XEN) 00000000babe0000 - 00000000babf0000 (ACPI data)
(XEN) 00000000babf0000 - 00000000bb958000 (ACPI NVS)
(XEN) 00000000bb958000 - 00000000bca35000 (reserved)
(XEN) 00000000bca35000 - 00000000bca36000 (usable)
(XEN) 00000000bca36000 - 00000000bcc3c000 (ACPI NVS)
(XEN) 00000000bcc3c000 - 00000000bd083000 (usable)
(XEN) 00000000bd083000 - 00000000bd7f4000 (reserved)
(XEN) 00000000bd7f4000 - 00000000bd800000 (usable)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec01000 (reserved)
(XEN) 00000000fec10000 - 00000000fec11000 (reserved)
(XEN) 00000000fec20000 - 00000000fec21000 (reserved)
(XEN) 00000000fed00000 - 00000000fed01000 (reserved)
(XEN) 00000000fed61000 - 00000000fed71000 (reserved)
(XEN) 00000000fed80000 - 00000000fed90000 (reserved)
(XEN) 00000000fef00000 - 0000000100000000 (reserved)
(XEN) 0000000100001000 - 000000043f000000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT BABE7078, 0064 (r1 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: FACP BABEE118, 010C (r5 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI Warning (tbfadt-0232): FADT (revision 5) is longer than
ACPI 2.0 version, truncating length 0x10C to 0xF4 [20070126]
(XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has
zero address or length: 0000000000000000/1 [20070126]
(XEN) ACPI: DSDT BABE7170, 6FA8 (r2 ALASKA A M I 0 INTL 20051117)
(XEN) ACPI: FACS BB952F80, 0040
(XEN) ACPI: APIC BABEE228, 009E (r3 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: FPDT BABEE2C8, 0044 (r1 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: MCFG BABEE310, 003C (r1 ALASKA A M I 1072009 MSFT 10013)
(XEN) ACPI: HPET BABEE350, 0038 (r1 ALASKA A M I 1072009 AMI 5)
(XEN) ACPI: BGRT BABEFBF8, 0038 (r0 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: IVRS BABEE3E0, 0100 (r1 AMD RD890S 202031 AMD 0)
(XEN) ACPI: SSDT BABEE4E0, 1714 (r1 AMD POWERNOW 1 AMD 1)
(XEN) System RAM: 16283MB (16674420kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
bb952f80/0000000000000000, using 32
(XEN) Processor #16 5:1 APIC version 16
(XEN) Processor #17 5:1 APIC version 16
(XEN) Processor #18 5:1 APIC version 16
(XEN) Processor #19 5:1 APIC version 16
(XEN) Processor #20 5:1 APIC version 16
(XEN) Processor #21 5:1 APIC version 16
(XEN) Processor #22 5:1 APIC version 16
(XEN) Processor #23 5:1 APIC version 16
(XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
(XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
(XEN) Table is not found!
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 3110.540 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
(XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
(XEN) IVHD Error: Invalid IO-APIC 0xff
(XEN) AMD-Vi: Error initialization
(XEN) I/O virtualisation disabled
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) HVM: ASIDs enabled.
(XEN) SVM: Supported advanced features:
(XEN) - Nested Page Tables (NPT)
(XEN) - Last Branch Record (LBR) Virtualisation
(XEN) - Next-RIP Saved on #VMEXIT
(XEN) - VMCB Clean Bits
(XEN) - DecodeAssists
(XEN) - Pause-Intercept Filter
(XEN) - TSC Rate MSR
(XEN) HVM: SVM enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 8 CPUs
(XEN) mtrr: your CPUs had inconsistent variable MTRR settings
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2346000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4035583
pages to be allocated)
(XEN) Init. ramdisk: 0000000439dc6000->000000043efff800
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff82346000
(XEN) Init. ramdisk: ffffffff82346000->ffffffff8757f800
(XEN) Phys-Mach map: ffffffff87580000->ffffffff894b31c8
(XEN) Start info: ffffffff894b4000->ffffffff894b44b4
(XEN) Page tables: ffffffff894b5000->ffffffff89504000
(XEN) Boot stack: ffffffff89504000->ffffffff89505000
(XEN) TOTAL: ffffffff80000000->ffffffff89800000
(XEN) ENTRY ADDRESS: ffffffff81d06210
(XEN) Dom0 has maximum 8 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
switch input to Xen)
(XEN) Freed 244kB init memory.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
Feral,

I have the R1.0 version of that motherboard. The problem is with the
BIOS; its returning bad information in the IVRS table for the IO-APICs,
which the new parser is catching and causing AMD-Vi initialization to fail.
I've already put in a support ticket via the online form, included all the
information such as links to the document which shows what the valid
details should be. The ticket was basically closed with a note saying to
look out for BIOS updates. I'm currently using 4.2.1 (which was before the
table checking was added) and am able to use AMD-Vi. If you want to fix
this, you're either going to have to get ASUS to issue a fixed BIOS (which
doesn't feel likely at this point) or you'd have to either disable
interrupt remapping or edit the source code to disable that check (and deal
with any potential issue that might cause later)

Regards,

David


On Mon, May 27, 2013 at 11:42 PM, feral <blistovmhz@gmail.com> wrote:

> This issue has been discussed many times before, but I haven't found
> an answer in my specific problem.
>
> Motherboard: Asus Sabertooth 990fx R2.0
> Linux 3.2.0-35 through 3.8.x (tested 5 kernels to be ridiculously thorough)
>
> Affected: Xen-hypervisor >4.1.2
> Not Affected: Xen-hypervisor <4.1.2
>
>
> Xen 4.1.2 and earlier works fine regardless of the BIOS version of the
> Mobo. After upgrading Xen hypervisor beyond 4.1.2, we get the
> following:
> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
> (XEN) Table is not found!
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 3110.540 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
> (XEN) IVHD Error: Invalid IO-APIC 0xff
> (XEN) AMD-Vi: Error initialization
> (XEN) I/O virtualisation disabled
>
>
> From what I understand, Xen used to just care less about the state of
> your ACPI/IVRS but that this was potentially a security concern and
> checks were implemented to disable IOMMU whenever it would be unsafe
> (potentially) to have it enabled.
> Now, I understand the logic here, but I disagree with the
> implementation. There are a fairly large number of people who are
> affected and we know Asus isn't going to fix the IVRS table. I
> recognize the importance of implementing these checks and hell, even
> automatically disabling IOMMU without notice (though on a consumer
> board, this is a bit paranoid).
> I understand there are kernel command line options you can pass to
> forcibly bypass the checks and leave IOMMU enabled in most cases (ie:
> iommu=no-amd-iommu-perdev-intremap) but thus far none of them seem to
> be enough to force IOMMU back on.
> I read somewhere in one of the many discussions on the subject, that
> there are cases where you simply aren't allowed to disable the checks
> (I believe this was when the IVRS table indicies don't line up or
> something, which seems to be my issue).
>
> As I said, I know other people have precisely this problem with a
> whole line of mid range Asus boards, but haven't seen a lot of
> discussion on the list. Anyone reading this run into the same
> problem? I bought this board specifically because of it's IOMMU
> implementation, which worked fine... at the time... Ultimately I know
> for a fact that the boards ACPI/IVRS can't be TOO wrong as it's been
> working for 6 months without an issue. I'm just sorta in a situation
> where it's time to upgrade my base OS and hacking Xen 4.1.2 in seems
> like a silly amount of work when all I really need is an option to
> disable the checks.
>
>
> xm dmesg from a working vs. borked system
> Working:
> (XEN) Xen version 4.1.2 (Ubuntu 4.1.2-2ubuntu2)
> (stefan.bader@canonical.com) (gcc version 4.6.3 (Ubuntu/Linaro
> 4.6.3-1ubuntu2) ) Thu Mar 8 14:31:11 UTC 2012
> (XEN) Bootloader: GRUB 1.99-21ubuntu3.9
> (XEN) Command line: pci_msitranslate=0 xen-pciback=passthrough
> xen-pciback.hide=(06:00.0)(06.00.1)
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
> (XEN) Disc information:
> (XEN) Found 6 MBR signatures
> (XEN) Found 6 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009e800 (usable)
> (XEN) 000000000009e800 - 00000000000a0000 (reserved)
> (XEN) 00000000000e0000 - 0000000000100000 (reserved)
> (XEN) 0000000000100000 - 00000000bc348000 (usable)
> (XEN) 00000000bc348000 - 00000000bc78c000 (reserved)
> (XEN) 00000000bc78c000 - 00000000bc797000 (ACPI data)
> (XEN) 00000000bc797000 - 00000000bd4f8000 (ACPI NVS)
> (XEN) 00000000bd4f8000 - 00000000bd8d5000 (reserved)
> (XEN) 00000000bd8d5000 - 00000000bd8d6000 (usable)
> (XEN) 00000000bd8d6000 - 00000000bdad9000 (ACPI NVS)
> (XEN) 00000000bdad9000 - 00000000bdf00000 (usable)
> (XEN) 00000000f8000000 - 00000000fc000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec01000 (reserved)
> (XEN) 00000000fec10000 - 00000000fec11000 (reserved)
> (XEN) 00000000fec20000 - 00000000fec21000 (reserved)
> (XEN) 00000000fed00000 - 00000000fed01000 (reserved)
> (XEN) 00000000fed61000 - 00000000fed71000 (reserved)
> (XEN) 00000000fed80000 - 00000000fed90000 (reserved)
> (XEN) 00000000fef00000 - 0000000100000000 (reserved)
> (XEN) 0000000100001000 - 000000043f000000 (usable)
> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
> (XEN) ACPI: XSDT BC78E078, 0064 (r1 ALASKA A M I 1072009 AMI 10013)
> (XEN) ACPI: FACP BC795BA8, 00F4 (r4 ALASKA A M I 1072009 AMI 10013)
> (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has
> zero address or length: 0000000000000000/1 [20070126]
> (XEN) ACPI: DSDT BC78E170, 7A38 (r2 ALASKA A M I 0 INTL 20051117)
> (XEN) ACPI: FACS BD4F2F80, 0040
> (XEN) ACPI: APIC BC795CA0, 009E (r3 ALASKA A M I 1072009 AMI 10013)
> (XEN) ACPI: FPDT BC795D40, 0044 (r1 ALASKA A M I 1072009 AMI 10013)
> (XEN) ACPI: MCFG BC795D88, 003C (r1 ALASKA A M I 1072009 MSFT 10013)
> (XEN) ACPI: HPET BC795DC8, 0038 (r1 ALASKA A M I 1072009 AMI 5)
> (XEN) ACPI: BGRT BC796260, 0038 (r0 ALASKA A M I 1072009 AMI 10013)
> (XEN) ACPI: IVRS BC795E58, 0100 (r1 AMD RD890S 202031 AMD 0)
> (XEN) ACPI: SSDT BC795F58, 0304 (r1 AMD POWERNOW 1 AMD 1)
> (XEN) System RAM: 16311MB (16702516kB)
> (XEN) Domain heap initialised
> (XEN) ACPI: 32/64X FACS address mismatch in FADT -
> bd4f2f80/0000000000000000, using 32
> (XEN) Processor #16 5:1 APIC version 16
> (XEN) Processor #17 5:1 APIC version 16
> (XEN) Processor #18 5:1 APIC version 16
> (XEN) Processor #19 5:1 APIC version 16
> (XEN) Processor #20 5:1 APIC version 16
> (XEN) Processor #21 5:1 APIC version 16
> (XEN) Processor #22 5:1 APIC version 16
> (XEN) Processor #23 5:1 APIC version 16
> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
> (XEN) Table is not found!
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 3110.553 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) AMD-Vi: IOMMU 0 Enabled.
> (XEN) I/O virtualisation enabled
> (XEN) - Dom0 mode: Relaxed
> (XEN) ENABLING IO-APIC IRQs
> (XEN) -> Using new ACK method
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Allocated console ring of 16 KiB.
> (XEN) HVM: ASIDs enabled.
> (XEN) SVM: Supported advanced features:
> (XEN) - Nested Page Tables (NPT)
> (XEN) - Last Branch Record (LBR) Virtualisation
> (XEN) - Next-RIP Saved on #VMEXIT
> (XEN) - VMCB Clean Bits
> (XEN) - Pause-Intercept Filter
> (XEN) HVM: SVM enabled
> (XEN) HVM: Hardware Assisted Paging detected.
> (XEN) Brought up 8 CPUs
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Xen kernel: 64-bit, lsb, compat32
> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x205f000
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4044532
> pages to be allocated)
> (XEN) Init. ramdisk: 000000043c7dd000->000000043efffa00
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN) Loaded kernel: ffffffff81000000->ffffffff8205f000
> (XEN) Init. ramdisk: ffffffff8205f000->ffffffff84881a00
> (XEN) Phys-Mach map: ffffffff84882000->ffffffff867b18b8
> (XEN) Start info: ffffffff867b2000->ffffffff867b24b4
> (XEN) Page tables: ffffffff867b3000->ffffffff867ec000
> (XEN) Boot stack: ffffffff867ec000->ffffffff867ed000
> (XEN) TOTAL: ffffffff80000000->ffffffff86c00000
> (XEN) ENTRY ADDRESS: ffffffff81cfb200
> (XEN) Dom0 has maximum 8 VCPUs
> (XEN) Scrubbing Free RAM: .done.
> (XEN) Xen trace buffers: disabled
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
> switch input to Xen)
> (XEN) Freed 220kB init memory.
> (XEN) traps.c:2432:d0 Domain attempted WRMSR 00000000c0010201 from
> 0x0000000000000000 to 0x000000000000abcd.
> (XEN) physdev.c:155: dom0: wrong map_pirq type 3
>
>
> BORKED:
> (XEN) Xen version 4.2.1 (Ubuntu 4.2.1-0ubuntu3.1)
> (stefan.bader@canonical.com) (gcc (Ubuntu/Linaro 4.7.3-1ubuntu1)
> 4.7.3) Mon Apr 29 19:35:31 UTC 2013
> (XEN) Bootloader: GRUB 2.00-13ubuntu3
> (XEN) Command line: iommu=no-amd-iommu-perdev-intremap x2apic=off
> xen-pciback.hide=(06:00.0)(06.00.1)
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
> (XEN) Disc information:
> (XEN) Found 7 MBR signatures
> (XEN) Found 6 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009e800 (usable)
> (XEN) 000000000009e800 - 00000000000a0000 (reserved)
> (XEN) 00000000000e0000 - 0000000000100000 (reserved)
> (XEN) 0000000000100000 - 00000000ba7ac000 (usable)
> (XEN) 00000000ba7ac000 - 00000000babe0000 (reserved)
> (XEN) 00000000babe0000 - 00000000babf0000 (ACPI data)
> (XEN) 00000000babf0000 - 00000000bb958000 (ACPI NVS)
> (XEN) 00000000bb958000 - 00000000bca35000 (reserved)
> (XEN) 00000000bca35000 - 00000000bca36000 (usable)
> (XEN) 00000000bca36000 - 00000000bcc3c000 (ACPI NVS)
> (XEN) 00000000bcc3c000 - 00000000bd083000 (usable)
> (XEN) 00000000bd083000 - 00000000bd7f4000 (reserved)
> (XEN) 00000000bd7f4000 - 00000000bd800000 (usable)
> (XEN) 00000000f8000000 - 00000000fc000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec01000 (reserved)
> (XEN) 00000000fec10000 - 00000000fec11000 (reserved)
> (XEN) 00000000fec20000 - 00000000fec21000 (reserved)
> (XEN) 00000000fed00000 - 00000000fed01000 (reserved)
> (XEN) 00000000fed61000 - 00000000fed71000 (reserved)
> (XEN) 00000000fed80000 - 00000000fed90000 (reserved)
> (XEN) 00000000fef00000 - 0000000100000000 (reserved)
> (XEN) 0000000100001000 - 000000043f000000 (usable)
> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
> (XEN) ACPI: XSDT BABE7078, 0064 (r1 ALASKA A M I 1072009 AMI 10013)
> (XEN) ACPI: FACP BABEE118, 010C (r5 ALASKA A M I 1072009 AMI 10013)
> (XEN) ACPI Warning (tbfadt-0232): FADT (revision 5) is longer than
> ACPI 2.0 version, truncating length 0x10C to 0xF4 [20070126]
> (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has
> zero address or length: 0000000000000000/1 [20070126]
> (XEN) ACPI: DSDT BABE7170, 6FA8 (r2 ALASKA A M I 0 INTL 20051117)
> (XEN) ACPI: FACS BB952F80, 0040
> (XEN) ACPI: APIC BABEE228, 009E (r3 ALASKA A M I 1072009 AMI 10013)
> (XEN) ACPI: FPDT BABEE2C8, 0044 (r1 ALASKA A M I 1072009 AMI 10013)
> (XEN) ACPI: MCFG BABEE310, 003C (r1 ALASKA A M I 1072009 MSFT 10013)
> (XEN) ACPI: HPET BABEE350, 0038 (r1 ALASKA A M I 1072009 AMI 5)
> (XEN) ACPI: BGRT BABEFBF8, 0038 (r0 ALASKA A M I 1072009 AMI 10013)
> (XEN) ACPI: IVRS BABEE3E0, 0100 (r1 AMD RD890S 202031 AMD 0)
> (XEN) ACPI: SSDT BABEE4E0, 1714 (r1 AMD POWERNOW 1 AMD 1)
> (XEN) System RAM: 16283MB (16674420kB)
> (XEN) Domain heap initialised
> (XEN) ACPI: 32/64X FACS address mismatch in FADT -
> bb952f80/0000000000000000, using 32
> (XEN) Processor #16 5:1 APIC version 16
> (XEN) Processor #17 5:1 APIC version 16
> (XEN) Processor #18 5:1 APIC version 16
> (XEN) Processor #19 5:1 APIC version 16
> (XEN) Processor #20 5:1 APIC version 16
> (XEN) Processor #21 5:1 APIC version 16
> (XEN) Processor #22 5:1 APIC version 16
> (XEN) Processor #23 5:1 APIC version 16
> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
> (XEN) Table is not found!
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 3110.540 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
> (XEN) IVHD Error: Invalid IO-APIC 0xff
> (XEN) AMD-Vi: Error initialization
> (XEN) I/O virtualisation disabled
> (XEN) ENABLING IO-APIC IRQs
> (XEN) -> Using new ACK method
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Allocated console ring of 16 KiB.
> (XEN) HVM: ASIDs enabled.
> (XEN) SVM: Supported advanced features:
> (XEN) - Nested Page Tables (NPT)
> (XEN) - Last Branch Record (LBR) Virtualisation
> (XEN) - Next-RIP Saved on #VMEXIT
> (XEN) - VMCB Clean Bits
> (XEN) - DecodeAssists
> (XEN) - Pause-Intercept Filter
> (XEN) - TSC Rate MSR
> (XEN) HVM: SVM enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) detected
> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> (XEN) Brought up 8 CPUs
> (XEN) mtrr: your CPUs had inconsistent variable MTRR settings
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Xen kernel: 64-bit, lsb, compat32
> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2346000
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4035583
> pages to be allocated)
> (XEN) Init. ramdisk: 0000000439dc6000->000000043efff800
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN) Loaded kernel: ffffffff81000000->ffffffff82346000
> (XEN) Init. ramdisk: ffffffff82346000->ffffffff8757f800
> (XEN) Phys-Mach map: ffffffff87580000->ffffffff894b31c8
> (XEN) Start info: ffffffff894b4000->ffffffff894b44b4
> (XEN) Page tables: ffffffff894b5000->ffffffff89504000
> (XEN) Boot stack: ffffffff89504000->ffffffff89505000
> (XEN) TOTAL: ffffffff80000000->ffffffff89800000
> (XEN) ENTRY ADDRESS: ffffffff81d06210
> (XEN) Dom0 has maximum 8 VCPUs
> (XEN) Scrubbing Free RAM: .done.
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
> switch input to Xen)
> (XEN) Freed 244kB init memory.
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
David and feral,

I have an Asus M5A99FX Pro 2.0 motherboard and, at least as far as I can
tell, IOMMU works well with that motherboard and Xen 4.1. (I got VGA
passthrough working with a Windows guest.) Although I did some
preliminary testing with Xen 4.2 on that motherboard, because of some other
issues, I never got around to creating a VM that required IOMMU support,
and so can't say for certain whether IOMMU is working with that motherboard
and Xen 4.2.

Apart from creating a VM that requires IOMMU support (which would take more
time than I'd prefer to spend right now) to see if it works, what would
good way to determine whether my Asus M5A99FX Pro 2.0 motherboard suffers,
or if free from, the IVRS table issue that you describe?

Thanks much for any help with this!

Best regards,
GizmoChicken


On Tue, May 28, 2013 at 3:31 AM, David Sutton <kantras@gmail.com> wrote:

> Feral,
>
> I have the R1.0 version of that motherboard. The problem is with the
> BIOS; its returning bad information in the IVRS table for the IO-APICs,
> which the new parser is catching and causing AMD-Vi initialization to fail.
> I've already put in a support ticket via the online form, included all the
> information such as links to the document which shows what the valid
> details should be. The ticket was basically closed with a note saying to
> look out for BIOS updates. I'm currently using 4.2.1 (which was before the
> table checking was added) and am able to use AMD-Vi. If you want to fix
> this, you're either going to have to get ASUS to issue a fixed BIOS (which
> doesn't feel likely at this point) or you'd have to either disable
> interrupt remapping or edit the source code to disable that check (and deal
> with any potential issue that might cause later)
>
> Regards,
>
> David
>
>
> On Mon, May 27, 2013 at 11:42 PM, feral <blistovmhz@gmail.com> wrote:
>
>> This issue has been discussed many times before, but I haven't found
>> an answer in my specific problem.
>>
>> Motherboard: Asus Sabertooth 990fx R2.0
>> Linux 3.2.0-35 through 3.8.x (tested 5 kernels to be ridiculously
>> thorough)
>>
>> Affected: Xen-hypervisor >4.1.2
>> Not Affected: Xen-hypervisor <4.1.2
>>
>>
>> Xen 4.1.2 and earlier works fine regardless of the BIOS version of the
>> Mobo. After upgrading Xen hypervisor beyond 4.1.2, we get the
>> following:
>> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
>> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
>> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
>> (XEN) Table is not found!
>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>> (XEN) Detected 3110.540 MHz processor.
>> (XEN) Initing memory sharing.
>> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
>> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
>> (XEN) IVHD Error: Invalid IO-APIC 0xff
>> (XEN) AMD-Vi: Error initialization
>> (XEN) I/O virtualisation disabled
>>
>>
>> From what I understand, Xen used to just care less about the state of
>> your ACPI/IVRS but that this was potentially a security concern and
>> checks were implemented to disable IOMMU whenever it would be unsafe
>> (potentially) to have it enabled.
>> Now, I understand the logic here, but I disagree with the
>> implementation. There are a fairly large number of people who are
>> affected and we know Asus isn't going to fix the IVRS table. I
>> recognize the importance of implementing these checks and hell, even
>> automatically disabling IOMMU without notice (though on a consumer
>> board, this is a bit paranoid).
>> I understand there are kernel command line options you can pass to
>> forcibly bypass the checks and leave IOMMU enabled in most cases (ie:
>> iommu=no-amd-iommu-perdev-intremap) but thus far none of them seem to
>> be enough to force IOMMU back on.
>> I read somewhere in one of the many discussions on the subject, that
>> there are cases where you simply aren't allowed to disable the checks
>> (I believe this was when the IVRS table indicies don't line up or
>> something, which seems to be my issue).
>>
>> As I said, I know other people have precisely this problem with a
>> whole line of mid range Asus boards, but haven't seen a lot of
>> discussion on the list. Anyone reading this run into the same
>> problem? I bought this board specifically because of it's IOMMU
>> implementation, which worked fine... at the time... Ultimately I know
>> for a fact that the boards ACPI/IVRS can't be TOO wrong as it's been
>> working for 6 months without an issue. I'm just sorta in a situation
>> where it's time to upgrade my base OS and hacking Xen 4.1.2 in seems
>> like a silly amount of work when all I really need is an option to
>> disable the checks.
>>
>>
>> xm dmesg from a working vs. borked system
>> Working:
>> (XEN) Xen version 4.1.2 (Ubuntu 4.1.2-2ubuntu2)
>> (stefan.bader@canonical.com) (gcc version 4.6.3 (Ubuntu/Linaro
>> 4.6.3-1ubuntu2) ) Thu Mar 8 14:31:11 UTC 2012
>> (XEN) Bootloader: GRUB 1.99-21ubuntu3.9
>> (XEN) Command line: pci_msitranslate=0 xen-pciback=passthrough
>> xen-pciback.hide=(06:00.0)(06.00.1)
>> (XEN) Video information:
>> (XEN) VGA is text mode 80x25, font 8x16
>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
>> (XEN) Disc information:
>> (XEN) Found 6 MBR signatures
>> (XEN) Found 6 EDD information structures
>> (XEN) Xen-e820 RAM map:
>> (XEN) 0000000000000000 - 000000000009e800 (usable)
>> (XEN) 000000000009e800 - 00000000000a0000 (reserved)
>> (XEN) 00000000000e0000 - 0000000000100000 (reserved)
>> (XEN) 0000000000100000 - 00000000bc348000 (usable)
>> (XEN) 00000000bc348000 - 00000000bc78c000 (reserved)
>> (XEN) 00000000bc78c000 - 00000000bc797000 (ACPI data)
>> (XEN) 00000000bc797000 - 00000000bd4f8000 (ACPI NVS)
>> (XEN) 00000000bd4f8000 - 00000000bd8d5000 (reserved)
>> (XEN) 00000000bd8d5000 - 00000000bd8d6000 (usable)
>> (XEN) 00000000bd8d6000 - 00000000bdad9000 (ACPI NVS)
>> (XEN) 00000000bdad9000 - 00000000bdf00000 (usable)
>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved)
>> (XEN) 00000000fec00000 - 00000000fec01000 (reserved)
>> (XEN) 00000000fec10000 - 00000000fec11000 (reserved)
>> (XEN) 00000000fec20000 - 00000000fec21000 (reserved)
>> (XEN) 00000000fed00000 - 00000000fed01000 (reserved)
>> (XEN) 00000000fed61000 - 00000000fed71000 (reserved)
>> (XEN) 00000000fed80000 - 00000000fed90000 (reserved)
>> (XEN) 00000000fef00000 - 0000000100000000 (reserved)
>> (XEN) 0000000100001000 - 000000043f000000 (usable)
>> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
>> (XEN) ACPI: XSDT BC78E078, 0064 (r1 ALASKA A M I 1072009 AMI
>> 10013)
>> (XEN) ACPI: FACP BC795BA8, 00F4 (r4 ALASKA A M I 1072009 AMI
>> 10013)
>> (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has
>> zero address or length: 0000000000000000/1 [20070126]
>> (XEN) ACPI: DSDT BC78E170, 7A38 (r2 ALASKA A M I 0 INTL
>> 20051117)
>> (XEN) ACPI: FACS BD4F2F80, 0040
>> (XEN) ACPI: APIC BC795CA0, 009E (r3 ALASKA A M I 1072009 AMI
>> 10013)
>> (XEN) ACPI: FPDT BC795D40, 0044 (r1 ALASKA A M I 1072009 AMI
>> 10013)
>> (XEN) ACPI: MCFG BC795D88, 003C (r1 ALASKA A M I 1072009 MSFT
>> 10013)
>> (XEN) ACPI: HPET BC795DC8, 0038 (r1 ALASKA A M I 1072009 AMI
>> 5)
>> (XEN) ACPI: BGRT BC796260, 0038 (r0 ALASKA A M I 1072009 AMI
>> 10013)
>> (XEN) ACPI: IVRS BC795E58, 0100 (r1 AMD RD890S 202031 AMD
>> 0)
>> (XEN) ACPI: SSDT BC795F58, 0304 (r1 AMD POWERNOW 1 AMD
>> 1)
>> (XEN) System RAM: 16311MB (16702516kB)
>> (XEN) Domain heap initialised
>> (XEN) ACPI: 32/64X FACS address mismatch in FADT -
>> bd4f2f80/0000000000000000, using 32
>> (XEN) Processor #16 5:1 APIC version 16
>> (XEN) Processor #17 5:1 APIC version 16
>> (XEN) Processor #18 5:1 APIC version 16
>> (XEN) Processor #19 5:1 APIC version 16
>> (XEN) Processor #20 5:1 APIC version 16
>> (XEN) Processor #21 5:1 APIC version 16
>> (XEN) Processor #22 5:1 APIC version 16
>> (XEN) Processor #23 5:1 APIC version 16
>> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
>> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
>> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
>> (XEN) Table is not found!
>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>> (XEN) Detected 3110.553 MHz processor.
>> (XEN) Initing memory sharing.
>> (XEN) AMD-Vi: IOMMU 0 Enabled.
>> (XEN) I/O virtualisation enabled
>> (XEN) - Dom0 mode: Relaxed
>> (XEN) ENABLING IO-APIC IRQs
>> (XEN) -> Using new ACK method
>> (XEN) Platform timer is 14.318MHz HPET
>> (XEN) Allocated console ring of 16 KiB.
>> (XEN) HVM: ASIDs enabled.
>> (XEN) SVM: Supported advanced features:
>> (XEN) - Nested Page Tables (NPT)
>> (XEN) - Last Branch Record (LBR) Virtualisation
>> (XEN) - Next-RIP Saved on #VMEXIT
>> (XEN) - VMCB Clean Bits
>> (XEN) - Pause-Intercept Filter
>> (XEN) HVM: SVM enabled
>> (XEN) HVM: Hardware Assisted Paging detected.
>> (XEN) Brought up 8 CPUs
>> (XEN) *** LOADING DOMAIN 0 ***
>> (XEN) Xen kernel: 64-bit, lsb, compat32
>> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x205f000
>> (XEN) PHYSICAL MEMORY ARRANGEMENT:
>> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4044532
>> pages to be allocated)
>> (XEN) Init. ramdisk: 000000043c7dd000->000000043efffa00
>> (XEN) VIRTUAL MEMORY ARRANGEMENT:
>> (XEN) Loaded kernel: ffffffff81000000->ffffffff8205f000
>> (XEN) Init. ramdisk: ffffffff8205f000->ffffffff84881a00
>> (XEN) Phys-Mach map: ffffffff84882000->ffffffff867b18b8
>> (XEN) Start info: ffffffff867b2000->ffffffff867b24b4
>> (XEN) Page tables: ffffffff867b3000->ffffffff867ec000
>> (XEN) Boot stack: ffffffff867ec000->ffffffff867ed000
>> (XEN) TOTAL: ffffffff80000000->ffffffff86c00000
>> (XEN) ENTRY ADDRESS: ffffffff81cfb200
>> (XEN) Dom0 has maximum 8 VCPUs
>> (XEN) Scrubbing Free RAM: .done.
>> (XEN) Xen trace buffers: disabled
>> (XEN) Std. Loglevel: Errors and warnings
>> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
>> (XEN) Xen is relinquishing VGA console.
>> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
>> switch input to Xen)
>> (XEN) Freed 220kB init memory.
>> (XEN) traps.c:2432:d0 Domain attempted WRMSR 00000000c0010201 from
>> 0x0000000000000000 to 0x000000000000abcd.
>> (XEN) physdev.c:155: dom0: wrong map_pirq type 3
>>
>>
>> BORKED:
>> (XEN) Xen version 4.2.1 (Ubuntu 4.2.1-0ubuntu3.1)
>> (stefan.bader@canonical.com) (gcc (Ubuntu/Linaro 4.7.3-1ubuntu1)
>> 4.7.3) Mon Apr 29 19:35:31 UTC 2013
>> (XEN) Bootloader: GRUB 2.00-13ubuntu3
>> (XEN) Command line: iommu=no-amd-iommu-perdev-intremap x2apic=off
>> xen-pciback.hide=(06:00.0)(06.00.1)
>> (XEN) Video information:
>> (XEN) VGA is text mode 80x25, font 8x16
>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
>> (XEN) Disc information:
>> (XEN) Found 7 MBR signatures
>> (XEN) Found 6 EDD information structures
>> (XEN) Xen-e820 RAM map:
>> (XEN) 0000000000000000 - 000000000009e800 (usable)
>> (XEN) 000000000009e800 - 00000000000a0000 (reserved)
>> (XEN) 00000000000e0000 - 0000000000100000 (reserved)
>> (XEN) 0000000000100000 - 00000000ba7ac000 (usable)
>> (XEN) 00000000ba7ac000 - 00000000babe0000 (reserved)
>> (XEN) 00000000babe0000 - 00000000babf0000 (ACPI data)
>> (XEN) 00000000babf0000 - 00000000bb958000 (ACPI NVS)
>> (XEN) 00000000bb958000 - 00000000bca35000 (reserved)
>> (XEN) 00000000bca35000 - 00000000bca36000 (usable)
>> (XEN) 00000000bca36000 - 00000000bcc3c000 (ACPI NVS)
>> (XEN) 00000000bcc3c000 - 00000000bd083000 (usable)
>> (XEN) 00000000bd083000 - 00000000bd7f4000 (reserved)
>> (XEN) 00000000bd7f4000 - 00000000bd800000 (usable)
>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved)
>> (XEN) 00000000fec00000 - 00000000fec01000 (reserved)
>> (XEN) 00000000fec10000 - 00000000fec11000 (reserved)
>> (XEN) 00000000fec20000 - 00000000fec21000 (reserved)
>> (XEN) 00000000fed00000 - 00000000fed01000 (reserved)
>> (XEN) 00000000fed61000 - 00000000fed71000 (reserved)
>> (XEN) 00000000fed80000 - 00000000fed90000 (reserved)
>> (XEN) 00000000fef00000 - 0000000100000000 (reserved)
>> (XEN) 0000000100001000 - 000000043f000000 (usable)
>> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
>> (XEN) ACPI: XSDT BABE7078, 0064 (r1 ALASKA A M I 1072009 AMI
>> 10013)
>> (XEN) ACPI: FACP BABEE118, 010C (r5 ALASKA A M I 1072009 AMI
>> 10013)
>> (XEN) ACPI Warning (tbfadt-0232): FADT (revision 5) is longer than
>> ACPI 2.0 version, truncating length 0x10C to 0xF4 [20070126]
>> (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has
>> zero address or length: 0000000000000000/1 [20070126]
>> (XEN) ACPI: DSDT BABE7170, 6FA8 (r2 ALASKA A M I 0 INTL
>> 20051117)
>> (XEN) ACPI: FACS BB952F80, 0040
>> (XEN) ACPI: APIC BABEE228, 009E (r3 ALASKA A M I 1072009 AMI
>> 10013)
>> (XEN) ACPI: FPDT BABEE2C8, 0044 (r1 ALASKA A M I 1072009 AMI
>> 10013)
>> (XEN) ACPI: MCFG BABEE310, 003C (r1 ALASKA A M I 1072009 MSFT
>> 10013)
>> (XEN) ACPI: HPET BABEE350, 0038 (r1 ALASKA A M I 1072009 AMI
>> 5)
>> (XEN) ACPI: BGRT BABEFBF8, 0038 (r0 ALASKA A M I 1072009 AMI
>> 10013)
>> (XEN) ACPI: IVRS BABEE3E0, 0100 (r1 AMD RD890S 202031 AMD
>> 0)
>> (XEN) ACPI: SSDT BABEE4E0, 1714 (r1 AMD POWERNOW 1 AMD
>> 1)
>> (XEN) System RAM: 16283MB (16674420kB)
>> (XEN) Domain heap initialised
>> (XEN) ACPI: 32/64X FACS address mismatch in FADT -
>> bb952f80/0000000000000000, using 32
>> (XEN) Processor #16 5:1 APIC version 16
>> (XEN) Processor #17 5:1 APIC version 16
>> (XEN) Processor #18 5:1 APIC version 16
>> (XEN) Processor #19 5:1 APIC version 16
>> (XEN) Processor #20 5:1 APIC version 16
>> (XEN) Processor #21 5:1 APIC version 16
>> (XEN) Processor #22 5:1 APIC version 16
>> (XEN) Processor #23 5:1 APIC version 16
>> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
>> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
>> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
>> (XEN) Table is not found!
>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>> (XEN) Detected 3110.540 MHz processor.
>> (XEN) Initing memory sharing.
>> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
>> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
>> (XEN) IVHD Error: Invalid IO-APIC 0xff
>> (XEN) AMD-Vi: Error initialization
>> (XEN) I/O virtualisation disabled
>> (XEN) ENABLING IO-APIC IRQs
>> (XEN) -> Using new ACK method
>> (XEN) Platform timer is 14.318MHz HPET
>> (XEN) Allocated console ring of 16 KiB.
>> (XEN) HVM: ASIDs enabled.
>> (XEN) SVM: Supported advanced features:
>> (XEN) - Nested Page Tables (NPT)
>> (XEN) - Last Branch Record (LBR) Virtualisation
>> (XEN) - Next-RIP Saved on #VMEXIT
>> (XEN) - VMCB Clean Bits
>> (XEN) - DecodeAssists
>> (XEN) - Pause-Intercept Filter
>> (XEN) - TSC Rate MSR
>> (XEN) HVM: SVM enabled
>> (XEN) HVM: Hardware Assisted Paging (HAP) detected
>> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
>> (XEN) Brought up 8 CPUs
>> (XEN) mtrr: your CPUs had inconsistent variable MTRR settings
>> (XEN) *** LOADING DOMAIN 0 ***
>> (XEN) Xen kernel: 64-bit, lsb, compat32
>> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2346000
>> (XEN) PHYSICAL MEMORY ARRANGEMENT:
>> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4035583
>> pages to be allocated)
>> (XEN) Init. ramdisk: 0000000439dc6000->000000043efff800
>> (XEN) VIRTUAL MEMORY ARRANGEMENT:
>> (XEN) Loaded kernel: ffffffff81000000->ffffffff82346000
>> (XEN) Init. ramdisk: ffffffff82346000->ffffffff8757f800
>> (XEN) Phys-Mach map: ffffffff87580000->ffffffff894b31c8
>> (XEN) Start info: ffffffff894b4000->ffffffff894b44b4
>> (XEN) Page tables: ffffffff894b5000->ffffffff89504000
>> (XEN) Boot stack: ffffffff89504000->ffffffff89505000
>> (XEN) TOTAL: ffffffff80000000->ffffffff89800000
>> (XEN) ENTRY ADDRESS: ffffffff81d06210
>> (XEN) Dom0 has maximum 8 VCPUs
>> (XEN) Scrubbing Free RAM: .done.
>> (XEN) Initial low memory virq threshold set at 0x4000 pages.
>> (XEN) Std. Loglevel: Errors and warnings
>> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
>> (XEN) Xen is relinquishing VGA console.
>> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
>> switch input to Xen)
>> (XEN) Freed 244kB init memory.
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xen.org
>> http://lists.xen.org/xen-users
>>
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
Hi guys,

i'm also experiencing this problem with M5A97 EVO R2.0. It has been
"solved" in Xen 4.2.2 (discused here
http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html).
But i agree on that that it should be repaired on ASUS side.

And I'm also facing with prolem about Adaptec 3805 Raid card with
enabled IOMMU on this motherbord. Without IOMMU storage adapter works
perfectly, but with enabled IOMMU in BIOS i see the device under linux
as block device but it's unusable - no data can read or written. Does
anyone know what to do here, please? Could it be related with bad IOMMU
implementation?

Thanks in advance
Jan

Dne 28.5.2013 12:11, Gizmo Chicken napsal(a):
> David and feral,
>
> I have an Asus M5A99FX Pro 2.0 motherboard and, at least as far as I
> can tell, IOMMU works well with that motherboard and Xen 4.1. (I got
> VGA passthrough working with a Windows guest.) Although I did some
> preliminary testing with Xen 4.2 on that motherboard, because of some
> other issues, I never got around to creating a VM that required IOMMU
> support, and so can't say for certain whether IOMMU is working with
> that motherboard and Xen 4.2.
>
> Apart from creating a VM that requires IOMMU support (which would take
> more time than I'd prefer to spend right now) to see if it works, what
> would good way to determine whether my Asus M5A99FX Pro 2.0
> motherboard suffers, or if free from, the IVRS table issue that you
> describe?
>
> Thanks much for any help with this!
>
> Best regards,
> GizmoChicken
>
>
> On Tue, May 28, 2013 at 3:31 AM, David Sutton <kantras@gmail.com
> <mailto:kantras@gmail.com>> wrote:
>
> Feral,
>
> I have the R1.0 version of that motherboard. The problem is with
> the BIOS; its returning bad information in the IVRS table for the
> IO-APICs, which the new parser is catching and causing AMD-Vi
> initialization to fail. I've already put in a support ticket via
> the online form, included all the information such as links to the
> document which shows what the valid details should be. The ticket
> was basically closed with a note saying to look out for BIOS
> updates. I'm currently using 4.2.1 (which was before the table
> checking was added) and am able to use AMD-Vi. If you want to fix
> this, you're either going to have to get ASUS to issue a fixed
> BIOS (which doesn't feel likely at this point) or you'd have to
> either disable interrupt remapping or edit the source code to
> disable that check (and deal with any potential issue that might
> cause later)
>
> Regards,
>
> David
>
>
> On Mon, May 27, 2013 at 11:42 PM, feral <blistovmhz@gmail.com
> <mailto:blistovmhz@gmail.com>> wrote:
>
> This issue has been discussed many times before, but I haven't
> found
> an answer in my specific problem.
>
> Motherboard: Asus Sabertooth 990fx R2.0
> Linux 3.2.0-35 through 3.8.x (tested 5 kernels to be
> ridiculously thorough)
>
> Affected: Xen-hypervisor >4.1.2
> Not Affected: Xen-hypervisor <4.1.2
>
>
> Xen 4.1.2 and earlier works fine regardless of the BIOS
> version of the
> Mobo. After upgrading Xen hypervisor beyond 4.1.2, we get the
> following:
> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000,
> GSI 0-23
> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000,
> GSI 24-55
> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
> (XEN) Table is not found!
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 3110.540 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) xstate_init: using cntxt_size: 0x3c0 and states:
> 0x4000000000000007
> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
> (XEN) IVHD Error: Invalid IO-APIC 0xff
> (XEN) AMD-Vi: Error initialization
> (XEN) I/O virtualisation disabled
>
>
> From what I understand, Xen used to just care less about the
> state of
> your ACPI/IVRS but that this was potentially a security
> concern and
> checks were implemented to disable IOMMU whenever it would be
> unsafe
> (potentially) to have it enabled.
> Now, I understand the logic here, but I disagree with the
> implementation. There are a fairly large number of people who are
> affected and we know Asus isn't going to fix the IVRS table. I
> recognize the importance of implementing these checks and
> hell, even
> automatically disabling IOMMU without notice (though on a consumer
> board, this is a bit paranoid).
> I understand there are kernel command line options you can pass to
> forcibly bypass the checks and leave IOMMU enabled in most
> cases (ie:
> iommu=no-amd-iommu-perdev-intremap) but thus far none of them
> seem to
> be enough to force IOMMU back on.
> I read somewhere in one of the many discussions on the
> subject, that
> there are cases where you simply aren't allowed to disable the
> checks
> (I believe this was when the IVRS table indicies don't line up or
> something, which seems to be my issue).
>
> As I said, I know other people have precisely this problem with a
> whole line of mid range Asus boards, but haven't seen a lot of
> discussion on the list. Anyone reading this run into the same
> problem? I bought this board specifically because of it's IOMMU
> implementation, which worked fine... at the time...
> Ultimately I know
> for a fact that the boards ACPI/IVRS can't be TOO wrong as
> it's been
> working for 6 months without an issue. I'm just sorta in a
> situation
> where it's time to upgrade my base OS and hacking Xen 4.1.2 in
> seems
> like a silly amount of work when all I really need is an option to
> disable the checks.
>
>
> xm dmesg from a working vs. borked system
> Working:
> (XEN) Xen version 4.1.2 (Ubuntu 4.1.2-2ubuntu2)
> (stefan.bader@canonical.com
> <mailto:stefan.bader@canonical.com>) (gcc version 4.6.3
> (Ubuntu/Linaro
> 4.6.3-1ubuntu2) ) Thu Mar 8 14:31:11 UTC 2012
> (XEN) Bootloader: GRUB 1.99-21ubuntu3.9
> (XEN) Command line: pci_msitranslate=0 xen-pciback=passthrough
> xen-pciback.hide=(06:00.0)(06.00.1)
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
> (XEN) Disc information:
> (XEN) Found 6 MBR signatures
> (XEN) Found 6 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009e800 (usable)
> (XEN) 000000000009e800 - 00000000000a0000 (reserved)
> (XEN) 00000000000e0000 - 0000000000100000 (reserved)
> (XEN) 0000000000100000 - 00000000bc348000 (usable)
> (XEN) 00000000bc348000 - 00000000bc78c000 (reserved)
> (XEN) 00000000bc78c000 - 00000000bc797000 (ACPI data)
> (XEN) 00000000bc797000 - 00000000bd4f8000 (ACPI NVS)
> (XEN) 00000000bd4f8000 - 00000000bd8d5000 (reserved)
> (XEN) 00000000bd8d5000 - 00000000bd8d6000 (usable)
> (XEN) 00000000bd8d6000 - 00000000bdad9000 (ACPI NVS)
> (XEN) 00000000bdad9000 - 00000000bdf00000 (usable)
> (XEN) 00000000f8000000 - 00000000fc000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec01000 (reserved)
> (XEN) 00000000fec10000 - 00000000fec11000 (reserved)
> (XEN) 00000000fec20000 - 00000000fec21000 (reserved)
> (XEN) 00000000fed00000 - 00000000fed01000 (reserved)
> (XEN) 00000000fed61000 - 00000000fed71000 (reserved)
> (XEN) 00000000fed80000 - 00000000fed90000 (reserved)
> (XEN) 00000000fef00000 - 0000000100000000 (reserved)
> (XEN) 0000000100001000 - 000000043f000000 (usable)
> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
> (XEN) ACPI: XSDT BC78E078, 0064 (r1 ALASKA A M I 1072009
> AMI 10013)
> (XEN) ACPI: FACP BC795BA8, 00F4 (r4 ALASKA A M I 1072009
> AMI 10013)
> (XEN) ACPI Warning (tbfadt-0444): Optional field
> "Pm2ControlBlock" has
> zero address or length: 0000000000000000/1 [20070126]
> (XEN) ACPI: DSDT BC78E170, 7A38 (r2 ALASKA A M I 0
> INTL 20051117)
> (XEN) ACPI: FACS BD4F2F80, 0040
> (XEN) ACPI: APIC BC795CA0, 009E (r3 ALASKA A M I 1072009
> AMI 10013)
> (XEN) ACPI: FPDT BC795D40, 0044 (r1 ALASKA A M I 1072009
> AMI 10013)
> (XEN) ACPI: MCFG BC795D88, 003C (r1 ALASKA A M I 1072009
> MSFT 10013)
> (XEN) ACPI: HPET BC795DC8, 0038 (r1 ALASKA A M I 1072009
> AMI 5)
> (XEN) ACPI: BGRT BC796260, 0038 (r0 ALASKA A M I 1072009
> AMI 10013)
> (XEN) ACPI: IVRS BC795E58, 0100 (r1 AMD RD890S 202031 AMD
> 0)
> (XEN) ACPI: SSDT BC795F58, 0304 (r1 AMD POWERNOW 1 AMD
> 1)
> (XEN) System RAM: 16311MB (16702516kB)
> (XEN) Domain heap initialised
> (XEN) ACPI: 32/64X FACS address mismatch in FADT -
> bd4f2f80/0000000000000000, using 32
> (XEN) Processor #16 5:1 APIC version 16
> (XEN) Processor #17 5:1 APIC version 16
> (XEN) Processor #18 5:1 APIC version 16
> (XEN) Processor #19 5:1 APIC version 16
> (XEN) Processor #20 5:1 APIC version 16
> (XEN) Processor #21 5:1 APIC version 16
> (XEN) Processor #22 5:1 APIC version 16
> (XEN) Processor #23 5:1 APIC version 16
> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000,
> GSI 0-23
> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000,
> GSI 24-55
> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
> (XEN) Table is not found!
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 3110.553 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) AMD-Vi: IOMMU 0 Enabled.
> (XEN) I/O virtualisation enabled
> (XEN) - Dom0 mode: Relaxed
> (XEN) ENABLING IO-APIC IRQs
> (XEN) -> Using new ACK method
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Allocated console ring of 16 KiB.
> (XEN) HVM: ASIDs enabled.
> (XEN) SVM: Supported advanced features:
> (XEN) - Nested Page Tables (NPT)
> (XEN) - Last Branch Record (LBR) Virtualisation
> (XEN) - Next-RIP Saved on #VMEXIT
> (XEN) - VMCB Clean Bits
> (XEN) - Pause-Intercept Filter
> (XEN) HVM: SVM enabled
> (XEN) HVM: Hardware Assisted Paging detected.
> (XEN) Brought up 8 CPUs
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Xen kernel: 64-bit, lsb, compat32
> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x205f000
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4044532
> pages to be allocated)
> (XEN) Init. ramdisk: 000000043c7dd000->000000043efffa00
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN) Loaded kernel: ffffffff81000000->ffffffff8205f000
> (XEN) Init. ramdisk: ffffffff8205f000->ffffffff84881a00
> (XEN) Phys-Mach map: ffffffff84882000->ffffffff867b18b8
> (XEN) Start info: ffffffff867b2000->ffffffff867b24b4
> (XEN) Page tables: ffffffff867b3000->ffffffff867ec000
> (XEN) Boot stack: ffffffff867ec000->ffffffff867ed000
> (XEN) TOTAL: ffffffff80000000->ffffffff86c00000
> (XEN) ENTRY ADDRESS: ffffffff81cfb200
> (XEN) Dom0 has maximum 8 VCPUs
> (XEN) Scrubbing Free RAM: .done.
> (XEN) Xen trace buffers: disabled
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
> switch input to Xen)
> (XEN) Freed 220kB init memory.
> (XEN) traps.c:2432:d0 Domain attempted WRMSR 00000000c0010201 from
> 0x0000000000000000 to 0x000000000000abcd.
> (XEN) physdev.c:155: dom0: wrong map_pirq type 3
>
>
> BORKED:
> (XEN) Xen version 4.2.1 (Ubuntu 4.2.1-0ubuntu3.1)
> (stefan.bader@canonical.com
> <mailto:stefan.bader@canonical.com>) (gcc (Ubuntu/Linaro
> 4.7.3-1ubuntu1)
> 4.7.3) Mon Apr 29 19:35:31 UTC 2013
> (XEN) Bootloader: GRUB 2.00-13ubuntu3
> (XEN) Command line: iommu=no-amd-iommu-perdev-intremap x2apic=off
> xen-pciback.hide=(06:00.0)(06.00.1)
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
> (XEN) Disc information:
> (XEN) Found 7 MBR signatures
> (XEN) Found 6 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009e800 (usable)
> (XEN) 000000000009e800 - 00000000000a0000 (reserved)
> (XEN) 00000000000e0000 - 0000000000100000 (reserved)
> (XEN) 0000000000100000 - 00000000ba7ac000 (usable)
> (XEN) 00000000ba7ac000 - 00000000babe0000 (reserved)
> (XEN) 00000000babe0000 - 00000000babf0000 (ACPI data)
> (XEN) 00000000babf0000 - 00000000bb958000 (ACPI NVS)
> (XEN) 00000000bb958000 - 00000000bca35000 (reserved)
> (XEN) 00000000bca35000 - 00000000bca36000 (usable)
> (XEN) 00000000bca36000 - 00000000bcc3c000 (ACPI NVS)
> (XEN) 00000000bcc3c000 - 00000000bd083000 (usable)
> (XEN) 00000000bd083000 - 00000000bd7f4000 (reserved)
> (XEN) 00000000bd7f4000 - 00000000bd800000 (usable)
> (XEN) 00000000f8000000 - 00000000fc000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec01000 (reserved)
> (XEN) 00000000fec10000 - 00000000fec11000 (reserved)
> (XEN) 00000000fec20000 - 00000000fec21000 (reserved)
> (XEN) 00000000fed00000 - 00000000fed01000 (reserved)
> (XEN) 00000000fed61000 - 00000000fed71000 (reserved)
> (XEN) 00000000fed80000 - 00000000fed90000 (reserved)
> (XEN) 00000000fef00000 - 0000000100000000 (reserved)
> (XEN) 0000000100001000 - 000000043f000000 (usable)
> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
> (XEN) ACPI: XSDT BABE7078, 0064 (r1 ALASKA A M I 1072009
> AMI 10013)
> (XEN) ACPI: FACP BABEE118, 010C (r5 ALASKA A M I 1072009
> AMI 10013)
> (XEN) ACPI Warning (tbfadt-0232): FADT (revision 5) is longer than
> ACPI 2.0 version, truncating length 0x10C to 0xF4 [20070126]
> (XEN) ACPI Warning (tbfadt-0444): Optional field
> "Pm2ControlBlock" has
> zero address or length: 0000000000000000/1 [20070126]
> (XEN) ACPI: DSDT BABE7170, 6FA8 (r2 ALASKA A M I 0
> INTL 20051117)
> (XEN) ACPI: FACS BB952F80, 0040
> (XEN) ACPI: APIC BABEE228, 009E (r3 ALASKA A M I 1072009
> AMI 10013)
> (XEN) ACPI: FPDT BABEE2C8, 0044 (r1 ALASKA A M I 1072009
> AMI 10013)
> (XEN) ACPI: MCFG BABEE310, 003C (r1 ALASKA A M I 1072009
> MSFT 10013)
> (XEN) ACPI: HPET BABEE350, 0038 (r1 ALASKA A M I 1072009
> AMI 5)
> (XEN) ACPI: BGRT BABEFBF8, 0038 (r0 ALASKA A M I 1072009
> AMI 10013)
> (XEN) ACPI: IVRS BABEE3E0, 0100 (r1 AMD RD890S 202031 AMD
> 0)
> (XEN) ACPI: SSDT BABEE4E0, 1714 (r1 AMD POWERNOW 1 AMD
> 1)
> (XEN) System RAM: 16283MB (16674420kB)
> (XEN) Domain heap initialised
> (XEN) ACPI: 32/64X FACS address mismatch in FADT -
> bb952f80/0000000000000000, using 32
> (XEN) Processor #16 5:1 APIC version 16
> (XEN) Processor #17 5:1 APIC version 16
> (XEN) Processor #18 5:1 APIC version 16
> (XEN) Processor #19 5:1 APIC version 16
> (XEN) Processor #20 5:1 APIC version 16
> (XEN) Processor #21 5:1 APIC version 16
> (XEN) Processor #22 5:1 APIC version 16
> (XEN) Processor #23 5:1 APIC version 16
> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000,
> GSI 0-23
> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000,
> GSI 24-55
> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
> (XEN) Table is not found!
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 3110.540 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) xstate_init: using cntxt_size: 0x3c0 and states:
> 0x4000000000000007
> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
> (XEN) IVHD Error: Invalid IO-APIC 0xff
> (XEN) AMD-Vi: Error initialization
> (XEN) I/O virtualisation disabled
> (XEN) ENABLING IO-APIC IRQs
> (XEN) -> Using new ACK method
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Allocated console ring of 16 KiB.
> (XEN) HVM: ASIDs enabled.
> (XEN) SVM: Supported advanced features:
> (XEN) - Nested Page Tables (NPT)
> (XEN) - Last Branch Record (LBR) Virtualisation
> (XEN) - Next-RIP Saved on #VMEXIT
> (XEN) - VMCB Clean Bits
> (XEN) - DecodeAssists
> (XEN) - Pause-Intercept Filter
> (XEN) - TSC Rate MSR
> (XEN) HVM: SVM enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) detected
> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> (XEN) Brought up 8 CPUs
> (XEN) mtrr: your CPUs had inconsistent variable MTRR settings
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Xen kernel: 64-bit, lsb, compat32
> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2346000
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4035583
> pages to be allocated)
> (XEN) Init. ramdisk: 0000000439dc6000->000000043efff800
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN) Loaded kernel: ffffffff81000000->ffffffff82346000
> (XEN) Init. ramdisk: ffffffff82346000->ffffffff8757f800
> (XEN) Phys-Mach map: ffffffff87580000->ffffffff894b31c8
> (XEN) Start info: ffffffff894b4000->ffffffff894b44b4
> (XEN) Page tables: ffffffff894b5000->ffffffff89504000
> (XEN) Boot stack: ffffffff89504000->ffffffff89505000
> (XEN) TOTAL: ffffffff80000000->ffffffff89800000
> (XEN) ENTRY ADDRESS: ffffffff81d06210
> (XEN) Dom0 has maximum 8 VCPUs
> (XEN) Scrubbing Free RAM: .done.
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
> switch input to Xen)
> (XEN) Freed 244kB init memory.
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org <mailto:Xen-users@lists.xen.org>
> http://lists.xen.org/xen-users
>
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org <mailto:Xen-users@lists.xen.org>
> http://lists.xen.org/xen-users
>
>
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
GizmoChicken,

To debug the issue when I first ran across it, I made sure I added
"iommu=debug,verbose" onto the boot options in my grub.conf file. I then
used "xl dmesg" to see the log when booting up - You'd be looking for
something like:

--
(XEN) AMD-Vi: Found MSI capability block at 0x54
(XEN) AMD-Vi: ACPI Table:
(XEN) AMD-Vi: Signature IVRS
(XEN) AMD-Vi: Length 0xf0
(XEN) AMD-Vi: Revision 0x1
(XEN) AMD-Vi: CheckSum 0x6e
(XEN) AMD-Vi: OEM_Id AMD
(XEN) AMD-Vi: OEM_Table_Id RD890S
(XEN) AMD-Vi: OEM_Revision 0x202031
(XEN) AMD-Vi: Creator_Id AMD
(XEN) AMD-Vi: Creator_Revision 0x0
(XEN) AMD-Vi: IVRS Block: type 0x10 flags 0x3e len 0xc0 id 0x2
(XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0x0 flags 0x0
(XEN) AMD-Vi: Dev_Id Range: 0x0 -> 0x2
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x10 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0x100 flags 0x0
(XEN) AMD-Vi: Dev_Id Range: 0x100 -> 0x101
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x20 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x200 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x28 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x300 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x30 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x400 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x48 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x500 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x58 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0x600 flags 0x0
(XEN) AMD-Vi: Dev_Id Range: 0x600 -> 0x601
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x88 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0x90 flags 0x0
(XEN) AMD-Vi: Dev_Id Range: 0x90 -> 0x92
(XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0x98 flags 0x0
(XEN) AMD-Vi: Dev_Id Range: 0x98 -> 0x9a
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa0 flags 0xd7
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa2 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa3 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa4 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x0 id 0x0 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x43 id 0x700 flags 0x0
(XEN) AMD-Vi: Dev_Id Range: 0x700 -> 0x7ff alias 0xa4
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa5 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa8 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa9 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x900 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0xb0 flags 0x0
(XEN) AMD-Vi: Dev_Id Range: 0xb0 -> 0xb2
(XEN) AMD-Vi: IVHD Device Entry: type 0x0 id 0x0 flags 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0x0 flags 0xd7
(XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x1 handle 0x0
(XEN) IVHD Error: Invalid IO-APIC 0x0
(XEN) AMD-Vi: Error initialization
--

In my case, the handle id is wrong ( you can see what it should be on the
support document http://support.amd.com/us/Processor_TechDocs/48882.pdf -
per tables 77/79 it should be the ID of the IO-APIC). Unlike the conflict
error, which can be worked around by passing another command line option,
invalid IO-APIC entries in the IVRS table will always cause AMD-Vi to fail
to be enabled if you have the XSA-36 patch installed.

From the notes in the patch for 4.2:
--
In addition, because some BIOSes may incorrectly program IVRS
entries for IOAPIC try to check for entry's consistency. Specifically,
if conflicting entries are found disable IOMMU if per-device
remapping table is used. If entries refer to bogus IOAPIC IDs
disable IOMMU unconditionally
--

Regards,

David


On Tue, May 28, 2013 at 5:11 AM, Gizmo Chicken <gizmochicken@gmail.com>wrote:

> David and feral,
>
> I have an Asus M5A99FX Pro 2.0 motherboard and, at least as far as I can
> tell, IOMMU works well with that motherboard and Xen 4.1. (I got VGA
> passthrough working with a Windows guest.) Although I did some
> preliminary testing with Xen 4.2 on that motherboard, because of some other
> issues, I never got around to creating a VM that required IOMMU support,
> and so can't say for certain whether IOMMU is working with that motherboard
> and Xen 4.2.
>
> Apart from creating a VM that requires IOMMU support (which would take
> more time than I'd prefer to spend right now) to see if it works, what
> would good way to determine whether my Asus M5A99FX Pro 2.0 motherboard
> suffers, or if free from, the IVRS table issue that you describe?
>
> Thanks much for any help with this!
>
> Best regards,
> GizmoChicken
>
>
> On Tue, May 28, 2013 at 3:31 AM, David Sutton <kantras@gmail.com> wrote:
>
>> Feral,
>>
>> I have the R1.0 version of that motherboard. The problem is with the
>> BIOS; its returning bad information in the IVRS table for the IO-APICs,
>> which the new parser is catching and causing AMD-Vi initialization to fail.
>> I've already put in a support ticket via the online form, included all the
>> information such as links to the document which shows what the valid
>> details should be. The ticket was basically closed with a note saying to
>> look out for BIOS updates. I'm currently using 4.2.1 (which was before the
>> table checking was added) and am able to use AMD-Vi. If you want to fix
>> this, you're either going to have to get ASUS to issue a fixed BIOS (which
>> doesn't feel likely at this point) or you'd have to either disable
>> interrupt remapping or edit the source code to disable that check (and deal
>> with any potential issue that might cause later)
>>
>> Regards,
>>
>> David
>>
>>
>> On Mon, May 27, 2013 at 11:42 PM, feral <blistovmhz@gmail.com> wrote:
>>
>>> This issue has been discussed many times before, but I haven't found
>>> an answer in my specific problem.
>>>
>>> Motherboard: Asus Sabertooth 990fx R2.0
>>> Linux 3.2.0-35 through 3.8.x (tested 5 kernels to be ridiculously
>>> thorough)
>>>
>>> Affected: Xen-hypervisor >4.1.2
>>> Not Affected: Xen-hypervisor <4.1.2
>>>
>>>
>>> Xen 4.1.2 and earlier works fine regardless of the BIOS version of the
>>> Mobo. After upgrading Xen hypervisor beyond 4.1.2, we get the
>>> following:
>>> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
>>> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
>>> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
>>> (XEN) Table is not found!
>>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>>> (XEN) Detected 3110.540 MHz processor.
>>> (XEN) Initing memory sharing.
>>> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
>>> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
>>> (XEN) IVHD Error: Invalid IO-APIC 0xff
>>> (XEN) AMD-Vi: Error initialization
>>> (XEN) I/O virtualisation disabled
>>>
>>>
>>> From what I understand, Xen used to just care less about the state of
>>> your ACPI/IVRS but that this was potentially a security concern and
>>> checks were implemented to disable IOMMU whenever it would be unsafe
>>> (potentially) to have it enabled.
>>> Now, I understand the logic here, but I disagree with the
>>> implementation. There are a fairly large number of people who are
>>> affected and we know Asus isn't going to fix the IVRS table. I
>>> recognize the importance of implementing these checks and hell, even
>>> automatically disabling IOMMU without notice (though on a consumer
>>> board, this is a bit paranoid).
>>> I understand there are kernel command line options you can pass to
>>> forcibly bypass the checks and leave IOMMU enabled in most cases (ie:
>>> iommu=no-amd-iommu-perdev-intremap) but thus far none of them seem to
>>> be enough to force IOMMU back on.
>>> I read somewhere in one of the many discussions on the subject, that
>>> there are cases where you simply aren't allowed to disable the checks
>>> (I believe this was when the IVRS table indicies don't line up or
>>> something, which seems to be my issue).
>>>
>>> As I said, I know other people have precisely this problem with a
>>> whole line of mid range Asus boards, but haven't seen a lot of
>>> discussion on the list. Anyone reading this run into the same
>>> problem? I bought this board specifically because of it's IOMMU
>>> implementation, which worked fine... at the time... Ultimately I know
>>> for a fact that the boards ACPI/IVRS can't be TOO wrong as it's been
>>> working for 6 months without an issue. I'm just sorta in a situation
>>> where it's time to upgrade my base OS and hacking Xen 4.1.2 in seems
>>> like a silly amount of work when all I really need is an option to
>>> disable the checks.
>>>
>>>
>>> xm dmesg from a working vs. borked system
>>> Working:
>>> (XEN) Xen version 4.1.2 (Ubuntu 4.1.2-2ubuntu2)
>>> (stefan.bader@canonical.com) (gcc version 4.6.3 (Ubuntu/Linaro
>>> 4.6.3-1ubuntu2) ) Thu Mar 8 14:31:11 UTC 2012
>>> (XEN) Bootloader: GRUB 1.99-21ubuntu3.9
>>> (XEN) Command line: pci_msitranslate=0 xen-pciback=passthrough
>>> xen-pciback.hide=(06:00.0)(06.00.1)
>>> (XEN) Video information:
>>> (XEN) VGA is text mode 80x25, font 8x16
>>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
>>> (XEN) Disc information:
>>> (XEN) Found 6 MBR signatures
>>> (XEN) Found 6 EDD information structures
>>> (XEN) Xen-e820 RAM map:
>>> (XEN) 0000000000000000 - 000000000009e800 (usable)
>>> (XEN) 000000000009e800 - 00000000000a0000 (reserved)
>>> (XEN) 00000000000e0000 - 0000000000100000 (reserved)
>>> (XEN) 0000000000100000 - 00000000bc348000 (usable)
>>> (XEN) 00000000bc348000 - 00000000bc78c000 (reserved)
>>> (XEN) 00000000bc78c000 - 00000000bc797000 (ACPI data)
>>> (XEN) 00000000bc797000 - 00000000bd4f8000 (ACPI NVS)
>>> (XEN) 00000000bd4f8000 - 00000000bd8d5000 (reserved)
>>> (XEN) 00000000bd8d5000 - 00000000bd8d6000 (usable)
>>> (XEN) 00000000bd8d6000 - 00000000bdad9000 (ACPI NVS)
>>> (XEN) 00000000bdad9000 - 00000000bdf00000 (usable)
>>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved)
>>> (XEN) 00000000fec00000 - 00000000fec01000 (reserved)
>>> (XEN) 00000000fec10000 - 00000000fec11000 (reserved)
>>> (XEN) 00000000fec20000 - 00000000fec21000 (reserved)
>>> (XEN) 00000000fed00000 - 00000000fed01000 (reserved)
>>> (XEN) 00000000fed61000 - 00000000fed71000 (reserved)
>>> (XEN) 00000000fed80000 - 00000000fed90000 (reserved)
>>> (XEN) 00000000fef00000 - 0000000100000000 (reserved)
>>> (XEN) 0000000100001000 - 000000043f000000 (usable)
>>> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
>>> (XEN) ACPI: XSDT BC78E078, 0064 (r1 ALASKA A M I 1072009 AMI
>>> 10013)
>>> (XEN) ACPI: FACP BC795BA8, 00F4 (r4 ALASKA A M I 1072009 AMI
>>> 10013)
>>> (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has
>>> zero address or length: 0000000000000000/1 [20070126]
>>> (XEN) ACPI: DSDT BC78E170, 7A38 (r2 ALASKA A M I 0 INTL
>>> 20051117)
>>> (XEN) ACPI: FACS BD4F2F80, 0040
>>> (XEN) ACPI: APIC BC795CA0, 009E (r3 ALASKA A M I 1072009 AMI
>>> 10013)
>>> (XEN) ACPI: FPDT BC795D40, 0044 (r1 ALASKA A M I 1072009 AMI
>>> 10013)
>>> (XEN) ACPI: MCFG BC795D88, 003C (r1 ALASKA A M I 1072009 MSFT
>>> 10013)
>>> (XEN) ACPI: HPET BC795DC8, 0038 (r1 ALASKA A M I 1072009 AMI
>>> 5)
>>> (XEN) ACPI: BGRT BC796260, 0038 (r0 ALASKA A M I 1072009 AMI
>>> 10013)
>>> (XEN) ACPI: IVRS BC795E58, 0100 (r1 AMD RD890S 202031 AMD
>>> 0)
>>> (XEN) ACPI: SSDT BC795F58, 0304 (r1 AMD POWERNOW 1 AMD
>>> 1)
>>> (XEN) System RAM: 16311MB (16702516kB)
>>> (XEN) Domain heap initialised
>>> (XEN) ACPI: 32/64X FACS address mismatch in FADT -
>>> bd4f2f80/0000000000000000, using 32
>>> (XEN) Processor #16 5:1 APIC version 16
>>> (XEN) Processor #17 5:1 APIC version 16
>>> (XEN) Processor #18 5:1 APIC version 16
>>> (XEN) Processor #19 5:1 APIC version 16
>>> (XEN) Processor #20 5:1 APIC version 16
>>> (XEN) Processor #21 5:1 APIC version 16
>>> (XEN) Processor #22 5:1 APIC version 16
>>> (XEN) Processor #23 5:1 APIC version 16
>>> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
>>> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
>>> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
>>> (XEN) Table is not found!
>>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>>> (XEN) Detected 3110.553 MHz processor.
>>> (XEN) Initing memory sharing.
>>> (XEN) AMD-Vi: IOMMU 0 Enabled.
>>> (XEN) I/O virtualisation enabled
>>> (XEN) - Dom0 mode: Relaxed
>>> (XEN) ENABLING IO-APIC IRQs
>>> (XEN) -> Using new ACK method
>>> (XEN) Platform timer is 14.318MHz HPET
>>> (XEN) Allocated console ring of 16 KiB.
>>> (XEN) HVM: ASIDs enabled.
>>> (XEN) SVM: Supported advanced features:
>>> (XEN) - Nested Page Tables (NPT)
>>> (XEN) - Last Branch Record (LBR) Virtualisation
>>> (XEN) - Next-RIP Saved on #VMEXIT
>>> (XEN) - VMCB Clean Bits
>>> (XEN) - Pause-Intercept Filter
>>> (XEN) HVM: SVM enabled
>>> (XEN) HVM: Hardware Assisted Paging detected.
>>> (XEN) Brought up 8 CPUs
>>> (XEN) *** LOADING DOMAIN 0 ***
>>> (XEN) Xen kernel: 64-bit, lsb, compat32
>>> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x205f000
>>> (XEN) PHYSICAL MEMORY ARRANGEMENT:
>>> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4044532
>>> pages to be allocated)
>>> (XEN) Init. ramdisk: 000000043c7dd000->000000043efffa00
>>> (XEN) VIRTUAL MEMORY ARRANGEMENT:
>>> (XEN) Loaded kernel: ffffffff81000000->ffffffff8205f000
>>> (XEN) Init. ramdisk: ffffffff8205f000->ffffffff84881a00
>>> (XEN) Phys-Mach map: ffffffff84882000->ffffffff867b18b8
>>> (XEN) Start info: ffffffff867b2000->ffffffff867b24b4
>>> (XEN) Page tables: ffffffff867b3000->ffffffff867ec000
>>> (XEN) Boot stack: ffffffff867ec000->ffffffff867ed000
>>> (XEN) TOTAL: ffffffff80000000->ffffffff86c00000
>>> (XEN) ENTRY ADDRESS: ffffffff81cfb200
>>> (XEN) Dom0 has maximum 8 VCPUs
>>> (XEN) Scrubbing Free RAM: .done.
>>> (XEN) Xen trace buffers: disabled
>>> (XEN) Std. Loglevel: Errors and warnings
>>> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
>>> (XEN) Xen is relinquishing VGA console.
>>> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
>>> switch input to Xen)
>>> (XEN) Freed 220kB init memory.
>>> (XEN) traps.c:2432:d0 Domain attempted WRMSR 00000000c0010201 from
>>> 0x0000000000000000 to 0x000000000000abcd.
>>> (XEN) physdev.c:155: dom0: wrong map_pirq type 3
>>>
>>>
>>> BORKED:
>>> (XEN) Xen version 4.2.1 (Ubuntu 4.2.1-0ubuntu3.1)
>>> (stefan.bader@canonical.com) (gcc (Ubuntu/Linaro 4.7.3-1ubuntu1)
>>> 4.7.3) Mon Apr 29 19:35:31 UTC 2013
>>> (XEN) Bootloader: GRUB 2.00-13ubuntu3
>>> (XEN) Command line: iommu=no-amd-iommu-perdev-intremap x2apic=off
>>> xen-pciback.hide=(06:00.0)(06.00.1)
>>> (XEN) Video information:
>>> (XEN) VGA is text mode 80x25, font 8x16
>>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
>>> (XEN) Disc information:
>>> (XEN) Found 7 MBR signatures
>>> (XEN) Found 6 EDD information structures
>>> (XEN) Xen-e820 RAM map:
>>> (XEN) 0000000000000000 - 000000000009e800 (usable)
>>> (XEN) 000000000009e800 - 00000000000a0000 (reserved)
>>> (XEN) 00000000000e0000 - 0000000000100000 (reserved)
>>> (XEN) 0000000000100000 - 00000000ba7ac000 (usable)
>>> (XEN) 00000000ba7ac000 - 00000000babe0000 (reserved)
>>> (XEN) 00000000babe0000 - 00000000babf0000 (ACPI data)
>>> (XEN) 00000000babf0000 - 00000000bb958000 (ACPI NVS)
>>> (XEN) 00000000bb958000 - 00000000bca35000 (reserved)
>>> (XEN) 00000000bca35000 - 00000000bca36000 (usable)
>>> (XEN) 00000000bca36000 - 00000000bcc3c000 (ACPI NVS)
>>> (XEN) 00000000bcc3c000 - 00000000bd083000 (usable)
>>> (XEN) 00000000bd083000 - 00000000bd7f4000 (reserved)
>>> (XEN) 00000000bd7f4000 - 00000000bd800000 (usable)
>>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved)
>>> (XEN) 00000000fec00000 - 00000000fec01000 (reserved)
>>> (XEN) 00000000fec10000 - 00000000fec11000 (reserved)
>>> (XEN) 00000000fec20000 - 00000000fec21000 (reserved)
>>> (XEN) 00000000fed00000 - 00000000fed01000 (reserved)
>>> (XEN) 00000000fed61000 - 00000000fed71000 (reserved)
>>> (XEN) 00000000fed80000 - 00000000fed90000 (reserved)
>>> (XEN) 00000000fef00000 - 0000000100000000 (reserved)
>>> (XEN) 0000000100001000 - 000000043f000000 (usable)
>>> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
>>> (XEN) ACPI: XSDT BABE7078, 0064 (r1 ALASKA A M I 1072009 AMI
>>> 10013)
>>> (XEN) ACPI: FACP BABEE118, 010C (r5 ALASKA A M I 1072009 AMI
>>> 10013)
>>> (XEN) ACPI Warning (tbfadt-0232): FADT (revision 5) is longer than
>>> ACPI 2.0 version, truncating length 0x10C to 0xF4 [20070126]
>>> (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has
>>> zero address or length: 0000000000000000/1 [20070126]
>>> (XEN) ACPI: DSDT BABE7170, 6FA8 (r2 ALASKA A M I 0 INTL
>>> 20051117)
>>> (XEN) ACPI: FACS BB952F80, 0040
>>> (XEN) ACPI: APIC BABEE228, 009E (r3 ALASKA A M I 1072009 AMI
>>> 10013)
>>> (XEN) ACPI: FPDT BABEE2C8, 0044 (r1 ALASKA A M I 1072009 AMI
>>> 10013)
>>> (XEN) ACPI: MCFG BABEE310, 003C (r1 ALASKA A M I 1072009 MSFT
>>> 10013)
>>> (XEN) ACPI: HPET BABEE350, 0038 (r1 ALASKA A M I 1072009 AMI
>>> 5)
>>> (XEN) ACPI: BGRT BABEFBF8, 0038 (r0 ALASKA A M I 1072009 AMI
>>> 10013)
>>> (XEN) ACPI: IVRS BABEE3E0, 0100 (r1 AMD RD890S 202031 AMD
>>> 0)
>>> (XEN) ACPI: SSDT BABEE4E0, 1714 (r1 AMD POWERNOW 1 AMD
>>> 1)
>>> (XEN) System RAM: 16283MB (16674420kB)
>>> (XEN) Domain heap initialised
>>> (XEN) ACPI: 32/64X FACS address mismatch in FADT -
>>> bb952f80/0000000000000000, using 32
>>> (XEN) Processor #16 5:1 APIC version 16
>>> (XEN) Processor #17 5:1 APIC version 16
>>> (XEN) Processor #18 5:1 APIC version 16
>>> (XEN) Processor #19 5:1 APIC version 16
>>> (XEN) Processor #20 5:1 APIC version 16
>>> (XEN) Processor #21 5:1 APIC version 16
>>> (XEN) Processor #22 5:1 APIC version 16
>>> (XEN) Processor #23 5:1 APIC version 16
>>> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
>>> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
>>> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
>>> (XEN) Table is not found!
>>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>>> (XEN) Detected 3110.540 MHz processor.
>>> (XEN) Initing memory sharing.
>>> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
>>> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
>>> (XEN) IVHD Error: Invalid IO-APIC 0xff
>>> (XEN) AMD-Vi: Error initialization
>>> (XEN) I/O virtualisation disabled
>>> (XEN) ENABLING IO-APIC IRQs
>>> (XEN) -> Using new ACK method
>>> (XEN) Platform timer is 14.318MHz HPET
>>> (XEN) Allocated console ring of 16 KiB.
>>> (XEN) HVM: ASIDs enabled.
>>> (XEN) SVM: Supported advanced features:
>>> (XEN) - Nested Page Tables (NPT)
>>> (XEN) - Last Branch Record (LBR) Virtualisation
>>> (XEN) - Next-RIP Saved on #VMEXIT
>>> (XEN) - VMCB Clean Bits
>>> (XEN) - DecodeAssists
>>> (XEN) - Pause-Intercept Filter
>>> (XEN) - TSC Rate MSR
>>> (XEN) HVM: SVM enabled
>>> (XEN) HVM: Hardware Assisted Paging (HAP) detected
>>> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
>>> (XEN) Brought up 8 CPUs
>>> (XEN) mtrr: your CPUs had inconsistent variable MTRR settings
>>> (XEN) *** LOADING DOMAIN 0 ***
>>> (XEN) Xen kernel: 64-bit, lsb, compat32
>>> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2346000
>>> (XEN) PHYSICAL MEMORY ARRANGEMENT:
>>> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4035583
>>> pages to be allocated)
>>> (XEN) Init. ramdisk: 0000000439dc6000->000000043efff800
>>> (XEN) VIRTUAL MEMORY ARRANGEMENT:
>>> (XEN) Loaded kernel: ffffffff81000000->ffffffff82346000
>>> (XEN) Init. ramdisk: ffffffff82346000->ffffffff8757f800
>>> (XEN) Phys-Mach map: ffffffff87580000->ffffffff894b31c8
>>> (XEN) Start info: ffffffff894b4000->ffffffff894b44b4
>>> (XEN) Page tables: ffffffff894b5000->ffffffff89504000
>>> (XEN) Boot stack: ffffffff89504000->ffffffff89505000
>>> (XEN) TOTAL: ffffffff80000000->ffffffff89800000
>>> (XEN) ENTRY ADDRESS: ffffffff81d06210
>>> (XEN) Dom0 has maximum 8 VCPUs
>>> (XEN) Scrubbing Free RAM: .done.
>>> (XEN) Initial low memory virq threshold set at 0x4000 pages.
>>> (XEN) Std. Loglevel: Errors and warnings
>>> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
>>> (XEN) Xen is relinquishing VGA console.
>>> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
>>> switch input to Xen)
>>> (XEN) Freed 244kB init memory.
>>>
>>> _______________________________________________
>>> Xen-users mailing list
>>> Xen-users@lists.xen.org
>>> http://lists.xen.org/xen-users
>>>
>>
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xen.org
>> http://lists.xen.org/xen-users
>>
>
>
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
> In my case, the handle id is wrong ( you can see what it should be on the
> support document http://support.amd.com/us/Processor_TechDocs/48882.pdf -
> per tables 77/79 it should be the ID of the IO-APIC). Unlike the conflict
> error, which can be worked around by passing another command line option,
> invalid IO-APIC entries in the IVRS table will always cause AMD-Vi to fail
> to be enabled if you have the XSA-36 patch installed.

This is the part I have an angry nerd rage issue with. I'm pretty
sure I fall into the latter category where there is no option to
forcibly enable IOMMU. I'd understand if the patch wasn't actually
removing functionality but in our case, it seems the patch does just
this even though the "bug" being addressed doesn't concern me (and
probably quite a few others) whatsoever. The machine in question is
my gaming rig primarily, and gets a lot of use in QA, but nothing
mission critical. I'd rather hobble along with a known bad IVRS table
if it has no affect on my work (or games) than be locked out entirely
:p.

I'm still a little confused though as I've heard a few people say that
this patch wasn't introduced until Xen 4.2 but I saw the issue first
pop up in Xen 4.1.2 and later. Am I even looking at the right bug?
And what options are available to forcibly disable the checks other
than the "iommu=no-amd-iommu-perdev-intremap" ?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
On Tue, May 28, 2013 at 4:24 AM, Jan Hejl <jh@excello.cz> wrote:
> Hi guys,
>
> i'm also experiencing this problem with M5A97 EVO R2.0. It has been "solved"
> in Xen 4.2.2 (discused here
> http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html). But
> i agree on that that it should be repaired on ASUS side.
>
> And I'm also facing with prolem about Adaptec 3805 Raid card with enabled
> IOMMU on this motherbord. Without IOMMU storage adapter works perfectly, but
> with enabled IOMMU in BIOS i see the device under linux as block device but
> it's unusable - no data can read or written. Does anyone know what to do
> here, please? Could it be related with bad IOMMU implementation?
>
> Thanks in advance
> Jan

What am I missing :p ? What does "solved" in Xen 4.2.2 mean? The
linked message discusses the original issue of the broken IVRS tables
and the "fix" is to implement checks to disable IOMMU on buggy bios's
after version 4.1.3. In this case, the "fix" is the problem, unless I
missed something?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
Missing? May be some irony and sarcasm about that "solved". Sure this could be done better in Xen implementation, but primarly ASUS should be the first to take a care. These boards are not cheapest so i suppose that they should work properly. May be i'm wrong.

28. 5. 2013 v 18:56, feral <blistovmhz@gmail.com>:

> On Tue, May 28, 2013 at 4:24 AM, Jan Hejl <jh@excello.cz> wrote:
>> Hi guys,
>>
>> i'm also experiencing this problem with M5A97 EVO R2.0. It has been "solved"
>> in Xen 4.2.2 (discused here
>> http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html). But
>> i agree on that that it should be repaired on ASUS side.
>>
>> And I'm also facing with prolem about Adaptec 3805 Raid card with enabled
>> IOMMU on this motherbord. Without IOMMU storage adapter works perfectly, but
>> with enabled IOMMU in BIOS i see the device under linux as block device but
>> it's unusable - no data can read or written. Does anyone know what to do
>> here, please? Could it be related with bad IOMMU implementation?
>>
>> Thanks in advance
>> Jan
>
> What am I missing :p ? What does "solved" in Xen 4.2.2 mean? The
> linked message discusses the original issue of the broken IVRS tables
> and the "fix" is to implement checks to disable IOMMU on buggy bios's
> after version 4.1.3. In this case, the "fix" is the problem, unless I
> missed something?
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
I agree entirely about who should be taking responsibility, but we all
know of course, that Asus will never fix this. There are probably
less than 5000 people in the world affected by this, and if we assume
most of them will still buy Asus's next board that has working IVRS
tables, Asus doesn't care.

I'm more interested in knowing why the
"iommu=no-amd-iommu-perdev-intremap" option doesn't cover my issue
though. It seems odd to me that this option was implemented precisely
to work around the regression issue, but somehow it didn't catch them
all. Goddamn I can't wait for the Xen Documentation to take form :)

On Tue, May 28, 2013 at 10:35 AM, Jan Hejl <jh@excello.cz> wrote:
> Missing? May be some irony and sarcasm about that "solved". Sure this could be done better in Xen implementation, but primarly ASUS should be the first to take a care. These boards are not cheapest so i suppose that they should work properly. May be i'm wrong.
>
> 28. 5. 2013 v 18:56, feral <blistovmhz@gmail.com>:
>
>> On Tue, May 28, 2013 at 4:24 AM, Jan Hejl <jh@excello.cz> wrote:
>>> Hi guys,
>>>
>>> i'm also experiencing this problem with M5A97 EVO R2.0. It has been "solved"
>>> in Xen 4.2.2 (discused here
>>> http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html). But
>>> i agree on that that it should be repaired on ASUS side.
>>>
>>> And I'm also facing with prolem about Adaptec 3805 Raid card with enabled
>>> IOMMU on this motherbord. Without IOMMU storage adapter works perfectly, but
>>> with enabled IOMMU in BIOS i see the device under linux as block device but
>>> it's unusable - no data can read or written. Does anyone know what to do
>>> here, please? Could it be related with bad IOMMU implementation?
>>>
>>> Thanks in advance
>>> Jan
>>
>> What am I missing :p ? What does "solved" in Xen 4.2.2 mean? The
>> linked message discusses the original issue of the broken IVRS tables
>> and the "fix" is to implement checks to disable IOMMU on buggy bios's
>> after version 4.1.3. In this case, the "fix" is the problem, unless I
>> missed something?



--
_____
Fact:
1. Ninjas are mammals.
2. Ninjas fight ALL the time.
3. The purpose of the ninja is to flip out and kill people.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
Also, I'm pretty much n00b when it comes to IVRS so I'm not entirely
sure what I'm talking about. Meanwhile I've got my Asus rep trying to
fix the issue. Does anyone have a technical explanation of what
specifically, Asus has to do to fix this issue?

On Tue, May 28, 2013 at 10:40 AM, feral <blistovmhz@gmail.com> wrote:
> I agree entirely about who should be taking responsibility, but we all
> know of course, that Asus will never fix this. There are probably
> less than 5000 people in the world affected by this, and if we assume
> most of them will still buy Asus's next board that has working IVRS
> tables, Asus doesn't care.
>
> I'm more interested in knowing why the
> "iommu=no-amd-iommu-perdev-intremap" option doesn't cover my issue
> though. It seems odd to me that this option was implemented precisely
> to work around the regression issue, but somehow it didn't catch them
> all. Goddamn I can't wait for the Xen Documentation to take form :)
>
> On Tue, May 28, 2013 at 10:35 AM, Jan Hejl <jh@excello.cz> wrote:
>> Missing? May be some irony and sarcasm about that "solved". Sure this could be done better in Xen implementation, but primarly ASUS should be the first to take a care. These boards are not cheapest so i suppose that they should work properly. May be i'm wrong.
>>
>> 28. 5. 2013 v 18:56, feral <blistovmhz@gmail.com>:
>>
>>> On Tue, May 28, 2013 at 4:24 AM, Jan Hejl <jh@excello.cz> wrote:
>>>> Hi guys,
>>>>
>>>> i'm also experiencing this problem with M5A97 EVO R2.0. It has been "solved"
>>>> in Xen 4.2.2 (discused here
>>>> http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html). But
>>>> i agree on that that it should be repaired on ASUS side.
>>>>
>>>> And I'm also facing with prolem about Adaptec 3805 Raid card with enabled
>>>> IOMMU on this motherbord. Without IOMMU storage adapter works perfectly, but
>>>> with enabled IOMMU in BIOS i see the device under linux as block device but
>>>> it's unusable - no data can read or written. Does anyone know what to do
>>>> here, please? Could it be related with bad IOMMU implementation?
>>>>
>>>> Thanks in advance
>>>> Jan
>>>
>>> What am I missing :p ? What does "solved" in Xen 4.2.2 mean? The
>>> linked message discusses the original issue of the broken IVRS tables
>>> and the "fix" is to implement checks to disable IOMMU on buggy bios's
>>> after version 4.1.3. In this case, the "fix" is the problem, unless I
>>> missed something?
>
>
>
> --
> _____
> Fact:
> 1. Ninjas are mammals.
> 2. Ninjas fight ALL the time.
> 3. The purpose of the ninja is to flip out and kill people.



--
_____
Fact:
1. Ninjas are mammals.
2. Ninjas fight ALL the time.
3. The purpose of the ninja is to flip out and kill people.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
Feral,

On Tue, May 28, 2013 at 11:49 AM, feral <blistovmhz@gmail.com> wrote:

> > In my case, the handle id is wrong ( you can see what it should be on the
> > support document http://support.amd.com/us/Processor_TechDocs/48882.pdf-
> > per tables 77/79 it should be the ID of the IO-APIC). Unlike the conflict
> > error, which can be worked around by passing another command line option,
> > invalid IO-APIC entries in the IVRS table will always cause AMD-Vi to
> fail
> > to be enabled if you have the XSA-36 patch installed.
>
> This is the part I have an angry nerd rage issue with. I'm pretty
> sure I fall into the latter category where there is no option to
> forcibly enable IOMMU. I'd understand if the patch wasn't actually
> removing functionality but in our case, it seems the patch does just
> this even though the "bug" being addressed doesn't concern me (and
> probably quite a few others) whatsoever. The machine in question is
> my gaming rig primarily, and gets a lot of use in QA, but nothing
> mission critical. I'd rather hobble along with a known bad IVRS table
> if it has no affect on my work (or games) than be locked out entirely
> :p.
>
> I'm still a little confused though as I've heard a few people say that
> this patch wasn't introduced until Xen 4.2 but I saw the issue first
> pop up in Xen 4.1.2 and later. Am I even looking at the right bug?
> And what options are available to forcibly disable the checks other
> than the "iommu=no-amd-iommu-perdev-intremap" ?
>

The issue will occur with both 4.1 and 4.2 - there was a security advisory
( XSA-36 ), the patch for which added in parsing of the IVRS table to
validate certain information being returned (such as the IO-APIC details).
Patches were released for both 4.1 and 4.2.

Regards,

David
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
Feral,

On Tue, May 28, 2013 at 12:40 PM, feral <blistovmhz@gmail.com> wrote:

> I agree entirely about who should be taking responsibility, but we all
> know of course, that Asus will never fix this. There are probably
> less than 5000 people in the world affected by this, and if we assume
> most of them will still buy Asus's next board that has working IVRS
> tables, Asus doesn't care.
>
> I'm more interested in knowing why the
> "iommu=no-amd-iommu-perdev-intremap" option doesn't cover my issue
> though. It seems odd to me that this option was implemented precisely
> to work around the regression issue, but somehow it didn't catch them
> all. Goddamn I can't wait for the Xen Documentation to take form :)
>
>
The problem is that there were two issues which were identified, and that
flag only deals with one of them; the first issue is where the IVRS has two
entries for the IO-APIC (for NB and SB) but has the same id listed for both
(causing a conflict). This means you can't split across the different
IO-APICs because its not sure which is which - the option allows it to use
a global pool, decreasing security but allowing things to work. The second
issue is where the IVRS table doesn't have a valid id listed for the
IO-APIC at all. This second case, because there isn't an id listed, is the
one where it just disables AMD-Vi support.

Regards,

David
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
So next question then. How did IOMMU work before these checks if I'm
affected by the latter bug, where the ID simply isn't listed? Did we
fall back on global pool anyway?

On Tue, May 28, 2013 at 11:25 AM, David Sutton <kantras@gmail.com> wrote:
> Feral,
>
>
> On Tue, May 28, 2013 at 12:40 PM, feral <blistovmhz@gmail.com> wrote:
>>
>> I agree entirely about who should be taking responsibility, but we all
>> know of course, that Asus will never fix this. There are probably
>> less than 5000 people in the world affected by this, and if we assume
>> most of them will still buy Asus's next board that has working IVRS
>> tables, Asus doesn't care.
>>
>> I'm more interested in knowing why the
>> "iommu=no-amd-iommu-perdev-intremap" option doesn't cover my issue
>> though. It seems odd to me that this option was implemented precisely
>> to work around the regression issue, but somehow it didn't catch them
>> all. Goddamn I can't wait for the Xen Documentation to take form :)
>>
>
> The problem is that there were two issues which were identified, and that
> flag only deals with one of them; the first issue is where the IVRS has two
> entries for the IO-APIC (for NB and SB) but has the same id listed for both
> (causing a conflict). This means you can't split across the different
> IO-APICs because its not sure which is which - the option allows it to use a
> global pool, decreasing security but allowing things to work. The second
> issue is where the IVRS table doesn't have a valid id listed for the IO-APIC
> at all. This second case, because there isn't an id listed, is the one where
> it just disables AMD-Vi support.
>
> Regards,
>
> David



--
_____
Fact:
1. Ninjas are mammals.
2. Ninjas fight ALL the time.
3. The purpose of the ninja is to flip out and kill people.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
I own a crosshair v formula with this issue, the problem seems to be the
reported io-apics handles are wrong
so here it is, this hack is for 4.1.5 though it can be adapted for 4.2.2

--- a/xen/drivers/passthrough/amd/iommu_acpi.c 2013-04-23 16:44:20.000000000
+0000
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c 2013-05-18 22:19:18.838434000
+0000
@@ -674,18 +674,18 @@ static u16 __init parse_ivhd_device_spec
*/
for ( apic = 0; apic < nr_ioapics; apic++ )
{
- if ( IO_APIC_ID(apic) != ivhd_device->special.handle )
+ if ( ioapic_bdf[IO_APIC_ID(apic)].bdf !=
ioapic_bdf[ivhd_device->special.handle].bdf )
continue;

- if ( ioapic_bdf[ivhd_device->special.handle].pin_setup )
+ if ( ioapic_bdf[IO_APIC_ID(apic)].pin_setup )
{
- if ( ioapic_bdf[ivhd_device->special.handle].bdf == bdf )
+ if ( ioapic_bdf[IO_APIC_ID(apic)].bdf == bdf )
AMD_IOMMU_DEBUG("IVHD Warning: Duplicate IO-APIC %#x
entries\n",
- ivhd_device->special.handle);
+ IO_APIC_ID(apic));
else
{
printk(XENLOG_ERR "IVHD Error: Conflicting IO-APIC %#x
entries\n",
- ivhd_device->special.handle);
+ IO_APIC_ID(apic));
if ( amd_iommu_perdev_intremap )
return 0;
}
@@ -693,9 +693,9 @@ static u16 __init parse_ivhd_device_spec
else
{
/* set device id of ioapic */
- ioapic_bdf[ivhd_device->special.handle].bdf = bdf;
+ ioapic_bdf[IO_APIC_ID(apic)].bdf = bdf;

- ioapic_bdf[ivhd_device->special.handle].pin_setup =
xzalloc_array(
+ ioapic_bdf[IO_APIC_ID(apic)].pin_setup = xzalloc_array(
unsigned long, BITS_TO_LONGS(nr_ioapic_registers[apic]));
if ( nr_ioapic_registers[apic] &&
!ioapic_bdf[IO_APIC_ID(apic)].pin_setup )




--
View this message in context: http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5716579.html
Sent from the Xen - User mailing list archive at Nabble.com.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
Forgot to add, after this hack:

(XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
(XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 4414.882 MHz processor.
(XEN) Initing memory sharing.
(XEN) AMD-Vi: IOMMU 0 Enabled.
(XEN) AMD-Vi: Enabling per-device vector maps
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled





--
View this message in context: http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5716582.html
Sent from the Xen - User mailing list archive at Nabble.com.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
for 4.2.2:

--- a/xen/drivers/passthrough/amd/iommu_acpi.c 2013-04-23 16:42:55.000000000
+0000
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c 2013-05-31 16:21:14.733159937
+0000
@@ -674,19 +674,19 @@ static u16 __init parse_ivhd_device_spec
*/
for ( apic = 0; apic < nr_ioapics; apic++ )
{
- if ( IO_APIC_ID(apic) != special->handle )
+ if ( ioapic_sbdf[IO_APIC_ID(apic)].bdf !=
ioapic_sbdf[special->handle].bdf )
continue;

- if ( ioapic_sbdf[special->handle].pin_setup )
+ if ( ioapic_sbdf[IO_APIC_ID(apic)].pin_setup )
{
- if ( ioapic_sbdf[special->handle].bdf == bdf &&
- ioapic_sbdf[special->handle].seg == seg )
+ if ( ioapic_sbdf[IO_APIC_ID(apic)].bdf == bdf &&
+ ioapic_sbdf[IO_APIC_ID(apic)].seg == seg )
AMD_IOMMU_DEBUG("IVHD Warning: Duplicate IO-APIC %#x
entries\n",
- special->handle);
+ IO_APIC_ID(apic));
else
{
printk(XENLOG_ERR "IVHD Error: Conflicting IO-APIC %#x
entries\n",
- special->handle);
+ IO_APIC_ID(apic));
if ( amd_iommu_perdev_intremap )
return 0;
}
@@ -694,10 +694,10 @@ static u16 __init parse_ivhd_device_spec
else
{
/* set device id of ioapic */
- ioapic_sbdf[special->handle].bdf = bdf;
- ioapic_sbdf[special->handle].seg = seg;
+ ioapic_sbdf[IO_APIC_ID(apic)].bdf = bdf;
+ ioapic_sbdf[IO_APIC_ID(apic)].seg = seg;

- ioapic_sbdf[special->handle].pin_setup = xzalloc_array(
+ ioapic_sbdf[IO_APIC_ID(apic)].pin_setup = xzalloc_array(
unsigned long, BITS_TO_LONGS(nr_ioapic_entries[apic]));
if ( nr_ioapic_entries[apic] &&
!ioapic_sbdf[IO_APIC_ID(apic)].pin_setup )




--
View this message in context: http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5716583.html
Sent from the Xen - User mailing list archive at Nabble.com.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
Perhaps my brain is stupid today, but I just noticed this reply and am
trying to apply the patch, but both hunks are failing against 4.2.2.

bens@octillion ~/Downloads/xen-4.2.2 $ patch -p1 < ivrs.patch
patching file xen/drivers/passthrough/amd/iommu_acpi.c
Hunk #1 FAILED at 674.
Hunk #2 FAILED at 694.
2 out of 2 hunks FAILED -- saving rejects to file
xen/drivers/passthrough/amd/iommu_acpi.c.rej



On Fri, May 31, 2013 at 12:21 PM, nbhs <santellads@gmail.com> wrote:

> for 4.2.2:
>
> --- a/xen/drivers/passthrough/amd/iommu_acpi.c 2013-04-23
> 16:42:55.000000000
> +0000
> +++ b/xen/drivers/passthrough/amd/iommu_acpi.c 2013-05-31
> 16:21:14.733159937
> +0000
> @@ -674,19 +674,19 @@ static u16 __init parse_ivhd_device_spec
> */
> for ( apic = 0; apic < nr_ioapics; apic++ )
> {
> - if ( IO_APIC_ID(apic) != special->handle )
> + if ( ioapic_sbdf[IO_APIC_ID(apic)].bdf !=
> ioapic_sbdf[special->handle].bdf )
> continue;
>
> - if ( ioapic_sbdf[special->handle].pin_setup )
> + if ( ioapic_sbdf[IO_APIC_ID(apic)].pin_setup )
> {
> - if ( ioapic_sbdf[special->handle].bdf == bdf &&
> - ioapic_sbdf[special->handle].seg == seg )
> + if ( ioapic_sbdf[IO_APIC_ID(apic)].bdf == bdf &&
> + ioapic_sbdf[IO_APIC_ID(apic)].seg == seg )
> AMD_IOMMU_DEBUG("IVHD Warning: Duplicate IO-APIC %#x
> entries\n",
> - special->handle);
> + IO_APIC_ID(apic));
> else
> {
> printk(XENLOG_ERR "IVHD Error: Conflicting IO-APIC %#x
> entries\n",
> - special->handle);
> + IO_APIC_ID(apic));
> if ( amd_iommu_perdev_intremap )
> return 0;
> }
> @@ -694,10 +694,10 @@ static u16 __init parse_ivhd_device_spec
> else
> {
> /* set device id of ioapic */
> - ioapic_sbdf[special->handle].bdf = bdf;
> - ioapic_sbdf[special->handle].seg = seg;
> + ioapic_sbdf[IO_APIC_ID(apic)].bdf = bdf;
> + ioapic_sbdf[IO_APIC_ID(apic)].seg = seg;
>
> - ioapic_sbdf[special->handle].pin_setup = xzalloc_array(
> + ioapic_sbdf[IO_APIC_ID(apic)].pin_setup = xzalloc_array(
> unsigned long, BITS_TO_LONGS(nr_ioapic_entries[apic]));
> if ( nr_ioapic_entries[apic] &&
> !ioapic_sbdf[IO_APIC_ID(apic)].pin_setup )
>
>
>
>
> --
> View this message in context:
> http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5716583.html
> Sent from the Xen - User mailing list archive at Nabble.com.
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>



--
_____
Fact:
1. Ninjas are mammals.
2. Ninjas fight ALL the time.
3. The purpose of the ninja is to flip out and kill people.
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
Given that this is a thread from our fellow AMD users, and all this
talk about IOMMU, I just could not resist. Are there any server based
chipsets, cpus, bios... that stably support IOMMU, and can play nice
with XEN (i.e., pci-passthrough, iommu...)?

We were looking at the SR5690 chipset and the Opteron. Not sure if
this combo is dated now? Please forgive me Intel vt-d guy here, trying
to make the transition.

Kind Regards,

Nick.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
I uploaded it here
http://www.filesend.net/download.php?f=6c7d9422ca9fd1d9be4042fbc6f7fbed



--
View this message in context: http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5717048.html
Sent from the Xen - User mailing list archive at Nabble.com.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
I haven't yet tried it with vanilla Xen, but at least with XenServer 6.2
(which relies on Xen 4.1.5), IOMMU can be forced to initiate on my Asus
M5A99FX Pro R2.0 motherboard (which suffers from an erroneous IVRS table)
using the following:

iommu=no-intremap

More information about this option, which seems to have been released for
Intel setups, can be found here:
http://support.citrix.com/article/CTX136517







On Tue, May 28, 2013 at 12:49 PM, feral <blistovmhz@gmail.com> wrote:

> > In my case, the handle id is wrong ( you can see what it should be on the
> > support document http://support.amd.com/us/Processor_TechDocs/48882.pdf-
> > per tables 77/79 it should be the ID of the IO-APIC). Unlike the conflict
> > error, which can be worked around by passing another command line option,
> > invalid IO-APIC entries in the IVRS table will always cause AMD-Vi to
> fail
> > to be enabled if you have the XSA-36 patch installed.
>
> This is the part I have an angry nerd rage issue with. I'm pretty
> sure I fall into the latter category where there is no option to
> forcibly enable IOMMU. I'd understand if the patch wasn't actually
> removing functionality but in our case, it seems the patch does just
> this even though the "bug" being addressed doesn't concern me (and
> probably quite a few others) whatsoever. The machine in question is
> my gaming rig primarily, and gets a lot of use in QA, but nothing
> mission critical. I'd rather hobble along with a known bad IVRS table
> if it has no affect on my work (or games) than be locked out entirely
> :p.
>
> I'm still a little confused though as I've heard a few people say that
> this patch wasn't introduced until Xen 4.2 but I saw the issue first
> pop up in Xen 4.1.2 and later. Am I even looking at the right bug?
> And what options are available to forcibly disable the checks other
> than the "iommu=no-amd-iommu-perdev-intremap" ?
>
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
feral,

On Thu, Sep 5, 2013 at 2:05 PM, feral <blistovmhz@gmail.com> wrote:

> FYI, I finally got a response from Asus Engineering. Unfortunately
> they're all speaking something I am not, so we had to do 3 layers of
> translation both ways to get it done, but I believe the issue may now be
> resolved.
> I sent them the ACPI dump and gave them the correct addresses for IOMMU 1
> and 2 (or 0 and 1?).
> I'm in the middle of moving into my new place, so haven't had a heap of
> time to test yet, but passthrough did work correctly (HDMI audio even works
> now) and no errors in dmesg. I'm currently testing with KVM.
>
> So that said, attached is the test BIOS we got hacked together. Flash at
> your own risk. I'm running it without any noticeable issues, but as I
> said, I haven't had time to do any rigorous testing or deep eval. on the
> actual IOMMU addresses (ie: haven't dumped ACPI/IVRS yet to confirm
> anything).
>
> Thanks for getting this - unfortunately I have the R1.0 version of this
board, so most likely won't be able to make use of it. Having said that,
I'm watching progress of a patch which could help with this on the
xen-devel list.

Regards,

David
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah [ In reply to ]
I'm very interested to see whether or not Asus has fixed this BIOS bug. How
is the test BIOS working?

Does anyone have an idea which boards are affected, or does it affect all
Asus AMD boards?

How would I get this BIOS fix? To whom would I need to turn to at Asus?

For reference, I've written a little Xen VGA passthrough how-to on the Linux
Mint forum and at least one user has encountered this exact issue - see
</a> <http://forums.linuxmint.com/viewtopic.php?f=42&t=112013> and go to
the last post.



--
View this message in context: http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5718631.html
Sent from the Xen - User mailing list archive at Nabble.com.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users