SCAPP / RMDA Problems and Solutions

DMA Timeouts

User program reports timeouts during DMA transfer. Check the output of dmesg. If it contains lines like

spcm4_cards 0000:02:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT

then probably the IOMMU (Input output memory management unit) is the reason.
Possible fixes:

1. Check with the manufacturer of the mainboard if a BIOS/UEFI update is available.
2. Disable the IOMMU in BIOS/UEFI. On Intel Systems it might be called VT-d.
3. Pass either “iommu=off” or “iommu=soft” to the kernel by editing the GRUB command line
   
   

Timeouts with AWG cards

On multiple systems RDMA with AWG cards ended in timeouts, but without the above-mentioned IO_PAGE_FAULT log entry. In this case, please check the manual of the mainboard if the PCIe slot of the AWG card can share its lanes with another slot. Common setups include “One slot with 16 lanes or two slots with eight lanes each” or “One slot with eight lanes and two slots with four lanes each”. The manual of the mainboard may mention this with entries like “2 x PCI Express 3.0/2.0 x16 slots (single at x16 or dual at x8&x8 mode)” or “Two PCIe 3.0 x8/x16 Slots (CPU SLOT4, CPU SLOT6: Supports auto switch)”. If possible, move the AWG card to a slot that cannot share its lanes with another slot.