GTS Transceivers with PCI-Express hard-IP in Agilex 3

Agilex 3 FPGA supports PCI-Express via its four GTS transceiver links. Each GTS physical link (tx+rx) supports up to 12.5Gbps transfer speed. On top of the GTS transceivers the FPGA implements hardened IPs for PCI-Express 3.0 x1-x2-x4, Ethernet 10 Gbe link, and USB 3.0 (only with HPS).

Available PCI-Express Throughput

PCI-Express speed is specified in generations: from 1.0 to 8.0 (planned in 2028), and in port width: from x1 to x16. The table below, copied from wikipedia, shows the expected throughput for the given generation and port width:

Each generation (‘version’ in the table) doubles the available throughput for user payload. Between Generations 1 and 2 the transfer rate (in GT/s) was doubled from 2.5GT/s to 5.0GT/s and thus also the payload throughput was doubled. Between the Generations 2 and 3 the transfer rate was increased from 5GT/s only to 8GT/s (+60%), but the line encoding was much improved from the 8b/10b method (i.e. 20% code overhead) to the 128b/130b method (1.5% overhead), so the total throughput did increase nearly twice.

Agilex 3 natively supports up to four links (x4), but the resulting data rate in Gen3 mode might be a bit too much for some variants of the FPGA. The table below, reproduced from GTS PCIe guide, shows that for the slowest FPGA speed grade -7 (which is the most affordable) it is recommended to not go beyond x2 link width:

Connecting PCIe to the FPGA

The official Agilex 3 Devkit implements PCI-Express x1 end-point. The GTS transceivers are located in FPGA bank 1A, shown below. The x1 interface is highlighted in green. The pins highlighted in yellow need to be connected to support the x4 width.

The photo below shows a simple passive adapter from PCIe to M.2 Key-M connector (also known as NGFF - Next Generation Form Factor). The PCB has markings for standard lengths of M.2 cards: 42mm, 60mm, 80mm.

The PCIe M.2 connector has 75 pins, but most of them are basically optional. In the pinout table of M.2 Key-M below I marked the signals that are necessary:

The strictly necessary signals:

  • REFCLK = reference clock input, 100MHz

  • PERST# = cold reset

  • PERxxx = PCIe receive pair

  • PETxxx = PCIe transmit pair

  • CLKREQ# = clock request, should be pulled low to enable the REFCLK from the host

  • VIO_CFG = should be connected to GND to signalize adapter using 3.3V on sideband signals

  • GND and 3.3V supply

The not-so-necessary signals:

  • SUSCLK = suspend clock 32.768kHz

  • MFG_CLOCK, MFG_DATA = free pins for manufactoring hadling

  • PEWAKE# = wake-up request

  • ALERT#, SMB_DATA, SMB_CLK = the PC SM-Bus, basically an I2C

  • USB_D+/- = USB 2.0 bus

  • PLA_S3 = power-loss signal

  • VIO 1.8V = 1.8V supply input

  • LED_1# = status LED output

  • PLN# = power-loss notification

  • PWRDIS = power disable of the adapter

  • PEDET = host interface indication; GND for SATA, no connect for PCIe

Overview of PCIe Demo Design in the FPGA

The picture below from Platform Designer (QSYS) shows a demo test-design for the PCIe IP in Agilex 3. The DUT is GTS AXI Streaming IP. Its FPGA-exported signals are refclock, the PCIe serial tx/rx data and the perstn signal; these are just the ‘strictly necessary’ signals mentioned above.

The GTS AXI Streaming IP handles config space registers including BARs and capabilities:

Internally the IP presents a lot of signals and interfaces. The most important are the AXI4 Streams that provide (RX) and accept (TX) the entire Transaction Layer Packets (TLP). The TLP includes request header with memory address, length, and optional payload (in case of writes). It is the task of upstream user logic to map addresses from TLP to target functions (or memories) in the design.