Skip to content

PCIe: Update documentation and platform support #636

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AkshGarg-19 wants to merge 6 commits into
base: master
Choose a base branch
from
Open

PCIe: Update documentation and platform support #636

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8c4f7b5
fix(configs): AM68x: Remove PCIe Endpoint and Backplane Documentation
AkshGarg-19 Feb 27, 2026
5a85fc6
fix(configs): AM69/AM69A: Remove PCIe Backplane Documentation
AkshGarg-19 Feb 27, 2026
82ad38f
fix(configs): J721S2: Remove PCIe Backplane Documentation
AkshGarg-19 Feb 27, 2026
dffa06a
fix(configs): TDA4VM: Remove PCIe Endpoint and Backplane Documentation
AkshGarg-19 Feb 27, 2026
905581b
fix(linux): Enhance PCIe Backplane documentation for multiple SoCs
AkshGarg-19 Feb 27, 2026
8f0b3e5
fix(linux): Add PCIe resource reallocation note for VF enumeration
AkshGarg-19 Feb 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/styles/config/vocabularies/PSDK/accept.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Vulkan
Weston
Yocto
Zink
[Bb]ackplane
[Cc]odec
[Dd]unfell
[Ee]thernet
Expand Down
2 changes: 0 additions & 2 deletions configs/AM68/AM68_linux_toc.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,6 @@ linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-EST
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-CBS
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-IET
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-TSN-Tuning
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_End_Point
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Backplane
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Root_Complex
linux/Foundational_Components/Kernel/Kernel_Drivers/PMIC/pmic_tps6594
linux/Foundational_Components_Power_Management
Expand Down
2 changes: 0 additions & 2 deletions configs/AM68A/AM68A_linux_toc.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,6 @@ linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-EST
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-CBS
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-IET
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-TSN-Tuning
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_End_Point
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Backplane
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Root_Complex
linux/Foundational_Components/Kernel/Kernel_Drivers/PMIC/pmic_tps6594
linux/Foundational_Components_Power_Management
Expand Down
1 change: 0 additions & 1 deletion configs/AM69/AM69_linux_toc.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-CBS
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-IET
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-TSN-Tuning
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_End_Point
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Backplane
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Root_Complex
linux/Foundational_Components/Kernel/Kernel_Drivers/PMIC/pmic_tps6594
linux/Foundational_Components_Power_Management
Expand Down
1 change: 0 additions & 1 deletion configs/AM69A/AM69A_linux_toc.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-CBS
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-IET
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-TSN-Tuning
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_End_Point
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Backplane
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Root_Complex
linux/Foundational_Components/Kernel/Kernel_Drivers/PMIC/pmic_tps6594
linux/Foundational_Components_Power_Management
Expand Down
1 change: 0 additions & 1 deletion configs/J721S2/J721S2_linux_toc.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,6 @@ linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-CBS
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-IET
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-TSN-Tuning
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_End_Point
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Backplane
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Root_Complex
linux/Foundational_Components/Kernel/Kernel_Drivers/PMIC/pmic_tps6594
linux/Foundational_Components_Power_Management
Expand Down
2 changes: 0 additions & 2 deletions configs/TDA4VM/TDA4VM_linux_toc.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,6 @@ linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-EST
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-CBS
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-IET
linux/Foundational_Components/Kernel/Kernel_Drivers/Network/CPSW-TSN-Tuning
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_End_Point
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Backplane
linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Root_Complex
linux/Foundational_Components/Kernel/Kernel_Drivers/PMIC/pmic_tps6594
linux/Foundational_Components_Power_Management
Expand Down
78 changes: 44 additions & 34 deletions source/linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Backplane.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@
controller instance and each host will enumerate the other host as an
independent function.

PCIe uses NTB (non transparent bridge) for two hosts to communicate with each
other. Though J721E doesn't have an explicit NTB controller, NTB functionality
can be achieved using multiple endpoint controller instances. And for PCIe
backplane (to connect more than 2 hosts), aggregation of NTB controllers
can be modeled using multiple instances of multi-function endpoint controller.
PCIe uses Non Transparent Bridge (NTB) for two hosts to communicate with each
other. Though |__PART_FAMILY_DEVICE_NAMES__| does not have an explicit NTB
controller, multiple endpoint controller instances offer NTB functionality.

Check warning on line 19 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Backplane.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.SimpleWords] Use simple language. Consider using 'many' rather than 'multiple'.

Raw Output:
{"message": "[RedHat.SimpleWords] Use simple language. Consider using 'many' rather than 'multiple'.", "location": {"path": "source/linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Backplane.rst", "range": {"start": {"line": 19, "column": 13}}}, "severity": "INFO"}
For PCIe backplane (to connect more than 2 hosts), multiple instances of

Check warning on line 20 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Backplane.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.SimpleWords] Use simple language. Consider using 'many' rather than 'multiple'.

Raw Output:
{"message": "[RedHat.SimpleWords] Use simple language. Consider using 'many' rather than 'multiple'.", "location": {"path": "source/linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Backplane.rst", "range": {"start": {"line": 20, "column": 52}}}, "severity": "INFO"}
multi-function endpoint controller create the aggregation of NTB controllers.

In the below diagram, PCI NTB function configures the SoC with multiple
PCIe Endpoint (EP) instances in such a way that transaction from one EP
Expand Down Expand Up @@ -63,6 +63,12 @@
The following picture shows J721E EVM connected to two DRA7 EVMs. Here the two
DRA7x boards communicate with each other using J721E as backplane.

.. ifconfig:: CONFIG_part_variant in ('J784S4','J742S2')

Similarly, you can connect |__PART_FAMILY_DEVICE_NAMES__| to two hosts, who
can communicate with each other using |__PART_FAMILY_DEVICE_NAMES__|
as backplane.

.. Image:: /images/j721e-backplane.jpg


Expand All @@ -71,21 +77,33 @@

.. rubric:: *Backplane DTS Overlay File*

The following DTS overlay file configures the PCIe controller in EP mode and
also contains a device tree node to create a NTB function device:
The following device tree overlay files configure both the PCIe controller in EP mode
required for NTB functionality:

::
.. ifconfig:: CONFIG_part_variant in ('J784S4','J742S2')

arch/arm64/boot/dts/ti/k3-j721e-pcie-backplane.dtso
* :file:`arch/arm64/boot/dts/ti/k3-j784s4-evm-pcie0-pcie1-ep.dtso`

In order to apply the dts overlay file, the following command should be given
in u-boot prompt:
.. ifconfig:: CONFIG_part_variant not in ('J784S4','J742S2')

::
* :file:`arch/arm64/boot/dts/ti/k3-j721e-evm-pcie0-ep.dtso`
* :file:`arch/arm64/boot/dts/ti/k3-j721e-evm-pcie1-ep.dtso`

To apply the device tree overlay files, run the following command in the u-boot prompt:

.. ifconfig:: CONFIG_part_variant in ('J784S4','J742S2')

#setenv name_overlays ti/k3-j721e-pcie-backplane.dtbo
.. code-block:: console

.. rubric:: *EP Side Configuration (J721E Backplane)*
setenv name_overlays ti/k3-j784s4-evm-pcie0-pcie1-ep.dtbo

.. ifconfig:: CONFIG_part_variant not in ('J784S4','J742S2')

.. code-block:: console

setenv name_overlays ti/k3-j721e-evm-pcie0-ep.dtbo ti/k3-j721e-evm-pcie1-ep.dtbo

.. rubric:: *EP Side Configuration (Backplane)*
:name: ep-side-configuration

.. rubric:: **Dip switch settings**
Expand All @@ -100,9 +118,9 @@
Both PCIe instances should be configured in EP mode by setting
PCIE_1L_MODE_SEL (switch 5) and PCIE_2L_MODE_SEL (switch 6) in sw3 to '1'.

.. rubric:: **8.x SDK (5.10 Kernel)**
.. rubric:: **EP Backplane Configuration Steps**

The following set of steps is required only for 5.10 Kernel
Follow these steps to configure the PCIe endpoints as a backplane:

.. rubric:: Creating pci-epf-ntb device

Expand All @@ -111,6 +129,7 @@

::

# modprobe pci_epf_ntb
# mount -t configfs none /sys/kernel/config
# cd /sys/kernel/config/pci_ep/
# mkdir functions/pci_epf_ntb/func1
Expand Down Expand Up @@ -151,15 +170,8 @@
# echo 0x104c > functions/pci_epf_ntb/func1/vendorid
# echo 0xb00d > functions/pci_epf_ntb/func1/deviceid

In order to configure NTB specific attributes, a new sub-directory to func1
should be created

::

# mkdir functions/pci_epf_ntb/func1/pci_epf_ntb.0/

The NTB function driver will populate this directory with various attributes
that can be configured by the user
The NTB function driver also populates :file:`func1/pci_epf_ntb.0` directory with
various attributes that can be configured by the user

::

Expand Down Expand Up @@ -188,13 +200,13 @@

::

# ln -s controllers/2900000.pcie-ep/ functions/pci-epf-ntb/func1/primary
# ln -s controllers/2910000.pcie-ep/ functions/pci-epf-ntb/func1/secondary
# ln -s controllers/2900000.pcie-ep/ functions/pci_epf_ntb/func1/primary
# ln -s controllers/2910000.pcie-ep/ functions/pci_epf_ntb/func1/secondary

Once the above step is completed, both the PCI endpoint controllers are ready to
establish a link with the host.

.. rubric:: Start the Link: 7.x and 8.x SDK (5.4 and 5.10 Kernel)
.. rubric:: Start the Link:

In order for the endpoint device to establish a link with the host, the _start_
field should be populated with '1'. For NTB, both the PCI endpoint controllers
Expand All @@ -205,9 +217,6 @@
# echo 1 > controllers/2900000.pcie-ep/start
# echo 1 > controllers/2910000.pcie-ep/start

(PCIe2 can also be configured for NTB, but that is not
tested yet).

.. rubric:: *RC Side Configuration*
:name: rc-side-configuration

Expand All @@ -221,6 +230,7 @@

::

modprobe ntb_hw_epf
echo 0000:01:00.0 > /sys/bus/pci/devices/0000\:01\:00.0/driver/unbind

After unbinding from existing driver, it should be bound to ntb_hw_epf driver.
Expand All @@ -241,14 +251,14 @@
.. rubric:: **Kernel Configs**
:name: kernel-configs

.. rubric:: *EP Side (J721E Backplane)*
.. rubric:: *EP Side (Backplane)*
:name: ep-side-configs

::

CONFIG_PCI_ENDPOINT=y
CONFIG_PCI_ENDPOINT_CONFIGFS=y
CONFIG_PCI_EPF_NTB=y
CONFIG_PCI_EPF_NTB=m
CONFIG_PCI_J721E=y
CONFIG_PCIE_CADENCE=y
CONFIG_PCIE_CADENCE_EP=y
Expand All @@ -274,5 +284,5 @@

::

<Processor_SDK_install_dir>/board-support/linux-[ver]/Documentation/PCI/endpoint/pci-test-ntb.txt
<Processor_SDK_install_dir>/board-support/linux-[ver]/Documentation/PCI/endpoint/pci-ntb-function.rst

106 changes: 60 additions & 46 deletions .../linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Root_Complex.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,70 +146,84 @@ Following is a brief explanation of layers shown in the diagram:

.. ifconfig:: CONFIG_part_family in ('AM64X_family','J7_family')

.. rubric:: **RC Device Configuration**
:name: rc-device-configuration
.. rubric:: **RC Device Configuration**
:name: rc-device-configuration

.. rubric:: *DTS Modification*
:name: rc-dts-modification
.. rubric:: *DTS Modification*
:name: rc-dts-modification

The default dts for |__PART_FAMILY_DEVICE_NAMES__| is configured to be used in
root complex mode.
The default dts for |__PART_FAMILY_DEVICE_NAMES__| is configured to be used in
root complex mode.

.. rubric:: *Linux Driver Configuration*
:name: linux-driver-configuration
.. rubric:: *Linux Driver Configuration*
:name: linux-driver-configuration

The following config options have to be enabled in order to configure the
PCI controller to be used in Root Complex mode.
The following config options have to be enabled in order to configure the
PCI controller to be used in Root Complex mode.

::
::

CONFIG_SOCIONEXT_SYNQUACER_PREITS=y
CONFIG_PCI=y
CONFIG_PCI_MSI=y
CONFIG_PCI_J721E=y
CONFIG_PCIE_CADENCE=y
CONFIG_PCIE_CADENCE_HOST=y

.. rubric:: **Compliance Mode**
:name: compliance-mode

In RC mode of operation, the Endpoint device can be forced to enter
Compliance Mode for PCIe compliance testing by setting the "EC" bit
in the LINK_CTRL_STATUS_2 register of the respective PCIe RC instance.
Setting "EC" to 1 initiates a hot reset thereby forcing the Endpoint
device into Compliance mode. The "EC" bit can be set from command-line
by using devmem2 utility as follows:

devmem2 <address> w <value>

where <address> is the address of LINK_CTRL_STATUS_2 register and
<value> is the resulting value to be written with "EC" bit of the
register set.

CONFIG_SOCIONEXT_SYNQUACER_PREITS=y
CONFIG_PCI=y
CONFIG_PCI_MSI=y
CONFIG_PCI_J721E=y
CONFIG_PCIE_CADENCE=y
CONFIG_PCIE_CADENCE_HOST=y
.. rubric:: **64-Bit Address Space with 4 GB Size**
:name: 64-bit-address-space

.. rubric:: **Compliance Mode**
:name: compliance-mode
The PCIe Controller support for 64-Bit addressing in the System's
Address Space with 4 GB Size is enabled in the device-tree.
The 4 GB region is split as:

In RC mode of operation, the Endpoint device can be forced to enter
Compliance Mode for PCIe compliance testing by setting the "EC" bit
in the LINK_CTRL_STATUS_2 register of the respective PCIe RC instance.
Setting "EC" to 1 initiates a hot reset thereby forcing the Endpoint
device into Compliance mode. The "EC" bit can be set from command-line
by using devmem2 utility as follows:
1. 4 KB ECAM region for Configuration Accesses
2. 1 MB IO region
3. Remaining region (4 GB - 1 MB - 4 KB) as 32-bit Non-Prefetchable MEM

devmem2 <address> w <value>
.. note::

where <address> is the address of LINK_CTRL_STATUS_2 register and
<value> is the resulting value to be written with "EC" bit of the
register set.
If the root complex is connected to an endpoint with virtual functions,
and the kernel panics during enumeration, add ``pci=realloc`` to ``optargs``
at the U-Boot prompt using the following command:

.. rubric:: **64-Bit Address Space with 4 GB Size**
:name: 64-bit-address-space
.. code-block:: console

The PCIe Controller support for 64-Bit addressing in the System's
Address Space with 4 GB Size is enabled in the device-tree.
The 4 GB region is split as:
setenv optargs 'pci=realloc'

1. 4 KB ECAM region for Configuration Accesses
2. 1 MB IO region
3. Remaining region (4 GB - 1 MB - 4 KB) as 32-bit Non-Prefetchable MEM
The ``pci=realloc`` parameter instructs the Linux kernel to reallocate PCI
bridge resources. This helps resolve resource conflicts during enumeration
of PCIe devices by allowing the kernel to reassign memory and I/O addresses.

.. rubric:: **Testing Details**
:name: testing-details
.. rubric:: **Testing Details**
:name: testing-details

The RC should enumerate any off-the-shelf PCIe cards. It has been tested
with Ethernet cards, NVMe cards, PCIe USB card, PCIe WiFi card, PCIe SATA
card and also to |__PART_FAMILY_DEVICE_NAMES__| in loopback mode.
The RC should enumerate any off-the-shelf PCIe cards. It has been tested
with Ethernet cards, NVMe cards, PCIe USB card, PCIe WiFi card, PCIe SATA
card and also to |__PART_FAMILY_DEVICE_NAMES__| in loopback mode.

In order to see if the connected card is detected, lspci utility should be
used. Different utilities can be used depending on the cards.
In order to see if the connected card is detected, lspci utility should be
used. Different utilities can be used depending on the cards.

Following are the outputs for some of them:
Following are the outputs for some of them:

- Loopback mode (|__PART_FAMILY_DEVICE_NAMES__| EVM to |__PART_FAMILY_DEVICE_NAMES__| EVM)
- Loopback mode (|__PART_FAMILY_DEVICE_NAMES__| EVM to |__PART_FAMILY_DEVICE_NAMES__| EVM)

Two |__PART_FAMILY_DEVICE_NAMES__| EVMs can be connected in loopback mode by following
the steps explained in
Expand Down
Loading