Question Regarding Using RTEMS with Custom FPGA Ip Cores on ZynqMPSOC

Architecture AArch64
1

Good Morning,

I wanted to learn what the flow and process looks like to interact with custom ip cores with RTEMS on a ZynqMPSoC. So I have 3 ip cores connected to an interconnect that expose an Axi interface with their base addresses. On a baremetal build with Vitis I am able to interact with them by just using the base addresses provided in the xparameters.h file from xilinx.

So what I am wondering is how can I go from that to be able to do something similar with rtems? Is this something where I would need to make a custom bsp for the ZynqMPSoc or can I use the generic BSP and just somehow add custom driver logic using my xparameters file?

2

There are a number of ways to do this so this is some I have used. Sorry I only have Zynq MMU code at hand to paste here. The ZynqMP should similar.

MMU Configuration

For direct AXI access you need to add the address space into the MMU. Unlike Vitis bare metal we run the MMU to support caching and relaxed memory alignment. We do not populate the MMU with any of the possible AXI bus interfaces to avoid opening windows to hardware that does not exist. As a result you need to add your IP address spaces.

You could make some calls or you can do what I normally do, provide an mmu.c file similar to this (this is for a Zynq, ie ARM) linked into my app:

/*
 * MMU regions for our design
 */

#include <bsp.h>
#include <bsp/start.h>

#if __arm__

#include <bsp/arm-cp15-start.h>
#include <bsp/arm-a9mpcore-start.h>

BSP_START_DATA_SECTION static const arm_cp15_start_section_config
zynq_mmu_config_table[] = {
  ARMV7_CP15_START_DEFAULT_SECTIONS,
  {
    .begin = 0x0080000U,
    .end = 0x0100000U,
    .flags = ARMV7_MMU_DEVICE
  },
#if defined(RTEMS_SMP)
  {
    .begin = 0xffff0000U,
    .end = 0xffffffffU,
    .flags = ARMV7_MMU_DEVICE
  },
#endif
  {
    .begin = 0xe0000000U,
    .end = 0xe0200000U,
    .flags = ARMV7_MMU_DEVICE
  }, {
    .begin = 0xf8000000U,
    .end = 0xf9000000U,
    .flags = ARMV7_MMU_DEVICE
  }, {
    .begin = 0x40000000U,
    .end   = 0x80000000U,
    .flags = ARMV7_MMU_DEVICE
  }
};

BSP_START_TEXT_SECTION void zynq_setup_mmu_and_cache(void)
{
  uint32_t ctrl = arm_cp15_start_setup_mmu_and_cache(
    ARM_CP15_CTRL_A,
    ARM_CP15_CTRL_AFE | ARM_CP15_CTRL_Z
  );

  arm_cp15_start_setup_translation_table_and_enable_mmu_and_cache(
    ctrl,
    (uint32_t *) bsp_translation_table_base,
    ARM_MMU_DEFAULT_CLIENT_DOMAIN,
    &zynq_mmu_config_table[0],
    RTEMS_ARRAY_SIZE(zynq_mmu_config_table)
  );
}

#endif /* __arm__ */

Accessing the IP

I use FDT blobs I load with the PL image at runtime. We agree as a team on a register model and format to document the registers and that data generates RTL code with the address map built into the bitfile and a FDT DTS source compiled to a DTB. This avoids software/FPGA mismatches when registers move or get removed.

IP driver initialization asks rtems_fdt for the register address using a string path and that is saved as a pointer to access the register in the driver. For example:

uintptr_t address = 0;
int r = rtems_fdt_get_value(path, "reg", sizeof(address), &address);
if (r != 0) {
  printf("fw:bitfile: invalid address path: %s: %s", 
            path, rtems_fdt_strerror(r));
  return;
}
uint32_t val = *((volatile uint32_t*) address);

Note: I always load the bitfiles at runtime within RTEMS and do not use BOOT.BIN to hold them. I do not like the idea of needing to update boot blocks in flash in fielded equipment. We have Platform Manager (pm) support for ZynqMP to load bitfiles at runtime.

1 Like
3

I forgot to mention, using FDT and rtems_fdt means you can use the RTEMS FDT shell commands to list the FDT nodes as well as read and write from registers. We have the FPGA developers use these commands to test their designs before handing over to software for integration.

1 Like
4

Thank you so much for the information!! I had a follow up question regarding to the mmu.c file, is the file i need to edit: mmu-config.c? I am not exactly sure which BSP i should be using and how I can overwrite the mmu configs with this mmu.c. Or would this mmu.c file be something i build in with my main.c file? Sorry I am kinda lost.

5

Good morning, I had another question aswell, how can I go about adding the shell functions for rtems_fdt and the pm ?

6

ZynqMPSOC BSPs

The file I posed above should work with any of the ZynqMPSOC BSPs. That BSP is a generic aarch64/zynqmp_apu for the APU or Application Processing Unit. I use it to make a BSP for specific hardware. For the K26 it is:

[DEFAULT]
RTEMS_POSIX_API = True

#
# KR26 BSP configuration
#
[aarch64/k26]
INHERIT = zynqmp_apu
RTEMS_SMP = True
BSP_XILINX_ZYNQMP_RAM_LENGTH = 0xFFFF0000
ZYNQ_UART_KERNEL_IO_BASE_ADDR = ZYNQ_UART_1_BASE_ADDR

Custom ZynqMP MMU

I include the mmu.c file I posted at the end of my init.c:

/*
 * Include the MMU set up. This code is in a library so we need to
 * force this code in the executable. This is a simple hack to do
 * that.
 */
#include "mmu.c"

FDT and PM Shell Commands

To add the FDT shell you can make the call:

  rtems_fdt_add_shell_command();

You may need to register the blob with the shell command with:

 auto cmd_handle = ::rtems_fdt_get_shell_handle();
 if (cmd_handle != nullptr) {
   rtems_fdt_release_handle(cmd_handle);
 }
 ::rtems_fdt_dup_handle(bsp_dtb, cmd_handle);

You may need to make a BSP call to the get the blob.

The pm can be added with:

rtems_shell_add_cmd_struct(&bsp_pm_load_shell_command);
7

Good Morning,

Thank you so much for the help!! I was able to get the pm and fdt stood up!!

I had a few more questions, sorry. So first, I was wondering if there was a way to adjust the firmware running on the pm from rtems using the pm commands? Also I was wondering if there was more documentation on the pm commands and utilities, id would like to be able to use that processor in my application.

8
  • The PM firmware is part of Vivado and is loaded at boot from BOOT.BIN. You would need to look into Vivado and how it creates that code to make changes. I believe it is a Microblaze and Vivado ships with the PM sources and a Microblaze compiler.
  • The RTEMS pm command accesses the PM via the ATF module and reports on silicon partner API Xilinx exposes. If you try the pm commnd’s features sub-command the list of features Xilinx exposes can be listed.
1 Like
9

Thank you so much,

Okay last question I promise, for this post at least. How did you get the
value for BSP_XILINX_ZYNQMP_RAM_LENGTH ?

I do not understand how to find out information like that for boards

10

I look at the hardware specifications for the board I am to run RTEMS on and it should tell how much DDR memory you have.

1 Like
11

Good Morning,

I had a few more questions after playing around some more. I am not sure what I am not doing right with regards to loading the bitfile into the fpga using the pm fpga load command:

shell0 [/fpga] # pm fpga load ./core_wrapper.bit
load: ./core_wrapper.bit size: 557056 bytes
error: fpga: load: Invalid argument
shell0 [/fpga] #

Is there something special I need to be doing or something to my bit file from vivado?

Also sorry for more questions regarding how to use stuff but how does DMA work with rtems, are there any drivers or anything that exist that I can use for interacting with regions of memory? Is that something that also falls under the mmu.c configs?

Also I did not quite understand how the mmu.c code you pasted above would work? Would I not need to use the AARCH64_MMU_DEVICE flag instead of the ARMV7_MMU_DEVICE flag? Also would I not be overwriting this function zynqmp_setup_mmu_and_cache instead of zynq_setup_mmu_and_cache ?

12

PM FPGA Loading

The load is performed by the PMU. All the pm command does is dispatch the call to the PMU via the ARM Trusted Firmware (ATF) so you need to make sure that is installed, running and can do the job. The PMU firmware is configured and built buy Vivardo.

The masked ROM in the SoC’s PMU loads the PMU code and then the FSBL for the boot processor, ie APU and that should load the ATF passing control to a second stage bootloader that loads RTEMS and starts it running. The PMU code should be in BOOT.BIN in the boot media.

AARCH64 MMU Configuration

I suggest you review and copy the Xilinx ZynqMP MMU code. As I stated the code I pasted was for an ARM Zynq as it was all I had on hand.

If you create a file called mmu.c in your application and include it at the end of your init.c file the code and symbols will be forced into your application and the linker will ignore any symbols that match.

ZynqMP DMA

It depends on the type of DMA you have implemented in your FPGA and that can be varied. There no generic drivers in RTEMS at the moment that can be reused cleanly.