Modifications of files to configure BSP have no effect

Dears, I am working on implementation of RTEMS on a custom-made board based on Microsemi Polarfire SoC. This SoC contains five Risc-V harts: One e51 hart of rv64imac architecture and four u54 harts of rv64gc architecture.

For the final solution, I will need new fully customized BSP. However, for the first tests, I decided to modify beaglevfire or rv64imac to allow run of a simple code on e51 hart. However, the modifications have no effect.

  • The question is: What to do to appear the changes in BSP?

More detailed description of the problem:

The beaglevfire BSP looks fine but it is configured to have rv64imafdc architecture on all harts. It is wrong (!) as e51 core is rv64 imac. The rv64imac BSP also looks good but it is configured to use u-boot but I run the code by debugger.

To configure named BSPs for my purpose, I modified files:
bsps/riscv/riscv/config/beaglevfire.cfg
bsps/riscv/riscv/config/rv64imac.cfg
bsps/riscv/shared/start/start.S
spec/build/bsps/optfdtuboot.yml

However, the changes have absolutely no effect.
I tried to make an obvious error in some of named files but the build was still successful! It seems that sb-set-builder takes these files from elsewhere - but where?

To build the tool suite, I use the command ran from $HOME/sandbox/src/rsb/rtems folder as recommended by RTOS 6.1 user manual (I use folder sandbox instead of quick-start):

…/source-builder/sb-set-builder --prefix=$HOME/sandbox/rtems/6 6/rtems-riscv

To build the BSP, I use commands:

…/source-builder/sb-set-builder --prefix=$HOME/sandbox/rtems/6 --target=riscv-rtems6 --with-rtems-bsp=riscv/beaglevfire --with-rtems-tests=yes 6/rtems-kernel

respective

…/source-builder/sb-set-builder --prefix=$HOME/sandbox/rtems/6 --target=riscv-rtems6 --with-rtems-bsp=riscv/rv64imac --with-rtems-tests=yes 6/rtems-kernel

I tried to add option --always-clean which had absolutely no effect.

The only thing I successfuly changed is linkcmds to run the binary in LIM.

How are you modifying the files in RTEMS if you’re building the BSP using RSB?

Generally, if you need to make changes to the BSPs, you’ll need to build RTEMS by hand using the waf builder. RSB will always pull a particular hash from the public repositories to build its packages.

Hello, I did this exercise a while ago, you can find the relevant commits here : Commits · fcuzzocrea/rtems · GitHub

I tested it on the icicle kit.

I use a plain ascii text editor to modify the files. Why do you ask that?

It is very strange because I tried to disconnect the computer from internet before the build but the resulting binary is completely the same as with original configuration files.

Of course, there can exist a local hash. However, I would expect that there should exist some command or option to clean all results of the previous build and start new complete build with current files.

RSB will always reset its build sources to a known condition for consistent results. If you need to modify the RTEMS sources then you’ll need to build manually outside of RSB. For RTEMS:

  • clone the repo
  • make your changes
  • run ./waf configure --prefix=ABS_INSTALL_DIR
  • run ./waf install

Hello,

even manual RSB build does not work in the way you described.

I still try to customize beaglevfire BSP for my application.

The comand ./waf configure --prefix=… overwrites some files and I cannot find the origin where their contents is taken from.

The examples of the overwritten files are:
…/src/build/riscv/beaglevfire/beaglevfire.cfg
…/src/build/riscv/beaglevfire/target.cfg
…/src/build/riscv/beaglevfire/linkcmds
…/src/build/riscv/beaglevfire/riscv-rtems6-beaglevfire.pc
However, I need to modify the content of these files or modify the source of which the contents of these files is based on.

I would expect, for example, that …/src/build/riscv/beaglevfire/beaglevfire.cfg is a copy of …/src/bsps/riscv/riscv/config/beaglevfire.cfg
However, the file …/src/bsps/riscv/riscv/config/beaglevfire.cfg is not used for anything. I can delete this file and BSP build as well as application build normally works.

I would expect, for example, that the contents of the linker script linkcmds is based on the file …/src/bsps/riscv/riscv/dts/beaglevfire.dts as it contains definition of the memory. However, I can delete the file beaglevfire.dts. This file is not used for anything, too.

Therefore, I work in this way:

  1. run ./waf configure --prefix=…
  2. make my changes
  3. run ./waf
  4. run ./waf install

However, I consider it a workaround.

I tried to search the documentation but I did not find a sufficient help how to customize a BSPs or how to make a custom-made BSP.

Can somebody help, please?

The .cfg files are not relevant for building the BSP manually using RTEMS. This information is now contained within the spec/ tree. For the beaglevfire, you need to look at spec/build/bsps/riscv/riscv/abi.yml. The linker script (linkcmds) is also generated from the spec tree at spec/build/bsps/riscv/linkcmds.yml. Note that both of these files will require a new ./waf configure ... and a project clean (./waf clean) to take effect.

The package-config (.pc) is generated from information in the spec/ tree.

The beaglevfire.dts file does not appear to be automatically converted, but is instead manually compiled into bsps/riscv/riscv/include/bsp/beaglevfire.h as needed. This is a deficiency in our build system that we have not yet corrected.

The workaround you describe is the normal manual way of building RTEMS. This is used when you need to modify RTEMS beyond the changes possible in config.ini.

Typically, you customize a BSP using config.ini. If you need changes beyond that, we can look at adding configuration parameters for other settings or creation of whole new BSP that contains presets and additions for your hardware variant. What board are you targeting?

Dear,

thank you for the helpful answer. A port to the custom-made board based on Microchip Polarfire SoC containing RISC-V based microcontroller subsystem (MSS) represents the final goal. However, I can use a Beagle-V Fire board for the first experiments. RTEMS 6.1 contains a BSP for this board but this BSP seems unfinished.

So, as the first step, I would like to repair the existing RTEMS 6.1 BSP for the Beagle-V board and run hello world or flashing LED on hart 0 of the Beagle-V Fire board.

Particular problems:

  1. gcc options:

Problem:
1a) Instruction set. MSS contains e51 hart of rv64imac architecture and four u54 harts of rv64imafdc architecture. However, by default, BSP as well as the application is compiled with -march=rv64imafdc option.

1b) Switch off optimization. To be able to step the code in the debugger, it is better to switch off optimization. Default optimization option is -O2 but I would like to change it to -O0.

Used solution:
I modified beaglevfire_cache.cfg, riscv-rtems6-beaglevfire.pc and target.cfg to use -O0 and -march=rv64imac gcc options. With this option, I am not able to use extensions of u54 harts but it does not matter at this moment. At this moment, I would like to run hello world on e51 hart and suspend other harts.

  1. Memory mapping.

Problem:
The current Beagle V Fire port uses RAM starting at 0x10_0000_0000 which represents cached DDR DRAM. However, the DRAM must be trained before the use and the start-up code does not perform DRAM training. Therefore, the application had no chance to run and really, in the debugger I saw that the code failed on the first instruction. Therefore, I decided to change memory mapping to LIM staring at 0x0800_0000. LIM does not need any initialization or training and it represents the ideal memory for the first experiments with MSS. In the Microchip PFSoC MSS Configurator, I configured MSS to have size of 0x10_0000 bytes (1MB). I am sure that LIM is well configured because I use it for other applications which do not use RTEMS.

Used solution:
To configure RTEMS to use LIM, I modified linkcmds and beaglevfire.dts to use memory starting 0x0800_0000 of 0x10_0000 bytes. I used dtc and xxd to create new beaglevfire.h. Than, the code was possible to start but it failed in filesystem initialization. I switched it off using #define CONFIGURE_APPLICATON_DISABLE_FILESYSTEM in init.c
It seems I did everything well as the code passed until the function _Thread_Start_multitasking.

  1. To start symmetric multiprocessoring (SMP) or not to start SMP (and how to start SMP)?

After named mods, code passed until this function

void _Thread_Start_multitasking( void )
{
Per_CPU_Control *cpu_self = _Per_CPU_Get();
Thread_Control *heir;

#if defined(RTEMS_SMP)
_Per_CPU_Set_state( cpu_self, PER_CPU_STATE_UP );
_SMP_Try_to_process_message( cpu_self, SMP_MESSAGE_FORCE_PROCESSING );
#endif

heir = _Thread_Get_heir_and_make_it_executing( cpu_self );

_Profiling_Thread_dispatch_disable( cpu_self, 0 );

#if defined(RTEMS_SMP)
_CPU_SMP_Prepare_start_multitasking();
#endif

#if defined(_CPU_Start_multitasking)
_CPU_Start_multitasking( &heir->Registers );
#elif defined(RTEMS_SMP)
#if CPU_ENABLE_ROBUST_THREAD_DISPATCH == TRUE
#error “The CPU port has to provide _CPU_Start_multitasking()”
#endif
{
Context_Control trash;

/*
 * Mark the trash context as executing to not confuse the
 * _CPU_Context_switch().  On SMP configurations the context switch
 * contains a special hand over section to atomically switch from the
 * executing to the currently selected heir thread.
 */
_CPU_Context_Set_is_executing( &trash, true );
_CPU_Context_switch_no_return( &trash, &heir->Registers );
RTEMS_UNREACHABLE();

}
#else
_CPU_Context_Restart_self( &heir->Registers );
#endif
}

As by default, SMP is off, macro RTEMS_SMP is not defined. At next, if the macro RTEMS_SMP is not defined, the function _CPU_Start_multitasking is not defined, too. The function _Thread_Start_multitasking passes to the line _CPU_Context_Restart_self( &heir->Registers ); The code fails (jumps to exception handler) in the function _CPU_Context_Restart_self.

I tried to build few other BSPs (sparc, arm, …) and all of them define the function _CPU_Start_multitasking.

I tried to use SMP but not sure how to do it. I did not found the proper place, where to place #define RTEMS_SMP. I tried to define it via gcc parameter -DRTEMS_SMP. However, BSP build failed because of plenty of missing references.

I tried to set RTEMS_HAS_MULTIPROCESSORING = yes in target.cfg. BSP build and application build both passed. The function _CPU_Start_multitasking was defined. However, the code failed in this function:
static inline void _Scheduler_priority_Ready_queue_initialize(
Chain_Control *ready_queues,
Priority_Control maximum_priority
)
{
size_t index;

for ( index = 0 ; index <= (size_t) maximum_priority ; ++index ) {
_Chain_Initialize_empty( &ready_queues[ index ] );
}
}

I found that maximum_priority had an excessive value greater than 100000000. Next, I found that some sources were compiled with the macro RTEMS_SMP and some sources were compiled without this macro. So, the build is inconsistent and probably it is the reason of the uncanny behavior.

Currently the principal question is if to start SMP and how or what to do to make the code operational if SMP is off.

Other question if SMP is on - how to define codes for u54 harts. At this point, I would use e.g. wfi to let u54 harts wait forever.

1a: As per the RTEMS RISC-V PolarFire documentation, RTEMS does not currently support operation on the E51 core. Any modifications necessary to support this core are not fixes for things that are broken, but new work.

1b: Optimization flags can be overridden in config.ini using the OPTIMIZATION_FLAGS option. The replacement optimization flags should include at least -fdata-sections -ffunction-sections.

2: If you need RTEMS to run from LIM, this can be set by changing the RAM origin and size using RISCV_RAM_REGION_BEGIN and RISCV_RAM_REGION_SIZE in config.ini.

3: If you are running on the E51 core, you will not be running SMP. For the E54 cores, SMP is started automatically. The CONFIGURE_MAXIMUM_PROCESSORS define in your application (before the confdefs.h include) determines the number of cores your application can use. There are SMP tests you can run in the testsuite that will exercise these. If you need SMP, you can enable RTEMS_SMP in config.ini.

Dear Opticron,

thank you for the answer.

However, I tried to start with the memory configuration which represents one of key problems and your hint does not work:

This is my config.ini - memory region you advised and recommended settings by manual.

[riscv/beaglevfire]
BUILD_TESTS = True

RTEMS_POSIX_API = True
RTEMS_SMP = False
BSP_START_COPY_FDT_FROM_U_BOOT = False
BSP_VERBOSE_FATAL_EXTENSION = False

RISCV_RAM_REGION_BEGIN = 134217728
RISCV_RAM_REGION_SIZE = 1048576

I am not sure if I can use hexadecimal values.

This is the error I got after ./waf

/home/ladinek/sandbox/rtems/6/lib/gcc/riscv-rtems6/13.3.0/…/…/…/…/riscv-rtems6/bin/ld:linkcmds.base:330 cannot move location counter backwards (from 000000000818a028 to 0000000008100000)
collect2: error: ld returned 1 exit status

Build failed
→ task in ‘testsuites/fstests/fsdosfsname01.exe’ failed with exit status 1:
{task 124634732390768: cprogram init.c.98.o → fsdosfsname01.exe}
[‘/home/ladinek/sandbox/rtems/6/bin/riscv-rtems6-gcc’, ‘testsuites/fstests/fsdosfsname01/init.c.98.o’, ‘-o/home/ladinek/sandbox/src/build/riscv/beaglevfire/testsuites/fstests/fsdosfsname01.exe’, ‘-Wl,-Bstatic’, ‘-L.’, ‘-lrtemstest’, ‘-lrtemscpu’, ‘-lrtemsbsp’, ‘-Wl,-Bdynamic’, ‘-qrtems’, ‘-march=rv64imafdc’, ‘-mabi=lp64d’, ‘-mcmodel=medany’, ‘-Wl,–gc-sections’, ‘-L/home/ladinek/sandbox/src/bsps/riscv/shared/start’, ‘-L/home/ladinek/sandbox/src/bsps/riscv/riscv/start’, ‘-Wl,–wrap=printf’, ‘-Wl,–wrap=puts’, ‘-Wl,–wrap=putchar’]

I tried it with new RTEMS installation which is not affected by my previous experiments.

The manual is confusing about harts. I read that e51 is reserved for the first stage bootloader but it is written in chapter 8.13.2 Microchip Polarfire SoC which is related to mpfs64imafdc BSP (which is not included in RTOS 6.1). If e51 is reserved for bootloader in all RISC-V BSPs, it shoud be written in section 8.13.1.

Anyway, what do you think about these problems:

  • What will be easier to get operational code - start SMP or not?
  • Is it a nominal situation when the function _CPU_Start_multitasking is missing?

Thanks

I tried to increase RISCV_RAM_REGION_SIZE to 2MB and it solved the problem with the error “ld:linkcmds.base:330 cannot move location counter backwards”

However, LIM cannot be configured to 2MB size as at least one of L2 cache ways must be on and therefore the maximal available size of LIM is 1920KB. When I tried to change RISCV_RAM_REGION_SIZE to something between 1MB and 2MB, ./waf configure… reported that the memory region size must be aligned to 1MB. So, it seems that there is no other option but run the code in DDR DRAM.

I will try to build the code with pre-defined options and run it in DDR DRAM. I will tell if I report some problems.

Some tests are known to be too large to fit on small memory systems. In the error you show, the test is fsdosfsname01 and this is definitely one of the tests that is too large to fit in small memory spaces. If you want to exclude such tests, you can add a link to ../../tstsmallmem in the BSP configuration spec file. The remainder of the tests should build and link correctly and be runnable.

When you change settings in config.ini, you need to reconfigure (./waf configure) and then do a clean build (./waf clean build) to ensure there are no stale objects with the old configuration. This is likely what caused the missing function.

1 Like

Thanks,

meanwhile, I tried to set BUILD_TEST = False which helped, too. I do not understand what is “BSP configuration spec file”.

At next, problems with the architecture setting persists. The BSP as well as the application is built with -march=rv64imafdc option regardless RTEMS_SMP is set to True or False. However, you wrote that if SMP is off then the build is done for e51 hart. It does not look so as the used architecture does not match the architecture of this hart.

Please, what is the nominal way how to set the architecture option? For the simplicity, I would like to use -march=rv64imac anytime and on any hart to prevent problems. I do not need floating point operations at this moment.

The function _CPU_Start_multitasking is missing anytime SMP is off. It is not a problem of the cleaning.

Take a look at cpu.h:

(line 460) #ifdef RTEMS_SMP

(line 504) #define _CPU_Start_multitasking( _heir ) _RISCV_Start_multitasking( _heir )

#endif /* RTEMS_SMP */

Maybe, it is fine. Maybe, it is a bug in the BSP.

No, the E51 hart on that platform is not currently supported regardless of other settings.

The “BSP configuration spec file” that I’m referring to is spec/build/bsps/riscv/riscv/bspmpfs64imafdc.yml. You could add a reference to tstsmallmem to turn off building larger tests.

If you want to try getting something to work on the E51 hart, you should be using the rv64imac generic BSP. You may need to create a new BSP to support this part of the PolarFire chip with appropriate peripherals. If you really want to use the existing mpfs64imafdc BSP, you need to edit its abi.yml under the spec/ tree.

I tried compiling the mpfs64imafdc BSP without SMP on latest 6 branch and main and got no errors. My config.ini is:

[riscv/mpfs64imafdc]
RTEMS_POSIX_API=True
BUILD_TESTS=True

The question is not if compiling reports an error. It does not even the function _CPU_Start_multitasking is missing. The question is if the resulting binary runs on the hardware even it is compiled with SMP off and consequently, it does not contain function _CPU_Start_multitasking.

Thanks for other answers, I go step by step through them.

I don’t think I understand your question. The MPFS contains 5 harts. Hart 0 is the E51 that RTEMS does not currently support. Harts 1-4 are the E54 that RTEMS supports.

If SMP is enabled, then _CPU_Start_multitasking exists and is used to perform the first task switch on a core.

If SMP is not enabled, then _CPU_Start_multitasking can be used if it exists, but is not required.

The question is, for example, why BSPs for sparc and arm use the function _CPU_Start_multitasking always and why BSPs for riscv use this function only when SMP is on. However, it is a question for the expert which perfectly knows how RTEMS starts. For me, it sounds suspicious.

As the function _CPU_Start_multitasking is not defined when SMP is off, then _CPU_Context_Restart_self is executed and the code crashes there.

To identify, where is the problem, I need to know, what is the nominal behavior. Should the code go to _CPU_Context_Restart_self or not? Depending on this, the problem will be searched in different places. If _CPU_Start_multitasking should be defined, I must search why it is not defined. If this function is not necessary, I must search why _CPU_Context_Restart_self crashes.

However, if you are not 100% sure what is the nominal behavior, please let this question unanswered and let us wait if some expert looks here and she/he will give the answer.

The things I spoke about represent reasons why I have a doubt the beaglevfire BSP is right.

I found that memory setting in config.ini you advised is not embracive. I still had to modify DTB table in file beaglevfire.h. Isn’t it possible that the memory must be set even on some other place? It could represent the cause of the problems.