Guidance on Testing Changes in RTEMS

General
,
1

Hi all,

I’m currently working on a contribution and would like to ask about the recommended approaches for testing changes in RTEMS.

Are there established testing workflows or best practices followed within the community? For example, how do contributors usually validate their changes are there specific tools, test suites, or environments that are commonly used?

Additionally, in cases where testing may require hardware or runtime validation, what is the typical approach? Is there any guidance on how to handle such scenarios if specific hardware is not available?

If there are any guides, documentation, or examples that outline the expected testing process, I would really appreciate being pointed to them.

Thanks in advance for your help.

2

Excellent question. We have a testing chapter that should address some of this, and feel free to suggest or send improvements. There are generally three approaches depending on the situation:

  • Run the full testsuite (use BUILD_TESTS in config.ini and rtems-test pointed at build/${arch}/${bspcfg}/testsuites. This is best for broad testing where you want to make sure you don’t break anything new. Typically you would do this before and after your commit(s), i.e., on a clean main branch and on your feature branch.
  • Run a partial testsuite or specific test. Like above, but maybe you know there is a specific set of tests that fail such as in smptests or in validation. In this case, you can limit your config.ini to only build those tests (e.g., BUILD_SMPTESTS) or only point rtems-test at their subdirectory like build/.../testsuites/validation, or use rtems-run on the specific test that is failing such as rtems-run build/.../testsuites/samples/hello.exe. This can be useful to reduce the build and test runtimes.
  • Custom testing. This might be what you would need to do if you have a hardware or BSP-specific failure. In this case you might write your own small application or derive a test just for your purpose. We do not have a great way of integrating hardware-specific tests in our testsuites currently.
1 Like