Scheduler Maximum Regression in RTEMS 6

Real-time SMP
,
1

I have created an issue in Gitlab called RTEMS Priority range API change (regression) to track the change made to RTEMS that effects users. I am creating this topic to discuss the design decisions that have lead to this regression we have and how we resolve it.

Summary

RTEMS 6 is the first release of RTEMS with production quality SMP. All previous versions of RTEMS have a priority range of 255. The introduction of plugin schedulers and the addition of new schedulers such as EDF has allowed for different ranges. These are both welcomed changes however these changes need to respect the range that existed for decades, justify any changes to user facing interfaces and provide means for users to manage and migrate applications. I do not believe we have done this.

My focus is on existing applications moving to SMP and building RTEMS on a BSP that supports SMP. I would like to examine and discuss how users do this and what the issues are. We can then decide on changes if any we need to make. I believe the changing of priority ranges in SMP is a regression and I feel it is up to those who have a vested interest in this change or those with academic and technical skill to comment and make a case for what is present. The period of time the changes have existed in RTEMS is not as important as providing a well reasoned and document approach for users.

My concerns are not related to advanced applications that implement clustered scheduling on SMP. I feel the users building those applications will be on development versions of RTEMS 6. They will already have carefully considered build and configuration options.

There are a number of ways to resolve this issue. A resolution needs to consider:

  1. Existing RTEMS 4 default API behavior. Silently changing behavior of the application is a negative outcome.
  2. User’s perspective and complexity.
  3. Technical merit.
  4. Documentation.

Scheduler Range Definition

POSIX has defined min and max calls to report the range for a scheduler. POSIX is careful to not define how the range effects the scheduling however Linux does clearly stating a higher number schedules threads before low priorities numbers. This has resulted in the max being associated to high priority threads.

The RTEMS API now provides the call rtems_scheduler_get_maximum_priority() however this value is reserved for the IDLE thread. This is confusing and I think our documentation for this call should clarify this.

Effected Applications and Frameworks

Effected applications and frameworks are:

  1. Using SMP. An application or framework may be common to SMP and non-SMP builds of RTEMS. The simplest configuration different is the addition of the required SMP confdef settings.
  2. Multiple priority ranges. This means the RTEMS Classic API priority range and the POSIX API priority range active in an application or framework.
  3. Use of confdefs configured RTEMS services. These are statically defined priorities for RTEMS Classic API priority range.

Single Priority Range

We should recommend users adopt a single priority range for their applications. Multiple ranges adds complexity and scaling of ranges to multiple priority ranges can result in clashes and unexpected results.

EPICS

EPICS is a major framework that supports RTEMS. EPICS moved to POSIX for RTEMS 5 and later which means EPICS with RTEMS 6 can support SMP.

EPICS supports an internal priority range of 0-99. I guess this maps to Linux’s POSIX priority range. The EPICS POSIX thread adaption layer linearly maps the EPICS priority range to the POSIX range as reported by sched_get_priority_min() and sched_get_priority_max() calls. This approach sort of works on RTEMS when the priority range for the RTEMS Classic API and POSIX are close, ie non-SMP and the default scheduler but I have not confirmed this.

SMP defaults to the EDF scheduler and the priority range is 0 to INT32_MAX and that means a single EPIC priority level covers over 2 million scheduler priorities. EPICS uses networking and the networking priorites are RTEMS Classic API. LibBSD is fixed between 96 to 100 so this means all EPICS created threads will have a priority lower than any default RTEMS services configured using confdefs.

Asking EPICS to define priorities with values such as 2147483547 is unreasonable and impractical and it is not clear from the documentation if the EDF range changes on 64 bit architectures.

Configurable Schedulers

RTEMS allows for configurable schedulers and these schedulers can have different ranges. The ability to get the priority range at runtime does not take into account statically set and compiled services such as networking services or libblock tyoe services configured using confdefs.

Other issues are:

  1. What are the pros and cons of the available SMP schedulers?
  2. Why is the user facing EDF priority range to large? Just because it can be is not a valid reason to have this range.
  3. Can scheduler priority ranges be limited by CONFIGURE_MAXIMUM_PRIORITY without effecting the performance and behavour of the scheduler?

LibBSD

LibBSD only works with EDF on SMP because the other SMP schedulers are missing a scheduler call.

  1. Why is there no issue for this?
  2. Could adding these calls be a GSoC project?

API Change Policy

I would like us to develop an API change policy.

  1. Changes that effect an API need to be labelled and discussed.
  2. Changes maybe approved and merged that changes an API without it being known. Those changes however long ago should be considered a regression and may be reverted.
  3. A change that effects an API interface may be reverted.
  4. A change may require a detailed discussion and investigation about its effect on users to determines a suitable outcome.
  5. Existing behavior needs to be maintained where possible. Existing API interfaces and related default settings take priority over new changes.
  6. API changes require a compelling case.
  7. Changes in behavior cannot be done silently.