linuxcnc's thread model is "rate monotonic scheduling".
- the thread with the smallest period (highest frequency) must be
created first. It is created with the highest available priority
in the underlying RTOS
- subsequent threads must be ordered by frequency and get
descending priority
- all threads are bound to the same CPU/core/thread
- the RTOS needs to guarantee that faster/higher priority threads
are allowed to preempt slower/lower priority threads, but not the
other way around
Remember, rtapi/hal was designed in a day where dual processor
systems were rare unobtainium, and for 2 CPUs it made perfect sense
to imagine the UI and task as running on one CPU and realtime on the
other (isolcpus= in older kernels). Today you might choose a
different design, but remember also that for the (rare!) components
which expose multiple functions that are intended to be executed in
different threads (e.g., software stepgen), rate monotonic
scheduling can be a great simplifying assumption.
It is mainly these components which need to be reviewed and fixed if
a different thread model (such as each HAL thread being bound to a
distinct processor thread/core) is to be adopted. However, another
detail that is important is the speed (or lack of it!) to transfer a
cache line of data from one processor/thread's L1 cache to another.
For instance, if you put "fast" thread on CPU1 and "slow" thread on
CPU2, then any data that is read by "fast" thread and written by
"slow" thread has its associated cache line bounce between the two
CPUs. Furthermore, any HAL data on the same cache line would have
to "bounce" with it, potentially making for even more cache line
bounces than necessary. I hear that machinekit made each pin's data
reside on a separate cache line for this reason, but I'm unaware of
the details or the exact microbenchmarks that showed this was a good
decision.
So, yes, long story short: if your main thread has marginal timing,
adding another faster thread is going to make it start missing
deadlines.
Jeff