Wow finally, it seems like yesterday when first encountered the real-time Linux kernel proposal.
It's a long time overdue and congrats to the real-time Linux time for the tenacity. The fact that it's included after eBPF being accepted to the Linux kernel but better late than never. Now we can preempt eBPF codes to make it even faster and responsive.
It will be very interesting to run real-time Linux on Raspberry Pi interfacing with Pi Pico for IoT monitoring, sensing and actuating communicating over bidirectional BiSS interface [1].
[1] BiSS: High speed open-source digital interface for IoT sensors and actuators:
Wow that sounds like a hell of an achievement. Here is the commit to the print_k function they mentioned as the final hurdle [1]. I have no idea what it does but maybe someone else here does. I wonder if the RT stuff will remain behind a config flag or become the default eventually. Is there any reason to not just have RT on by default?
I remember I had some nasty sound distortion issues on a Windows VFIO VM and it came down to events/interrupts not being handled quickly enough. I wonder what happens with an RT kernel and the VM thread having high priority in that situation.
> Is there any reason to not just have RT on by default?
It's a latency vs throughput trade-off usually. Ideally you'd want both to improve, but the more you switch between the processes, the more time you waste on busywork and cache invalidation rather than what the processes want to achieve. For example if RT wants to guarantee that your audio is more in sync, you're going to switch from your app to the audio system more often.
cwillu 291 days ago [-]
That's a bit of a red-herring though, as the audio system will only preempt other processes if it's configured for very small buffers and given real-time permissions. A server's audio subsystem won't preempt actual work because a server won't _have_ an audio subsystem running, let alone be configured for sub-millisecond latency.
The real trade-off is the overhead required to make all kernel operations potentially safe WRT maximum wait times.
viraptor 291 days ago [-]
This is a qualified example. Yes, if you're not doing audio, audio is not going to interrupt. But that's a moot point...
ttshaw1 290 days ago [-]
Anyone know if this will have any impact on low-latency linux audio? The article calls out JACK, but I'd guess the new standard for musicians recording on Linux is pipewire. So if pipewire either found some workaround or has been enabling a real-time module, I suppose those end users wouldn't see a difference.
orbital-decay 290 days ago [-]
There are already realtime/low latency kernels in many distros, very useful for low-latency audio processing. Usually you have to install them separately.
seba_dos1 290 days ago [-]
> Usually you have to install them separately.
...and that won't change now either. You'll just be able to build the RT kernel from the same upstream sources as the regular one.
cwillu 290 days ago [-]
Pipewire will benefit in exactly the same way as jack.
tetris11 290 days ago [-]
If anyone wants to try multiple kernels on their machine, here's how you'd do it with systemd-boot
## install mainline, bleeding, secure, stable, and stable realtime, mainline realtime
pacman -S linux linux-zen linux-hardened linux-lts linux-rt-lts, linux-rt
## and I guess, soon to be: linux-rt
And then add the following 5 bootloader entries as separate files
#/boot/loader/entries/linux{,-zen,-hardened,-lts,-rt-lts,-rt}.conf
title Linux {Main,Zen,Hard,LTS,RT-LTS,RT}
linux /vmlinuz-linux{,-zen,-hardened,-lts,-rt-lts,-rt}
initrd /initramfs-linux{,-zen,-hardened,-lts,-rt-lts,-rt}.img
options root=PARTUUID="the-part-uuid-of-your-root-partition" rw
and after those 5 files are in place, do a `bootctl update`
josephcsible 290 days ago [-]
Does Arch really not integrate bootloader config into the package manager? Or is this a shortcoming of systemd-boot? Because on, e.g., Ubuntu and Fedora, when I install a new kernel from a .deb or .rpm, it automatically does everything with GRUB for it to be there when I reboot.
tetris11 289 days ago [-]
I do agree that systemd-boot should have a post installation hook that generates these automatically.
enva2712 290 days ago [-]
you install the bootloader yourself on arch
Animats 291 days ago [-]
Nice. Real-time Linux used to be a joke, but apparently now it works.
The sheer complexity of making much of kernel space preemptable is scary. There's too much running in kernel space.
goodcanadian 291 days ago [-]
Real-time Linux used to be a joke, but apparently now it works.
When was that? I used it over a decade ago, and it worked pretty well even then.
> All it would take is one malicious process to execute and take advantage of the real-time code to completely lock-out a user from their machine, turning that machine into part of a botnet or other malicious purpose. Real-Time processes have the potential to completely take-over a machine. This is the number one reason Ubuntu does not carry a Real-Time kernel.
Snild 290 days ago [-]
That page seems to be describing SCHED_FIFO processes, which are already a thing without PREEMPT_RT. Maybe they weren't back in the pre-2.6 days? Anyway, they are usually limited to 95% of total runtime by the sched_rt_runtime_us tunable, to avoid accidental self-DoSing. Maybe that, too, was later invention -- 2.6 is very very old.
The page goes on:
> A patch does exist to enable process to have real-time process access to any process requesting it.
According to the sched(7) man page, this has never been the case: before 2.6.12, the process had to have CAP_SYS_NICE; after, it was limited by policy through RLIMIT_RTPRIO. I guess it's possible that this was not the case for the original out-of-tree patch set.
But it's been there for many years, well before the 2020 edit that added the bulk of the current text on that wiki page.
PhilipRoman 290 days ago [-]
>completely lock-out a user from their machine, turning that machine into part of a botnet or other malicious purpose
There seems to be a pretty big leap from beginning of that sentence to the end, I personally wouldn't consider local DoS a problem.
So this means there are no critical sections or interrupts being disabled anywhere in kernel code when PREEMPT_RT is enabled?
oasisaimlessly 290 days ago [-]
I would assume it means something a bit weaker: that all critical sections have a bounded length, being guaranteed to end after a given finite number of clock cycles.
ahazred8ta 290 days ago [-]
A buddy of mine used Trinux RTK and Wind River RTLinux in a commercial audio console product back in the 00's. There were several efforts back then.
nirav72 291 days ago [-]
I don’t know much about RTOS. But am aware that JPL uses VxWorks for planetary rovers and orbiters. So wondering if JPL or other entities will replace VxWorks with a Linux RTOS implementation at some point.
skovati 291 days ago [-]
If you haven't already, look into the avionics and flight software of the Ingenuity helicopter. First time Linux ran on Mars!
291 days ago [-]
penguin_booze 290 days ago [-]
> Unlike general-purpose operating systems like Windows or macOS.
FYI, there appears to be minor one, too: it's called Linux.
jellykid 290 days ago [-]
I wonder if this will improve emulation compatibility?
Rendered at 21:28:12 GMT+0000 (UTC) with Wasmer Edge.
It's a long time overdue and congrats to the real-time Linux time for the tenacity. The fact that it's included after eBPF being accepted to the Linux kernel but better late than never. Now we can preempt eBPF codes to make it even faster and responsive.
It will be very interesting to run real-time Linux on Raspberry Pi interfacing with Pi Pico for IoT monitoring, sensing and actuating communicating over bidirectional BiSS interface [1].
[1] BiSS: High speed open-source digital interface for IoT sensors and actuators:
https://news.ycombinator.com/item?id=41516826
I remember I had some nasty sound distortion issues on a Windows VFIO VM and it came down to events/interrupts not being handled quickly enough. I wonder what happens with an RT kernel and the VM thread having high priority in that situation.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
It's a latency vs throughput trade-off usually. Ideally you'd want both to improve, but the more you switch between the processes, the more time you waste on busywork and cache invalidation rather than what the processes want to achieve. For example if RT wants to guarantee that your audio is more in sync, you're going to switch from your app to the audio system more often.
The real trade-off is the overhead required to make all kernel operations potentially safe WRT maximum wait times.
...and that won't change now either. You'll just be able to build the RT kernel from the same upstream sources as the regular one.
The sheer complexity of making much of kernel space preemptable is scary. There's too much running in kernel space.
When was that? I used it over a decade ago, and it worked pretty well even then.
https://help.ubuntu.com/community/UbuntuStudio/RealTimeKerne...
> Security Implications
> All it would take is one malicious process to execute and take advantage of the real-time code to completely lock-out a user from their machine, turning that machine into part of a botnet or other malicious purpose. Real-Time processes have the potential to completely take-over a machine. This is the number one reason Ubuntu does not carry a Real-Time kernel.
The page goes on:
> A patch does exist to enable process to have real-time process access to any process requesting it.
According to the sched(7) man page, this has never been the case: before 2.6.12, the process had to have CAP_SYS_NICE; after, it was limited by policy through RLIMIT_RTPRIO. I guess it's possible that this was not the case for the original out-of-tree patch set.
But it's been there for many years, well before the 2020 edit that added the bulk of the current text on that wiki page.
There seems to be a pretty big leap from beginning of that sentence to the end, I personally wouldn't consider local DoS a problem.
So this means there are no critical sections or interrupts being disabled anywhere in kernel code when PREEMPT_RT is enabled?
FYI, there appears to be minor one, too: it's called Linux.