I’d be glad, if someone could help me with a problem that I noticed a few days ago:
On two different devices running Manjaro jack soundserver suddenly lost the permission to operate in realtime mode. On both devices this had been just working for at least one year. I noticed that on both devices the package realtime-privileges had vanished, but reinstalling it didn’t change anything. I crawled my way through Arch wiki professional audio, but nothing helped.
The user is member of groups realtime and audio.
/etc/security/limits.conf is set accordingly.
jack runs when I uncheck the ‘Realtime’ box (I’m using cadence)
The fact that the problem occured on two - quiet different - devices made me believe that the probleme might be more wide spread. Does anyone have an idea how to solve this?
Does cadence-logs show any errors in ~/.log/jack/jackdbus.log ?
Package realtime-privileges last update was Jan 2022 and cadence last update was Jun 2022, and there are no conflicting packages that would need them removed, and no logical reason why a package might disappear without any warning.
I suggest you check pacman log to see if that shows when or why package was removed
grep realtime-privileges */var/log/pacman.log*
Editing /etc/security/limits.conf should not be necessary. realtime-privileges creates file /etc/security/limits.d/99-realtime-privileges.conf for realtime pam_limits
I suggest using rtcqs to check system realtime configuration
Install script from AUR for GUI version – pamac build rtcqs
or download the python script rtcs.py to run in terminal – ./rtcqs.py
(If you can run script in terminal, response can be copy/pasted here easily)
You might have a look here:
Not completely sure if you get it working.
In all honesty - i gave up at a certain point and reverted back to a non-rt kernel
I probably made a mistake when writing that realtime-privileges had vanished. At my job I have to work with APT which tells you, if a package you want to install is already there, while pacman and yay just reinstall without further notice as I learned now.
Anyway - I reinstalled and it is not working. I get these outputs:
$ cat ~/.log/jack/jackdbus.log | grep ERROR
Sun Jan 15 13:58:59 2023: ERROR: Cannot use real-time scheduling (RR/10) (1: Operation not permitted)
Sun Jan 15 13:58:59 2023: ERROR: AcquireSelfRealTime error
$ id
uid=1000(zaches) gid=1000(zaches) Gruppen=1000(zaches),6(disk),7(lp),10(wheel),90(network),91(video),92(audio),93(optical),95(storage),96(scanner),98(power),100(users),618(autologin),991(realtime),992(docker)
$ rtcqs
rtcqs - version 0.5.3
Root User
=========
[ OK ] Not running as root.
Audio Group
===========
[ OK ] User zaches is in the audio group.
CPU Frequency Scaling
=====================
[ WARNING ] The scaling governor of one or more CPU's is not set to 'performance'. You can set the scaling governor to 'performance' with 'cpupower frequency-set -g performance' or 'cpufreq-set -r -g performance' (Debian/Ubuntu). See also https://wiki.linuxaudio.org/wiki/system_configuration#cpu_frequency_scaling
Kernel Configuration
====================
[ OK ] Valid kernel configuration found.
High Resolution Timers
======================
[ OK ] High resolution timers are enabled.
Tickless Kernel
===============
[ OK ] System is using a tickless kernel.
Preempt RT
==========
[ OK ] Kernel 6.0.5-2-rt14-MANJARO is a real-time kernel.
Spectre/Meltdown Mitigations
============================
[ WARNING ] Kernel with Spectre/Meltdown mitigations found. This could have a negative impact on the performance of your system. See also https://wiki.linuxaudio.org/wiki/system_configuration#disabling_spectre_and_meltdown_mitigations
RT Priorities
=============
[ WARNING ] Could not assign a 80 rtprio SCHED_FIFO value due to the following error: [Errno 1] Operation not permitted. Set up limits.conf. See also https://wiki.linuxaudio.org/wiki/system_configuration#limitsconfaudioconf
Swappiness
==========
[ OK ] Swappiness is set at 1.
Filesystems
===========
[ OK ] The following mounts can be used for audio purposes: /, /home
[ WARNING ] The following mounts should be avoided for audio purposes: /run/user/1000/gvfs, /run/user/1000/doc. See also https://wiki.linuxaudio.org/wiki/system_configuration#filesystems
IRQs
=====
[ OK ] Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Soundcard snd_hda_intel:card1 with IRQ does not share its IRQ.
Power Management
================
[ OK ] Power management can be controlled from user space. This enables DAW's like Ardour and Reaper to set CPU DMA latency which could help prevent xruns.
I suspect setting real-time priority to 95 for audio group conflicts with 98 for realtime and system is ignoring both values (and probably using default value 0)
I suggest you delete the /etc/security/limits.d/*.conf file containing the two audio group settings;
reboot system and check if realtime priority is set to 98 – ulimit -r
I don’t know what @users group is for. I don’t see the point of setting memlock twice for this group and third setting for realtime group
I suggest you check system memlock setting – ulimit -l
If response shows value of 1024, delete the /etc/security/limits.d/*.conf file for users group settings
If response shows value of unlimited, settings for users group are redundant and not needed
Arch package manager pacman warns if a package is already installed and requests user to reinstall package
warning: realtime-privileges-4-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...
Packages (1) realtime-privileges-4-1
Total Installed Size: 0.00 MiB
Net Upgrade Size: 0.00 MiB
:: Proceed with installation? [Y/n]
Manjaro package manager pamac has a different response:
$ pamac install realtime-privileges
Preparing...
Warning: realtime-privileges-4-1 is up to date -- skipping
Nothing to do.
Transaction successfully finished.
I didn’t know the ulimit command. Without having changed anything it showed me
$ ulimit -r
95
I edited the files as you suggested and now I get:
$ ulimit -r
98
$ ulimit -l
unlimited
Unfortunately rtcqs and jack still don’t get rtprio.
$ rtcqs
...
RT Priorities
=============
[ WARNING ] Could not assign a 80 rtprio SCHED_FIFO value due to the following error: [Errno 1] Operation not permitted. Set up limits.conf. See also https://wiki.linuxaudio.org/wiki/system_configuration#limitsconfaudioconf
...
In an articel on some forum I read that there might be service definitions in /etc/systemd that limit the assignable value of rtprio if jack relies on that service, but I have no idea which service that might be.
Another thread I follow: On my second device I use ffado drivers. The program ffado-test complains that it can’t read /dev/rtc. On both devices /dev/rtc is a link to /dev/rtc0 and both files cannot be read by my user.
Well, probably I don’t. On my second device - a quiet old one, connected to a quiet old firewire Mackie interface - I had to move around audio regions anyway after punch in recordings.
But, as it worked until some weeks ago (last verified December 21st), it’s hard to accept to be downgraded like this without knowing the reason.
Then restart system and post response to this command to show jack engine parameters
jack_control ep
In an articel on some forum I read that there might be service definitions in /etc/systemd that limit the assignable value of rtprio if jack relies on that service
jack2 includes 2 files for systemd integration /usr/lib/systemd/user/jack@.service and a configuration file /etc/jack/example.conf
Comment in the configuration file advise that this is a template a user can modify to create a working systemd.service if required
If the systemd service has not been modified it cannot work, so it is unlikely to cause any conflict with realtime group configuration
On my second device I use ffado drivers. The program ffado-test complains that it can’t read /dev/rtc. On both devices /dev/rtc is a link to /dev/rtc0 and both files cannot be read by my user.
Realtime configuration used to include a udev rule to grant user access to rtc0 realtime clock and hpet high precision event timer.
But the udev rule was dropped from realtime-privileges and removed from wiki.linuxaudio.org/system_configuration > 5 years ago because it is no longer needed with recent Linux kernels
I suggest you check with ffado developers if their test script is up to date. If access to timer is still needed for Firewire device, they should be able to advise the udev rule needed
I removed my user from those groups. Here is the output:
$ groups
disk lp wheel network video optical storage scanner power autologin realtime docker <myuser> <some samba related groups>
$ jack_control ep
--- get engine parameters (type:isset:default:value)
driver: Driver to use (str:set:dummy:alsa)
name: Server name to use. (str:notset:default:default)
realtime: Whether to use realtime mode. (bool:set:True:True)
realtime-priority: Scheduler priority when running in realtime mode. (sint:set:10:10)
temporary: Exit once all clients have closed their connections. (bool:notset:False:False)
verbose: Verbose mode. (bool:set:False:True)
client-timeout: Client timeout limit in milliseconds. (sint:notset:0:0)
clock-source: Clocksource type : c(ycle) | h(pet) | s(ystem). (uint:notset:0:0)
port-max: Maximum number of ports. (uint:notset:2048:2048)
replace-registry: Replace shared memory registry. (bool:notset:False:False)
sync: Use server synchronous mode. (bool:notset:False:False)
self-connect-mode: Self connect mode. (char:set: : )
slave-drivers: Slave drivers to use (str:notset::)
I still can’t get rtprio:
RT Priorities
=============
[ WARNING ] Could not assign a 80 rtprio SCHED_FIFO value due to the following error: [Errno 1] Operation not permitted. Set up limits.conf. See also https://wiki.linuxaudio.org/wiki/system_configuration#limitsconfaudioconf
I noticed that even root cannot get rtprio on my computer now:
# rtcqs
rtcqs - version 0.5.3
Root User
=========
[ WARNING ] You are running this script as root. Please run it as a regular user for the most reliable results.
...
RT Priorities
=============
[ WARNING ] Could not assign a 80 rtprio SCHED_FIFO value due to the following error: [Errno 1] Operation not permitted. Set up limits.conf. See also https://wiki.linuxaudio.org/wiki/system_configuration#limitsconfaudioconf
I didn’t find time to check the ffado driver thing again. The second device is in a rehearse room.
The first check in rtcqs is to ensure user is not running as root because realtime scheduling is only configured for normal user access
Root access to system files is needed for configuring system but trying to run JACK as root is likely to fail
If realtime group settings were working as expected, jack_control command should show realtime-priority configuration (sint:set:10:98)
I suggest you remove user from realtime group and uninstall realtime-privileges
I know you shouldn’t run jack as root, but I think root should have the necessary permissions. Also this doesn’t work:
# chrt 10 ls /
chrt: Festlegen der Richtlinie für PID 0 ist fehlgeschlagen: Die Operation ist nicht erlaubt
(in English: … ~operation not permitted~)
I think the output of jack_control has nothing to do with the user permissions. The first number seems to be the default value and the second the value set in ~/.config/jack/conf.xml which is modified by cadence. Whatever value I fill into the realtime priority field in the cadence gui will appear in that file and later in the output of jack_control, e.g.:
realtime-priority: Scheduler priority when running in realtime mode. (sint:set:10:42)
My users permission is still
$ ulimit -r
98
and as much as I understand by now, I should be able to start any command with any rtprio up to 98. rtcqs tries to get rtprio 80, which is less than 98 but not permitted and also the following command doesn’t work:
$ chrt 42 ls
chrt: Festlegen der Richtlinie für PID 0 ist fehlgeschlagen: Die Operation ist nicht erlaubt
I double checked again: There are no other limits defined in limits.conf or limits.d/* than those for realtime and audio group. I tried to omit realtime with no success and I tried to omit audio with no success. Values for both groups are equal now:
Comments in post #10 and #13 about jack_control command are related to requested value of realtime-priority
Data in post #11 showed realtime-priority set to default value 10
jack_control ep shows the engine parameters in a better format than screenshots of Cadence GUI or contents of ~/.config/jack/config.xml
~/.config/jack/config.xml is created by jackdbus. and can be accessed/rewritten by Cadence, jack_control and other JACK packages
The first and last message in my ~/.log/jackdbus.log when Cadence starts and stops is:
Saving settings to “/home/nik/.config/jack/conf.xml”
jack_control can also change /.config/jack/conf.xml
for example, this can change realtime-priority requested when JACK starts
jack_control eps realtime-priority 95
I have have tried multiple combinations of rtprio settings for realtime/audio group on 2 different systems
(2nd system has never been configured for JACK use and was only a member of audio group for using PulseAudio)
In all tests I cannot get chrt -f 98 echo success to fail if ulimit -r reports value for rtprio greater than or equal to 98
The number of spaces in rtprio configuration makes no difference
Using both realtime/audio groups does not appear to cause conflict
(Cadence has a warning if user is not in audio group but can still start JACK in realtime mode with the requested realtime-priority level)
The only thing I can think of now is to check all possible sources of rtprio for errors
Please post full contents of system configuration
cat /etc/limits.conf
and this to show file permissions for group configuration files
ls -l /etc/security/limits.d/*
and this to show group configurations within the files
Something is really broken with RT priority assignments SCHED_FIFO; it has now become impossible to set up a fresh Manjaro system with RT kernel that would be usable for robotic purposes.
I was able to set up the last executable system on 12/15/2022.
I have tried to use both Manjaro 21.3 and 22.x images for system setup in the last 2 weeks, using both several Linux 5 and one Linux 6 RT kernels. No system will let me use the Robotic programs with RT-Prio anymore, because access to the scheduler and RT-Prio is denied. However, this is not an actual access or configuration problem.
The only remedy is a prepended command, which is not the final solution, but only points to the actual problem.
sudo sysctl -w kernel.sched_rt_runtime_us=-1
At least it can be used to execute robot programs.
Some further research brought me to a thread on the ardour forum. The problem there is the kernel option CONFIG_RT_GROUP_SCHED=y. I searched my system et voila:
Unfortunately until now I didn’t manage to reproduce the solution described there using cgroups on my computer. A short description of this can also be found in the jackaudio wiki.