Launching X with GPU within Docker

maddieM4 · 19 February 2024 21:42

Alright. Fair warning that the thing I’m trying to do is straddling a line of being stupid and brilliant at the same time. I’m trying to make a proof of concept for the following idea: have some sort of minimal Linux distro running on my desktop tower, which is kind of headless on its own (just ttys on CTRL-ALT-F*), but from which I can run various other distros as privileged Docker containers that are able to drive my GPU/screen/other peripherals. If I do it right, I can essentially distro-hop without reboots.

Before fully committing to this plan, I’m trying to make a basic demo work with my existing Manjaro installation as the outer layer. I’ve dropped it down to multi-user.target, I’m tinkering over SSH, and I have an Ubuntu docker container based on Bedrin’s docker-x-server container, where the only tweak I’ve done is bump the version number for Ubuntu so that packages actually download. I do have the proprietary kernel module for Nvidia installed on Manjaro, but for simplicity, I’m trying to use Nouveau for now from Ubuntu, and not having any issues loading the kernel module.

From the container, if I run X :0 -nolisten tcp vt7, I get an output like this, which I’ll have to paste in entirety because I’m not allowed to link files:

[  7147.739] 
X.Org X Server 1.21.1.4
X Protocol Version 11, Revision 0
[  7147.739] Current Operating System: Linux e205f18935e7 6.5.13-7-MANJARO #1 SMP PREEMPT_DYNAMIC Wed Dec 20 07:15:58 UTC 2023 x86_64
[  7147.739] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.5-x86_64 root=UUID=388e2d24-f4cd-4f1e-b756-00217d32fada rw quiet splash apparmor=1 security=apparmor resume=UUID=15d85720-dbee-449b-9c7f-ca304999a877 udev.log_priority=3
[  7147.739] xorg-server 2:21.1.4-2ubuntu1.7~22.04.8 (For technical support please see http://www.ubuntu.com/support) 
[  7147.739] Current version of pixman: 0.40.0
[  7147.739] 	Before reporting problems, check http://wiki.x.org
	to make sure that you have the latest version.
[  7147.739] Markers: (--) probed, (**) from config file, (==) default setting,
	(++) from command line, (!!) notice, (II) informational,
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[  7147.739] (==) Log file: "/var/log/Xorg.0.log", Time: Mon Feb 19 21:33:24 2024
[  7147.740] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[  7147.740] (==) No Layout section.  Using the first Screen section.
[  7147.740] (==) No screen section available. Using defaults.
[  7147.740] (**) |-->Screen "Default Screen Section" (0)
[  7147.740] (**) |   |-->Monitor "<default monitor>"
[  7147.740] (==) No monitor specified for screen "Default Screen Section".
	Using a default monitor configuration.
[  7147.740] (==) Automatically adding devices
[  7147.740] (==) Automatically enabling devices
[  7147.740] (==) Automatically adding GPU devices
[  7147.740] (==) Automatically binding GPU devices
[  7147.740] (==) Max clients allowed: 256, resource mask: 0x1fffff
[  7147.740] (WW) The directory "/usr/share/fonts/X11/cyrillic" does not exist.
[  7147.740] 	Entry deleted from font path.
[  7147.740] (WW) The directory "/usr/share/fonts/X11/100dpi/" does not exist.
[  7147.740] 	Entry deleted from font path.
[  7147.740] (WW) The directory "/usr/share/fonts/X11/75dpi/" does not exist.
[  7147.740] 	Entry deleted from font path.
[  7147.740] (WW) The directory "/usr/share/fonts/X11/Type1" does not exist.
[  7147.740] 	Entry deleted from font path.
[  7147.740] (WW) The directory "/usr/share/fonts/X11/100dpi" does not exist.
[  7147.740] 	Entry deleted from font path.
[  7147.740] (WW) The directory "/usr/share/fonts/X11/75dpi" does not exist.
[  7147.740] 	Entry deleted from font path.
[  7147.740] (==) FontPath set to:
	/usr/share/fonts/X11/misc,
	built-ins
[  7147.740] (==) ModulePath set to "/usr/lib/xorg/modules"
[  7147.740] (II) The server relies on udev to provide the list of input devices.
	If no devices become available, reconfigure udev or disable AutoAddDevices.
[  7147.740] (II) Loader magic: 0x5616b16fe020
[  7147.740] (II) Module ABI versions:
[  7147.740] 	X.Org ANSI C Emulation: 0.4
[  7147.740] 	X.Org Video Driver: 25.2
[  7147.740] 	X.Org XInput driver : 24.4
[  7147.740] 	X.Org Server Extension : 10.0
[  7147.740] (EE) dbus-core: error connecting to system bus: org.freedesktop.DBus.Error.FileNotFound (Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory)
[  7147.740] (++) using VT number 7

[  7147.740] (II) systemd-logind: logind integration requires -keeptty and -keeptty was not provided, disabling logind integration
[  7147.740] (II) xfree86: Adding drm device (/dev/dri/card1)
[  7147.740] (II) Platform probe for /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card1
[  7147.740] (II) xfree86: Adding drm device (/dev/dri/card2)
[  7147.740] (II) Platform probe for /sys/devices/pci0000:00/0000:00:1c.4/0000:06:00.0/drm/card2
[  7147.741] (II) xfree86: Adding drm device (/dev/dri/card0)
[  7147.741] (II) Platform probe for /sys/devices/platform/simple-framebuffer.0/drm/card0
[  7147.744] (--) PCI:*(1@0:0:0) 10de:1c82:1458:3729 rev 161, Mem @ 0xf6000000/16777216, 0xd0000000/268435456, 0xe0000000/33554432, I/O @ 0x0000e000/128, BIOS @ 0x????????/131072
[  7147.744] (--) PCI: (6@0:0:0) 10de:1187:3842:2765 rev 161, Mem @ 0xf4000000/16777216, 0xe8000000/134217728, 0xf0000000/33554432, I/O @ 0x0000c000/128, BIOS @ 0x????????/524288
[  7147.744] (II) LoadModule: "glx"
[  7147.744] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[  7147.744] (II) Module glx: vendor="X.Org Foundation"
[  7147.744] 	compiled for 1.21.1.4, module version = 1.0.0
[  7147.744] 	ABI class: X.Org Server Extension, version 10.0
[  7148.058] (==) Matched nouveau as autoconfigured driver 0
[  7148.058] (==) Matched modesetting as autoconfigured driver 1
[  7148.058] (==) Matched fbdev as autoconfigured driver 2
[  7148.058] (==) Matched vesa as autoconfigured driver 3
[  7148.058] (==) Assigned the driver to the xf86ConfigLayout
[  7148.058] (II) LoadModule: "nouveau"
[  7148.058] (II) Loading /usr/lib/xorg/modules/drivers/nouveau_drv.so
[  7148.059] (II) Module nouveau: vendor="X.Org Foundation"
[  7148.059] 	compiled for 1.21.1.3, module version = 1.0.17
[  7148.059] 	Module class: X.Org Video Driver
[  7148.059] 	ABI class: X.Org Video Driver, version 25.2
[  7148.059] (II) LoadModule: "modesetting"
[  7148.059] (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
[  7148.059] (II) Module modesetting: vendor="X.Org Foundation"
[  7148.059] 	compiled for 1.21.1.4, module version = 1.21.1
[  7148.059] 	Module class: X.Org Video Driver
[  7148.059] 	ABI class: X.Org Video Driver, version 25.2
[  7148.059] (II) LoadModule: "fbdev"
[  7148.059] (WW) Warning, couldn't open module fbdev
[  7148.059] (EE) Failed to load module "fbdev" (module does not exist, 0)
[  7148.059] (II) LoadModule: "vesa"
[  7148.059] (WW) Warning, couldn't open module vesa
[  7148.059] (EE) Failed to load module "vesa" (module does not exist, 0)
[  7148.373] (==) Matched nouveau as autoconfigured driver 0
[  7148.373] (==) Matched modesetting as autoconfigured driver 1
[  7148.373] (==) Matched fbdev as autoconfigured driver 2
[  7148.373] (==) Matched vesa as autoconfigured driver 3
[  7148.373] (==) Assigned the driver to the xf86ConfigLayout
[  7148.373] (II) LoadModule: "nouveau"
[  7148.373] (II) Loading /usr/lib/xorg/modules/drivers/nouveau_drv.so
[  7148.373] (II) Module nouveau: vendor="X.Org Foundation"
[  7148.373] 	compiled for 1.21.1.3, module version = 1.0.17
[  7148.373] 	Module class: X.Org Video Driver
[  7148.373] 	ABI class: X.Org Video Driver, version 25.2
[  7148.373] (II) UnloadModule: "nouveau"
[  7148.373] (II) Unloading nouveau
[  7148.373] (II) Failed to load module "nouveau" (already loaded, 0)
[  7148.373] (II) LoadModule: "modesetting"
[  7148.373] (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
[  7148.373] (II) Module modesetting: vendor="X.Org Foundation"
[  7148.373] 	compiled for 1.21.1.4, module version = 1.21.1
[  7148.373] 	Module class: X.Org Video Driver
[  7148.373] 	ABI class: X.Org Video Driver, version 25.2
[  7148.373] (II) UnloadModule: "modesetting"
[  7148.373] (II) Unloading modesetting
[  7148.373] (II) Failed to load module "modesetting" (already loaded, 0)
[  7148.373] (II) LoadModule: "fbdev"
[  7148.373] (WW) Warning, couldn't open module fbdev
[  7148.373] (EE) Failed to load module "fbdev" (module does not exist, 0)
[  7148.373] (II) LoadModule: "vesa"
[  7148.373] (WW) Warning, couldn't open module vesa
[  7148.373] (EE) Failed to load module "vesa" (module does not exist, 0)
[  7148.373] (II) NOUVEAU driver Date:   Sat Jan 23 12:24:42 2021 -0500
[  7148.373] (II) NOUVEAU driver for NVIDIA chipset families :
[  7148.373] 	RIVA TNT            (NV04)
[  7148.373] 	RIVA TNT2           (NV05)
[  7148.373] 	GeForce 256         (NV10)
[  7148.373] 	GeForce 2           (NV11, NV15)
[  7148.373] 	GeForce 4MX         (NV17, NV18)
[  7148.373] 	GeForce 3           (NV20)
[  7148.373] 	GeForce 4Ti         (NV25, NV28)
[  7148.373] 	GeForce FX          (NV3x)
[  7148.373] 	GeForce 6           (NV4x)
[  7148.373] 	GeForce 7           (G7x)
[  7148.373] 	GeForce 8           (G8x)
[  7148.373] 	GeForce 9           (G9x)
[  7148.373] 	GeForce GTX 2xx/3xx (GT2xx)
[  7148.373] 	GeForce GTX 4xx/5xx (GFxxx)
[  7148.373] 	GeForce GTX 6xx/7xx (GKxxx)
[  7148.373] 	GeForce GTX 9xx     (GMxxx)
[  7148.373] 	GeForce GTX 10xx    (GPxxx)
[  7148.373] (II) modesetting: Driver for Modesetting Kernel Drivers: kms
[  7148.376] (EE) [drm] Failed to open DRM device for (null): -2
[  7148.378] (EE) [drm] Failed to open DRM device for pci:0000:01:00.0: -2
[  7148.379] (WW) Falling back to old probe method for modesetting
[  7148.380] (II) modeset(1): using default device
[  7148.380] (EE) [drm] Failed to open DRM device for (null): -2
[  7148.380] (EE) [drm] Failed to open DRM device for (null): -2
[  7148.381] (EE) [drm] Failed to open DRM device for (null): -22
[  7148.382] (II) modeset(G0): using drv /dev/dri/card0
[  7148.382] (EE) Screen 0 deleted because of no matching config section.
[  7148.382] (II) UnloadModule: "modesetting"
[  7148.382] (EE) 
Fatal server error:
[  7148.382] (EE) Cannot run in framebuffer mode. Please specify busIDs        for all framebuffer devices
[  7148.382] (EE) 
[  7148.382] (EE) 
Please consult the The X.Org Foundation support 
	 at http://wiki.x.org
 for help. 
[  7148.382] (EE) Please also check the log file at "/var/log/Xorg.0.log" for additional information.
[  7148.382] (EE) 
[  7148.395] (EE) Server terminated with error (1). Closing log file.

I know this project is fundamentally goofy. But if I can snipe any fellow nerds with free time and kind hearts, I’d be boundlessly grateful.

linux-aarhus · 20 February 2024 05:20

I am nerdy but not that nerdy - as what you picture - is next to impossible - at least with my limited knowledge of running docker containers

If I understand your picture correct - you want to …

Install a basic command line linux system on some hardware with a monitor.
Switch between VT and run different containers from the command line
Those containers should ideally be able to run X for distro hopping
Distro hopping without reboot requires a good deal more than a docker container

To my - admittedly limited understanding - Docker containers isolate single processes based on the host system - which makes it impossible to jump start Ubuntu, Fedora and Manjaro on the same host.

Perhaps I am wrong - I suggest searching the hub.docker.com for e.g. lxqt or xfce

That may give you something to work with as you can pick them apart to see how they work.

There is easier ways to do distro hopping without reboot - I my opinion distro hopping quickly becomes confusing as you have to adapt back and forth between apt, dpkg, dnf, zypper and pacman.

The overall experience is the same whether it is Gnome, Plasma, Xfce or any of window manager solutions existing - and as my personal interest is especially for theming and layouts - Virtual Machines is my go to when I feel the urge to peek into other distributions.

You can create a minimal GUI taking minimal memory using Openbox window manager. You can build the Manjaro ISO profile for e.g. lxqt - getting a full desktop (using Openbox)

If you are careful you can get as low as 140M for the GUI.

I think what you are looking for may be something like https://kasmweb.com/ or https://www.shells.com

maddieM4 · 1 March 2024 00:26

Thank you! I’ve worked on it a bit further, and actually gotten an approach I’m happy with, but it did require some concessions. You do understand my original goals correctly though!

Using containers for this definitely imposes some specific constraints - I’ve accepted a need to run X and Pipewire at the host layer, though I’m trying to keep the full set of packages pretty small. I wanted to avoid even that much host software, but it’s so much easier with those compromises. I’ve also decided, for matching purposes across the host kernel and guest software, that I’m probably only going to put effort into proprietary driver acceleration on ONE guest which intentionally matches the host. Everything else, where I’m just dinkin’ around, can be on Nouveau.

VM support may actually be a legit secondary goal later! I’m prioritizing Docker first because it makes it easier for my desktop environments to be recipe-based, rebuildable, and committed into a personal git repo. They’re also pretty high performance if I put the work in to make them so. It’s also the case that, with all the same DEs available on every distro, comparing package management systems is actually the main pro I’m pursuing, rather than being a con at all!

I’ll keep poking at this, but I’m pretty happy with my proof of concept so far. I think your ideas are going to be very helpful when I get to the VM support stage.