Penguins-eggs: help needed for Manjaro compatibility

Yes, pnpm is not only interesting, it is extremely fast and neater than npm, I just recently discovered it–about a month–and it allows me to avoid the long waits. In the transition a few little problems arose, but it is beneficial.

This morning I was trying to redo the iso, I saw that you already removed penguins-eggs and added pnpm.

So I started from an xfce-minimun, created a script to install vscode, nodejs and pnpm and the various dependencies, and am testing it.

Everything seems okay for this “developers” edition!

download egg-of-manjarolinux-qonos-colibri-amd64_2022-06-05_0757.iso 1,8G-

On a system never seen eggs - full command chain

[projects]$ git clone https://github.com/pieroproietti/penguins-eggs
 ...
[projects]$ cd penguins-eggs
[penguins-eggs]$ ./install-dependencies-manjaro
 ...
[penguins-eggs]$ pnpm i
  ...
[penguins-eggs]$ ./install-dependecies-manjaro
  ...
[penguins-eggs]$ sudo ./eggs dad -d -v
 ...
Daddy, what else did you leave for me?
- creating configuration dir...
- distro template install...
distroTemplateInstall
cp -r /a/projects/penguins-eggs/conf/distros/rolling/* /etc/penguins-eggs.d/distros/rolling/

[penguins-eggs]$ sudo ./eggs produce -v
 ...
eggs >>> Produce an egg....
Disk space used: 182 GB
Space available: 262.1 GB
There are 0 snapshots taking 0 GB of disk space.

The free space should  be sufficient to hold the
compressed data from the system
? Select yes to continue... Yes
Overy: liveCreateStructure
eggs >>> Creating egg in /home/eggs/ovarium/.
mkdir -p /home/eggs/ovarium/
cp /a/projects/penguins-eggs/conf/README.md /home/eggs/ovarium/README.md}
mkdir -p /home/eggs/ovarium/.overlay/lowerdir
mkdir -p /home/eggs/ovarium/.overlay/upperdir
mkdir -p /home/eggs/ovarium/.overlay/workdir
mkdir -p /home/eggs/ovarium/filesystem.squashfs
mkdir -p /home/eggs/ovarium/iso//boot/grub/x86_64-efi
mkdir -p /home/eggs/ovarium/iso//efi/boot
mkdir -p /home/eggs/ovarium/iso//isolinux
mkdir -p /home/eggs/ovarium/iso//live
eggs >>> eggs will remove users accounts and datas from live.
fisherman: partition module in /etc/calamares/modules/partition.conf
fisherman: mount module in /etc/calamares/modules/mount.conf
fisherman: unpackfs module in /etc/calamares/modules/unpackfs.conf
fisherman: fstab module in /etc/calamares/modules/fstab.conf
fisherman: locale module in /etc/calamares/modules/locale.conf
unchanged: keyboard
unchanged: localecfg
fisherman: users module in /etc/calamares/modules/users.conf
fisherman: displaymanager module in /etc/calamares/modules/displaymanager.conf
unchanged: networkcfg
unchanged: hwclock
unchanged: grubcf
unchanged: bootloader
fisherman: packages module in /etc/calamares/modules/packages.conf
unchanged: luksbootkeyfile
unchanged: plymouthcfg
fisherman: removeuser module in /etc/calamares/modules/removeuser.conf
unchanged: umount
fisherman: finished module in /etc/calamares/modules/finished.conf
calamares: create /etc/calamares/branding/eggs/branding.desc
ovary: syslinux
syslinux path: /usr/lib/syslinux/bios/
cp /usr/lib/syslinux/bios//vesamenu.c32 /home/eggs/ovarium/iso//isolinux/
cp /usr/lib/syslinux/bios//chain.c32 /home/eggs/ovarium/iso//isolinux/
cp /usr/lib/syslinux/bios//ldlinux.c32 /home/eggs/ovarium/iso//isolinux/
cp /usr/lib/syslinux/bios//libcom32.c32 /home/eggs/ovarium/iso//isolinux/
cp /usr/lib/syslinux/bios//libutil.c32 /home/eggs/ovarium/iso//isolinux/
ovary: isolinux
cp /usr/lib/syslinux/bios//isolinux.bin /home/eggs/ovarium/iso//isolinux/
ovary: kernelCopy
cp /boot /home/eggs/ovarium/iso//live/
cp: -r not specified; omitting directory '/boot'

There is a script error with the produce command - excerpt

src/classes/ovary.ts → line #641

await exec(`cp ${this.settings.kernel_image} ${this.settings.work_dir.pathIso}/live/`, this.echo)
 ...
ovary: kernelCopy
cp /boot /home/eggs/ovarium/iso//live/
cp: -r not specified; omitting directory '/boot'

Retrying after changing above error ends with the following error

ovary: bindLiveFs
ovary: makeIfNotExist(/home/eggs/ovarium/filesystem.squashfs/a)
mount --bind --make-slave /a /home/eggs/ovarium/filesystem.squashfs/a
mount -o remount,bind,ro /home/eggs/ovarium/filesystem.squashfs/a
cp -r /bin /home/eggs/ovarium/filesystem.squashfs
mount --bind --make-slave /boot /home/eggs/ovarium/.overlay/lowerdir/boot
mount -o remount,bind,ro /home/eggs/ovarium/.overlay/lowerdir/boot
ovary: makeIfNotExist(/home/eggs/ovarium/.overlay/upperdir/boot)
ovary: makeIfNotExist(/home/eggs/ovarium/.overlay/workdir/boot)
ovary: makeIfNotExist(/home/eggs/ovarium/filesystem.squashfs/boot)
mount -t overlay overlay -o lowerdir=/home/eggs/ovarium/.overlay/lowerdir/boot,upperdir=/home/eggs/ovarium/.overlay/upperdir/boot,workdir=/home/eggs/ovarium/.overlay/workdir/boot /home/eggs/ovarium/filesystem.squashfs/boot
mount: /home/eggs/ovarium/filesystem.squashfs/boot: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error.
       dmesg(1) may have more information after failed mount system call.

This part strikes me as odd

mount --bind --make-slave /a /home/eggs/ovarium/filesystem.squashfs/a

Is your script failing on an assumption that the root path to the repo is something else which it - in my case - is not?

Is your script failing because the path /a is pre-existing on my system but not yours?

Let’s me to investigate… I’m using a clean machine like you, formatted ext4 branch stable, and don’t get this error. Until now I tried selecting fast compression.

regarding

mount --bind --make-slave /a /home/eggs/ovarium/filesystem.squashfs/a

I tried creating a /pippo on root, and don’t evidences result.
I discovered a problem in calamares configuration /etc/calamares/modules/packages.conf when I use --fast option, the resulting installation fail

I think I will solve this in the afternoon, it’s just a misconfigured yaml. Anyway becouse the problem arise at the end of installation, the system was installed with the presence of /pippo.

BTW: it’s nice to have finally feedback. Good sunday @linux-aarhus!

@artisan @Yochanan, i think we have a misunderstanding here: i think is not necessary to remove penguins-eggs from our repo because if we remove this we have to remove 99% of our program from the repo since anyone can build itself from the source. I think what @artisan wants to say with “hack” is to build directly from the source when a developer wants instantly test the changes made by him for helping the development of penguins-eggs. Indeed if the packaging is not the best because the directive of npm ( or pnpm ) for building the project is wrong from a system-wide perspective we can improve it…

1 Like

@ste74, @Yochanan
Yes, this is exactly that I want to say and tryed to explain.

I used npm to install pnpm just becouse pnpm it was not in the repository and recently I discover it and it’s quite fast compared to npm.

In all the ways for me it’s not a big problem to not have eggs in the community repo, perhaps can attrack more developers than just users.

In all ways we will respect all the limits of the Law! :slight_smile:

Yep i understand but if your program is in our repo the program have more visibility and is plus easy to attract more developers. In other hands you don’t have visibility :face_with_peeking_eye:

For now it is until we can package it properly. Dumping the whole repo into /usr/lib/ is not acceptable. This project is not meant to be installed to the system at all. The only thing that should be in /usr/lib/ is the node_modules. Perhaps the main files can be installed to /opt/.

That’s not why as I already corrected that, remember? :wink:

In the meantime, a user should do the following:

git clone https://github.com/pieroproietti/penguins-eggs
cd penguins-eggs
pnpm i
pnpm run build
2 Likes

I’m trying to create a new version of BUILDPKG for eggs, so I forked from the original-one on penguins-eggs-manjaro

Just tested and seem to work, my idea was to take manjaro calamares configuration too, but at the moment I’m not able, so still remain mine settings.

my dependencies are:

depends=('arch-install-scripts' 'erofs-utils' 'manjaro-tools-iso' 'mtools' 'nodejs' 'npm' 'python' 'syslinux' 'xdg-utils')

Why I get a lot of dependecies?

  • apparmor-3.0.4-1
  • calamares-tools-0.1.0-1
  • libburn-1.5.4-1
  • libisoburn-1.5.4-2 *
  • libisofs-1.5.4-1 libuv-1.43.0-1
  • manjaro-iso-profiles-base-17.1.12-1
  • manjaro-tools-base-git-r2996.d3ab091-1
  • manjaro-tools-yaml-git-r2996.d3ab091-1
  • mktorrent-1.1-4
  • ruby-3.0.4-1
  • ruby-irb-1.4.1-1
  • ruby-kwalify-0.7.2-3
  • ruby-reline-0.3.1-1
  • rubygems-3.3.8-1
  • snapd-2.55.5-1
  • arch-install-scripts-24-2
  • erofs-utils-1.4-2
  • manjaro-tools-iso-git-r2996.d3ab091-1
  • mtools-1:4.0.39-1
  • nodejs-18.2.0-1
  • syslinux-6.04.pre2.r11.gbf6db5b4-3

most of then I think I don’t need at all. How optimize it?

@Yochanan @Ste74 @codesardine

Those are dependencies required by manjaro-tools-iso-git:

❯ pacman -Sii manjaro-tools-iso-git | grep 'Depends On'
Depends On      : dosfstools  libisoburn  squashfs-tools  manjaro-tools-yaml-git  mkinitcpio  mktorrent  grub  snapd  git

You can also use pactree for each package.

1 Like

Ok, thanks @yochanan

Trying to remove manjaro-tools-iso from dependencies

We use very little from manjaro-tools-iso, so I’m trying to remove it from dependencies.

Mostly we need hooks for mkinitcpio, I’m getting this error:

==> ERROR: Hook 'miso_shutdown' cannot be found
==> ERROR: Hook 'miso' cannot be found
==> ERROR: Hook 'miso_loop_mnt' cannot be found
==> ERROR: Hook 'miso_kms' cannot be found

how to fix it?

We can find them on manjaro-tools. under /usr/lib/inipcpio

It’s neccessary to copy manjaro-tools/initcpio

sudo cp manjaro-tools/initcpio/ /usr/lib -R

We need too /etc/initcpio/miso_shutdown. I don’t remember now it from where come, I just copied it inside this repo, so:

cp miso_shutdown /etc/initcpio

Not too orthodox, but It seem to work!

Advices for doing it in a more orthodox way are welcome, :slight_smile:

Resulting iso egg-of-manjarolinux-qonos-xfce-minimun-amd64_2022-06-12_0920.iso 1,8G, can be found in Penguin's eggs - Browse /iso/manjaro at SourceForge.net we reduced iso size compared to previuos version about 100M.

@Yochanan @Ste74 @codesardine

Ok, let me do It…

1 Like

manjaro-tools-iso

To not depend from manjaro-tools-iso, who bring a lot of stuffs not strictly necessary for eggs, we need:

cut and paste

git clone https://gitlab.manjaro.org/tools/development-tools/manjaro-tools
sudo cp manjaro-tools/initcpio/hooks/ /usr/lib/initcpio/ -R
sudo cp manjaro-tools/initcpio/install/ /usr/lib/initcpio/ -R
sudo cp manjaro-tools/initcpio/script/miso_shutdown /etc/initcpio/

We must do that, probably during eggs installation.

No. Then penguins-eggs will conflict with manjaro-iso-tools-git.

OK Yochanan, but that I’m trying is to find a solution!

How will be possible to get that?

We need just: hooks and install contents from manjaro-iso-tools and the script miso_shutdown.

I just arranged this page on sourceforge with the main indications for eggs on manjaro.

this week trying to optimize the PKGBUILD of the package I made a lot of progress and tested eggs with the new stable and and testing updated.

On my PKGBUILD I cleaned it up, reintroduced penguins-eggs.install and removal of the various files that eggs generates for its configuration.

Also, I configured for manjaro, the automatic removal of both calamares and penguins-eggs at the end of installation.

That leaves, the problem of how to “inherit” from manjaro-tools-iso the various hooks that eggs needs without taking everything else with it. I rely on those who know more about this than I do, but it should not be impossible.

@ste74 @Yochanan @codesardine

My answer remains the same as above.

I’ve attempted to improve the PKGBUILD, but I can only do so much when the upstream repo is neither organized nor actually designed to be installed to the system. A Makefile to assist packagers would also be nice.

I still think it would be best to use this tool in userspace as it is. However, it is nice to be able to have all dependencies tied to the package that requires them instead of having to explicitly install things for user space program.

1 Like

Of course, you are right… Unfortunately, I’m used to work mostly alone so, this is my foult!

My problem in all the ways is not so much - to have or not to have the package on community - but to make eggs usefull for users.

The idea is to bring here, where I see a little more interest, the capabilitis of eggs: give to the system the capability of reproduce and reinstall. In short:

  • remaster/install both CLI/Desktop base or finished system;

I want Include too a wardrobe of costumes too: a repo with collection of preconfigurated configurations.

I did that on Debian - and it’s already included but not working in manjaro version - I have another repo penguins-wardrobe with pre-created costumes (desktop configurations), servers (servers configurations) and accessories (used for common things).

Nothing so great, customizing a system takes time and need to do so, mostly just samples except for my development machine “colibri” who I really use every day, and other birds who can be build on Debian/Devuan or Ubuntu.

The “great hope” is to add manjaro, spread eggs and assist to an evolution… I’m just a dreamer perhaps.

Thanx a lot for your answers, I will try to define a Makefile to install eggs, to help mantainers.

Ok, I was finally able to get a naked version of manjaro.

This is just a CLI system, but capable of reproduction. The image is about 800M and can be used to create a server, to install and try a different desktop and so on.


ruah-boot-live

Of course, we need again to clean and improve things, I hope here to get interest, feedback and help.

I think eggs can be already good to build servers, and remaster custom GUI systems (something like the old systemback in Debian or the venerable Remastersys). Here I tryed to replicate that and dress with a nice metaphora: penguins, eggs and reproduction.

Well, the composition is starting to work and be usable, there are again things to do but it’s possible to install and remaster a manjaro CLI system using eggs.

I put the resulting ISO image on my Penguin’s eggs page on sourceforge, and made a short video to give an idea of that I’m speacking: Just a installation of a naked manjaro system remaster with eggs - YouTube

I think can be usable enought, I heard about peoples wanting use manjaro as server, eggs let to install, change and remaster your CLI system as much you need. BTW: it work with desktop too and in that case, it’s prerefible to use GUI installer Calamares.

@codesardine @Ste74 @Yochanan