Manjaro quality assurance (QA)

I don’t get your point @mbod. Phil reacted very quickly and fixed the faulty package.

Mistakes can and will eventually always happen , since humans are involved. Manjaro has a ‘stable-staging’ branch, where most updates go after testing to be tested again before rolling out to stable.

You just can’t rely on any other branch than stable to get a perfect running system.

2 Likes

I am not talking about the support process.

This is my point: Preventing mistakes by using automated installation&boot test process.
I am surprised that the value is not obvious.

1 Like

I get your point and my post wasn’t aimed at you it really wasn’t, it was just a general observation. There are always posts about "why so long for updates " but when mistakes happen some people jump up and down.
The bot idea is a good idea but I have no idea how difficult it is to implement

Lets get specific. I start.

Other open source projects are using “real” tools to do automated testing. ZFS for example uses buildbot:
http://build.zfsonlinux.org/
http://buildbot.net/

This is probably overkill for what I am proposing here. I was more thinking about a “quick&dirty” solution with virtualbox, shell scripts, etc.

It could look like this

  1. Create a virtual image of a current Manjaro Testing installation. This could be one image per main desktop environment (gnome, kde, xfce, etc.)

  2. Make use of snapshots to be able to restore a default state before any tests.

  3. run the virtual image headless, login with ssh, execute an update. The content of this update is most likely just a subset of the Unstable branch. Capture log information

  4. reboot the virtual image. Capture boot log information.

  5. Forward all log information to a self made parser script which detects relevant errors and notifies the Manjaro team.

  6. This could be done nightly

2 Likes

Are there any people here who have experience with openQA? Or want to learn it?
I think Manjaro Team would welcome such contributors.
http://open.qa/
https://fedoraproject.org/wiki/OpenQA

3 Likes

My (main) unstable machine didn’t have this issue when updating today but half an hour later my (secondary) testing one was affected.
That package/breakage simply wasn’t present on unstable for a long enough time to spot this error. :man_shrugging:

2 posts were split to a new topic: Updated laptop after a long break - now it won’t boot

2 posts were merged into an existing topic: Updated laptop after a long break - now it won’t boot

I’ve done a good bit of automation over the years and will take a look at OpenQA to see if I can wrap my head around it.

1 Like

Why not build out a proof of concept and then contribute to the process…

2 Likes

A post was merged into an existing topic: Welcome and introduce yourself - 2020

I agree, but for now Manjaro would need people to get involved in the process of creating such automation, let just hope some people will join the discussion to help the team.

2 Likes

Ah, so this is more about a workflow proposal rather than a (contract?) for quality assurance … Probably the title might need rephrasing. :slight_smile:

I love when people send me links to wikipedia. It must be a thing this days … in reference to ISO 9000 and ISO/IEC 17025 in the context where:

ISO/IEC 9126 - for software development or software engineering indeed is meant to:

was replaced by ISO/IEC 25010:2011

Anyway, in Romanian the word assurance can be used and interpreted as a pompous thing, patronizing when in a non contractual circumstance. In something “official” is fine and you become bound by it by signing a contract.

1 Like

Having some qa-automation would be helpful and especially manjaro-architect would benefit from it. If someone has time to do a poc for this, it would be greatly appreciated. :slightly_smiling_face:

4 Likes

I think I will rename the topic Manjaro quality assurance (QA) so it is clear that it is a technical term and not an assurance for which the users can sue us. :wink:

2 Likes

I hope my comments didn’t come as derogatory in any way to what the OP intended to convey. While wiki is a nice resource of information, explanations, definitions, etc. i don’t agree with everything from there, and not all the conventions work for me :slight_smile:

Due to life circumstances, I’m not going to be in a position to pursue this for the foreseeable future. Apologies.

1 Like