Responsible use of AUR

[linux-aarhus]

@alven

the command of course actually installs updates

which is excatly what I said

@linux-aarhus

it works just like pacman -Syu - without actually installing the packages.

[banjo]

@linux-aarhus

I know - it was a reference to Manjaro users seeing AUR as a part of Manjaro - which it most certainly is not.

AGAIN the user ??
Pamac is a efault, installed even on every minimal iso. So please manjaro, don’t blame the users, instead provide deault tools that do not break other sites !
Damned, I beginn to run hot. What a shame.

[sawdoctor]

Unfortunately there will be a lot of users who just don’t read the forum. As manjaro is so user friendly there will be a lot of users who just use pamac and have no idea the problems they are causing for the aur. Just for that reason alone we probably need to make adjustments to the way pamac works

[thingsiplay]

EDIT: Don’t bother reading this reply. It seems the problem is something else and my brainstorming idea doesn’t change anything.

A suggestion: Let us specify an independent setting for automatic check for updates on AUR. This could be something like

1. General: “Check for Updates” - pamac checkupdates --no-aur every 12 hours.
2. If it finds a something new, then in the background automatically do a pamac checkupdates --aur before showing anything to the user. Maybe a message: “Update detected, checking again with AUR”
3. Force AUR check, after every 3 General “Check for Updates” with pamac checkupdates --aur. This would be at default every 36 hours, if previously nothing was found at step 2 already.

The user could in example set in how many steps the AUR will be checked, at default every 3 normal checks. In example if General Check for Updates is set to 6 hours, then the AUR will be checked every 18 hours (or earlier as described in step 2).

If forgot to mention, this is just the default automatic check. Users are still able to check by running the commandline whenever they like.

[omano]

I think people misunderstand the issue. It is not really the sporadic updates check from AUR that cause issues, it is the number of queries Pamac does when people have AUR enabled, when using Pamac, it sends too many requests to the AUR server every time people look for something in Pamac.

[thingsiplay]

You mean even if no explicit check on the AUR for updates is done, the Pamac client will do requrests to AUR?

Would a global cache be useful here? If same request comes again from a different user, then Manjaro AUR Cache would just read there (until expiration, maybe every 6 hour refresh or so).

[omano]

It is the search and suggestion that is the culprit. I don’t think brainstorming about other minor issues is really relevant personally.

[banjo]

@sawdoctor

…a lot of users who just use pamac and have no idea…

No idea of what, that pamac causes issues? That this is a kind of DDOS-tool?
The problem is pamac (waitng for an update), not “the user”.
People just use the tools they were provided. And all the new users doing this intensivly and test out until their installs are settled for their needs. They are not to blame in any way.

[thingsiplay]

So only the GUI version is affected then? I thought it is a general issue with commandline interface too.
If you are right, then there is nothing the user can do actually. The topic is “Responsible use of AUR” with the content that the user should be aware of the issue and not check for updates that often, in example in automation scripts. So you suggest this is not the issue at all, but running the GUI version of Pamac?

And btw, I am not brainstorming about other minor issues. I am talking about solutions for this problem. And a Manjaro AUR Cache could address this issue, regardless of where it comes from.

[winnie]

This is what is hammering the AUR, via the Pamac GUI:

---

Because Pamac defaults to searching the official repositories plus the AUR, it hammers the AUR with every character you type in the search box.

---

Two possible solutions to address this in Pamac itself:

1. Do not default to searching in all sources: only default to searching the official repositories. Make it up to the user to search the AUR.
2. Do not send a query with every character typed: only send a query after the user hits Enter or there is an obvious “end” to their typing, such as a 2-second delay.

---

UPDATE: Therefore, automatically checking for updates (even every 6-hours) is a red-herring. We needn’t worry about automatic updates. The real culprit is the peppering of queries due to sending a request for every character typed, which the AUR is not designed to handle.

[winnie]

I was responding directly to @thingsiplay with a video demonstration, since they needed clarification.

[Takei]

The first solution is not practical and sounds annoying, but the second one (pressing enter) seems practical and is in line with everyday use of other searches like google etc. and as such should be intuitive. Also prevents unnecessary requests simply because the user had to stop and think for 1 or 2 seconds.

[winnie]

It might not be so straight-forward, since I believe they want Pamac to make use of “auto suggestions”, which is not possible if pressing Enter is the only action that will search for something.

[Takei]

I know it isn’t… even google is struggling with that one.
For that to work one would need to prevent the search from happening until enter is pressed a second time after a suggestion is selected. By default no suggestion should be selected and as such the search should be able to be done with only one keypress of Enter.

[thingsiplay]

Autosuggestions could be on for Manjaro repository and the AUR suggestions will only appear after Majaro repos are loaded in or after the user has ended typing.

[alven]

Just resumed some ideas I met:

---

Long-duration fix suggestion

@Evernow

What would be the technical limitation of Manjaro hosting a copy of the AUR which Manjaro users use? It would simply sync the PKGBUILDs at the same time Manjaro syncs with the Arch repos (so would go through the Unstable → Testing → Stable process)

Doesn’t mean official support for the AUR, but that way Manjaro is not spending Arch resources.

@Yochanan

@Evernow

Unstable → Testing → Stable process

Three copies, two stale? No way. Arch would blacklist us all for such a monstrosity.

I think it is one of good idea of fixing. More detailed architecture goes like this:

1. Manjaro team adds another one db: AUR-snapshot alongside with existed (core, extra, comm, multi).

2. Mirror servers syncs that DB with AUR service directly a few times a day as they do it for classical Manjaro repos.

3. Users gets their local copies of AUR-snaphot db only from Manjaro mirrors servers and can do a local search on each keystroke, use autocomplete, etc. 'till the end of Time: all search queries will be local PC only.

What info to grab into AUR-snapshot.db? Name, version, description. What else?

4. After user select a package to build, than only after that Pamac on a user machine do direct network requests to AUR service to grab all needed info: package build file, etc.

So that AUR-snapshop works like cache.

@Yochanan

Three copies, two stale? No way. Arch would blacklist us all for such a monstrosity.

It will be a single AUR-snapshot db for all update branches (unstable, testing, stable-staging, stable).

The architecture requires not a single-hour developing process and hard testing before for release.
May be it is good, but not as fast solution.

---

Fast fix suggestion

@omano

Maybe the default search should also be only in the repositories, instead of searching the AUR by default (clicking the AUR button would then trigger the search)?

Or as illustration:

@alven

The suggested fix probably needs only to change default flag from default section “all” into “repositories”. That’s it.

It will greatly reduce request count to AUR.

Also autocomplete could be removed temporarily.

---

Did we decide which solution exactly to implement? Are we ready to share exact technical task to our friends-developers? Or we are still looking for other possible solutions?
At what stage we are currently?

[openminded]

Interesting. In Russian we use colon for describing what’s before it. Dashes are used to designate a pause or to make a short – maybe explanatory – digression.

PS: I just wanted to say that I got what Frede said, but of course, with “but” it could have been more obvious what he meant.

[andreas85]

@winnie

Do not send a query with every character typed: only send a query after the user hits Enter or there is an obvious “end” to their typing, such as a 2-second delay.

This is a very good suggestion !! (and may reduce the traffic by factor 20 or more)

This sort of problem is not new ! It also exist with local programs, but is often ignored, because todays CPUs are so fast.

Sending a query with every keystroke is always bad practice.

• It searches for a lot of things, the user really does not want
• With a small number of characters (1…4) it will produce a lot of useless hits and create a lot of unwanted traffic
• When the answers arrive, they must be thrown away (hopefully they are not also processed, because the user added a few keys already)

If you look for the worst example of this behavior i have ever seen, look into eclipse installer. There every keypress will block until everything is completely searched for the first keystroke. Then it will throw away everything and starts again with the next added keystroke

[tuxmanjarino]

Imo forget the 1st one as Pamac Manager (GUI) is used by other distros as well and everyone would be forced to do the same, plus it doesn’t address the software deficit directly, so this has to be addressed cleanly “in the code”, which means until the devs find the necessary time and way to properly patch it one should better use pacman or pamac in CLI mode or at the very least disable AUR/untick AUR updates in the GUI like @linux-aarhus has suggested so that the traffic could be somewhat mitigated.

[openminded]

Yep I actually hate the entire Gnome project for this “feature”, especially Nautilus whose window content gets filtered as you type lol. Then you need 2 “Back” button presses or gestures to get out of results view. Who “invented” that I wonder.