Potential file conflict with new python-pyqt5 - Fix it with manjaro-system?

#1

In the last update release, several users (fortunately not everyone) reported having a file conflict with /usr/lib/python3.7/site-packages/PyQt5/__pycache__/__init__.cpython-37.pyc, a file that was automatically generated by Python and that is now included in the newest python-pyqt5 package. People that got the issue fixed it simply by deleting the file or renaming it to something else.

Some users in Arch Linux also got the same issue too. For example:
https://bbs.archlinux.org/viewtopic.php?id=243825
https://bbs.archlinux.org/viewtopic.php?id=243795 (Although for this guy, he got many conflicting files, but on the other post as well as what was reported on Testing here, there was only the file mentionned above. So I guess this case is much rarer? At least, I hope so?)

Considering that there was some users, both on Manjaro and on Arch, reporting a file conflict with the exact same package with the exact same file, I think it isn’t just an isolated case and that it is likely that it’ll affect a couple of users when it’ll hit stable if it is left like that.

@philm To make the upgrade smoother, perhaps some code should be added in manjaro-system in order to delete the conflicting file before python-pyqt5 get upgraded. A simple rm (-f ?) /usr/lib/python3.7/site-packages/PyQt5/__pycache__/__init__.cpython-37.pyc should be enough to deal with the issue without asking a manual intervention from the user, at least for the majority of affected people (hopefully there aren’t too much people that are as messed up as the guy in the second thread I mentioned from the Arch Linux forum).

3 Likes
[Testing Update] 2019-02-17 - Brave, Deepin, Wine, Python, Haskell
[Testing Update] 2019-02-15 - Kernels, Grub, Systemd, Haskell, Python
[Testing Update] 2019-02-10 - KDE Apps, Gambas, Qt5, VirtualBox, LibreOffice, Wine, Firefox
#2

This particular file might be only the first of many: every sub-directory within the /usr/lib/python3.7/site-packages/PyQt5/ tree could have its own __pycache__ directory, each containing many .pyc files.

Would it be easy enough to remove all __pycache__ dirs that exist there?

[ I can never remember the details of how to do such complex operations in bash; I use python for that kind of thing. ]

EDIT:
Have just noticed that there are not many subdirs in that tree – only ‘uic’ and its subdirs. I mis-read that folder in Dolphin; it has lot’s of files with CamelCase names, and I’m used to Uppercase being a directory … sorry.

#4

I guess it could be doable? With find /usr/lib/python3.7/site-packages/PyQt5 -type d -name __pycache__, I get the following result.

[awesome@i56400 ~]$ find /usr/lib/python3.7/site-packages/PyQt5 -type d -name __pycache__
/usr/lib/python3.7/site-packages/PyQt5/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__

Perhaps the output could be used for a For loop that execute a rm -r $i for each of those directories. Maybe use -f if deemed necessary.

#5

So yes, it is doable to remove all __pycache__ folder in /usr/lib/python3.7/site-packages/PyQt5/ with bash with the idea I had. Here’s my proof of concept.

#!/bin/bash

### For debugging purpose ###
echo 'Search for __pycache__ folders related to PyQt5'
find /usr/lib/python3.7/site-packages/PyQt5 -type d -name __pycache__
echo "Found: $(find /usr/lib/python3.7/site-packages/PyQt5 -type d -name __pycache__ | wc -l)"
echo " "
#############################

echo 'Deleting all __pycache__ folders in /usr/lib/python3.7/site-packages/PyQt5'
echo 'to prevent potential file conflicts.'
for i in $(find /usr/lib/python3.7/site-packages/PyQt5 -type d -name __pycache__)
  do
    echo "Deleting $i"
    rm -rf $i
  done  
  
### For debugging purpose ###
echo " "
echo 'Search for __pycache__ folders related to PyQt5'
find /usr/lib/python3.7/site-packages/PyQt5 -type d -name __pycache__
echo "Found: $(find /usr/lib/python3.7/site-packages/PyQt5 -type d -name __pycache__ | wc -l)"
#############################

Of course, if I run this without elevated privileges, it fails to delete any files.

Output of script without elevated privileges
[ami@mjrocin ~]$ ./rm-pycache-pyqt5 
Search for __pycache__ folders related to PyQt5
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/__pycache__
Found: 6
 
Deleting all __pycache__ folders in /usr/lib/python3.7/site-packages/PyQt5
to prevent potential file conflicts.
Deleting /usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/qtproxies.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/compiler.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/__init__.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/indenter.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/qobjectcreator.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/misc.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/compiler.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/proxy_metaclass.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/indenter.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/proxy_metaclass.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/misc.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/__init__.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/qtproxies.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/qobjectcreator.cpython-37.opt-1.pyc': Permission denied
Deleting /usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__/__init__.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__/qobjectcreator.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__/loader.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__/__init__.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__/loader.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__/qobjectcreator.cpython-37.opt-1.pyc': Permission denied
Deleting /usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/string_io.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/as_string.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/proxy_base.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/__init__.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/ascii_upper.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/as_string.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/string_io.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/ascii_upper.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/proxy_base.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/__init__.cpython-37.pyc': Permission denied
Deleting /usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/driver.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/uiparser.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/__init__.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/objcreator.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/exceptions.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/exceptions.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/properties.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/pyuic.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/objcreator.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/icon_cache.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/properties.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/__init__.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/driver.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/icon_cache.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/uiparser.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/pyuic.cpython-37.pyc': Permission denied
Deleting /usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qscintilla.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qtprintsupport.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qtquickwidgets.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qtwebenginewidgets.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qtcharts.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qscintilla.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qaxcontainer.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qaxcontainer.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qtwebenginewidgets.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qtquickwidgets.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qtwebkit.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qtcharts.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qtprintsupport.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__/qtwebkit.cpython-37.opt-1.pyc': Permission denied
Deleting /usr/lib/python3.7/site-packages/PyQt5/__pycache__
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/__pycache__/__init__.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/__pycache__/pyrcc_main.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/__pycache__/__init__.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/__pycache__/pylupdate_main.cpython-37.opt-1.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/__pycache__/pylupdate_main.cpython-37.pyc': Permission denied
rm: cannot remove '/usr/lib/python3.7/site-packages/PyQt5/__pycache__/pyrcc_main.cpython-37.pyc': Permission denied
 
Search for __pycache__ folders related to PyQt5
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/__pycache__
Found: 6
[ami@mjrocin ~]$ 

If I run the script with sudo, all the folders actually get deleted, as we wanted.

Output of the script when run with elevated privileges.
[ami@mjrocin ~]$ sudo ./rm-pycache-pyqt5 
[sudo] password for ami: 
Search for __pycache__ folders related to PyQt5
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/__pycache__
Found: 6
 
Deleting all __pycache__ folders in /usr/lib/python3.7/site-packages/PyQt5
to prevent potential file conflicts.
Deleting /usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__
Deleting /usr/lib/python3.7/site-packages/PyQt5/uic/Loader/__pycache__
Deleting /usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__
Deleting /usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__
Deleting /usr/lib/python3.7/site-packages/PyQt5/uic/widget-plugins/__pycache__
Deleting /usr/lib/python3.7/site-packages/PyQt5/__pycache__
 
Search for __pycache__ folders related to PyQt5
Found: 0
[ami@mjrocin ~]$ 

It wasn’t rocket science, but I wanted to be see if it was doable and not too complicated to create.

#6

FYI: This might impact more users than expected. Anyone who has used HP Device Manager to install a printer is prompted for root credentials to install/configure the printer. Since this software uses python and Qt5, the elevation request might create the cache file in question (I cannot currently test this due to a bug in the current version of hplip – the GUI does not accept user or root credentials; I’ve been having to use sudo hp-toolbox -i and manually download the plugin as well).

However, i can confirm that running sudo hp-toolbox generates that cache file.

I also have a quick question: if this is a cache file that is only created when python/qt5 apps are run with elevated permissions, why is it statically included in python-pyqt5? Is that a packaging error? Or does something need the included cache file?

1 Like
#7

Since it is a common and known issue people facing the matter should manually delete the file on their own. I don’t think a hack with manjaro-system is needed for this matter.

#8

Python tries to create a bytecode cache when system libraries are accessed. Generally it will fail because of insufficient rights.

The correct approach is to package the cache files along with the libraries (which is something Debian has done for years).

1 Like
closed #9

This topic was automatically closed after 90 days. New replies are no longer allowed.