I just switched to kernel 6.1.12-1 LTS (KDE) and I can confirm this weird behavior of random mouse jumps while scrolling. My mouse is Logitech Performance MX. The experience is just horrible and it just appeared, but until a few minutes back I never had this issue on kernels 5.x. Couldn’t find any helpful info out there, since people tend to point at mice hardware cleaning and what not, but I can assure you this has to be related to kernel 6.x + mouse driver somehow.
EDIT: Apparently, the following driver causes such issues: hid-logitech-hidpp.c and the last normally working driver was from linux 6.0.19. More info: