Ok, so it looks like it’s not actually downloading anything and it’s not really even possible to download anything there (it’s possible to download files elsewhere on disk), because it’s a network folder, so every time it creates a temporary file and hence the lags. 
This is convenient in terms of memory, but not really in usability: the lag when navigating between folders or opening a file is annoying. And since gdrive can not be navigated to (https://www.reddit.com/r/kde/comments/dkx5sz/accessing_google_drive_location_from_command_line/f4kohi8?utm_source=share&utm_medium=web2x&context=3) it’s a bit unclear what can it be used for. Maybe if all browsers fail, then you can resort to Dolphin view.
I hope my ignorance will be of help to someone.