Alright! I got it working!
The problem was (or is) that the cuda package is version 12.2 and pip install tensorflow[and-cuda] installs tensorflow 2.13.1 but with specifying the version for 2.14.0 at least the GPU is recognized. Unfortunately cudnn, cublas and cufft from 12.2 aren’t supported in 2.14.0. But yeah so far so good 
Innocent question: As I am fairly new to arch (as you might can tell), I am wondering, if in general it would make sense to update driver packages at the point in time when they can actually be used? Or am I not seeing something here?
Thanks for the help!
ps. headsup tf 2.14 only runs on python 3.10 as the required tensorrt package is not yet available for python 3.11