Access pinephone camera with opencv?

ARM Plasma Mobile
1

Is there a way to open either of the cameras with opencv? I have the phone turned off now, but I am pretty sure it had devices video0-video3. I tried them all in the following code, and was met was a basic error that it couldn’t open the v4l2 device, and couldn’t generate the window, or just that it can’t generate the window.

This is python.

import cv2
print(cv2.__version__)
cam=cv2.VideoCapture(0)
while True:
    _, frame=cam.read() 
    cv2.imshow('cam',frame)
    if cv2.waitKey(1)==ord('q'):
        break
cam.release()
cv2.destroyAllWindows()

On almost any system with python, opencv, and a camera at video 0, this should open a window called “cam” and display video from camera 0. Any advice to get it working on the pinephone would be appreciated.

Thanks!

1 Like
2

as far as I know, camera access via v4l2 does not work on the Pinephone (yet).

1 Like
3

Fair enough. I have waited this long for the phone to be kinda usable, I suppose I can wait longer. I just hope it’s on the todo list.

4

Try running:

media-ctl -d1 -l'5:0->1:0[0],7:0->1:0[1]'
media-ctl -d1 -V'7:0[fmt:UYVY8_2X8/1280x720@1/30]'

and then using OpenCV. At least the ffmpeg CLI accepts the video input if the camera is set up that way, but most applications still do not recognize the camera even with that.

1 Like
5

Thanks, I gave that a try and I added an option in opencv to force ffmpeg as the video api.
It didn’t seem to work when running the script from the terminal. I appreciate it though.

A while back I was using a gstreamer pipeline I found online, to make the pi camera work on the jetson nano. I am looking at some literature to see if I can figure out out to make it something like that work here.

Specifically this is what I used:

'nvarguscamerasrc sensor-id=0 wbmode=1 tnr-mode=2 tnr-strength=1 ee-mode=2 ee-strength=0  !  video/x-raw(memory:NVMM), width=3280, height=2464, format=NV12, framerate=21/1 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(dispW)+', height='+str(dispH)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! videobalance hue=-0 contrast=1.5 brightness=-.2 saturation=1.2 ! appsink drop=true'

maybe the right settings will work.

I do get 2 indexes returned from.

 v4l2-ctl -d /dev/video0 --list-formats-ext

I will mess with it later, it’s not really a huge concern at this second.
If I get it sorted, I’ll post an update.

6

Well, this is the complete script I used to record and encode video from the camera, which worked when I last tested it, at least:

media-ctl -d1 -l'5:0->1:0[0],7:0->1:0[1]'
media-ctl -d1 -V'7:0[fmt:UYVY8_2X8/1280x720@1/30]'
fmpeg -input_format yuv420p -s 1280x720 -f video4linux2 -thread_queue_size 4096 -i /dev/video3 -f pulse -thread_queue_size 256 -i alsa_input.platform-sound.HiFi___ucm0001.hw_PinePhone_0__source -c:a flac -c:v ffvhuff video.mkv
ffmpeg -i video.mkv -c:v libvpx-vp9 -c:a libopus video.webm

It would be great to have GStreamer working with the camera, because it would also make many applications work, but I do not think nvarguscamerasrc is going to work, that is not the model used on the PinePhone (and anyway, GStreamer needs to auto-detect the camera for applications to work with it).

2 Likes
7

I was just messing around with this a little bit, and come to find out that the opencv builds from PyPI aren’t built with gstreamer support.

I have built pip wheels on python 3.9.9 for opencv-python, and it’s required numpy. It took overnight to build, so I want to upload them in case someone wants to play around with it.

If it’s inappropriate to post external links, mods feel free to remove.

Screenshot_20220108_195450
Screenshot_20220108_195450720×1440 127 KB

8

And do the builds with GStreamer support recognize the PinePhone cameras?

And if yes: Out of the box? Or after running the correct media-ctl lines? Or how?

9

I have been out of town for work and haven’t touched this device for weeks. I built those wheels the night before I left and haven’t tried anything other than seeing that they were built successfully with gstreamer support. If they updated to python 3.10 those wheels probably won’t install anyway. I will probably take a look at it later in the week.

10