Start a new topic

Unity Camera Access on MiRZA: Failed to release frame from runtime - XR_ERROR_VALIDATION_FAILURE

We wanted to implement camera feed on MiRZA to send RGB camera images to our object detection & segmentation model. However, when trying to implement [standard `OnFrameReceived` event as described here](https://docs.spaces.qualcomm.com/unity/features/camera-frame-access-feature) we struggled to capture more than first 1-2 frames. Before `OnFrameReceived` stops firing we could see some initialisation issues (we enabled `ARCameraManager` on `Start()` as it was recommended to turn it off before building):


```Qualcomm.Snapdragon.Spaces.CameraAccessFeature:TryGetLatestCpuImage(Cinfo&)` printing `Tried to acquire latest CPU image, but no CPU image is available yet.

```


When looking through camera-related logs we would see repeatedly: 

```

2290 E QVRConnection: /dev/socket/qvrservice_camera:122: HandleDataPacket: failure

3921 E QVRClientConnection: /dev/socket/qvrservice_camera:1: SendReceivePacket: error occurred: QVR_ERROR (-1)

3921 E QVRClientImpl: /dev/socket/qvrservice_camera:122: GetParam Numerical failed

3921 E QVRCameraDeviceClient: /dev/socket/qvrservice_camera:122: GetCameraAEParameters: Error on getting the frame config size

3921 W QVRCameraClientImpl: /dev/socket/qvrservice_camera:122: AttachCamera[wireless-rgb] failed - Get service AE related prameters failed

2290 I QVRServiceCamDevice: wireless-rgb camera creating a new stream with ID 0

3921 I QVRCameraClientImpl: /dev/socket/qvrservice_camera:122: AttachCamera[wireless-rgb] - camIndex=0 / ring id=51

07-27 17:04:43.115 3826 3921 W QVRClientImpl: /dev/socket/qvrservice_camera:122: warning: value for 'wireless-rgb_camera_resolution_mode' is being truncated (64 < 255)

3921 E QVRCameraClientImpl: /dev/socket/qvrservice_camera:122: GetCameraStreamUriFromResFormat: Camera wireless-rgb Uri list not found

2290 E QVRServiceConfiguration: GetParam vendor/qcom/proprietary/qvr-vndr/service/src/QVRServiceConfiguration.cpp 3927: Invalid parameter wireless

```


After trying to debug  this for a while decided to check MiRZA's samples and stripped down the code from [MiRZA Camera image recognition sample](https://www.devices.nttqonoq.com/developer/doc/samples/samples-qonoq/unity-packages/function/image-recognition 

), class `ImageCapture`, which worked straightaway (with pretty much the same code as what we had tried but less scene and lifecycle/initalisation complexity).  We did not change anything in our project settings, and the sample was capturing frames reliably, except for... we noticed that `XR_ERROR_VALIDATION_FAILURE` error is flooding the logs, no frames get released by `CameraAccessFeature` class  with `Failed to release frame from runtime` happening for each frame


For now we made the initialisation of our code logic wait for glasses to be ready and then for `AR Camera Manager` (enabling the component like in `ImageCapture`, starting subsystems) processes to complete. Now camera frame capture almost always starts correctly, it still suffers from the frame release issue. Please find a sample logcat output of running the app attached. 

The frame release error leads to severe frame rate drop over a short period of time. Even when we throttle image capture (1/s), this performance is not feasible for users to accept.


Has anyone been looking into the frame release issue? We presume it has been known to the Spaces team given MiRZA's official sample is experiencing it?


We are using Snapdragon Spaces 1.0.2 (same results with Unity 2022.x and Unity 6.0x).

txt
1 Comment

For continued support, please forward the requested information to our team at support@spaces.qualcomm.com.

Login to post a comment