HDMI-in as camera color bleeding

VIM4
1

Which system do you use? Android, Ubuntu, OOWOW or others?

Android

Which version of system do you use? Khadas official images, self built images, or others?

Official latest image

Please describe your issue below:

We are seeing severe color bleeding (like a low-quality YUV transform or one of the color planes being offset ~4-8 pixels) in the HDMI-in camera preview. Issue is not visible in the HDMI input app in Khadas Settings.

IMG_7757
IMG_77571920×1440 204 KB

The artifact is visible as bands around the edges of all the single-color areas in the picture.

This seems to be limited to the camera preview stream.
Image capture looks ok.
How can we work around this? We are dependent on the realtime preview.

We use V-by-one video output btw, if that matters at all.

Edit: I noticed this conversion was added quite recently. Could it be involved? camera: hal support transform yuv to RGBA by ge2d [1/1] · khadas/android_hardware_amlogic@feb02c4 · GitHub

2

@cadahl Can you tell me the detailed reproduction steps?

3

Sure. We use CameraX with androidx.camera.view.PreviewView and get this artifact on the image. We connect e.g. a laptop displaying Powerpoint or Paint or something similar. We have tried different HDMI devices, such as laptops, screensharing boxes etc.

        mCameraProviderFuture = ProcessCameraProvider.getInstance(parentView.getContext());
        mCameraProviderFuture.addListener(() -> {
            try {
                mCameraProvider = mCameraProviderFuture.get();
                bindPreview(mCameraProvider);
            } catch (ExecutionException | InterruptedException e) {
                e.printStackTrace();
            }
        }, ContextCompat.getMainExecutor(parentView.getContext()));

 ...
bindPreview:

        Preview preview = new Preview.Builder()
                .setTargetResolution(new Size(1920, 1080))
                .build();

        CameraSelector cameraSelector = new CameraSelector.Builder()
                .build();

        preview.setSurfaceProvider(((PreviewView)mPreviewView).getSurfaceProvider());

        try {
            cameraProvider.bindToLifecycle(mLifecycleOwner, cameraSelector, preview);
        } catch (Exception e) {
            e.printStackTrace();
        }

I’ve tried both “implementation modes” of the PreviewView (which uses SurfaceView or TextureView).

I have also tried using a SurfaceView directly, instead of a PreviewView, and manually forced different PixelFormats on the Surface. I tried those that are supported by the camera device (as reported by “dumpsys media.camera”. e.g. NV12, YV12, YUV420_888), but it doesn’t make a difference.

Scaling the preview also makes the artifacts scale along with it, so my guess is that the artifacts are introduced earlier in the pipeline, in some internal conversion step (the color is definitely subsampled, but looks much worse than expected from 4:2:0 - as if one color plane is offset).

Another data point: We did not see this issue in the “HDMI-in-as-camera” builds posted earlier in this forum. But we see it now in the official OS builds.

4

I have made some progress:

As a test, I tried using an ImageAnalysis and copied the image to an ImageView. (Not using the Preview pipeline at all)
This shows the same artifacts, BUT:

If I force setOnePixelShiftEnabled(true) on the ImageAnalysis builder, the result is better:

image
image1920×1440 405 KB

Now the color planes seem to align with the luma plane. There’s still “fuzz” around the colors but that’s because of the chroma subsampling.

This “one pixel shift” flag is automatically set by Android, IF the camera lists the “OnePixelShiftQuirk”. Your camera implementation should do this, but doesn’t.

I don’t think I can force it in the Preview config, so it’s a change that would be needed on your side.

Ideally, I would like to not see chroma subsampling at all but this at least improves the situation.

5

@cadahl You’ve said so much, but you haven’t come to the point yet. Didn’t tell me how to simply reproduce it. For example, you can send me an APK for testing and teach me how to reproduce the problem.

6

I gave you the setup code we use for the camera, that’s all that should be necessary.

I can look into creating a small CameraX sample app later today.

7

@cadahl I see. But after work today, I will be on vacation for 3 days next week. So we can experiment with your APK very late next week.

8

Sorry about the long delay.

Here is a sample app created in Android Studio, it shows the problem.

Attaching another screenshot of how MS Paint looks on a laptop connected through the HDMI input.
To reproduce, simply draw some colored lines on a white background.

IMG_7673
IMG_76731920×1440 515 KB

You can clearly see the colors being shifted horizontally. They are shifted more than a simple RGB-to-YUV conversion would cause.

9

@cadahl Can you directly publish APK for us to verify?

10

Sure, here’s the debug APK: Filebin | 4xr6kq73atwi6uuw

11

@cadahl Before modifying according to this link, verify that it is indeed consistent with what you described. But when modified according to this link, the camera preview will be fine. And the APK you uploaded cannot be opened. Can you see if you can solve the problem of not being able to open it.

12

What does this patch have to do with the color bleeding?

I’ll have a look at the apk, but it’s a simple repro project built in Android Studio. You can easily build it using the previous .zip I linked.

Edit: I guess the link timed out, it will disappear again in 6 days, but here’s a new one: Filebin | tgj6wjgxk2nxytcz

13

@cadahl You misunderstood my meaning. The APK you uploaded can be opened without modification, but it cannot be opened after modifying the image. Please verify it yourself first. The problem you reported has been resolved. I verified it using a system camera.

14

I understand now. I’ve compiled an APK using LENS_FACING_BACK instead:

15

@cadahl What is the conclusion you verified on your end? I will close this post.

16

Sorry for the late reply. I will get back to you shortly on this, I’ve just received the os image.

17

It does indeed seem to be fixed now that the image isn’t mirrored anymore. Thank you! You can close this.