Migrate CameraX from SurfaceTexture to SurfaceProducer. (#6462)

_**WIP**: We do not plan to land this PR until the next stable release (>= April 3rd 2024)_.

Work towards flutter/flutter#145930.

## Details

Migrates uses of `createSurfaceTexture` to `createSurfaceProducer`, which is intended to have no change in behavior, but _does_ change the backend rendering path, so it will require more testing (and we're also open to minor API renames or changes before it becomes stable).

## Background 

Android plugins previously requested a `SurfaceTexture` from the Android embedder, and used that to produce a `Surface` to render external textures on (i.e. `video_player`).  This worked because 100% of Flutter applications on Android used OpenGLES (via our Skia backend), and `SurfaceTexture` is actually an (opaque) OpenGLES-texture.

Starting soon (roughly ~Q3, this is not a guarantee and just an estimate), Flutter on Android will start to use our new Impeller graphics backend, which on newer devices (`>= API_VERSION_28`), will default to the Vulkan, _not_ OpenGLES. In other words, `SurfaceTexture` will cease to work (it is possible, but non-trivial, to map an OpenGLES texture over to Vulkan).

After consultation with the Android team, they helped us understand that vending `SurfaceTexture` (the _consumer-side_ API) was never the right abstraction, and we should have been vending the _producer-side_ API, or `Surface` directly. The new `SurfaceProducer` API is exactly that - it generates a `Surface`, and similar to our platform view strategy, picks the "right" _consumer-side_ implementation details _for_ the user/plugin packages.

The new `SurfaceProducer` API has 2 possible rendering types (as an implementation detail):

- `SurfaceTexture`, for older OpenGLES devices, which works exactly as it does today.
- `ImageReader`, for newer OpenGLES _or_ Vulkan devices.

These are some subtle nuances in how these two APIs work differently (one example: flutter/flutter#144407), but our theory at this point is we don't expect these changes to be observed by any users, and we have other ideas if necessary. 

> [!NOTE]
> These invariants are [tested on CI in `flutter/engine`](https://github.com/flutter/engine/tree/main/testing/scenario_app/android#ci-configuration).

Points of contact: 
- @matanlurey or @jonahwilliams  (Flutter Engine)
- @johnmccutchan or @reidbaker  (Flutter on Android)

Author: matanlurey

packages/camera/camera_android_camerax/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.6.5+4
+
+* [Supports Impeller](https://docs.flutter.dev/release/breaking-changes/android-surface-plugins).
+
 ## 0.6.5+3
 
 * Updates minimum supported SDK version to Flutter 3.22/Dart 3.4.

packages/camera/camera_android_camerax/android/src/main/java/io/flutter/plugins/camerax/CameraXProxy.java

@@ -5,9 +5,7 @@
 package io.flutter.plugins.camerax;
 
 import android.app.Activity;
-import android.graphics.SurfaceTexture;
 import android.util.Size;
-import android.view.Surface;
 import androidx.annotation.NonNull;
 import androidx.camera.core.CameraSelector;
 import androidx.camera.core.ImageAnalysis;
@@ -51,11 +49,6 @@ public class CameraXProxy {
     return new Preview.Builder();
   }
 
-  /** Creates a {@link Surface} instance from the specified {@link SurfaceTexture}. */
-  public @NonNull Surface createSurface(@NonNull SurfaceTexture surfaceTexture) {
-    return new Surface(surfaceTexture);
-  }
-
   /**
    * Creates an instance of the {@link SystemServicesFlutterApiImpl}.
    *

packages/camera/camera_android_camerax/android/src/main/java/io/flutter/plugins/camerax/PreviewHostApiImpl.java

@@ -4,7 +4,6 @@
 
 package io.flutter.plugins.camerax;
 
-import android.graphics.SurfaceTexture;
 import android.util.Size;
 import android.view.Surface;
 import androidx.annotation.NonNull;
@@ -25,7 +24,7 @@ public class PreviewHostApiImpl implements PreviewHostApi {
   private final TextureRegistry textureRegistry;
 
   @VisibleForTesting public @NonNull CameraXProxy cameraXProxy = new CameraXProxy();
-  @VisibleForTesting public @Nullable TextureRegistry.SurfaceTextureEntry flutterSurfaceTexture;
+  @VisibleForTesting public @Nullable TextureRegistry.SurfaceProducer flutterSurfaceProducer;
 
   public PreviewHostApiImpl(
       @NonNull BinaryMessenger binaryMessenger,
@@ -62,12 +61,11 @@ public void create(
   @Override
   public @NonNull Long setSurfaceProvider(@NonNull Long identifier) {
     Preview preview = getPreviewInstance(identifier);
-    flutterSurfaceTexture = textureRegistry.createSurfaceTexture();
-    SurfaceTexture surfaceTexture = flutterSurfaceTexture.surfaceTexture();
-    Preview.SurfaceProvider surfaceProvider = createSurfaceProvider(surfaceTexture);
+    flutterSurfaceProducer = textureRegistry.createSurfaceProducer();
+    Preview.SurfaceProvider surfaceProvider = createSurfaceProvider(flutterSurfaceProducer);
     preview.setSurfaceProvider(surfaceProvider);
 
-    return flutterSurfaceTexture.id();
+    return flutterSurfaceProducer.id();
   }
 
   /**
@@ -76,13 +74,13 @@ public void create(
    */
   @VisibleForTesting
   public @NonNull Preview.SurfaceProvider createSurfaceProvider(
-      @NonNull SurfaceTexture surfaceTexture) {
+      @NonNull TextureRegistry.SurfaceProducer surfaceProducer) {
     return new Preview.SurfaceProvider() {
       @Override
       public void onSurfaceRequested(@NonNull SurfaceRequest request) {
-        surfaceTexture.setDefaultBufferSize(
+        surfaceProducer.setSize(
             request.getResolution().getWidth(), request.getResolution().getHeight());
-        Surface flutterSurface = cameraXProxy.createSurface(surfaceTexture);
+        Surface flutterSurface = surfaceProducer.getSurface();
         request.provideSurface(
             flutterSurface,
             Executors.newSingleThreadExecutor(),
@@ -133,8 +131,8 @@ String getProvideSurfaceErrorDescription(int resultCode) {
    */
   @Override
   public void releaseFlutterSurfaceTexture() {
-    if (flutterSurfaceTexture != null) {
-      flutterSurfaceTexture.release();
+    if (flutterSurfaceProducer != null) {
+      flutterSurfaceProducer.release();
     }
   }
 

packages/camera/camera_android_camerax/android/src/test/java/io/flutter/plugins/camerax/PreviewTest.java

@@ -13,7 +13,6 @@
 import static org.mockito.Mockito.verify;
 import static org.mockito.Mockito.when;
 
-import android.graphics.SurfaceTexture;
 import android.util.Size;
 import android.view.Surface;
 import androidx.camera.core.Preview;
@@ -86,59 +85,60 @@ public void create_createsPreviewWithCorrectConfiguration() {
   public void setSurfaceProviderTest_createsSurfaceProviderAndReturnsTextureEntryId() {
     final PreviewHostApiImpl previewHostApi =
         spy(new PreviewHostApiImpl(mockBinaryMessenger, testInstanceManager, mockTextureRegistry));
-    final TextureRegistry.SurfaceTextureEntry mockSurfaceTextureEntry =
-        mock(TextureRegistry.SurfaceTextureEntry.class);
-    final SurfaceTexture mockSurfaceTexture = mock(SurfaceTexture.class);
+    final TextureRegistry.SurfaceProducer mockSurfaceProducer =
+        mock(TextureRegistry.SurfaceProducer.class);
     final Long previewIdentifier = 5L;
-    final Long surfaceTextureEntryId = 120L;
+    final Long surfaceProducerEntryId = 120L;
 
     previewHostApi.cameraXProxy = mockCameraXProxy;
     testInstanceManager.addDartCreatedInstance(mockPreview, previewIdentifier);
 
-    when(mockTextureRegistry.createSurfaceTexture()).thenReturn(mockSurfaceTextureEntry);
-    when(mockSurfaceTextureEntry.surfaceTexture()).thenReturn(mockSurfaceTexture);
-    when(mockSurfaceTextureEntry.id()).thenReturn(surfaceTextureEntryId);
+    when(mockTextureRegistry.createSurfaceProducer()).thenReturn(mockSurfaceProducer);
+    when(mockSurfaceProducer.id()).thenReturn(surfaceProducerEntryId);
 
     final ArgumentCaptor<Preview.SurfaceProvider> surfaceProviderCaptor =
         ArgumentCaptor.forClass(Preview.SurfaceProvider.class);
-    final ArgumentCaptor<Surface> surfaceCaptor = ArgumentCaptor.forClass(Surface.class);
 
     // Test that surface provider was set and the surface texture ID was returned.
-    assertEquals(previewHostApi.setSurfaceProvider(previewIdentifier), surfaceTextureEntryId);
+    assertEquals(previewHostApi.setSurfaceProvider(previewIdentifier), surfaceProducerEntryId);
     verify(mockPreview).setSurfaceProvider(surfaceProviderCaptor.capture());
-    verify(previewHostApi).createSurfaceProvider(mockSurfaceTexture);
+    verify(previewHostApi).createSurfaceProvider(mockSurfaceProducer);
   }
 
   @Test
   public void createSurfaceProvider_createsExpectedPreviewSurfaceProvider() {
     final PreviewHostApiImpl previewHostApi =
         new PreviewHostApiImpl(mockBinaryMessenger, testInstanceManager, mockTextureRegistry);
-    final SurfaceTexture mockSurfaceTexture = mock(SurfaceTexture.class);
+    final TextureRegistry.SurfaceProducer mockSurfaceProducer =
+        mock(TextureRegistry.SurfaceProducer.class);
     final Surface mockSurface = mock(Surface.class);
     final SurfaceRequest mockSurfaceRequest = mock(SurfaceRequest.class);
     final SurfaceRequest.Result mockSurfaceRequestResult = mock(SurfaceRequest.Result.class);
     final SystemServicesFlutterApiImpl mockSystemServicesFlutterApi =
         mock(SystemServicesFlutterApiImpl.class);
     final int resolutionWidth = 200;
     final int resolutionHeight = 500;
+    final Long surfaceProducerEntryId = 120L;
 
     previewHostApi.cameraXProxy = mockCameraXProxy;
-    when(mockCameraXProxy.createSurface(mockSurfaceTexture)).thenReturn(mockSurface);
     when(mockSurfaceRequest.getResolution())
         .thenReturn(new Size(resolutionWidth, resolutionHeight));
     when(mockCameraXProxy.createSystemServicesFlutterApiImpl(mockBinaryMessenger))
         .thenReturn(mockSystemServicesFlutterApi);
+    when(mockTextureRegistry.createSurfaceProducer()).thenReturn(mockSurfaceProducer);
+    when(mockSurfaceProducer.id()).thenReturn(surfaceProducerEntryId);
+    when(mockSurfaceProducer.getSurface()).thenReturn(mockSurface);
 
     final ArgumentCaptor<Surface> surfaceCaptor = ArgumentCaptor.forClass(Surface.class);
     @SuppressWarnings("unchecked")
     final ArgumentCaptor<Consumer<SurfaceRequest.Result>> consumerCaptor =
         ArgumentCaptor.forClass(Consumer.class);
 
     Preview.SurfaceProvider previewSurfaceProvider =
-        previewHostApi.createSurfaceProvider(mockSurfaceTexture);
+        previewHostApi.createSurfaceProvider(mockSurfaceProducer);
     previewSurfaceProvider.onSurfaceRequested(mockSurfaceRequest);
 
-    verify(mockSurfaceTexture).setDefaultBufferSize(resolutionWidth, resolutionHeight);
+    verify(mockSurfaceProducer).setSize(resolutionWidth, resolutionHeight);
     verify(mockSurfaceRequest)
         .provideSurface(surfaceCaptor.capture(), any(Executor.class), consumerCaptor.capture());
 
@@ -189,13 +189,13 @@ public void createSurfaceProvider_createsExpectedPreviewSurfaceProvider() {
   public void releaseFlutterSurfaceTexture_makesCallToReleaseFlutterSurfaceTexture() {
     final PreviewHostApiImpl previewHostApi =
         new PreviewHostApiImpl(mockBinaryMessenger, testInstanceManager, mockTextureRegistry);
-    final TextureRegistry.SurfaceTextureEntry mockSurfaceTextureEntry =
-        mock(TextureRegistry.SurfaceTextureEntry.class);
+    final TextureRegistry.SurfaceProducer mockSurfaceProducer =
+        mock(TextureRegistry.SurfaceProducer.class);
 
-    previewHostApi.flutterSurfaceTexture = mockSurfaceTextureEntry;
+    previewHostApi.flutterSurfaceProducer = mockSurfaceProducer;
 
     previewHostApi.releaseFlutterSurfaceTexture();
-    verify(mockSurfaceTextureEntry).release();
+    verify(mockSurfaceProducer).release();
   }
 
   @Test

packages/camera/camera_android_camerax/pubspec.yaml

@@ -2,7 +2,7 @@ name: camera_android_camerax
 description: Android implementation of the camera plugin using the CameraX library.
 repository: https://github.com/flutter/packages/tree/main/packages/camera/camera_android_camerax
 issue_tracker: https://github.com/flutter/flutter/issues?q=is%3Aissue+is%3Aopen+label%3A%22p%3A+camera%22
-version: 0.6.5+3
+version: 0.6.5+4
 
 environment:
   sdk: ^3.4.0