Customize camera with Camera2 API for Android

Time:2021-9-18

preface

Because the project needs to customize the camera, the author learned about the Android API for camera. Android SDK 21 (lollipop) has abandoned the previous camera class and provided Camera2 related APIs. At present, there is little information about Camera2 API on the Internet. The author collects online information and makes a summary here in combination with his own practice.

technological process

Because Camera2 provides many interfaces, although it is very flexible, it also increases the complexity of use. First, let’s have a general understanding of the process of calling Camera2 to facilitate us to clarify our ideas.

Customize camera with Camera2 API for Android

To display the picture captured by the camera, you only need three steps: initialize the camera, preview and update the preview. That is, the part on the left in the figure above. To implement these three steps, the main interface classes and their action steps are shown in the right part of the above figure. Let’s explain it in detail with code.

case

First create a camera interface:

activity_camera.xml

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:orientation="vertical">

    <TextureView
            android:id="@+id/camera_texture_view"
            android:layout_width="match_parent"
            android:layout_height="match_parent" />

    <ImageButton
            android:id="@+id/capture_ib"
            android:layout_width="60dp"
            android:layout_height="60dp"
            android:layout_marginBottom="10dp"
            android:layout_gravity="bottom|center"
            android:background="@drawable/send_pres"/>

</LinearLayout>

The interface is very simple, with only one texureview and one button.
Next, initialize and display the picture captured by the camera in the activity.

The first problem to be solved is the problem of picture stretching.
To solve this problem, first start with TextureView.

CameraActivity.java

mTextureView.setSurfaceTextureListener(new TextureView.SurfaceTextureListener() {
            @Override
            public void onSurfaceTextureAvailable(SurfaceTexture surfaceTexture, int width, int height) {
                mWidth = width;
                mHeight = height;
                getCameraId();
                openCamera();
            }

            @Override
            public void onSurfaceTextureSizeChanged(SurfaceTexture surfaceTexture, int i, int i1) {

            }

            @Override
            public boolean onSurfaceTextureDestroyed(SurfaceTexture surfaceTexture) {
                return false;
            }

            @Override
            public void onSurfaceTextureUpdated(SurfaceTexture surfaceTexture) {

            }
        });

Initialize the camera in onsurfaceextenseavailable. Opencamera through the cameramanager object, which is the first step in the init step in the flowchart. Opencamera has three parameters. The first is cameraid of string type, the second is cameradevice.statecallback, and the third is handler. Here we want to declare a statecallback:

private CameraDevice.StateCallback mCameraDeviceStateCallback = new CameraDevice.StateCallback() {
        @Override
        public void onOpened(CameraDevice cameraDevice) {
            mCameraDevice = cameraDevice;
            createCameraPreview();
        }

        @Override
        public void onDisconnected(CameraDevice cameraDevice) {
            mCameraDevice.close();
            mCameraDevice = null;
        }

        @Override
        public void onError(CameraDevice cameraDevice, int i) {
            mCameraDevice.close();
            mCameraDevice = null;
        }
    };

You can see that after the camera is ready, you can create a preview interface. To solve the problem of picture stretching, you need to set an appropriate proportion of surface texture buffer size for the preview interface.

private void createCameraPreview() {
        try {
            SurfaceTexture texture = mTextureView.getSurfaceTexture();
            assert texture != null;
            CameraCharacteristics characteristics = mCameraManager.getCameraCharacteristics(mCameraId);
            StreamConfigurationMap map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
            int deviceOrientation = getWindowManager().getDefaultDisplay().getOrientation();
            int totalRotation = sensorToDeviceRotation(characteristics, deviceOrientation);
            boolean swapRotation = totalRotation == 90 || totalRotation == 270;
            int rotatedWidth = mWidth;
            int rotatedHeight = mHeight;
            if (swapRotation) {
                rotatedWidth = mHeight;
                rotatedHeight = mWidth;
            }
            mPreviewSize = getPreferredPreviewSize(map.getOutputSizes(SurfaceTexture.class), rotatedWidth, rotatedHeight);
            texture.setDefaultBufferSize(mPreviewSize.getWidth(), mPreviewSize.getHeight());
            Log.e("CameraActivity", "OptimalSize width: " + mPreviewSize.getWidth() + " height: " + mPreviewSize.getHeight());
            ...
        

Here, judge whether to exchange the width and height values according to the rotation angle of the current equipment and sensor, then obtain the width and height most suitable for the current size ratio through cameracharacteristics, and then set the width and height to the surface texture.

private Size getPreferredPreviewSize(Size[] sizes, int width, int height) {
        List<Size> collectorSizes = new ArrayList<>();
        for (Size option : sizes) {
            if (width > height) {
                if (option.getWidth() > width && option.getHeight() > height) {
                    collectorSizes.add(option);
                }
            } else {
                if (option.getHeight() > width && option.getWidth() > height) {
                    collectorSizes.add(option);
                }
            }
        }
        if (collectorSizes.size() > 0) {
            return Collections.min(collectorSizes, new Comparator<Size>() {
                @Override
                public int compare(Size s1, Size s2) {
                    return Long.signum(s1.getWidth() * s1.getHeight() - s2.getWidth() * s2.getHeight());
                }
            });
        }
        return sizes[0];
    }

Here, sizes is the supported resolution returned by the camera. Find the closest resolution from the parameters we passed.

Next, create and update the preview interface through capturerequest.builder and cameracapturesession.statecallback:

...
Surface surface = new Surface(texture);
            mBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
            //Set preview object
            mBuilder.addTarget(surface);
            mCameraDevice.createCaptureSession(Arrays.asList(surface), new CameraCaptureSession.StateCallback() {
                @Override
                public void onConfigured(CameraCaptureSession cameraCaptureSession) {
                    if (null == mCameraDevice) {
                        return;
                    }
                    mSession = cameraCaptureSession;
                    mBuilder.set(CaptureRequest.CONTROL_MODE, CameraMetadata.CONTROL_MODE_AUTO);
                    try {
                        //Constantly update the captured picture to TextureView
                        mSession.setRepeatingRequest(mBuilder.build(), mSessionCaptureCallback, mBackgroundHandler);
                    } catch (CameraAccessException e) {
                        e.printStackTrace();
                    }
                }

                @Override
                public void onConfigureFailed(CameraCaptureSession cameraCaptureSession) {
                    Toast.makeText(CameraActivity.this, "Camera configuration change", Toast.LENGTH_SHORT).show();
                }
            }, null);
        } catch (CameraAccessException e) {
            e.printStackTrace();
        }

This completes the first step of customizing the camera.Please stamp the source code address here.