Image Tuning

Image tuning is the process of adjusting various image parameters to achieve the intended image effect.

• Image parameters control image processing.
• Image effect is the viewer’s acceptance level of the image quality (IQ).
  • Simple tuning can be achieved with Tuya’s open APIs to adjust parameters such as brightness, sharpness, and saturation.
  • Complex tuning requires the chip vendor’s IQ tool to adjust underlying parameters such as exposure convergence speed and noise in specific scenes.

Scenarios

• Subjective tuning: Adjust image parameters to achieve the desired result. This can include increasing saturation for more intense colors, increasing sharpness for clearer images, or increasing brightness for an overall brighter appearance.

• Objective tuning: Fix IQ issues in specific scenes, for example, flicker, color shift, and noises.

Feature description

Method 1: Tune with Tuya’s APIs (image style customization)

In the root directory of tuyaos-ipc-$CHIPNAME, run the command make app_menuconfig to customize the image style. After saving your modifications, run make app_config to compile the modifications and generate a configuration file.

This method enables you to tune the following parameters:

• Brightness
• Saturation
• Sharpness
• Contrast
• Threshold for day/night mode switching.
• Others

This method works with Anyka’s AK39EV330L and Fullhan’s FH8636/FH8652 currently, with more compatible platforms on the way. For more information, consult your account manager.

Method 2: Tuning ISP parameters with IQ tools (fine-tuning)

This method enables you to tune the following parameters:

• AE and AWB
• Dynamic range, including BLC, gamma, WDR, and dehaze.
• Signal-to-noise ratio, including 2DNR, demosaicing, 3DNR, and sharpen.
• Color correction, including CCM.
• Other tuning modules, including LSC, LDC, and DPC.

This method works with all chips, but the tuning process may differ depending on the platform. See the tuning guide specific to your platform.

Acronyms

• Image signal processing (ISP)
• Auto exposure (AE)
• Auto white balance (AWB)
• Black level correction (BLC)
• Color correction matrix (CCM)
• 3D noise reduction (3DNR)
• Lens shading correction (LSC)
• Lens distortion correction (LDC)
• Defect pixel correction (DPC)

How it works

How image parameters work and how to implement them:

• How it works: When the IPC is running, isp.bin is the most important image file. isp.bin is reloaded when the IPC is powered on or switches between day and night mode to apply the appropriate image effects.

  The format of the isp.bin file varies by chip platform. For example, isp.conf for Anyka, isp.hex for Fullhan, and tag.bin for Ingenic.

• Implementation: Modify isp.bin to achieve the desired result.

  • Tuya’s open APIs unify the tuning functions of different chip platforms.

    To tune with Tuya’s APIs, modify and compile the configuration file and then check the tuning results.

  • Tune the underlying ISP parameters. Changes are applied to the IPC immediately. This method is used to fine-tune the image quality but requires coding knowledge.

    • After modifying the ISP parameters, you need to copy and replace the ISP configuration file in the target folder and then compile the file to check the tuning results.
      Using the Anyka platform as an example, replace the existing ISP configuration file in software/TuyaOS/apps/tuyaos-ipc-ak3918ev330-xxxxx/resource/isp with the new one isp.conf, while retaining the original name of the .conf file.
    • If you modify the sensor driver file, add or replace the driver in software/TuyaOS/vendor/anyka3918ev300s/tuyaos/components/linux-anyka3918ev300s/drivers/modules.

Image development

Concepts

The image component is the entry point for image configuration, which is named in the format tkl_media_CHIPNAME and usually located at /vendor/CHIPNAME/tuyaos/tuyaos_adapter/src/tkl_media_CHIPNAME.

Image component includes APIs for common ISP features, including ISP parameter initialization, converting color images to black and white, flipping, anti-flicker, calculating illuminance for day/night mode switching, and image customization.

Scenarios

When the IPC is running, the image component takes care of two things.

• When the IPC is powered on, the component runs isp_init and loads image parameters.
• When the IPC is running, the component invokes the corresponding API for configuration when the user sets features, such as day/night mode, anti-flicker, and flipping, on the mobile app.

You can simply call the API as needed, without modifying the image component.

API description

Initialize image component

/***********************************************************
* Function: tkl_vi_isp_init
* Description: ISP initialization
* Parameter: IN TKL_VI_CHN_E chn  The device ID
         IN TKL_ISP_PARAM_INFO_T *pisp_info  The struct of configurable ISP parameters, including ADN, WDR, AWB, and denoise.
*
* Return value: <0: Failure. =0: Success.
***********************************************************/
OPERATE_RET tkl_vi_isp_init(TKL_VI_CHN_E chn, TKL_ISP_PARAM_INFO_T *pisp_info)

Load image parameters

/***********************************************************
* Function: tkl_vi_isp_load_isp_file
* Description: Load the image parameters, typically a .bin or .ini file.
* Parameter: IN TKL_ISP_DN_MODE_E irmode,  The current day/night mode. 0: Auto mode. 1: Day mode. 2: Night mode. Specifying it to auto will force the day mode. Auto is not supported.
*
* Return value: <0: Failure. =0: Success.
  ***********************************************************/
OPERATE_RET tkl_vi_isp_load_isp_file(TKL_ISP_DN_MODE_E irmode)

Load custom image style configuration

/***********************************************************
* Function: tkl_vi_isp_load_user_isp_param
*    Description: Load user-defined image settings, such as style, white balance, WDR, and noise reduction.
    The image configurations configured by this function are all static, without real-time calculation.
* Parameter: IN TKL_VI_CHN_E chn The device ID
         IN TKL_ISP_PARAM_INFO_T *pisp_info, The index structure of all the required configurable ISP parameters during initialization.
*
* Return value: <0: Failure. =0: Success.
***********************************************************/
OPERATE_RET tkl_vi_isp_load_user_isp_param(TKL_VI_CHN_E chn, INT32_T irmode)

• tkl_vi_isp_load_isp_file calls tkl_vi_isp_load_user_isp_param automatically, without additional API request.

Flip images

/***********************************************************
* Function: tkl_vi_isp_get_mirror_flip
* Description: Get the current flipping status.
* Parameter: IN TKL_VI_CHN_E chn  The channel ID of the camera.
         OUT TKL_VI_MIRROR_FLIP_E *direction, 0: Default status. 1: Mirror. 2: Flip. 3: Mirror and flip.
*
* Return value: <0: Failure. =0: Success.
***********************************************************/
OPERATE_RET tkl_vi_isp_get_mirror_flip(TKL_VI_CHN_E chn, TKL_VI_MIRROR_FLIP_E *direction)

/***********************************************************
* Function: tkl_vi_isp_set_mirror_flip
* Description: Set image flipping.
* Parameter: IN TKL_VI_CHN_E chn The device ID
         IN TKL_VI_MIRROR_FLIP_E direction, 0: Default status. 1: Mirror. 2: Flip. 3: Mirror and flip.
*
* Return value: <0: Failure. =0: Success.
***********************************************************/
OPERATE_RET tkl_vi_isp_set_mirror_flip(TKL_VI_CHN_E chn, TKL_VI_MIRROR_FLIP_E direction)

Configure anti-flicker

/***********************************************************
* Function: tkl_vi_isp_get_antiflicker
* Description: Get the current anti-flicker status.
* Parameter: IN TKL_VI_CHN_E chn The device ID
         OUT TKL_ISP_FLICKER_TYPE_E *eAntiFlicker, 0: Disabled. 1: 50 Hz. 2: 60 Hz. 3: Auto.
*
* Return value: <0: Failure. =0/1/2: Success.
***********************************************************/
OPERATE_RET tkl_vi_isp_get_antiflicker(TKL_VI_CHN_E chn, TKL_ISP_FLICKER_TYPE_E *eAntiFlicker)

/***********************************************************
* Function: tkl_vi_isp_set_antiflicker
* Description: Set anti-flicker.
* Parameter: IN TKL_VI_CHN_E chn The device ID
         IN TKL_ISP_FLICKER_TYPE_E eAntiFlicker, 0: Disabled. 1: 50 Hz. 2: 60 Hz. 3: Auto.
*
* Return value: <0: Failure. =0: Success.
***********************************************************/
OPERATE_RET tkl_vi_isp_set_antiflicker(TKL_VI_CHN_E chn, TKL_ISP_FLICKER_TYPE_E eAntiFlicker)

Convert color images to black and white

/***********************************************************
* Function: tkl_vi_isp_set_color_to_gray
* Description: Convert color images to black and white, and vice versa.
* Parameter: IN TKL_VI_CHN_E chn The device ID
         IN int grey_flag, The current mode. 0: Full color in day mode. 1: Black and white in night mode. Other values are not supported.
*
* Return value: <0: Failure. =0: Success.
***********************************************************/
OPERATE_RET tkl_vi_isp_set_color_to_gray(TKL_VI_CHN_E chn, int grey_flag)

Calculate illuminance for day/night mode switching

/***********************************************************
* Function: tkl_vi_dnswitch_get_illumin
* Description: Get the calculated illuminance value for day/night switching.
* Parameter: IN TKL_VI_CHN_E chn The device ID
         IN TKL_ISP_DN_MODE_E irmode, The current day/night mode. 0: Auto mode.1: Day mode. 2: Night mode. Specifying it to auto will force the day mode. Auto is not supported.
         OUT INT32_T *illumin_result  The calculation result of the day/night switching algorithm is by default an integer. When there is a strong reflection in night vision, 1 is returned with a serial prompt.
* Return value: <0: Failure. =0: Success.
***********************************************************/
OPERATE_RET tkl_vi_dnswitch_get_illumin(TKL_VI_CHN_E chn, TKL_ISP_DN_MODE_E irmode, INT32_T *illumin_result)

Display ISP version number

/***********************************************************
* Function: tkl_vi_isp_show_fw_version
* Description: Display the ISP version number, which is output through the serial port every time the camera is powered on.
* Parameter: ISP_FW_VERSION, Macro definition, the ISP version number of the current camera.
         ISP_SENSOR_NAME, Macro definition, the name of the current camera's sensor.
*
* Return value: <0: Failure. =0: Success.
***********************************************************/
OPERATE_RET tkl_vi_isp_show_fw_version(PCHAR_T version, PCHAR_T name)

Enable tuning logs

/******************************************************************************
* Function: tkl_vi_isp_get_param
* Description: Get the parameters of various modules in ISP pipeline with the chip platform API for image analysis. Currently, only AE and AWB are supported.
* Parameter: IN TKL_VI_CHN_E chn The device ID
         OUT TKL_ISP_LOG_INFO_T *pstISPParam isp  Parameters to be displayed after logging is enabled.
* Return value:    <0: Failure. =0: Success.
******************************************************************************/
OPERATE_RET tkl_vi_isp_get_param(TKL_VI_CHN_E chn, TKL_ISP_LOG_INFO_T *pstISPParam)

Things to note

After you complete image tuning and compilation, be sure to check the result in different scenes.

You should comprehensively evaluate the tuning result because image parameters that fit a scene might not work well in a different one.

FAQs

Why can’t the IPC output images?

Troubleshooting steps:

1. Check if the I2C interface works fine.

   If I2C communication is normal, the IPC will print xxxxx Probed success,subdev:xxxxxx when power on. The sensor ID is returned by the function sensor_probe_id_func. If the sensor ID is not returned correctly, perform the steps below.

2. Check if you have written the correct sensor I2C address into the driver file according to the sensor datasheet.

3. Measure the I2C signal.

4. Check if the power-on sequence in the driver file matches the one specified on the sensor datasheet.

5. Check if the sensor ID in the ISP parameter configuration file is correct.

How many sets of configuration files are used for day and night mode?

For the Anyka platform, the parameters for day and night mode are saved in the configuration file isp.conf. The SubFileId displayed in the tuning tool is:

• 0: Day mode.
• 1: Night mode.

For Fullhan and Novatek platforms, the day mode and night mode each have their own separate configuration file.

Why is the image blurry and unclear?

1. Ensure that the sensor and lens of the IPC are clean.
2. Manually adjust the focus of the lens until the image is clear.
3. To further improve the quality, try increasing the sharpness and contrast or tuning the WDR and dehaze modules.

What causes a color shift to red or green?

1. Check if the lens has been replaced and if the IR cut filter works properly to identify the cause of red color shift.
2. If the lens has been replaced, perform an AWB calibration.
3. In case of green color shift, check if the black level is correct.