The most eye-catching "Red Maple Original Color Camera" on the latest Huawei Mate70 phone is the product of spectral technology - a 15-megapixel multi-spectral camera that improves color accuracy by 120% compared to the Mate 60 Pro+, achieving a qualitative leap in image color.

According to Huawei's official introduction, the Red Maple Original Color camera can accurately capture the environmental spectrum, providing more authentic color input for the main camera, ultra-wide angle, telephoto macro cameras, so as to take "what you see is what you get" photos and videos.

All major smartphone manufacturers have adopted multi-spectral color temperature sensors in their flagship models.

In 2017, the iPhone X was equipped with a 6-channel multicolor sensor developed byams for it, including ultraviolet, red, green, blue, and two near-infrared lights.

In 2018, the P20, launched by Huawei, was the first to feature a color temperature sensor provided byams, which is used to achieve accurate color and ambient light sensing under different lighting conditions, allowing for more precise white balance adjustment during photography.

In 2021, Huawei launched the P50 Pocket, the first to feature a super-spectral super-imaging unit, with a spectral hardware that consists of a 10-channel multi-spectral sensor, flash, 32MP super-spectral camera, and a super-spectral flash.

In 2022, with the iPhone 14 series, Apple first equipped it with a dual ambient light sensor, while improving brightness adjustment and rear camera exposure.

In 2022, OPPO Find X5 Pro was the first in the industry to feature a 13-channel spectrometer sensor, bringing precise color reproduction to the camera system, while also restoring more authentic environmental light information and improving the white balance effect.

In 2022, the vivo X90 adopts its self-developed VCS仿生光谱 technology, which continuously improves the color filter to make the sensor's raw information closer to the human eye, achieving better noise performance and color reproduction, and optimizes the light signal in advance.

View the interpretation of Smart Technology in the horizon

Multispectral technology fully into

The era of consumer electronics

Application prospects

Optical image acquisition and image display are symbiotic relationships. The performance of image sensors and display devices directly affects the user's perception of the real environment. How to match the color differences between acquisition and display to achieve the real "what you see is what you get" has always been an important issue in this field.

Chen Guanghao, technical director of the ViewAR platform, said:

In recent years, with the innovation of technology and process, many cutting-edge optoelectronic theories have been rapidly applied to the display end, greatly improving the display brightness, resolution, color reproduction, and refresh rate, but at the same time, it has also put forward higher requirements for the color reproduction performance of sensors and the post-color calibration process.

The color of the acquisition process is affected by the sensitivity of the sensor, the spectral response range, the consistency of processing, and so on. The current widely used RGB wide-spectral Bayer filter array cannot provide enough information to assist post-processing color processing. Not to mention that the human eye's spectral response changes with the change of environmental illumination.

The best way to solve these problems perfectly

It is multi-channel spectral perception.

Multispectral imaging technology (less than 10 channels) is a sub-field of hyperspectral imaging (100-1000 channels) technology. When using multispectral, it is necessary to have a clear understanding of the target application scenario and select a limited number of spectral channels accordingly. The detection results of multispectral are usually used to achieve the function of feature recognition, rather than accurate spectral reconstruction. It has been widely used in agriculture, remote sensing, remote sensing, environmental protection and other fields.

At present, due to the constraints of cost and application scenarios, the multi-spectral technology搭载ed on the consumer electronics side is still relatively simple, and the target application scenarios are focused on environmental light spectrum sensing to assist the color reproduction algorithm to work normally. Single-channel multi-spectral components are widely used in the color adjustment of display devices.

These changes are subtle for consumers, but revolutionary for the industry. 

Consumers, after experiencing more realistic image technology, will quickly abandon products using old technology and are willing to pay a higher cost for new technology, driving the industry to rapidly transform.

Such examples are everywhere, such as the high-resolution Retina Display and Haptic Engine that Apple has pioneered on its own devices and quickly popularized on all kinds of electronic products.

Dr. Chen Guanghao: "The widespread use of multi-spectral imaging has had a profound impact on the development of artificial intelligence."

The leap in AI development and the proliferation and improvement of image capture devices are closely related, thanks to the endless stream of real-world images generated by digital devices all over the world.

Data is the fuel of the current big data and artificial intelligence development, and high-quality data brings higher accuracy of machine learning.

At present, deep learning models based on monochrome or RGB image data are encountering bottlenecks in development, and for scenarios with high real-time requirements (such as automotive autonomous driving, robots, drones, augmented reality, etc.), engineers need to compromise between the volume and accuracy of neural networks.

多光谱成像则另辟蹊径。由于光谱和物质有密切的关系，将物体少量特征性的光谱信息加入模型中，反而降低了模型对空间维度的要求，最终实现更快的推理速度和更高的准确度，降低计算端的功耗和成本。

多光谱技术的应用不限于色彩校准。在消费电子端，多光谱成像还可以用于开发更安全的安全验证机制、更丰富的健康监测应用、更新奇的交互方式等。

多种应用场景的集成需要更高的通道数，不可避免将多光谱技术逐渐推向高光谱成像技术。

如何实现便携、低成本的高光谱成像，一直是领域里的重大挑战。