标签归档 南京外卖工作室微信群

Writing | Huang Longqian

Recently, from the School of Optoelectronics, Zhejiang University　Hao Xiang　The team of researchers is "Spectral Imaging with Deep Learning"The topic is　Lighting: Science & Applications　Published a review paper, sorted out the spectral imaging technology based on deep learning, explained the principle and research summary of its various technical routes, and organized the current spectral imaging dataset, outlining the possible future trends and challenges.

First, spectral images

The image we usually see in the display screen is an RGB three-color image, which has three channels of red, green and blue, that is, the color of each pixel is formed by superimposing the three colors of red, green and blue. If the three-color image is expanded to ten colors, hundred colors, and thousand colors, then our control and rendering of the image color will be extremely fine. An image with so many bands is a spectral image (term explanation >): it has more than three bands, which can be thirty or even hundreds.

Figure 1 Schematic diagram of a spectral image composed of 9 bands

The spectral information of a substance can fully reflect its internal physical structure and chemical composition. When there are enough wavelength bands in the image, each part is a spectral curve – reflecting the structural composition of the place. Therefore, the spectral image not only reflects the spatial characteristics of the imaging target, but also contains its structure and composition characteristics. Therefore, in addition to fine image rendering, the key application of spectral images is material composition analysis, which is widely used in remote sensing, medical detection, food detection and other fields.

Second, spectral imaging

Spectral imaging is a combination of imaging technology and spectral detection technology, and the main technical goal is to obtain spectral images. Traditional spectral imaging technology is scanning(Including point scanning, line scanning, band scanning)By constantly moving the spectrometer to obtain the spectrum of each point, a two-dimensional image is finally obtained. This scanning spectral imaging system is bulky and slow, which has been criticized.

Figure 2 Scanning spectral imaging (from left to right, point scan, first scan, and band scan)

With the development of computational optics, computational spectral imaging has become a new spectral imaging method. The method uses some coding to make the hidden spectral information enter the detector in a certain way, and finally uses the optimization algorithm to reconstruct the spectral image. The traditional computational spectral imaging method is iterative. Although it has effectively reduced the size of the system compared with the scanning spectral imaging, it has a great computational burden, often requires several minutes or even hours of iteration, and the reconstruction accuracy is relatively average.

The huge burden of data computing prompts people to think and look for more effective spectral reconstruction algorithms. In recent years, deep learning has developed rapidly, and the powerful pattern recognition and feature extraction capabilities of deep neural networks after good training have been favored by spectral imaging researchers. Deep learning methods have become the object of exploration, and deep learning has gradually empowered spectral imaging.

Methods of deep learning

The process of computational spectral imaging coding sampling system design and spectral image reconstruction, deep learning technology can be applied in every aspect of the process. How to effectively classify these different techniques is a key issue. The review proposes to classify according to the properties of light, that is, the computational spectral imaging system is divided into amplitude coding, phase coding and wavelength coding according to different coding methods.

(1) Amplitude-encoded spectral imaging

Figure 3: The spectral coding process in a coding aperture system

Amplitude-coded spectral imaging is achieved through a coded aperture system(CASSI)Conducted, using coding aperture(Amplitude mask)Amplitude coded spectral imaging based on deep learning replaces the iterative algorithm of compressed sensing recovery with deep neural networks, and performs efficient spectral reconstruction through codec co-optimization, iterative expansion neural networks, and untrained networks.

(2) Phase-encoded spectral imaging

Figure 4: A phase-encoded spectral imaging system

Phase-encoded spectral imaging through diffractive optics(Diffraction Optical Elements, DOE)Phase coding is achieved by designing the two-dimensional height profile of the DOE to achieve specific phase delays at different positions. Phase coding affects different spectral components through Fresnel diffraction, and then by modeling the diffraction process, the original spectral image can be reconstructed by the algorithm. Due to the complexity of diffraction calculation after phase coding, traditional iterative algorithms are difficult to achieve effective recovery of spectral images. This problem has been solved after the emergence of deep learning. At present, the spectral recovery of phase coding is mainly carried out through deep neural networks. Compared with amplitude coding, phase-encoded spectral imaging has the advantages of small light energy loss and compact system.

(3) Wavelength-encoded spectral imaging

Wavelength-encoded spectral imaging directly encodes the image in the spectral dimension and can be performed through optical filters. RGB images can be regarded as a kind of spectral coding. At present, the mainstream wavelength coding methods use existing RGB or design optical filters, and the encoded spectral reconstruction mostly relies on deep learning technology.

Direct spectral reconstruction based on RGB images is a very hot direction. With the NTIRE 2018 and NTIRE 2020 spectral reconstruction competitions, many deep learning technology teams participated, greatly expanding the existing deep learning spectral restoration techniques. The researchers analyzed the RGB spectral reconstruction and filter design spectral reconstruction involved in deep learning, and divided the reconstruction methods into point reconstruction and block reconstruction, so as to introduce each reconstruction method.

Figure 5 Codec Cooperative Design Spectral Imaging Technology [
1] [
[2] (Source: Advanced Theory and Simulations)

Wavelength coding based on custom optical filters is an emerging spectral imaging technology in recent years. By designing wide-spectrum filters, greater coding freedom can be obtained than RGB filters. Combined with effective deep learning technology, compact, fast and accurate spectral recovery can be achieved.

IV. Future Outlook

Based on current deep learning techniques and spectral imaging techniques, the researchers have made some suggestions for the future development of deep learning-based spectral imaging. These include the large-scale and normalization of datasets, and the use of large neural networks(For example Transformers)Introduce additional information such as object categories, solve the problem of layer disappearance in the process of codec collaborative design, and use neural networks architecture search technology.

Paper information

Huang, L., Luo, R., Liu, X. et al. Spectral imaging with deep learning. Light Sci Appl 11, 61 (2022).　

https://doi.org/10.1038/s41377-022-00743-6

references

[1] Song H, Ma Y, Han Y, et al. Deep ? Learned Broadband Encoding Stochastic Filters for Computational Spectroscopic Instruments [J]. Advanced Theory and Simulations, 2021, 4 (3): 2000299.

[2] Zhang W, Song H, He X, et al. Deeply learned broadband encoding stochastic hyperspectral imaging [J]. Light: Science & Applications, 2021, 10 (1): 1-7.

Contributions from the research group are welcome – press release

Article reprint/business cooperation/research group submission, WeChat:447882024

Let’s read one article a day!Join >Light reading club

Source: China Daily China Watch Think Tank

Introduction: From ancient times to the present, Chinese civilization has learned from others and nurtured excellent values that transcend history and keep pace with the times. Chinese culture pays attention to ideological integration, self-governance, inheritance and reference, seeking truth from facts, beauty and beauty, and integrates these elements into the way of governing the country. The rise of China will not pose a threat to other countries. On the contrary, it will promote broader security, promote greater development and help all mankind create a better future.

Author: Joseph Gregory Mahoney

(Josef Gregory Mahoney)

Executive Director, International Center for Political Studies, East China Normal University

Deputy Editor-in-Chief of China Journal of Political Science.

No civilization or culture in the world can draw nutrition from its own history completely; None of them can make progress completely according to their own ideas; No one can fully release the creative potential of the whole society and achieve complete social justice.

Marxist view of history holds that progress is achieved by actively solving contradictions. Resolving conflicts can produce the result of promoting progress, and to achieve social progress, we must first promote human liberation and social justice. However, social progress does not mean that the past history can be erased or forgotten; On the contrary, there are some ancient values that can transcend history. In China, the concepts of "harmony" and "great harmony" are such values, which are essentially interlinked with the concepts of modern socialism and communism. Driven by the concept of good governance, these traditional Chinese civilizations and cultural values are also advancing with the times.

One of the ways of the continuous development of these values is mutual integration, which has long been a feature of China’s traditional thought and Marxism’s China-learning from each other’s strengths and adapting to changes. To some extent, this explains why China’s Marxism not only emphasizes Confucian concepts such as "well-off", but also advocates the Taoist principle of harmonious coexistence between man and nature, and even opens the door to all outstanding achievements of western civilization and other civilizations in the past and present.

Image source: China Daily

From this, we can sum up five key experiences related to governance in the history of China.

The first lesson: ideological integration

China is the first country in the world to create a similar modern national model, which started in the Qin Dynasty and accelerated to maturity in the Han Dynasty. During the Han Dynasty, China integrated the legalist thoughts of the Qin Dynasty with Confucianism, and established a professional national system. The most important thing was to attach importance to good governance and the rule of law. These achievements predate other civilizations and countries by hundreds or even thousands of years. In short, whether as a civilization or as a national system, China has created a national culture to protect itself from disputes over the excessive expansion of the empire.

The second experience: self-governance

Mind your own business, or you risk being ruled by others. In this regard, it is an eternal theme to establish a self-governance system, constantly promote self-improvement, and even carry out radical self-revolution when necessary, but it is often a difficult problem. It is unwise to try to rule others, it will only erode your self-governance ability, thus endangering your own sovereignty and security.

Indeed, throughout the history of imperialism, have we not seen such lessons again and again? Moreover, we can draw another conclusion from the history of imperialist development: aggression and conquest of others often rely on technological innovation, such as a new weapon or new tactics, rather than China’s emphasis on governance innovation since ancient times. For example, Alexander the Great, the ancient Macedonian king, relied on his father Philip II’s powerful military power to March eastward to India, but he could not rule and control all the territories he conquered.

At a more fundamental level, why are imperialist countries, whether it is the ancient Roman Empire, modern Britain or modern America, ultimately doomed to decline? The answer is: on the one hand, they have never mastered the art of self-governance-they can’t even manage themselves well, let alone others. On the other hand, their motivation to control, exploit and even plunder others is often to make up for the defects of domestic self-governance and externalize their internal failures to others. For example, the concept of protecting private property rights is deified. However, in the "zero-sum" game environment, it is this "golden rule" that leads to the fundamental inequality between people becoming the norm.

Image source: Xinhua News Agency

The third experience: inheritance and reference.

We must learn from our own history and culture, otherwise we may lose ourselves completely. As Okakura Tenjin, a Japanese art critic and author of The Book of Tea, warned, since the Meiji Restoration in the late 19th century, Japan has hastily embarked on the road of total westernization. At the same time of comprehensive modernization and transformation, it has almost completely abandoned its traditional values and culture, leaving only a superficial thing.

Countries should base their own development on their own culture and values, otherwise they may, like Japan, first repeat the mistakes of western imperialist countries’ history of expansion and aggression, and then, when a new era comes, they will be unable to carry out reforms to meet new needs and fall into the middle-income trap and security trap again.

In addition, many problems in today’s era, such as climate change, pandemic of new infectious diseases and other increasingly severe survival risks, are caused by some wrong values and behaviors. If they are not screened, some countries may easily get caught up in them. Or they may wander around for decades in the post-colonial era, unable to express their true identity or build a truly effective governance system for themselves, because they don’t really know who they are and where they come from, and they are still vulnerable to more powerful external forces, including former colonists. They will cultivate a sense of insecurity in these countries and use this sense of insecurity to achieve the goal of continuing to control them.

At the same time, countries must learn from others and share what they have learned. China learned a lot from Marxism, Leninism and the so-called Fordism, and benefited a lot from the Enlightenment, the industrial revolution and even the development history of the global capitalist economy. But the most important thing is to treat these past experiences and lessons with a rational and dialectical attitude, put them in an appropriate position and background, and take the essence and discard the dross.

Image source: China Daily

Article 4 Experience: Seeking truth from facts

No matter what role culture and values can play in an ideal state, no matter what lessons we learn from the past, the Chinese governance model has a law that transcends history: everything must start from the material foundation. Material determines consciousness and foundation determines architecture, which is an important principle of Marxism and the first lesson I learned when I came to work in China in 2010. What you see, hear, say and do should be consistent with the actual situation. In this regard, the development and progress of China’s governance model, both domestically and internationally, both in the past and now, have confirmed the correctness of this sentence.

Article 5 Experience: Beauty and Beauty Together

A rising and well-governed China is not a threat to other countries, but the opposite. The stronger and more efficient China becomes, the more it can reflect on its own historical experience, the more it can understand its own limitations, the more it can embody the deep-rooted cultural value of good governance, and the more it can treat other countries cautiously and support their progress as much as possible, because only in this way can it ensure wider security and greater development and create a common future for all mankind. In the past ten years, we have seen such a trend in the rapid development of the "the belt and road initiative" initiative, which is the only logical way forward for a country like China that emphasizes good governance.

Image source: China Daily

At 21: 00 pm on May 21st, Beijing time, in the 37th round of the Premier League, Brighton, who was competing for six, played against the relegated Southampton at home.

In the first half, Ferguson scored twice, mcallister and Mitsuki Kaoru sent assists, Mitsuki Kaoru hit the post with a single knife, and Brighton led 2-0 at halftime; In the second half, prowse assisted El Yunusi to pull back a city, and colville assisted Gross to seal the victory.

In the end, Brighton beat Southampton 3-1. Brighton led Aston Villa by 3 points and scored 16 goals on goal difference. In fact, it has locked in the top six of the Premier League, and the first European war in the history of Seagulls is coming.

Strength crushing! Brighton 3-1 Southampton, locked the top six in the Premier League 2 rounds ahead of schedule, and the first European game in team history came!

Brighton, who is unkind, is a strong team in the Premier League this season. Brighton has a great advantage against Southampton, which has been relegated early at home. If it weren’t for his poor shooting skills and bad luck, Brighton would have beaten the saints in the first half. Ferguson scored twice, and McCarthy Park Jung Su and Sam Kun Xun sent assists. Southampton was assisted by prowse and El Yunusi pulled back a city, and Gross scored a goal to seal the victory. In the end, Brighton beat Southampton 3-1 at home.

Brighton led Villa by 3 points in one game, with 20 goal difference and 4 goal difference, which means that Brighton has locked in the top six of the Premier League even if it lost all the last two rounds, as long as it is not a tragedy. At the end of the Premier League, Brighton played against Villa. In fact, Villa had to compete with Tottenham for the seventh place in the UEFA. Not surprisingly, Brighton will have the first European game in the team’s history.This season, the three flying beasts in the Premier League took off. First, the magpies competed for four, second, the seagulls competed for six, and third, the bees ranked ninth..

The tendency of crushing,Brighton’s possession rate was 62.3%, and there were 26 shots, 8 of which were right. Southampton made five shots, one of which was right..