This research focuses on the whole image acquisition system, and proposes to integrate the compressive sensing (CS) based measurement domain image sensing and the corresponding measurement domain data compression. Comparing with the conventional solution, the power of the whole system is expected to be reduced by more 90%.
Title: CS-MCNet: A Video Compressive Sensing Reconstruction Network with Interpretable Motion Compensation
Abstract: In this paper, a deep neural network with interpretable motion compensation called CS-MCNet is proposed to realize high-quality and real-time decoding of video compressive sensing. Firstly, explicit multi-hypothesis motion compensation is applied in our network to extract correlation information of adjacent frames(as shown in Fig. 1), which improves the recover performance. And then, a residual module further narrows down the gap between reconstruction result and original signal. The overall architecture is interpretable by using algorithm unrolling, which brings the benefits of being able to transfer prior knowledge about the conventional algorithms. As a result, a PSNR of 22dB can be achieved at 64x compression ratio, which is about 4% to 9% better than state-of-the-art methods. In addition, due to the feed-forward architecture, the reconstruction can be processed by our network in real time and up to three orders of magnitude faster than traditional iterative methods.
Published: Asian Conference on Computer Vision 2020 (ACCV), Virtual Tokyo. Nov. 2020. [論文] [コード]
Title: Bi-directional intra prediction based measurement coding for compressive sensing images
Abstract: This work proposes a bi-directional intra prediction-based measurement coding algorithm for compressive sensing images. Compressive sensing is capable of reducing the size of the sparse signals, in which the high-dimensional signals are represented by the under-determined linear measurements. In order to explore the spatial redundancy in measurements, the corresponding pixel domain information extracted using the structure of measurement matrix. Firstly, the mono-directional prediction modes (i.e. horizontal mode and vertical mode), which refer to the nearest information of neighboring pixel blocks, areobtained by the structure of the measurement matrix. Secondly, we design bi-directional intra prediction modes (i.e. Diagonal +Horizontal, Diagonal + Vertical) base on the already obtained mono-directional prediction modes. Experimental results show that this work improves 0.01 - 0.02 dB PSNR improvement and the birate reductions of on average 19%, up to 36% compared to the state-of-the-art.
Published: 2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP), Sep. 2020. [paper]
【法政の研究ブランドvol.9（動画）】膨大な動画像データを処理する 次世代の技術（大学院理工学研究科 周 金佳 准教授）
Related publications
-
Rentao Wan, Jinjia Zhou, Bowen Huang, Hui Zeng, Yibo Fan, ""Measurement Coding Framework with Adjacent Pixels based Measurement Matrix for Compressively Sensed Images", IEEE International Conference on Acoustics, Speach and Signal Processing (ICASSP) Jun. 2021.
-
Bowen Huang, Jinjia Zhou, Xiao Yan, Ming'e Jing, Rentao Wan, Yibo Fan, "CS-MCNet: A Video Compressive Sensing Reconstruction Network with Interpretable Motion Compensation", Asian Conference on Computer Vision 2020 (ACCV), Virtual Tokyo. Nov. 2020. (acceptance rate 33%).
-
Thuy Thi Thu Tran, Jirayu Peetakul, Chi Do Kim Pham, Jinjia Zhou, "Bi-directional intra prediction based measurement coding for compressive sensing images", 2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP), DOI: DOI: 10.1109/MMSP48831.2020.9287074, Sep. 2020.
-
Jiayao Xu, Jirayu Peetakul, Muchen Li, Jinjia Zhou, "High-speed Compressed Sensing Reconstruction using Zigzag Ordering based Parallel Processing", International Conference on Image and Video Processing (ICIVP 2020), Oct. 2020.(Best presentation)
-
Jirayu Peetakul and Jinjia Zhou, "Temporal Redundancy Reduction in Compressive Video Sensing by using Moving Detection and Inter-Coding," 2020 Data Compression Conference (DCC), Snowbird, UT, USA, 2020, pp. 387-387
-
Jirayu Peetakul, Jinjia Zhou, Koishi Wada, "A Measurement Coding System for Block-based Compressive Sensing Images by Using Pixel-Domain Features", IEEE Data Compression Conference (DCC 2019), Snowbird, USA, March 2019.