2022-03-09 更新
DeltaCNN: End-to-End CNN Inference of Sparse Frame Differences in Videos
Authors:Mathias Parger, Chengcheng Tang, Christopher D. Twigg, Cem Keskin, Robert Wang, Markus Steinberger
Convolutional neural network inference on video data requires powerful hardware for real-time processing. Given the inherent coherence across consecutive frames, large parts of a video typically change little. By skipping identical image regions and truncating insignificant pixel updates, computational redundancy can in theory be reduced significantly. However, these theoretical savings have been difficult to translate into practice, as sparse updates hamper computational consistency and memory access coherence; which are key for efficiency on real hardware. With DeltaCNN, we present a sparse convolutional neural network framework that enables sparse frame-by-frame updates to accelerate video inference in practice. We provide sparse implementations for all typical CNN layers and propagate sparse feature updates end-to-end - without accumulating errors over time. DeltaCNN is applicable to all convolutional neural networks without retraining. To the best of our knowledge, we are the first to significantly outperform the dense reference, cuDNN, in practical settings, achieving speedups of up to 7x with only marginal differences in accuracy.
PDF CVPR 2022
论文截图
Seeing BDD100K in dark: Single-Stage Night-time Object Detection via Continual Fourier Contrastive Learning
Authors:Ujjal Kr Dutta
In this paper, we study the lesser explored avenue of object detection at night-time. An object detector trained on abundant labeled daytime images often fails to perform well on night images, due to domain gap. As collecting more labeled data from night-time is expensive, unpaired generative image translation techniques seek to synthesize night-time images. However, unrealistic artifacts often arise on the synthetic images. Illuminating night-time inference images also does not work well in practice, as shown in our paper. To address these issues, we suggest a novel technique for enhancing the object detector via Contrastive Learning, which tries to group together embeddings of similar images. To provide anchor-positive image pairs for Contrastive Learning, we leverage Fourier Transformation, which is naturally good at preserving the semantics of an image. For practical benefits in real-time applications, we choose the recently proposed YOLOF single-stage detector, which provides a simple and clean encoder-decoder segregation of the detector network. However, merely trying to teach the encoder to perform well on the auxiliary Contrastive Learning task may lead to catastrophic forgetting of the knowledge essential for object detection. Hence, we train the encoder in a Continual Learning fashion. Our novel method by an elegant training framework achieves state-of-the-art performance on the large scale BDD100K dataset, in an uniform setting, chosen, to the best of our knowledge, for the first time.
PDF
论文截图