Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving
Jinlong Li1*,  Baolu Li1*,  Zhengzhong Tu2,  Xinyu Liu1,  Qing Guo3,  Felix Juefei-Xu4,  Runsheng Xu5 Hongkai Yu1
1 Cleveland State University, 2 University of Texas at Austin, 3 A*STAR, 4 New York University, 5 UCLA
* Equal contribution
CVPR 2024  [Paper]  [Code]
Overview
Without realistic paired day-night images, synthesizing dark driving images with vehicle lights is quite difficult, limiting the research in this field. This work introduces LightDiff, a domain-tailored framework designed to enhance the low-light image quality for autonomous driving applications, mitigating the challenges faced by vision-centric perception systems. By leveraging a dynamic data degradation process, a multi-condition adapter for diverse input modalities, and perception-specific score guided reward modeling using reinforcement learning, LightDiff significantly enhances the image quality and 3D vehicle detection in nighttime on the nuScenes dataset. This innovation not only eliminates the need for extensive nighttime data but also ensures semantic integrity in image transformation, demonstrating its potential to enhance safety and reliability in autonomous driving scenarios.
The architecture of LightDiff. During the training stage, a Training Data Generation pipeline enables the acquisition of triple-modality data without any human-collected paired data. Our LightDiff employs a Multi-Condition Adapter to dynamically weight multiple conditions, coupled with LiDAR and Distribution Reward Modeling (LDRM), allowing for perception-oriented control.
Qualitative Results
Visual comparison on the example nighttime images in the nuScenes validation set.
Qualitative Results
Quantitative comparison of image quality on the nuScenes nighttime validation set
Visualization of 3D detection results
We employ BEVDepth as the 3D detector and visualize both the front view of camera and the Bird's-Eye-View.
BibTeX
  @inproceedings{li2024light,
    title={Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving},
    author={Li, Jinlong and Li, Baolu and Tu, Zhengzhong and Liu, Xinyu and Guo, Qing and Juefei-Xu, Felix and Xu, Runsheng and Yu, Hongkai},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    pages={15205--15215},
    year={2024}
  }
Related Work
Lvmin Zhang, Anyi Rao, Maneesh Agrawala. Adding Conditional Control to Text-to-Image Diffusion Models. ICCV 2023.
Comment: Builds a addition encoder to add spatial conditioning controls to T2I diffusion models.
Ziteng Cui, Guo-Jun Qi, Lin Gu, Shaodi You, Zenghui Zhang, Tatsuya Harada. Multitask AET with orthogonal tangent regularity for dark object detection. ICCV 2021.
Comment: Deploys Low-Illumination Degrading Transformations for Dark Object Detection.