Refereed International Conference Publications

Real-Time Object Detection System with Multi-Path Neural Networks [abstract] (IEEE Xplore, PyTorch, Detectron, PDF)
Seonyeong Heo, Sungjun Cho, Youngsok Kim, and Hanjun Kim
Proceedings of the IEEE Real-Time And Embedded Technology And Applications Symposium (RTAS), April 2020.

Thanks to the recent advances in Deep Neural Networks (DNNs), DNN-based object detection systems becomes highly accurate and widely used in real-time environments such as autonomous vehicles, drones and security robots. Although the systems should detect objects within a certain time limit that can vary depending on their execution environments such as vehicle speeds, existing systems blindly execute the entire long- latency DNNs without reflecting the time-varying time limits, and thus they cannot guarantee real-time constraints. This work proposes a novel real-time object detection system that employs multi-path neural networks based on a new worst-case execution time (WCET) model for DNNs on a GPU. This work designs the WCET model for a single-layer of DNNs analyzing processor and memory contention on GPUs, and extends the WCET model to the end-to-end networks. This work also designs the multi- path networks with three new operators such as skip, switch, and dynamic generate proposals that dynamically change their execution paths and the number of target objects. Finally, this work proposes a path decision model that chooses the optimal execution path at run-time reflecting dynamically changing en- vironments and time constraints. Our detailed evaluation using widely-used driving datasets shows that the proposed real-time object detection system performs as good as a baseline object detection system without violating the time-varying time limits. Moreover, the WCET model predicts the worst-case execution latency of convolutional and group normalization layers with only 28% and 64% errors on average, respectively.