Energy Efficient Reinforcement Learning Accelerator for Cloud-Terminal IoT Applications

Speaker: Prof. Yongpan Liu
Date: 2018-03-27 (Tue) 14:00 - 15:30
Location: Meeting Room 124 at CITI
Host: Pi-Cheng Hsiu


Reinforcement learning (RL) has attracted abroad interests in artificial intelligent regions. Since RL is mostly based on Deep-Q- Learning (DQN) algorithms where the deep neural network is typically composed of convolutional neural network (CNN) and fully-connected layer (FC), implementing RL applications on self-powered IoT devices is challenged by limited power supply, serious computation, and memory overhead. Moreover, FC layers should be fine- tuned and re-trained to make the network more precise for mobile applications and adaptable for different users’ tastes. Therefore, an energy efficient online training accelerator is in demand. This paper proposes a kernel-optimized CNN architecture and an online training FC architecture. A 3x3 kernel-optimized architecture is implemented as a proof-of- concept, the energy efficiency of which is 4.01x better than the state-of- the-art CNN processor. The proposed online training architecture further reduces parameters update operations by five orders of magnitude for IoT applications. Finally, we present a framework to collaboratively train a neural network on cloud and terminal platforms.


Dr. Yongpan Liu is an associate professor in Tsinghua University. He has been a visiting scholar at Penn State University and City University of Hong Kong. He is a key member of Tsinghua-Rohm Research Center and Research Center of Future ICs. His research interests include low power design, emerging circuits and systems and design automation. He published over 100 papers and designed several sensor chips, including the first nonvolatile processor (THU1010N). He is an IEEE senior member and served on TPC of (DAC, ASP-DAC, ISLPED, A-SSCC, ICCD, VLSI-D, VLSI-DAT, etc) and has received Design Contest Awards from (ISLPED2012, ISLPED2013), IEEE Micro Top Pick 2016 and best paper awards of HPCA2015 and ASP-DAC2017 and Under 40 Innovation Award of DAC 2017. He served as the publicity co-chair of ICCD15,16 and the exhibition chair of A-SSCC15 and one of the founders of Asia Workshop on Smart Sensor System