Generative Artificial Intelligence (GenAI) has become everywhere nowadays. It has drawn even more attention due to the rapid development of Large Language Models (LLMs). However, most of the GenAI models have huge model sizes and are generally computationally heavy. Usually, a powerful centralized computation infrastructure, i.e., a cloud server, is required to process all user queries. As a result, users may experience high latency when using such GenAI systems due to the high volume of queries and the transmission delay. In addition, user privacy, power consumption, factuality, and the scalability of the systems are also concerns for current GenAI models. Therefore, it's worthwhile to design a distributed GenAI system under the edge-cloud computing paradigm that can address these concerns efficiently. In this talk, we first summarize the technical challenges of current GenAI models. Then, we identify some design considerations for GenAI systems under the edge-cloud computing paradigm. Finally, we demonstrate how to integrate those design considerations into an overall distributed system using several applications under different user scenarios.

Yun-Cheng (Joe) Wang obtained his Ph.D. in Electrical and Computer Engineering at the University of Southern California (USC) in December 2023, supervised by Prof. C.-C. Jay Kuo. Prior to joining USC, he received his B.S. in Electrical Engineering from National Taiwan University in 2018. His research focuses on developing lightweight, efficient, and transparent machine learning (ML) models with specific interests in diverse knowledge graph applications, such as KG completion, KG representation learning, and interpretable and efficient reasoning on KGs.