Large Language Models

Are you eager to explore the latest breakthrough in artificial intelligence, particularly the domain of large language models (LLMs)? This book is your go-to guide for understanding the core foundations and advanced techniques of LLMs.

This comprehensive resource offers a complete understanding of LLM developments, from pre-training to fine-tuning. It elaborates on the classic Transformer architecture, its adaptations for LLMs, and the full training process, including data collection, cleaning, and preparation. From the book, readers can also learn how to fine-tune LLMs to follow human instructions and align with human values and intentions, ensuring safer and more ethical AI behavior. Furthermore, it helps readers discover effective prompting strategies, such as in-context learning and chain-of-thought, to enhance LLM capabilities and solve complex tasks.

Suitable for both beginners and experienced professionals, this book is an invaluable resource for navigating the dynamic field of LLMs, offering a concise yet comprehensive exploration of the subject.

The translation was originally done using artificial intelligence. Subsequently, a comprehensive human revision was done to ensure content accuracy and coherence throughout the book.

Wayne Xin Zhao is a professor at Gaoling School of Artificial Intelligence, Renmin University of China. His research areas include natural language processing, information retrieval, and data mining, with a particular focus on large language models. Xin graduated from Harbin Institute of Technology in 2008 and earned his PhD from Peking University in 2014.  He has published more than 200 technical papers in top international conferences and journals, accumulating more than 29,000 citations according to Google Scholar. His contributions have been honored with awards, such as the ECIR 2021 Test-of-time award and EACL 2024 Evaluation and Model Insight Award. Xin has also regularly served as the area chair or senior program committee member for prominent conferences. He is the lead author of the survey paper "A survey of large language models," which provides a comprehensive overview of the field.

Kun Zhou obtained his Ph.D degree at School of Information, Renmin University of China in 2024. His research interests encompass natural language processing and multimodal systems, with focuses on large language models and their applications in complex scenarios. Kun has published more than 40 papers at top conferences and journals, gathering more than 9,000 citations according to Google Scholar. Kun has been awarded by MSRA Fellowship, Baidu Scholarship, Bytedance Scholarship, Baosteel Scholarship, and EACL 2024 Evaluation and Model Insight Award.

Junyi Li is a postdoctoral researcher at School of Computing, National University of Singapore, Singapore. His research interests center around natural language processing and multi-modal systems, with an emphasis on large language models and their applications. Junyi received his PhD degree from Renmin University of China, supervised by Prof. Xin Zhao and a second PhD degree from Université de Montréal, advised by Prof. Jian-Yun Nie. He has published several technical papers at top international conferences and journals including ACL, SIGIR, EMNLP, and NAACL, accumulating more than 6,500 citations according to Google Scholar. Junyi has been awarded National Scholarship at 2019 and 2021 and 2024 Outstanding Graduates. Junyi has also served as the program committee member for several prominent conferences and journals, including ACL, EMNLP, AAAI, and ACM Computing Survey.

Tianyi Tang is a senior algorithm engineer at the Qwen Team, Alibaba Group. His research interests include natural language processing and large language models. He received both his M.E. and B.E. degrees from Renmin University of China, under the supervision of Prof. Wayne Xin Zhao. Tianyi has authored over 20 research papers in top journals and conferences such as ACM Computing Surveys, ACL, EMNLP, and NAACL, amassing more than 6,900 citations according to Google Scholar. He leads the LLMBox project, a comprehensive code library that provides researchers with a convenient and effective toolkit for training and utilizing large language models. Additionally, he has achieved four silver medals in ACM-ICPC contests.

Ji-Rong Wen is a full professor, and Executive Dean of the Gaoling School of Artificial Intelligence at Renmin University of China. With extensive experience in big data and AI, he has an impressive publication record in renowned international conferences and journals, amassing than 41,000 citations. Prof. Wen served as the PC Chair of SIGIR 2020 and was the Associate Editor of ACM TOIS and IEEE TKDE. He spent 14 years at Microsoft Research Asia (MSRA), where he was a Senior Researcher and Group Manager of the Web Search and Mining Group. In 2013, he joined Renmin University of China to lead the big data and AI research, especially interdisciplinary research between AI and social sciences & humanities. He was elected as a National Distinguished Expert in 2013 and Beijing’s Distinguished Young Scientist in 2018. Prof. Wen also holds the position of Chief Scientist at the Beijing Academy of Artificial Intelligence.