题目：LOLA: LLM-Assisted Online Learning Algorithm for Content Experiments

时间：2024年7月8日 10:00-11:00

地点：suncitygroup太阳新城官网 F210会议室

邀请人：夏唐斌 长聘副教授（工业工程与管理系）

Biography

Zikun Ye is an Assistant Professor of Marketing at the Michael G. Foster School of Business, University of Washington. He received his Ph.D. in Operations Research from the University of Illinois Urbana Champaign in 2023 and B.S. in Industrial Engineering from Shanghai Jiao Tong University in 2018. His research focuses on data-driven causal inference, field experiments, machine learning, and optimization methodologies to evaluate and optimize the strategies in the contexts of digital platforms and marketplaces, especially their recommendation and advertising.

Abstract

This lecture will introduce the LLM-Assisted Online Learning Algorithm (LOLA), a novel framework that integrates Large Language Models (LLMs) with adaptive experimentation to optimize content delivery. Leveraging a large-scale dataset from Upworthy, we first investigate three broad pure-LLM approaches: prompt-based methods, embedding-based classification models, and fine-tuned open-source LLMs. We then introduce LOLA, which combines the best pure-LLM approach with the Upper Confidence Bound algorithm to adaptively allocate traffic and maximize clicks. Our numerical experiments on Upworthy data show that LOLA outperforms the standard A/B testing method (the current status quo at Upworthy), pure bandit algorithms, and pure-LLM approaches, particularly in scenarios with limited experimental traffic or numerous arms. Our approach is both scalable and broadly applicable to content experiments across a variety of digital settings where firms seek to optimize user engagement, including digital advertising and social media recommendations.