讲座时间：2024年7月18日（四）10:00-11:00

讲座地点：东校区财科馆316会议室

讲座题目：Simple Fixes that Accommodate Switching Costs in Multi-armed Bandits.

ABSTRACT:When switching costs are added to the multi-armed bandit (MAB) problem where the arms' random reward distributions are previously unknown, usually quite different techniques than those for pure MAB are required. We find that two simple fixes on the existing upper-confidence-bound (UCB) policy can work well for MAB with switching costs (MAB-SC). Two cases should be distinguished. One is with {\em positive-gap} ambiguity where the performance gap between the leading and lagging arms is known to be at least some $\delta>0$. For this, our fix is to erect barriers that discourage frivolous arm switchings. The other is with {\em zero-gap} ambiguity where absolutely nothing is known. We remedy this by forcing the same arms to be pulled in increasingly prolonged intervals. As usual, the effectivenesses of our fixes are measured by the worst average regrets over long time horizons $T$. When the barriers are fixed at $\delta/2$, we can accomplish a $\ln(T)$-sized regret bound for the positive-gap case. When intervals are such that $n$ of them occupy $n^2$ periods, we can achieve the best possible $T^{1/2}$-sized regret bound for the zero-gap case. Other than UCB, these fixes can be applied to a learning while doing (LWD) heuristic to reach satisfactory results as well. While not yet with the best theoretical guarantees, the LWD-based policies have empirically outperformed those based on UCB and other known alternatives. Numerically competitive policies still include ones resulting from interval-based fixes on Thompson sampling (TS).

主讲人简介：Dr. Jian Yang obtained his Ph.D. in Management Science from the University of Texas at Austin.After working for the Department of Mechanical and Industrial Engineering at New Jersey Institute of Technology, he is now a professor at the Department of Management Science and Information Systems, Rutgers Business School, Rutgers University.Dr. Yang’s research interests are in combinatorial optimization, logistics, production and inventory control, dynamic pricing, and game theory.At the present he is particularly interested in the role played by risk and ambiguity in dynamic inventory-price control and game-theoretical settings.