Large language models (LLMs) have led to ground-breaking improvements in the capabilities of generative AI (Gen-AI) applications, leading to their increased adoption, which in turn is leading to increasing volumes of user requests at LLM inference deployments. The existing common implementations of LLM inference engines perform a new prefill every time there is a prompt departure. We analytically model the inference system for a fixed batch size with large rate of prompt arrivals and scheduling prefills after a fixed number of prompt departures. We characterize the throughput of the system as number of prompts departing per unit time for different thresholds. We observe that to maximize throughput, there exists an optimal threshold on the number of prompt departures. We verify this observation with vLLM experiments, and compare the optimal threshold predicted theoretically to the experimentally observed ones.

Parimal Parag is currently an associate professor in department of electrical communication engineering at Indian Institute of Science at Bangalore. He is also the convenor of the centre for networked intelligence, a faculty participant at Robert Bosch centre for cyber-physical systems, and a member of the applied probability research group. He was working as senior systems engineer in R&D at ASSIA Inc. from October 2011 to November 2014. He received his B. Tech. and M. Tech. degrees from Indian Institute of Technology Madras in fall 2004; and the PhD degree from Texas A&M University in fall 2011. He was at Stanford University and Los Alamos National Laboratory, in autumn of 2010 and summer of 2007, respectively. His research interests are in design, performance evaluation, and control of large distributed and networked intelligent systems applying mathematical tools from queueing theory, information theory, coding theory, and optimization methods.

Continual learning under non-stationary data streams raises a fundamental modeling question: how should learning objectives be formulated when past performance must be preserved while adapting to future tasks? A central mechanism for addressing this challenge in practice is replay, which mitigates catastrophic forgetting by reintroducing past data or gradients during training. In this talk, we examine two formulations that address this challenge from distinct yet unifying perspectives, and we provide theoretical guarantees in non-convex settings. The first formulation arises in continual federated learning, where non-stationary data streams are distributed across clients. In this setting, the problem becomes one of distributed nonconvex optimization under temporal drift and client heterogeneity. The second formulation views continual learning as a long-term constrained optimization problem, where forgetting is modeled explicitly as a time-average constraint on past task performance. Essentially, both centralized and federated continual learning can be understood as instances of a broader principle: learning under objective drift with long-horizon performance guarantees. We highlight how task variation fundamentally limits achievable performance in both settings and suggest directions toward scalable lifelong learning systems.

Dr. Ranjitha Prasad is an Assistant Professor in the Department of Electronics and Communication Engineering at IIIT Delhi, where she is also Associate Faculty at the Center for Artificial Intelligence. She received her Ph.D. from the Indian Institute of Science (IISc), Bangalore, in 2015. Her research spans signal processing, Bayesian statistics, and machine learning, with current interests in Responsible AI, federated learning, and continual learning. Prior to joining IIIT Delhi, she was a Research Fellow at the National University of Singapore and a Research Associate at Nanyang Technological University, Singapore, and also worked as a Scientist at TCS Innovation Labs, Delhi. She has held visiting research positions at the University of California San Diego and Monash University. Dr. Prasad is the recipient of the Seshagiri Kaikini Medal for the Best Ph.D. Thesis at IISc (2015), the Best Paper Award in the Communications Track at IEEE NCC 2014, and the Research Excellence Award at IIIT Delhi (2021, 2023). She is leading multiple sponsored projects in Responsible and Explainable AI funded by MeitY, DST, DRDO, and SERB.

In two-sampling testing, one observes two independent sequences of independent and identically distributed random variables generated according to two unknown distributions $P_1$ and $P_2$ and the goal is to decide whether $P_1=P_2$ (null hypothesis) or $P_1\neq P_2$ (alternative hypothesis). A classical solution is the Gutman test, which compares the empirical distributions of the two sequences and accepts the null hypothesis if the Jensen–Shannon (JS) divergence between them is below a given threshold. In this talk, we consider a generalization of the Gutman test, referred to as the divergence test, which replaces the JS divergence with an arbitrary divergence. For this class of tests, we consider the exponential decay rate of the type-II error probability under a fixed type-I error constraint. First, we show that the divergence test achieves the optimal first-order error exponent, namely, twice the Bhattacharyya distance between $P_1$ and $P_2$ regardless of the specific divergence used. Second, we demonstrate that when the divergence is invariant, the divergence test attains the same second-order asymptotics as the Gutman test. Finally, we establish a connection between two-sample testing and the robust goodness-of-fit testing problem.

Jithin Ravi is an assistant professor in the Department of Electronics and Electrical Communication Engineering at the Indian Institute of Technology Kharagpur. He received his M.Tech. and Ph.D. degrees in Electrical Engineering from the Indian Institute of Technology Bombay, Mumbai, in 2017. He completed his first post-doctoral fellowship at the Tata Institute of Fundamental Research, Mumbai, in 2017, followed by a second post-doctoral position at the Signal Theory and Communications Department, Universidad Carlos III de Madrid, Leganés, Spain, from 2018 to 2021. His research interests include information theory, wireless communications, information-theoretic security, and statistical inference.