Adaptive Batch Normalization Techniques for Enhancing Model Tuning in Multi-Task Learning Environments

Abstract

Multi-task learning (MTL) is a subfield of machine learning where a single model is trained to perform multiple tasks simultaneously, leveraging shared information to improve performance across tasks. However, tuning models in MTL environments is challenging due to the conflicting objectives and varying data distributions associated with each task. This paper introduces Adaptive Batch Normalization (AdaBN) techniques that dynamically adjust normalization parameters to enhance model tuning and improve generalization in multi-task learning settings. We provide a comprehensive review of current batch normalization approaches, propose new adaptive strategies, and evaluate them on various benchmark datasets. Our results demonstrate that AdaBN significantly enhances model performance by efficiently managing task-specific data distributions, leading to better convergence and reduced task interference.