Attention modulates value normalization in human reinforcement learning by shaping reward encoding.

Abstract

Contextual valuation is a well-documented phenomenon in reinforcement learning, typically manifesting as range normalization in outcome representation. However, recent findings have revealed systematic deviations from this model, particularly when three options with equally spaced values are presented. In this study, we hypothesize that these distortions in outcome normalization arise from attentional processes. To test this, we conduct three experiments with 105 participants in total while simultaneously tracking their gaze position with eye-tracking. Furthermore, we systematically manipulate attention using both top-down and bottom-up approaches. These manipulations significantly increase the subjective valuation of attended options, supporting a causal role of attention in shaping value representation. To account for these effects, we develop a reinforcement learning model that integrates attentional mechanisms, wherein gaze duration directly modulates the absolute value of options prior to range normalization. This attentional range model outperforms attention-free and choice-repetition alternatives, underscoring the critical influence of attention in value computation.