The Effect of State Representations in Sequential Sensory Prediction: Introducing the Shape Sequence Task
- Henrik Siljebråt, Goldsmiths, University of London, London, United Kingdom
- Alan Pickering, Goldsmiths, University of London, London, United Kingdom
AbstractHow do humans learn models supporting decision making? Reinforcement learning (RL) is a success story both in artificial intelligence and neuroscience. Essential to these RL models are state representations. Based on what current state an animal or artificial agent is in, animals learn optimal actions by maximizing future expected reward. But in most implementations of RL models the states are pre-defined. How are humans able to learn and create such representations? We introduce a novel sequence prediction task with hidden structure where participants have to combine learning and memory to find the proper state representation, without the task explicitly indicating such structure. We show how humans are able to find this pattern, while a sensory prediction error version of RL cannot, unless equipped with appropriate state representations. Furthermore, in slight variations of the task, making it more difficult for humans, the RL-derived model with simple state representations sufficiently describes behaviour and suggests that how humans fall back on simple state representations when a more optimal task representation cannot be found. We argue this task grants the ability to investigate recently proposed models of state and task representations as well as supporting recent results indicating that RL describes a more general sensory prediction error function for dopamine, rather than predictions focussed solely on reward.