Computational, Neuroscientific, and Lifespan Perspectives on the Exploration-Exploitation Dilemma

摘要: Computational, Neuroscientific, and Lifespan Perspectives on the Exploration-Exploitation Dilemma A. Ross Otto 1 (Moderator) , Bradley W. Knox 2 C. Love Department of Psychology, Computer Science, University Texas at Austin Sam Gershman Yael Niv Princeton Darrell Worthy Todd Maddox AM computational modeling; cogni- tive neuroscience, information search Consider following real-life decisions that we make: deciding which route to take home minimize time spent traveling, choosing amongst a set known restaurants or new restaurant when dining out, between reading book by consistently good author versus an whose books vary widely in quality. All these involve balancing conflicting demands exploiting pre- vious knowledge order maximize payoffs explor- ing less-known options gain about currently optimal course action. Indeed, successfully competing is non-trivial problem interest artificial intelligence neural Reinforcement Learning (RL) research communities alike (Cohen, McClure, & Yu, 2007; Daw et al., 2006; Sutton Barto, 1998). There are adverse consequences for failing properly balance above examples: solely making exploita- choices entails possibility ignorance better courses action, while exploring too frequently incurs large opportunity costs. The goal proposed symposium bring together researchers from variety perspectives who working understand psychological neurobiological mechanisms underlying exploratory choice. In recent years, novel modeling approaches have been developed applied understanding how hu- mans incorporate gathering into their patterns These techniques yielded insight not only describing human choice behav- ior, but also physio- logical correlates decision-making humans (Daw Jepma Niewenhuis, press). re- searchers agreed participate this all applying models under- pinning peoples negotiation exploration-exploitation tradeoff. taken speakers indeed diverse, ranging uncovering hidden variables decision-makers’ unpack neu- robiological physiological measurements aging-related changes behavior. will provide forum highlighting advances applications symposium–while each performing elucidates decision-making—offer different issue. described includes 1) aging work examining lifespan decision-making, elucidating neurobiology types (Worthy Maddox), 2) Bayesian account effect novelty–when presented with new, potentially rewarding options–on choice, nov- elty signals represented brain compute values guide (Gershman Niv), 3) individu- als uncertainty costs planning action situations sequential de- pendencies outcomes (Hotaling, Buse- meyer, Shiffrin), 4) internally calculated uncer- tainty environment directs manifests itself physiologically over decision- (Otto, Knox, Love). addition proposing an- swers diverse important neuro- scientific questions, lines rely upon laboratory tasks monetary incentives that, own way, ecologically interest- reward dynamics. Belief-directed Exploration Human Decision-Makers: Behavioral Physiological Evidence A Otto, Decision-making uncertain environments poses con- flict goals past or- der rewards gather information. However, descriptive mod- eling framework utilized previous studies behavior characterizes exploration as result choices, rather than process reflecting beliefs and/or un-