On what basis do we learn to make effective decisions when faced with intermittent feedback from actions taken in a dynamic decision-making environment? In the present study we hypothesize that reward information (financial, social) may provide useful signals that can guide decision-making in these situations. To examine this, we present three experiments in which people make decisions directly towards controlling a dynamically uncertain output. We manipulate the framing of incentives (gains and losses) and the form of the incentives (financial, social), and measure their impact on decision-making performance (both in frequent and intermittent output feedback conditions). Overall, performance suffered under intermittent output feedback. Relative to social rewards, financial rewards generally improved control performance, and a gains framing (financial, social) leading to better performance than a losses framing (financial, social). To understand how rewards affect behavior in our tasks, we present a reinforcement learning model to capture the learning and performance profiles in each of our experiments. This study shows that information regarding incentives impacts the levels of exploration, the optimality of decisions-making, and the variability in the strategies people develop to control a dynamic output experienced frequently and intermittently.
KeywordsIntermittent vs. Frequent FeedbackRewardsOutcome FeedbackDynamic Decision-MakingUncertaintyControl
Aktas, N., De Bodt, E., & Cousin, J.-G. (2011). Do Financial Markets Care about SRI? Evidence from Mergers and Acquisitions. Journal of Banking & Finance, 35, 1753-1761.
Andersson, M., Hedesstrom, M., & Garling, T. (2014). A Social-Psychological Perspective on Herding in Stock Markets. Journal of Behavioral Finance, 15, 226-234. https://doi.org/10.1080/15427560.2014.941062
Ariely, D., Gneezy, U., Loewenstein, G., & Mazar, N. (2009). Large Stakes and Big Mistakes. The Review of Economic Studies, 76, 451-469. https://doi.org/10.1111/j.1467-937X.2009.00534.x
Bahrick, H. P. (1954). Incidental Learning under Two Incentive Conditions. Journal of Experimental Psychology, 47, 170. https://doi.org/10.1037/h0053619
Bandiera, O., Barankay, I., & Rasul, I. (2011). Field Experiments with Firms. The Journal of Economic Perspectives, 63-82. https://doi.org/10.1257/jep.25.3.63
Bank of England (2015). One Bank Research Agenda: Discussion Paper.
Bonner, S. E., & Sprinkle, G. B. (2002). The Effects of Monetary Incentives on Effort and Task Performance: Theories, Evidence, and a Framework for Research. Accounting, Organizations and Society, 27, 303-345.
Brehmer, B. (1992). Dynamic Decision Making: Human Control of Complex Systems. Actapsychologica, 81, 211-241.
Brooks, J. D., Wilson, N., & Sun, R. (2012). The Effects of Performance Motivation: A Computational Exploration of a Dynamic Decision Making Task. In Proceedings of the First International Conference on Brain-Mind (pp. 7-14).
Buchheit, S., Dalton, D., Downen, T., & Pippin, S. (2012). Outcome Feedback, Incentives, and Performance: Evidence from a Relatively Complex Forecasting Task. Behavioral Research in Accounting, 24, 1-20. https://doi.org/10.2308/bria-50151
Burns, B. D. & Vollmeyer, R. (2002). Goal Specificity Effects on Hypothesis Testing in Problem Solving. The Quarterly Journal of Experimental Psychology: Section A, 55, 241-261. https://doi.org/10.1080/02724980143000262
Bushman, B. J., Moeller, S. J., & Crocker, J. (2011). Sweets, Sex, or Self-Esteem? Comparing the Value of Self-Esteem Boosts with Other Pleasant Rewards. Journal of personality, 79, 993-1012. https://doi.org/10.1111/j.1467-6494.2011.00712.x
Camerer, C. & Ho, T.-H. (1999). Experienced-Weighted Attraction Learning in Normal Form games. Econometrica, 827-874. https://doi.org/10.1111/1468-0262.00054
Chase, H. W., Kumar, P., Eickhoff, S. B., & Dombrovski, A. Y. (2015). Reinforcement Learning Models and Their Neural Correlates: An Activation Likelihood Estimation Meta-Analysis. Cognitive, Affective & Behavioral Neuroscience, 15, 435-459. https://doi.org/10.3758/s13415-015-0338-7
Cisek, P. (2006). Integrated Neural Processes for Defining Potential Actions and decIding between Them: A Computational Model. The Journal of Neuroscience, 26, 9761-9770. https://doi.org/10.1523/JNEUROSCI.5605-05.2006
Daw, N. D., & Doya, K. (2006). The Computational Neurobiology of Learning and Reward. Current Opinion in Neurobiology, 16, 199-204.
Deci, E. L. (1976). Notes on the Theory and Metatheory of Intrinsic Motivation. Organizational Behavior and Human Performance, 15, 130-145.
Deci, E. L., & Ryan, R. M. (1985). Intrinsic Motivation and Self-Determination in Human Behavior. Berlin: Springer Science & Business Media. https://doi.org/10.1007/978-1-4899-2271-7
Eisenberger, R., & Cameron, J. (1996). Detrimental Effects of Reward: Reality or Myth? American Psychologist, 51, 1153. https://doi.org/10.1037/0003-066X.51.11.1153
Erev, I., & Barron, G. (2005). On Adaptation, Maximization, and Reinforcement Learning among Cognitive Strategies. Psychological Review, 112, 912. https://doi.org/10.1037/0033-295X.112.4.912
Erev, I., & Roth, A. E. (1998). Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria. American Economic Review, 848-881.
Feiler, D. C., Tong, J. D., & Larrick, R. P. (2013). Biased Judgment in Censored Environments. Management Science, 59, 573-591. https://doi.org/10.1287/mnsc.1120.1612
Frederickson, J. R., Peffer, S. A., & Pratt, J. (1999). Performance Evaluation Judgments: Effects of Prior Experience under Different Performance Evaluation Schemes and Feedback Frequencies. Journal of Accounting Research, 151-165. https://doi.org/10.2307/2491401
Frey, B. (1997). Not Just for the Money: An Economic Theory of Personal Motivation. Cheltenham: Elgar.
Gibson, F. P. (2007). Learning and Transfer in Dynamic Decision Environments. Computational and Mathematical Organization Theory, 13, 39-61. https://doi.org/10.1007/s10588-006-9010-7
González-Vallejo, C., & Bonham, A. (2007). Aligning Confidence with Accuracy: Revisiting the Role of Feedback. Acta Psychologica, 125, 221-239.
Grimm, L. R., Markman, A. B., Maddox, W. T., & Baldwin, G. C. (2008). Differential Effects of Regulatory fit on Category Learning. Journal of Experimental Social Psychology, 44, 920-927.
Hertwig, R., & Ortmann, A. (2001). Experimental Practices in Economics: A Methodological Challenge for Psychologists? Behavioral and Brain Sciences, 24, 383-403. https://doi.org/10.2139/ssrn.1129845
Higgins, E. T. (1997). Beyond Pleasure and Pain. American Psychologist, 52, 1280. https://doi.org/10.1037/0003-066x.52.12.1280
Hossain, T., & List, J. A. (2012). The Behavioralist Visits the Factory: Increasing Productivity Using Simple Framing Manipulations. Management Science, 58, 2151-2167. https://doi.org/10.1037/0003-066X.52.12.1280
Kamenica, E. (2012). Behavioral Economics and Psychology of Incentives. Annual Review of Economics, 4, 427-452. https://doi.org/10.1146/annurev-economics-080511-110909
Le Pelley, M. E. (2004). The Role of Associative History in Models of Associative Learning: A Selective Review and a Hybrid Model. Quarterly Journal of Experimental Psychology Section B, 57, 193-243. https://doi.org/10.1080/02724990344000141
Lepper, M. R., & Greene, D. (2015). The Hidden Costs of Reward: New Perspectives on the Psychology of Human Motivation. London: Psychology Press.
Lipshitz, R., & Strauss, O. (1997). Coping with Uncertainty: A Naturalistic Decision-Making Analysis. Organizational Behavior and Human Decision Processes, 69, 149-163. https://doi.org/10.1006/obhd.1997.2679
Luckett, P. F., & Eggleton, I. R. (1991). Feedback and Management Accounting: A Review of Research into Behavioral Consequences. Accounting, Organizations and Society, 16, 371-394.
Lurie, N. H., & Swaminathan, J. M. (2009). Is Timely Information Always Better? The Effect of Feedback Frequency on Decision Making. Organizational Behavior and Human Decision Processes, 108, 315-329.
Luce, R. D. (1959). On the Possible Psychophysical Laws. Psychological Review, 66, 81-95. https://doi.org/10.1037/h0043178
Margolis, J. D., Elfenbein, H. A., & Walsh, J. P. (2007). Does It Pay to Be Good? A Meta-Analysis and Redirection of Research on the Relationship between Corporate Social and Financial Performance. Ann Arbor, 1001, 48109-1234. https://doi.org/10.2139/ssrn.1866371
McGraw, K. O. (1978). The Detrimental Effects of Reward on Performance: A Literature Review and a Prediction Model. In M. Lepper, & D. Greene (Eds.), The Hidden Costs of Reward: New Perspectives on the Psychology of Human Motivation (pp. 33-60). London: Psychology Press.
Meder, B., Le Lec, F., & Osman, M. (2013). Decision Making in Uncertain Times: What Can Cognitive and Decision Sciences Say about or Learn from Economic Crises? Trends in Cognitive Sciences, 17, 257-260.
Melancon, J. P., Noble, S. M., & Noble, C. H. (2011). Managing Rewards to Enhance Relational Worth. Journal of the Academy of Marketing Science, 39, 341-362. https://doi.org/10.1007/s11747-010-0206-5
Melnyk, S. A., Bititci, U., Platts, K., Tobias, J., & Andersen, B. (2014). Is Performance Measurement and Management Fit for the Future? Management Accounting Research, 25, 173-186.
Oikonomou, I., Brooks, C., & Pavelin, S. (2014). The Effects of Corporate Social Performance on the Cost of Corporate Debt and Credit Ratings. Financial Review, 49, 49-75. https://doi.org/10.1111/fire.12025
Osman, M. (2008a). Observation Can Be as Effective as Action in Problem Solving. Cognitive Science, 32, 162-183. https://doi.org/10.1080/03640210701703683
Osman, M. (2008b). Positive Transfer and Negative Transfer/Antilearning of Problem-Solving Skills. Journal of Experimental Psychology: General, 137, 97. https://doi.org/10.1037/0096-3445.137.1.97
Osman, M. (2010). Controlling Uncertainty: A Review of Human Behavior in Complex Dynamic Environments. Psychological Bulletin, 136, 65. https://doi.org/10.1037/a0017815
Osman, M. (2012). The Effects of Self Set or Externally Set Goals on Learning in an Uncertain Environment. Learning and Individual Differences, 22, 575-584.
Osman, M. (2014). Future-Minded: The Psychology of Agency and Control. Palgrave Macmillan. https://doi.org/10.1007/978-1-137-02227-1
Osman, M., Glass, B. D., & Hola, Z. (2015). Approaches to Learning to Control Dynamic Uncertainty. Systems, 3, 211-236. https://doi.org/10.3390/systems3040211
Osman, M., & Speekenbrink, M. (2011). Cue Utilization and Strategy Application in Stable and Unstable Dynamic Environments. Cognitive Systems Research, 12, 355-364.
Osman, M., & Speekenbrink, M. (2012). Prediction and Control in a Dynamic Environment. Frontiers in Psychology, 3. https://doi.org/10.3389/fpsyg.2012.00068
Otto, A. R., Markman, A. B., Gureckis, T. M., & Love, B. C. (2010). Regulatory Fit and Systematic Exploration in a Dynamic Decision-Making Environment. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 797. https://doi.org/10.1037/a0018999
Rademacher, L., Krach, S., Kohls, G., Irmak, A., Gründer, G., & Spreckelmeyer, K. N. (2010). Dissociation of Neural Networks for Anticipation and Consumption of Monetary and Social Rewards. Neuroimage, 49, 3276-3285.
Rudski, J. M., Lischner, M. I., & Albert, L. M. (2012). Superstitious Rule Generation Is Affected by Probability and Type of Outcome. Psychological Record, 49, 245-260.
Smith, V. L., & Walker, J. M. (1993). Monetary Rewards and Decision Cost in Experimental Economics. Economic Inquiry, 31, 245-261. https://doi.org/10.1111/j.1465-7295.1993.tb00881.x
Sterman, J. D. (1989). Modeling Managerial Behavior: Misperceptions of Feedback in a Dynamic Decision Making Experiment. Management Science, 35, 321-339. https://doi.org/10.1287/mnsc.35.3.321
Sun, R., Slusarz, P., & Terry, C. (2005). The Interaction of the Explicit and the Implicit in Skill Learning: A Dual-Process Approach. Psychological Review, 112, 159. https://doi.org/10.1037/0033-295X.112.1.159
Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction. Cambridge: MIT Press. https://doi.org/10.1109/tnn.1998.712192
Van Pham, H., Tran, K. D., & Kamei, K. (2014). Applications Using Hybrid Intelligent Decision Support Systems for Selection of Alternatives under Uncertainty and Risk. International Journal of Innovative Computing, Information and Control, 10, 39-56.
Vera-Munoz, S. C. (1998). The Effects of Accounting Knowledge and Context on the Omission of Opportunity Costs in Resource Allocation Decisions. Accounting Review, 47-72.
Vollmeyer, R., Burns, B. D., & Holyoak, K. J. (1996). The Impact of Goal Specificity on Strategy Use and the Acquisition of Problem Structure. Cognitive Science, 20, 75-100. https://doi.org/10.1207/s15516709cog2001_3
Worthy, D. A., Maddox, W. T., & Markman, A. B. (2007). Regulatory Fit Effects in a Choice Task. Psychonomic Bulletin & Review, 14, 1125-1132. https://doi.org/10.3758/BF03193101
Yechiam, E., Busemeyer, J. R., Stout, J. C., & Bechara, A. (2005). Using Cognitive Models to Map Relations between Neuropsychological Disorders and Human Decision-Making Deficits. Psychological Science, 16, 973-978. https://doi.org/10.1111/j.1467-9280.2005.01646.x
Yechiam, E., & Hochman, G. (2014). Loss Attention in a Dual-Task Setting. Psychological Science, 25, 494-502. https://doi.org/10.1177/0956797613510725