(If you find IQ and RT boring, skip to paragraph 3, “How can one test…”)

The smartest people also tend to perform best on simple tasks, such as detecting the direction of an arrow. This effect has been interpreted as evidence that the same elementary processing capacity underlies behavior on both tasks. Interestingly, the association between higher order cognitive ability (IQ, working memory) and simple task-performance is most pronounced when the simple task-performance is summarized by people’s slowest responses: People’s 10% slowest response times are more predictive of their intelligence than their fastest 10%. This phenomenon is called the “worst performance rule”.

The diffusion model offers a very simple explanation for the worst performance rule. The diffusion model postulates that response time and accuracy in a simple multi-trial task are the result of one process that is governed by a number of parameters. One of those, the drift rate, quantifies the rate of information processing. The higher this drift rate, the quicker and more accurate the responses. More crucially for the worst performance rule: Drift rate affects mostly slow RTs. So, it seems that drift rate quantifies the elementary information processing speed that underlies IQ, working memory, and simple task performance. In our study, we use the very large data set of the Basel-Berlin Risk Study (a large study designed to understand the biological foundations of risk taking) to seek confirmative evidence for the connection between simple task drift rate and working memory capacity.

How can one test this (or any) effect in a truly confirmatory fashion?

First, we would want to publish the results irrespective of the outcomes (that is: prevent publication bias). Therefore, we chose to submit this study to an appropriate journal, specifically, the journal Attention, Perception and Psychophysics (AP&P) as a “registered report”. This new type of research report implies that you submit your method and analysis plan before carrying it out. This plan gets reviewed. If they like it, they commit to publishing it — no matter what you find — as long as you do precisely what you planned (BTW, we just passed this stage, woohoo!).

Second, we didn’t want to be able to fool ourselves (that is: prevent experimenter bias). Therefore, we came up with a pretty inventive blind analysis plan: The person who carries out the diffusion model analyses (modeling also the correlation of the drift rate with working memory capacity) gets the freedom to tweak around with the data until the model works and fits. However, there’s a twist: the working memory capacity variable is shuffled. This means that the modeler cannot be influenced by his own expectation of a positive correlation between drift rate and working memory. Once the model is bug-free and the modeler is happy, he will post the code on the Open Science Framework website. Only then we supply him with the unshuffled version of the working memory variable.

I am very curious about the results. The paper is on my website, waiting for the results section.

Szymon Wichary, Jörg Rieskamp, and I have a new publication out on how stress affects decision making. Our results suggest that decision makers under stress limit their information search (and possibly information integration). One important question that remains to be addressed is to what extent such effects are adaptive, in other words, in which circumstances these reductions in information search and integration can lead to good decisions. Here’s the abstract of our paper:

Probabilistic Inferences Under Emotional Stress: How Arousal Affects Decision Processes

Many models of decision making neglect emotional states that could affect individuals’ cognitive processes. The present work explores the effect of emotional stress on people’s cognitive processes when making probabilistic inferences. Two contrasting hypotheses are tested against one another: the uncertainty-reduction and attention-narrowing hypotheses of how emotional stress affects decision making. In the experimental study, emotional stress was induced through the use of highly aversive pictures immediately before each decision. Emotional state was assessed by both subjective (state anxiety, arousal, and pleasantness ratings) and objective (skin conductance) measures. The results show that emotional stress impacts decision making; in particular, emotionally aroused participants seem to have focused on the most important information and selected simpler decision strategies relative to participants in a control condition. The results are in line with the attention-narrowing hypothesis and suggest that emotional stress can impact decision making through limited predecisional information search and the selection of simpler decision strategies.

Wichary, S., Mata, R., and Rieskamp, J. (2015) Probabilistic inferences under emotional stress: How arousal affects decision processes. Journal of Behavioral Decision Making. doi: 10.1002/bdm.1896.

Dirk Wulff has a new paper out on decisions from experience that suggests that pre-decisional information search can provide a window into risk and decision making preferences, and perhaps do so better than choice, which seems to be more sensitive to task characteristics such as those explored in Dirk’s work: “Depending on which environment emerges, choices may or may not be informative about the underlying preferences or aspirations (…). The good news, however, is that decisions from experience paradigms offer an observable psychological dimension that appears to afford researchers another window onto preferences or aspirations: the appetite for information.”

How short- and long-run aspirations impact search and choice in decisions from experience

To what extent do people adapt their information search policies and subsequent decisions to the long- and short-run consequences of choice environments? To address this question, we investigated exploration and exploitation policies in choice environments that involved single or multiple plays. We further compared behavior in these environments with behavior in the standard sampling paradigm. Frequently used in research on decision from experience, this paradigm does not explicitly implement the choice in terms of the short or long run. Results showed that people searched more in the multi-play environment than in the single-play environment. Moreover, the substantial search effort in the multi-play environment was conducive to choices consistent with expected value maximization, whereas the lesser search effort in the single-play environment was compatible with the goal of maximizing the chance of winning something. Furthermore, choice and search behaviors in the sampling paradigm predominantly echoed those observed in the single-play environment. This suggests that, when not instructed otherwise, participants in the sampling paradigm appear to favor search and choice strategies that embody short-run aspirations. Finally, the present findings challenge the revealed preference approach in decisions from experience, while also suggesting that information search may be an important and potentially even better signal of preference or aspirations than choice.

Wulff, D., Hills, T, & Hertwig, R. (2015). How short- and long-run aspirations impact search and choice in decisions from experience. Cognition, 144, 29-57. doi:10.1016/j.cognition.2015.07.006

David Kellen has yet another paper out on the nature of recognition memory! His new paper offers a critical test to distinguish between continuous and discrete-state models of recognition memory, with the results favouring the latter. An important aspect of this study is that it compares models by identifying their predictions for specific parts of the data (in this case, confidence judgments for studied words that were not recognized). This approach minimizes the need for auxiliary assumptions and yields more general conclusions than traditional comparison methods.

Signal detection and threshold modeling of confidence-rating ROCs: A critical test with minimal assumptions

An ongoing discussion in the recognition-memory literature concerns the question of whether recognition judgments reflect a direct mapping of graded memory representations (a notion that is instantiated by signal detection theory) or whether they are mediated by a discrete-state representation with the possibility of complete information loss (a notion that is instantiated by threshold models). These 2 accounts are usually evaluated by comparing their (penalized) fits to receiver operating characteristic data, a procedure that is predicated on substantial auxiliary assumptions, which if violated can invalidate results. We show that the 2 accounts can be compared on the basis of critical tests that invoke only minimal assumptions. Using previously published receiver operating characteristic data, we show that confidence-rating judgments are consistent with a discrete-state account.

Kellen, D. & Klauer, K. C. (2015). Signal detection and threshold modeling of confidence-rating ROCs: A critical test with minimal assumptions. Psychological Review, 122, 542-547.

David Kellen has a new paper out in the Journal of Mathematical Psychology on recognition memory modelling. David’s paper deals with an old problem in model comparison, namely how to penalize models according to their flexibility (i.e., how to implement a reasonable Ockham’s razor?). Specifically, the paper reports an efficient method for the computation of the Normalized Maximum Likelihood statistic for models of categorical data. This statistic is then applied to the domain of recognition-memory models, in which model flexibility is known to play a large role when comparing model fits to empirical data.

The flexibility of models of recognition memory: The case of confidence ratings

The normalized maximum likelihood (NML) index is a model-selection index derived from the minimum-description length principle. In contrast to traditional model-selection indices, it also quantifies differences in flexibility between models related to their functional form. We present a new method for computing the NML index for models of categorical data that parameterize multinomial or product-multinomial distributions and apply it to comparing the flexibility of major models of recognition memory for confidence-rating based receiver-operating-characteristic (ROC) data. NML penalties are tabulated for datasets of typical sizes and interpolation functions are fitted that allow one to interpolate NML penalties for datasets with sizes between the tabulated ones. Recovery studies suggest that the NML index performs better than traditional model-selection indices in model selection from ROC data. In an NML-based meta-analysis of 850 ROC datasets, versions of the dual-process signal detection models received most support followed by the finite mixture signal detection model and constrained versions of two-high threshold models.

Klauer, K. C. & Kellen, D. (2015). The flexibility of models of recognition memory: The case of confidence ratings. Journal of Mathematical Psychology, 67, 8-25.

Renato Frey, Ralph Hertwig, and I have a new paper out on age differences in decision from experience (i.e., decisions which involve learning about the probabilities of different outcomes through sampling). The results suggest that rather than age differences being observed across the board, they are a function of task complexity: Older adults show similar sampling (see figure above) and choice behavior when making decisions between 2 options but show reduced search and poorer choices in more complex environments involving 4 or 8 options. In a nutshell, aging doesn’t affect decision making across the board but rather as a function of increased task demands…

The role of cognitive abilities in decisions from experience: Age differences emerge as a function of choice set size

“People seldom enjoy access to summarized information about risky options before making a decision. Instead, they may search for information and learn about environmental contingencies—thus making decisions from experience. Aging is associated with notable deficits in learning and memory—but do these translate into poorer decisions from experience? We report three studies that used a sampling paradigm to investigate younger (M = 24 years) and older (M = 71 years) adults’ decisions from experience. In Study 1 (N = 121) participants made 12 decisions between pairs of payoff distributions in the lab. Study 2 (N = 70) implemented the same paradigm using portable devices, collecting 84 decisions per individual over a week. Study 3 (N = 84) extended the sampling paradigm by asking participants to make 12 decisions between two, four, and eight payoff distributions (in the lab). Overall, the behavioral results suggest that younger and older adults are relatively similar in how they search and what they choose when facing two payoff distributions (Studies 1 and 2). With an increasing number of payoff distributions, however, age differences emerged (Study 3). A modeling analysis on the level of individual participants showed that a simple delta-learning rule model best described the learning processes of most participants. To the extent that ongoing updating processes unfold relatively automatically and effortlessly, older adults may be liberated from the detrimental consequences of cognitive aging in the case of decisions from experience with few decision options. We discuss implications for research on decisions from experience and choice performance over the lifespan.”

Frey, R., Mata, R., & Hertwig, R. (2015). The role of cognitive abilities in decisions from experience: Age differences emerge as a function of choice set size. Cognition, 142, 60-80. doi:10.1016/j.cognition.2015.05.004

Bettina von Helversen and I have a new paper out on age differences in search in TopiCS. The literature is still scarce on this issue but we try to take stock of the idea that age differences in internal and external search are due to age-related cognitive decline. The goal is to do more empirical work on this in the future…

Here’s the abstract:

Search is a prerequisite for successful performance in a broad range of tasks ranging from making decisions between consumer goods to memory retrieval. How does aging impact search processes in such disparate situations? Aging is associated with structural and neuromodulatory brain changes that underlie cognitive control processes, which in turn have been proposed as a domain-general mechanism controlling search in external environments as well as memory. We review the aging literature to evaluate the cognitive control hypothesis that suggests that age-related change in cognitive control underlies age differences in both external and internal search. We also consider the limits of the cognitive control hypothesis and propose additional mechanisms such as changes in strategy use and affect that may be necessary to understand how aging affects search.

http://onlinelibrary.wiley.com/doi/10.1111/tops.12139/full