(If you find IQ and RT boring, skip to paragraph 3, “How can one test…”)

The smartest people also tend to perform best on simple tasks, such as detecting the direction of an arrow. This effect has been interpreted as evidence that the same elementary processing capacity underlies behavior on both tasks. Interestingly, the association between higher order cognitive ability (IQ, working memory) and simple task-performance is most pronounced when the simple task-performance is summarized by people’s slowest responses: People’s 10% slowest response times are more predictive of their intelligence than their fastest 10%. This phenomenon is called the “worst performance rule”.

The diffusion model offers a very simple explanation for the worst performance rule. The diffusion model postulates that response time and accuracy in a simple multi-trial task are the result of one process that is governed by a number of parameters. One of those, the drift rate, quantifies the rate of information processing. The higher this drift rate, the quicker and more accurate the responses. More crucially for the worst performance rule: Drift rate affects mostly slow RTs. So, it seems that drift rate quantifies the elementary information processing speed that underlies IQ, working memory, and simple task performance. In our study, we use the very large data set of the Basel-Berlin Risk Study (a large study designed to understand the biological foundations of risk taking) to seek confirmative evidence for the connection between simple task drift rate and working memory capacity.

How can one test this (or any) effect in a truly confirmatory fashion?

First, we would want to publish the results irrespective of the outcomes (that is: prevent publication bias). Therefore, we chose to submit this study to an appropriate journal, specifically, the journal Attention, Perception and Psychophysics (AP&P) as a “registered report”. This new type of research report implies that you submit your method and analysis plan before carrying it out. This plan gets reviewed. If they like it, they commit to publishing it — no matter what you find — as long as you do precisely what you planned (BTW, we just passed this stage, woohoo!).

Second, we didn’t want to be able to fool ourselves (that is: prevent experimenter bias). Therefore, we came up with a pretty inventive blind analysis plan: The person who carries out the diffusion model analyses (modeling also the correlation of the drift rate with working memory capacity) gets the freedom to tweak around with the data until the model works and fits. However, there’s a twist: the working memory capacity variable is shuffled. This means that the modeler cannot be influenced by his own expectation of a positive correlation between drift rate and working memory. Once the model is bug-free and the modeler is happy, he will post the code on the Open Science Framework website. Only then we supply him with the unshuffled version of the working memory variable.

I am very curious about the results. The paper is on my website, waiting for the results section.