As for the response confidence scores, the opinion judgments are reported here as proportions rather than percentages. There was a positive relationship between standard question confidence and standard question opinion judgments for incorrect reasoners, r(225) = .480, p< .001, such that incorrect reasoners who were less confident in their response were also less likely to think other reasoners could answer the standard question correctly. This strong relationship lends support to the notion that opinion judgments and response confidence scores are reflecting similar cognitive processes. Agnoli F, & Krantz DH (1989). Suppressing natural heuristics by formal instruction: The case of the conjunction fallacy. Cognitive Psychology, 21, 515–550. doi: 10.1016/0010-0285(89)90017-0 [ CrossRef] [ Google Scholar]

Finally, latency responses to both the standard and control versions were collected. Correlations between response time and response confidence were expected, in line with previous work (e.g., Johnson et al., 2016; Kelley & Lindsay, 1993; Thompson, Prowse Turner, & Pennycook, 2011; Thompson et al., 2013).The question really belongs more to the science of the mind than it does to mathematics and logic— it is about the assumptions we make, rather than whether or not we have the ability to solve the question. Thompson VA, Prowse Turner JA, & Pennycook G (2011). Intuition, reason, and metacognition. Cognitive Psychology, 63, 107–140. doi: 10.1016/j.cogpsych.2011.06.001 [ PubMed] [ CrossRef] [ Google Scholar]

Koriat A, Lichtenstein S, & Fischhoff B (1980). Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory, 6, 107–118. doi: 10.1037/0278-7393.6.2.107 [ CrossRef] [ Google Scholar]Frey D, Johnson ED, & De Neys W (2017). Individual differences in conflict detection during reasoning. Quarterly Journal of Experimental Psychology, 71, 1188–1208. doi: 10.1080/17470218.2017.1313283 [ PubMed] [ CrossRef] [ Google Scholar] Fischhoff B, Slovic P, & Lichtenstein S (1977). Knowing with certainty: The appropriateness of extreme confidence. Journal of Experimental Psychology: Human Perception and Performance, 3, 552–564. doi: 10.1037/0096-1523.3.4.552 [ CrossRef] [ Google Scholar] B. T., & Stanovich KE (2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8, 223–241. doi: 10.1177/1745691612460685 [ PubMed] [ CrossRef] [ Google Scholar]

Pennycook G, Fugelsang JA, & Koehler DJ (2012). Are we good at detecting conflict during reasoning? Cognition, 124, 101–106. doi: 10.1016/j.cognition.2012.04.004 [ PubMed] [ CrossRef] [ Google Scholar]

Giving some thoughts to deliberation.

For recognition, all incorrect reasoners’ responses were included in the analyses because their answers came in the form of a multiple-choice forced response. A mixed effects logistic regression (with subject as the random variable) was again conducted due to the dichotomous dependent variable (with or without “more than”). Once again, incorrect reasoners usually recognized the standard problem, but not the control, as containing “more than” (see Table 1), with this effect of condition significant, b = 3.00, odds ratio ( OR) = 20.07, χ 2 = 50.34, p< .001, 95% confidence interval ( CI) [8.76, 45.94]. Israeli-American psychologist Daniel Kahneman examines what he calls the machinery of the mind — two distinct systems in our brain that dictate how we think and make decisions — in his book, Thinking Fast, and Slow. B. T. (2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 59, 255–278. doi: 10.1146/annurev.psych.59.103006.093629 [ PubMed] [ CrossRef] [ Google Scholar] Psychology of learning and motivation (Vol. 26, pp. 125–173). Academic Press. doi: 10.1016/S0079-7421(08)60053-5. [ CrossRef] [ Google Scholar]