101 Human Experiment 1
See Tversky & Kahneman (1974) Judgement under Uncertainty: Heuristics and Biases. Science 185:1124.
2 (& 7): 8 x 7 x 6 x 5 x 4 x 3 x 2 x 1 (& rev)
Our answers 2: 1000 4000 10000 20000 20000 35000 43000 500000
Our answers 7: 700 5040 10000 35000 36000 40000 45000 60000
Actual: 40,320 Reported median: 2250 descending; 512 ascending
3: 70 engineers and 30 lawyers; probability that Dick is an engineer
0.05 0.5 0.7 0.7 0.7 0.7 0.7 0.7
Actual: 0.7 Previous observations: 0.5
4: TTTTTTTTTT HHTHTTTTHH HTHTTTHHTH THHTTHTHHH HTTTHHTTTH HTTHHHTHTH
Previous observations: more than two in row are rare.
5: 10 people, # committees of 2 or 8 members?
(45, 45) (45, 45) (45, 45) 50
Actual: 45 Median for 2 was 70; for 8 was 20.
6: 4/5 vs 12/20 -- Odds that actually 2/3?
0 2 0, 0 100 "high"
Actual: [C(5,4)(2/3)^4 (1/3) / C(5,1)(2/3)(1/3)^4]= 8
[C(20,12)(2/3)^12 (1/3)^8 / C(20,8)(2/3)^8(1/3)^12]= 16
Most people feel that 4/5 is better evidence.
I can't remember what I actually wrote for most of these questions now, but on a few of them, more information is necessary than the questions provide, and so many different answers can be justified given different background assumptions. On #6, for example, asks what the odds are that the hat or barrel or whatever contains 2/3 white (or black, or whichever color) marbles, if I'm not mistaken. We have no idea how large the container is, or what an upper bound on the number of marbles it might reasonably contain is. As the number of marbles increases, then the probability that the odds are exactly 2/3 decreases, and as the N -> Inf, then p(odds are 2/3) -> 0, regardless of the color of the five or twenty marbles selected by the participants.
Also, I was one of the people who said something lower than 0.7 for the probability that Dick is an engineer, knowing full well that without any extra information, the probability would of course be 0.7. The probability that he is an engineer, however, remains 0.7 only if the extra information is irrelevant. Consider this more extreme example: We consider a specific person P, who is a member of a group of 90 women and 10 men. We are told that P has a Y chromosome. What is the probability that P is a man? The answer of course nearly 1.0, despite the fact that far more people in the group are women. Assigning some attribute to P that is less-strongly but still positively correlated with being a man will likewise raise the chance that P is a man. In the case of the Dick the engineer/lawyer, when we are told that people say that Dick has certain attributes that are popularly considered to be more fitting for lawyers than for engineers, this raises the chance that he is a lawyer. An illustration of this using Bayes's Theorem is:
- p(H|E&K) = p(E|H&K)/p(E|K) * p(H|K)
where H is the hypothesis that Dick is a lawyer, E is the knowledge that people say that he has certain lawyerly traits, and K is our background knowledge about the proportion of engineers and lawyers in our sample. The probability of X is denoted p(X); the probability of X given K is p(X|K)
- p(H|K) = 0.3 (without the evidence that Dick has certainly lawyerly traits, we assign a 0.3 probability to his being an engineer)
- p(E|H&K)/p(E|K) = something greater than 1 (the chance that we would hear that Dick has certain lawyerly traits is greater if we also know that he is in fact a lawyer than if we did not have such knowledge)
- p(H|E&K) = 0.3 * something greater than one, so p(~H|E&K) = 0.7 * something less than one, so knowing that he has certain lawyerly traits decreases the chance that Dick is an engineer.
We can't of course provide a precise value for "something greater than 1" in this example, and so the temptation might be to shunt it to the realm of the unquantifiable and forget about it. Perhaps a moral of the story is that hard-to-quantify information is often not irrelevant. I'm inclined to say, however, that since we were presented the questions in the context of being told about a study that showed how people's intuition often leads them astray when making mathematical judgements, we made sure to turn off our intuition as best we could, or to just stick to the mathematics. Anyway, apologies for the gratuitous use of Bayes's Theorem, and see you all at 9:45. --Jleith 01:21, 22 Sep 2005 (EDT)