We investigate optimal group member configurations for producing a maximally accurate group forecast. Our approach accounts for group members that may be biased in their forecasts and/or have errors that correlate with the criterion values being forecast. We show that for large forecasting groups, the diversity of individual forecasts linearly trades off with individual forecaster accuracy when determining optimal group composition. We develop a statistical model that estimates features of individual forecasters when making aggregate forecasts. We use this model to better understand the conditions under which researchers should pursue optimal group weighting versus simply averaging the individual forecasts.
SFI Seminar – Clintin Davis-Stober: Understanding the individual within the crowd: An analysis of how individual forecasters contribute to ideal group forecasts
Q4 hedge fund letters, conference, scoops etc
Transcript
Thank you. So yes. So the title of my talk today is "Understanding the individual within the crowd: An analysis of how individual forecasters contribute to ideal group forecasts." So this is all about the wisdom of the crowd and I'm sure you've all heard this term. I know a lot of you in this very room do a lot of really awesome work in this area. But just to kind of give a quick overview. Are these the history of the wisdom of the crowd. The idea is basically that somehow we have a whole bunch of people and they make some judgment some estimates some forecast somehow pooling them in different ways to do that somehow bringing all those estimates together bringing all those people together. In the end the group itself will do a better job than many individuals or maybe even the best individual or the average individual. There's different ways to define this kind of the classic paper that kind of started this all I'm sure you're all very familiar with this is that 1907 paper by Sir Francis Galton vox populi. And you know he did something pretty clever where this is I believe somewhere in England there was a country fair where people would go and there'd be livestock and there'd be fun things to do. And as you walked into this fair there was an ox alive ox which I'm sure everyone knows what an ox's is like a really big cow with horns. Is that fair. In one study oxen okay.
And so you'd see the ox and you have to make a guess how much the ox weighs fully dressed and by fully dressed I mean butchered and like just the meat not wearing a costume or anything like that. But this is a hard task right. And there was lots of little fun plays on it right. So we had to pay a little bit of money to participate. So if you were to actually buy a ticket and make a guess. So the idea was you'd guess how much the oxen weighed fully dressed and whoever got closest would win the ox meat which was quite a nice price. That's a lot of meat or some prize like this. I don't remember all the details but the basic idea that you have a lot a lot a lot of people made estimates and guesses throughout the day. Like I said you had to pay a little bit of money to play this game so it wasn't just hordes of people who'd just anybody and everybody but you had quite a large distribution of people with different experiences different backgrounds that were making these guesses and what was interesting about this is that Francis Scott like at the time when he was sort of carrying out this experiment what do you want to do is just not just how stupid the crowd is right how ridiculous some of these guesses would be how far off they would be and he was absolutely shocked when he looked at the distribution of the gases and found that the median estimate was almost within it has been like a half a percentage point of the true dressed weight. It's outrageous right. So the median not the mean that was a common error got to that paper.
The median estimate was very very close to the truth and that was kind of the whole point of the paper is that out of this crowd of uninformed many of them very uninformed people the crowd somehow just converged right on the true weight of address stocks which is a pretty impressive feat really. Does that make sense to everybody. Now you can actually play wisdom of the crowd at home. It's like a little parlor trick. You yourself can do. You don't need an. Right. So I do this little. I do this little game in my class. I've done this several times. I have a jar and I fill it with pennies. And I know that there's two hundred eleven pennies in the jar because that's how many pennies I put in the jar. And as long as I have a class of about 20 25 people or more I'll just let everyone in the class hold the weight of the penny jar. Look at it but they can't open it but it's glass. You can see through it. And then they make a guess how many pennies are in there. And I've done this several times the mean and the median are always with them two or three pennies of the true number and the distribution is always wild. Like I'll get somebody that guesses like six hundred pennies or something and somebody that guesses like fifty pennies. That's ridiculous. Like if you ever paid with money. But you know it the mean and the media and just converge very rapidly on that true value. It's sort of remarkable. It's fun to do.
I mean obviously I'm a lot of fun at parties right. We'll get like a jar and some pennies and jelly be something. These are also very simple tasks right. The thing you're trying to guess isn't random it's not like predicting the weather. Some fixed number of pennies or some finite weight of address stocks things like this and you know there's reasons that we understand why generally this works like for mathematicians describe this as the and equality at work. You know you can think about is bracketing where there might be some true signal there that somebody has some information like maybe some butchers attended the fair and they can make pretty good guesses and everyone.
