Nobody has met a city or a company. They have met a few people and filed them under the name
We were at it again last week. “You’ve met six of them,” he said. “I’ve met two, and I had to sit across a table from mine.” He thinks the people at a certain firm are arrogant. I think they’re fine. We’ve been having this argument, on and off, for about a year, and neither of us has moved.
His two interviewed him on the same afternoon, back to back. Which means they’d been at it since nine in the morning and had, I’d guess, run out of warmth somewhere around eleven. He turned down the offer on the strength of that afternoon. He now holds a settled view of a company of eight thousand people, and the view is two tired men in a conference room on a Thursday.
I’d have left it as his problem if I didn’t do the same thing. My opinion of at least three colleges rests on the one or two alumni I’ve shared a desk with, and I have no idea whether those people were the best their batch produced or the worst. My opinion of Bangalore is one work trip before the MBA, most of which I spent in autos. Whatever I think about the French comes from four exchange students in second year. Whatever I think about investment bankers comes from the handful of batchmates who went that way. Ask me what lawyers are like and I’ll describe one man who handled a property matter for my family, and handled it badly.
None of this felt like a mistake at the time. It felt like experience, which is the whole problem, because experience is what it was. I did meet those people. The interviewers were arrogant that afternoon. The lawyer was bad. What went wrong is what happened to the information afterwards. A handful of people got filed under a label that covers thousands, sometimes millions, and after that I couldn’t tell the label and the handful apart.
Lippmann saw this in 1922. “For the most part we do not first see, and then define, we define first and then see.” He was writing about newspapers and war, but the line holds for the two interviewers just as well. The label was waiting before they walked in.
Here’s the claim, as plainly as I can put it. You never perceive a group. You perceive the handful of its members who happened to cross your life, and your mind quietly stores that handful under the group’s name without mentioning it’s done so. From then on, every sentence that starts “people from that firm are” or “doctors are” is a statement about a sample, said with the confidence of a statement about a population.
Tversky and Kahneman wrote about the general version in 1971. People treat small samples as if they behave like the whole; they called it belief in the law of small numbers. What I keep running into is a social version of it, with two extra problems stacked on top of the small n.
The first is that the sample isn’t random. It’s drawn by where you’re standing. Who you meet from any group depends almost entirely on your own position, so what you’re sampling is never the group, it’s the group intersected with the set of people your life can actually reach. The alumni I’ve met from a given college all work in the same two industries I work in, because where else would I meet them. The French people I knew were the sort of French people who choose to spend a semester in Lucknow, which is a type. The bankers I know sat in the same classrooms as me for two years. They cleared the same exam I cleared and then, in the last month, made a different choice. In every case the group had been filtered through my own trajectory before I got a look at it. So what I’ve actually sampled is the thin slice of each that overlaps with people like me, and that slice, no surprise, looks a lot like me.
The second is that even within the handful, the weighting is off. The quiet alumnus who did his work and went home did nothing to my picture of his college. The one who contested every point in every meeting built the whole picture on his own. Memory keeps whoever stood out, so four people collapse to one, and the one is whoever was most annoying or most impressive.
If you knew all this was happening you’d discount the opinion accordingly. You don’t know, because it arrives as first-hand knowledge, and first-hand knowledge is about the last thing anyone gives up in an argument.
I should admit the complication, since it’s the one my friend throws at me. Sometimes the sample is right. One of those three colleges, I’ve since met maybe a dozen more alumni of, and the first two turned out to be typical. Tiny sample, badly drawn, correct. And his two interviewers weren’t random either, in a way that helps his case: a firm chooses who it puts in front of candidates, so two arrogant men on a Thursday do tell you something about the firm, even if it’s less than he concluded. All that shows, though, is that the method sometimes lands on the truth, and that from the inside you can’t tell when. The certainty feels identical either way.
I’ve started calling this positional sampling, because the position is where most of the damage comes from. Small numbers are a known problem. That the small numbers were chosen for you by your pin code and your CAT percentile is less obvious, and it explains something the small numbers alone can’t.
It explains the argument with my friend, for one. The six I met were in another city, in another function, and I met them as a colleague rather than as a candidate. His two met him across a table with his CV in front of them. Neither of us has met the firm. What we’re comparing is two samples with no members in common, drawn from different corners of the same organization under different conditions, while arguing as if the organization itself were in the room.
Scale it up and a group’s reputation stops looking like a fact about the group. It looks like an aggregate of millions of these micro-samples, each drawn by a different observer from whichever corner of the group their life happened to touch. Most of what one city believes about another, or one firm about a rival, travels through the small number of people who crossed between them, and whatever made them cross was rarely that they were typical. A city’s picture of migrants from another state is built from the migrants who could afford the move, who are a different population from the state. A firm’s picture of a competitor is built from the people who left the competitor to join it, who are, by definition, the ones who were unhappy there. Nobody has to intend any of this. The selection does it on its own.
And it runs both ways, which is the part I find hardest to shake. Somewhere there’s a person for whom I am the sample. I’m one of the two or three people from my hometown, or my college, or my firm that they’ve properly known, and some fraction of what they believe about that whole set is how I behaved on a day I was short on sleep. I’ll never find out which person, or which day.