Category Archives: Good Examples

Siddhartha Mukherjee on a common post hoc abuse of statistical analysis

Siddhartha Mukherjee:

then a second instinct takes over: Why not try to find the people for whom the drug did work?…

But it’s also a treacherous seduction…

Perhaps the most stinging reminder of these pitfalls comes from a timeless paper published by the statistician Richard Peto. In 1988, Peto and colleagues had finished an enormous randomized trial on 17,000 patients that proved the benefit of aspirin after a heart attack. The Lancet agreed to publish the data, but with a catch: The editors wanted to determine which patients had benefited the most. Older or younger subjects? Men or women?

Peto, a statistical rigorist, refused — such analyses would inevitably lead to artifactual conclusions — but the editors persisted, declining to advance the paper otherwise. Peto sent the paper back, but with a prank buried inside. The clinical subgroups were there, as requested — but he had inserted an additional one: “The patients were subdivided into 12 … groups according to their medieval astrological birth signs.” When the tongue-in-cheek zodiac subgroups were analyzed, Geminis and Libras were found to have no benefit from aspirin, but the drug “produced halving of risk if you were born under Capricorn.” Peto now insisted that the “astrological subgroups” also be included in the paper — in part to serve as a moral lesson for posterity. I’ve often thought of Peto’s paper as required reading for every medical student.

 

Chief Justice Roberts on learning life lessons

John Roberts

Now the commencement speakers will typically also wish you good luck and extend good wishes to you. I will not do that, and I’ll tell you why. From time to time in the years to come, I hope you will be treated unfairly, so that you will come to know the value of justice. I hope that you will suffer betrayal because that will teach you the importance of loyalty. Sorry to say, but I hope you will be lonely from time to time so that you don’t take friends for granted. I wish you bad luck, again, from time to time so that you will be conscious of the role of chance in life and understand that your success is not completely deserved and that the failure of others is not completely deserved either. And when you lose, as you will from time to time, I hope every now and then, your opponent will gloat over your failure. It is a way for you to understand the importance of sportsmanship. I hope you’ll be ignored so you know the importance of listening to others, and I hope you will have just enough pain to learn compassion. Whether I wish these things or not, they’re going to happen. And whether you benefit from them or not will depend upon your ability to see the message in your misfortunes.

Recalling when the NYT hinted that inventor Thomas Edison should be hanged

From a 1878 New York Times editorial:

Something ought to be done to Mr. EDISON, and there is a growing conviction that it had better be done with a hemp rope. Mr. EDISON has invented too many things, and almost without exception they are things of the most deleterious character. He has been addicted to electricity for many years, and it is not very long ago that he became notorious for having… invented the phone- graph, a machine that catches the lightest whisper of conversation and stores it up, so that at any future time it can be brought out…

Thanks to Mr. Edison’s perverted ingenuity, this has not only become a literal truth, but every shelf, closet, or floor may now have its concealed phonographic ears…

 The phonograph was, at the time of its invention, the most terrible example of depraved ingenuity which the world had seen; but Mr. EDISON has since reached a still more conspicuous peak of scientific infamy by inventing the aerophone–an instrument far more devastating in its effects and fraught with the destruction of human society…

 From morning till midnight our ears will be tortured with the uproar of aerophonic talk, and deaf men will be looked upon as the favored few to whom nature has made life tolerable…

 The result will be the complete disorganization of society. Men and women will flee from civilization and seek in the silence of the forest relief from the roar of count- less aerophones. Business, marriage, and all social amusements will be thrown aside, except by totally deaf men, and America will retrogade to the Stone Age with frightful rapidity... Far better is it to starve in solitude than to possess all the luxuries of civilization at the price of hearing every remark that is made within a radius of four miles…

Ralph Linton puts cultural appropriation in perspective

Jonathan Zimmerman unearths a classic piece by Ralph Linton that nicely illustrates the absurdity of obsessing over cultural appropriation:

Our solid American citizen… slips into his moccasins, invented by the Indians of the Eastern woodlands… He takes off his pajamas, a garment invented in India, and washes with soap invented by the ancient Gauls…

puts on shoes made from skins tanned by a process invented in ancient Egypt and cut to a pattern derived from the classical civilizations of the Mediterranean…

His plate is made of a form of pottery invented in China. His knife is of steel, an alloy first made in southern India, his fork a medieval Italian invention, and his spoon a derivative of a Roman original…

waffles, cakes made by a Scandinavian technique from wheat domesticated in Asia Minor. Over these he pours maple syrup, invented by the Indians of Eastern woodlands…

he settles back to smoke, an American Indian habit, consuming a plant domesticated in Brazil in either a pipe, derived from the Indians of Virginia, or a cigarette, derived from Mexico

he reads the news of the day, imprinted in characters invented by the Semites upon a material invented in China by a process invented in Germany. As he absorbs the account of foreign troubles he will, if he is a good conservative citizen, thank a Hebrew deity in a Indo-European language that he is 100 percent American.

Lisa Servon on predatory check cashing services

Lisa Servon via Alex Morrell:

The prevailing wisdom from bankers and policy makers went like this: People who used alternative financial services — like check cashers and payday lenders — were making expensive and unwise decisions…

But Servon, a professor of city and regional planning at the University of Pennsylvania and a former dean at the New School, spent 20 years studying low-income communities, and to her, that picture didn’t add up…

“The implication of that… was these people were making poor decisions,” Servon recently told Business Insider. “I knew that the people I had worked with closely who don’t have very much money know where every penny goes. They budget things. They know where to get the best deals on things. And so it struck me that if they were using check cashers, there must be a good reason for that.”…

Servon recounts her journey in her new book, “The Unbanking of America: How the New Middle Class Survives,”…

Over and over, Servon heard and observed that check cashers often met customers’ needs better than banks did.

She discovered there were three main reasons people used these services instead of banks: cost, transparency, and service.

This is a great example of the value of how familiarity with target population can yield better research. Prior to this, I had always thought these types of services were overpriced relative to a traditional bank, but I was not a good judge of that because I never really need the services they provide, so I naturally discounted their value to those that do need them.

Joel Spolsky on the importance of talent and the implications for scalability

Joel Spolsky:

the scalability problem. When you try to clone a restaurant, you have to decide between hiring another great chef of your caliber (in which case, that chef will probably want and expect to keep most of the extra profits that he created, so why bother), or else you’ll hire a cheaper, younger chef who’s not quite as good, but pretty soon your patrons will figure that out and they won’t go to the clone restaurant…

How many times have you heard this story?

Mike was unhappy. He had hired a huge company of IT consultants to build The System. The IT consultants he hired were incompetents who kept talking about “The Methodology” and who spent millions of dollars and had failed to produce a single thing.

Luckily, Mike found a youthful programmer who was really smart and talented. The youthful programmer built his whole system in one day for $20 and pizza. Mike was overjoyed. He recommended the youthful programmer to all his friends.

Youthful Programmer starts raking in the money. Soon, he has more work than he can handle, so he hires a bunch of people to help him. The good people want too many stock options, so he decides to hire even younger programmers right out of college and “train them” with a 6 week course.

The trouble is that the “training” doesn’t really produce consistent results, so Youthful Programmer starts creating rules and procedures that are meant to make more consistent results. Over the years, the rule book grows and grows. Soon it’s a six-volume manual called The Methodology.

After a few dozen years, Youthful Programmer is now a Huge Incompetent IT Consultant with a capital-M-methodology and a lot of people who blindly obey the Methodology, even when it doesn’t seem to be working, because they have no bloody idea whatsoever what else to do, and they’re not really talented programmers — they’re just well-meaning Poli Sci majors who attended the six-week course.

And Newly Huge Incompetent IT Consultant starts messing up. Their customers are unhappy. And another upstart talented programmer comes and takes away all their business, and the cycle begins anew…

What’s the moral of the story? Beware of Methodologies. They are a great way to bring everyone up to a dismal, but passable, level of performance, but at the same time, they are aggravating to more talented people who chafe at the restrictions that are placed on them.

Scott Alexander on how bad government regulation begets more goverment regulation (EpiPen edition)

Scott Alexander

EpiPens, useful medical devices which reverse potentially fatal allergic reactions, have recently quadrupled in price…

Let me ask Vox a question: when was the last time that America’s chair industry hiked the price of chairs 400% and suddenly nobody in the country could afford to sit down? When was the last time that the mug industry decided to charge $300 per cup, and everyone had to drink coffee straight from the pot or face bankruptcy? When was the last time greedy shoe executives forced most Americans to go barefoot? And why do you think that is?

The problem with the pharmaceutical industry isn’t that they’re unregulated just like chairs and mugs. The problem with the pharmaceutical industry is that they’re part of a highly-regulated cronyist system that works completely differently from chairs and mugs.

If a chair company decided to charge $300 for their chairs, somebody else would set up a woodshop, sell their chairs for $250, and make a killing – and so on until chairs cost normal-chair-prices again. When Mylan decided to sell EpiPens for $300, in any normal system somebody would have made their own EpiPens and sold them for less. It wouldn’t have been hard. Its active ingredient, epinephrine, is off-patent, was being synthesized as early as 1906, and costs about ten cents per EpiPen-load.

Why don’t they? They keep trying, and the FDA keeps refusing to approve them for human use…

EpiPen manufacturer Mylan Inc spends about a million dollars on lobbying per year.OpenSecrets.org tells us what bills got all that money. They seem to have given the most to defeat S.214, the “Preserve Access to Affordable Generics Act”. The bill would ban pharmaceutical companies from bribing generic companies not to create generic drugs…

So let me try to make this easier to understand.

Imagine that the government creates the Furniture and Desk Association, an agency which declares that only IKEA is allowed to sell chairs. IKEA responds by charging $300 per chair. Other companies try to sell stools or sofas, but get bogged down for years in litigation over whether these technically count as “chairs”. When a few of them win their court cases, the FDA shoots them down anyway for vague reasons it refuses to share…

Imagine that this whole system is going on at the same time that IKEA spends millions of dollars lobbying senators about chair-related issues, and that these same senators vote down a bill preventing IKEA from paying off other companies to stay out of the chair industry. Also, suppose that a bunch of people are dying each year of exhaustion from having to stand up all the time because chairs are too expensive unless you’ve got really good furniture insurance, which is totally a thing and which everybody is legally required to have.

And now imagine that a news site responds with an article saying the government doesn’t regulate chairs enough.

Socrates on how writing destroys our memory

Given all the concern about how computers may be making people dumber, I thought this quote from Socrates dug up by Abby Smith Rumsey was good to keep in mind:

Abby Smith Rumsey:

Socrates, the ur-techno-curmudgeon, warned that the invention of writing would lead to ignorance and ultimately the death of memory itself:

For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise.

Socrates got it wrong. It was the societies that entrusted their memories to papyrus, paper, and now computer chips that have immensely expanded the scope of human knowledge…

Christie Aschwanden on a great misconception about p-values in statistics

Christie Aschwanden:

the p-value cannot tell you if your hypothesis is correct. Instead, it’s the probability of your data given your hypothesis. That sounds tantalizingly similar to “the probability of your hypothesis given your data,” but they’re not the same thing, said Stephen Senn, a biostatistician at the Luxembourg Institute of Health. To understand why, consider this example. “Is the pope Catholic? The answer is yes,” said Senn. “Is a Catholic the pope? The answer is probably not. If you change the order, the statement doesn’t survive.”

A good example of: selection bias

Howard Bodenhorn, Timothy W. Guinnane, and Thomas Mroz provide an excellent example of selection bias. Selection bias occurs when the sample you analyze is not representative of the population you are trying to study. Bodenhorn, Guinnane, and Mroz argue that this problem arose for studies on the effects of industrialization on human height:

people appeared to get shorter during the period of early industrialisation… This result supports an extremely pessimistic understanding of how industrialisation affects human welfare.

During the 18th and 19th centuries, most of these sources were subject to important selection forces. Consider soldiers, a favourite source of height data. No studies account for the fact that most armies in the past relied heavily on volunteers. Men willing to join the army in the economically dynamic US in the 1830s and 1840s, for example, did not amount to a random draw from the US population – in fact, they were probably men whose labour market opportunities were inferior to those who remained civilians.

To the extent that labour market opportunities reflect the same kinds of background traits and ‘health human capital’ that account for height, we would expect the soldiers to be shorter than the civilians…

the only sources for historical height data that do not suffer this kind of bias come from the handful of countries that imposed a form of universal conscription. Figure 1 reports mean heights in five conscript armies, in addition to the US. The US did not have effective military conscription until World War I. The industrialisation puzzle, strikingly, does not appear in any country with conscription…

heights

 

The measured decline of mean height during industrialisation reflects in large part the nature of the data sources, not necessarily changes in the heights of the underlying populations.