It's Not Such a Small World After All

You get an anonymous letter on January 2nd informing you that the market will go up during the month. It proves to be true, but you disregard it, owing to the well-known January effect (stocks have gone up historically during January). Then you receive another one on Feb 1st telling you that the market will go down. Again, it proves to be true. Then you get another letter on March 1st – same story. By July you are intrigued by the prescience of the anonymous person and you are asked to invest in a special offshore fund. You pour all your savings into it. Two months later, your money is gone. You go spill your tears on your neighbor’s shoulder and he tells you that he remembers that he received two such mysterious letters. But the mailings stopped at the second letter. He recalls that the first one was correct in its prediction, the other incorrect.

What happened? The trick is as follows. The con-operator pulls 10,000 names out of a phone book. He mails a bullish letter to one half of the sample, and a bearish one to the other half. The following month he selects the names of the persons to whom he mailed the letter whose prediction turned out to be right, that is, 5,000 names. The next month he does the same with the remaining 2,500 names, until the list narrows down to 500 people. Of these there will be 200 victims. An investment in a few thousand dollars worth of postage stamps will turn into several million.

It is not uncommon for someone watching a tennis game on television to be bombarded by advertisements for funds that did (until that minute) outperform others by some percentage over some period. But, again, why would anybody advertise if he didn’t happen to outperform the market? There is a high probability of the investment coming to you if its success is caused entirely by randomness. This phenomenon is what economists and insurance people call adverse selection. Judging an investment that comes to you requires more stringent standards than judging an investment you seek, owing to such selection bias. For example, by going to a cohort composed of 10,000 managers, I have 2/100 chances of finding a spurious survivor. By staying home and answering my doorbell, the chance of the soliciting party being a spurious survivor is closer to 100%.

The same logic that applies to the spurious survivor, also applies to the skilled person who has the odds markedly stacked in her favor, but who still ends up going to the cemetery. This effect is the exact opposite to the survivorship bias. Consider that all one needs is two bad years in the investment industry to terminate a risk-taking career and that, even with great odds in one’s favor, such an outcome is very possible. What do people do to survive? They maximize their odds of staying in the game by taking black-swan risks; those that fare well most of the time, but incur a risk of blowing up.

Luck and Selection Bias: The Birthday Paradox

The most intuitive way to describe the data mining problem to a non-statistician is through what is called the birthday paradox, though it is not really a paradox, simply a perceptional oddity. If you meet someone randomly, there is a one in 365.25 chance of your sharing their birthday, and a considerably smaller one of having the exact birthday of the same year. So, sharing the same birthday would be a coincidental event that you would discuss at the dinner table. Now let us look at a situation where there are 23 people in a room. What is the chance of there being two people with the same birthday? About 50%. For we are not specifying which people need to share a birthday, any pair works.

A similar misconception of probabilities arises from the random encounters one may have with relatives or friends in highly unexpected places. "It’s a small world!" is often uttered with surprise. But these are not improbable occurrences – the world is much larger than we think. It is just that we are not truly testing for the odds of having an encounter with one specific person, in a specific location at a specific time. Rather, we are simply testing for any encounter, with any person we have ever met in the past, and in any place we will visit during the period concerned. The probability of the latter is considerably higher, perhaps several thousand times the magnitude of the former.

When the statistician looks at the data to test a given relationship, say to ferret out the correlation between the occurrence of a given event, like a political announcement, and stock market volatility, odds are that the results can be taken seriously. But when one throws the computer at data, looking for just about any relationship, it is certain that a spurious connection will emerge, such as the fate of the stock market being linked to the length of women’s skirts. And just like the birthday coincidences, it will amaze people.

What is your probability of winning the New Jersey lottery twice? One in 17 trillion. Yet it happened to Evelyn Adams, whom the reader might guess should feel particularly chosen by destiny. Using the method we developed above, researchers Percy Diaconis and Frederick Mosteller estimated at 30 to 1 the probability that someone, somewhere, in a totally unspecified way, gets so lucky!

Some people carry their data mining activities into theology – after all, ancient Mediterraneans used to read potent messages in the entrails of birds. Michael Drosnin provides an interesting extension of data mining into biblical exegesis in The Bible Code. Drosnin, a former journalist (seemingly innocent of any training in statistics), aided by the works of a "mathematician," helped "predict" the former Israeli Prime Minister Yitzhak Rabin’s assassination by deciphering a bible code. He informed Rabin, who obviously did not take it too seriously. The Bible Code finds statistical irregularities in the Bible; these help predict some such events. Needless to say, the book sold well enough to warrant a sequel predicting with hindsight even more such events.

The same mechanism is behind the formation of conspiracy theories. Like The Bible Code, they can seem perfect in their logic and can cause otherwise intelligent people to fall for them. I can create a conspiracy theory by downloading hundreds of paintings from an artist or group of artists and finding a constant among all those paintings (among the hundreds of thousand of traits). I would then concoct a conspiratorial theory around a secret message shared by these paintings. This is seemingly what the author of the bestselling The Da Vinci Code did.

Luck and Selection Bias: When Luck Doesn’t Matter

My favorite time is spent in bookstores, where I aimlessly move from book to book in an attempt to make a decision as to whether to invest the time in reading it. My buying is frequently made on impulse, based on superficial, but suggestive clues. Frequently, I have nothing but a book jacket as appendage to my decision making. Jackets often contain praise by someone, famous or not, or excerpts from a book review. Good praise by a famous and respected person or a well-known magazine would sway me into buying the book.

What is the problem? I tend to confuse a book review, which is supposed to be an assessment of the quality of the book, with the best book reviews, marred with the same survivorship biases. I mistake the distribution of the maximum of a variable with that of the variable itself. The publisher will never put on the jacket of the book anything but the best praise. Some authors go even a step beyond, taking a tepid or even unfavorable book review and selecting words in it that appear to praise the book. One such example came from one Paul Wilmott (an English financial mathematician of rare brilliance and irreverence) who managed to announce that I gave him his "first bad review," yet used excerpts from it as praise on the book jacket (we later became friends, which allowed me to extract an endorsement from him for my book).

The first time I was fooled by this bias was upon buying, when I was 16, Manhattan Transfer, a book by John Dos Passos, the American writer, based on praise on the jacket by the French writer and "philosopher" Jean-Paul Sartre, who claimed something to the effect that Dos Passos was the greatest writer of our time. This simple remark, possibly blurted out in a state of intoxication or extreme enthusiasm, caused Dos Passos to become required reading in European intellectual circles, as Sartre’s remark was mistaken for a consensus estimate of the quality of Dos Passos rather than what it was, the best remark. (In spite of such interest in his work, Dos Passos has reverted to obscurity.)

I am frequently asked the question: When is it truly not luck? There are professions in randomness for which performance is low in luck: Like casinos, which manage to tame randomness. In finance? Perhaps. All traders are not speculative traders: There exists a segment called market makers whose job is to derive, like bookmakers, or even like store owners, an income against a transaction. If they speculate, their dependence on the risks of such speculation remains too small compared to their overall volume. They buy at a price and sell to the public at a more favorable one, performing large numbers of transactions. Such income provides them some insulation from randomness. Such category includes floor traders on the exchanges, bank traders who "trade against order flow," moneychangers in the souks of the Levant. The skills involved are sometimes rare to find: Fast thinking, alertness, a high level of energy, an ability to guess from the voice of the seller her level of nervousness; those who have them make a long career (that is, perhaps a decade). They never make it big, as their income is constrained by the number of customers, but they do well probabilistically. They are, in a way, the dentists of the profession.

Regards,

Nassim Nicholas Taleb
for The Daily Reckoning

January 06, 2005

Nassim Nicholas Taleb is an essayist principally concerned with the problems of uncertainty and knowledge. Taleb’s interests lie at the intersection of philosophy, mathematics, finance, literature and cognitive science, but he has stayed extremely close to the ground, thanks to an uninterrupted two-decade career as a mathematical trader. Specializing in the risks of unpredicted rare events ("black swans"), he held senior trading positions in New York and London, before founding Empirica LLC, a trading firm and risk research laboratory. Fooled by Randomness has been published in 14 languages, and the author’s ideas on skeptical empiricism have been covered by hundreds of articles around the world.

That’s it. That’s enough.

We’re tired of this shilly-shallying…malingering…and indecision. We’ve said the Dow is going down. It ought to go down. It has no business doing anything other than going down. So, we’re not going to pay any attention to it until it does what it’s supposed to do. Until then, it will get the cold shoulder from us…

Besides, we were included in an article in this week’s MoneyWeek in a group called, derisively, as the "Stopped Clock Society." We’re only right twice a day, says the article.

Well, we’re such modest people. The thought of being right twice a day sounds like more than enough. If we were right even once a day, we would it a run of extraordinary good luck.

We’ve been saying the Dow would go down for the last five years. Nor have we wavered in our opinion that the price of gold would go up. Generally, we have been right – stocks have gone down; gold has gone up. Being a stopped clock has paid off.

The big trends in markets tend to be long-lived. Interest rate cycles last for 20 or 30 years. Bear markets in stocks tend to last for 15-20 years. You wouldn’t have lost out by selling stocks in 1966, for example, and staying out of the market for the next 15 years. Nor could you have done much better than buy stocks in 1982 – and hold for the next 18 years. Most likely, getting out of stocks in 2000 – and staying out for at least a decade – will turn out to be a shrewd move.

But most analysts like to pretend that they can tell you what will happen from month to month…week to week…even day-to-day! Investors don’t want to hear that they should "sit tight." They don’t want to wait, not when money is burning a hole in their pockets; they want to get rid of it as soon as possible.

Investors did get rid of a little money yesterday. The Dow went down. But not enough. The press, public, economists, analysts and investors are still overwhelmingly bullish…alarmingly complacent…and repulsively cheerful. We can barely stand it.

The LA TIMES reveals, for example, that a poll of 62 economists put next years’ growth rate at a healthy 3.6%. Which raises a number of interesting questions – such as, is a group of economists more likely to be right than a single one? We’ll get to these questions sooner or later…

Today, we turn our suspicious eyes on the consensus view itself. The experts, we are told, expect 2.22 million new jobs in America next year. They even think that the average jobholder may make more next year than he did the last (real incomes fell in 2004). The TIMES article on-line gives us a single example of where these new jobs will arise – in Texas, it turns out, where America’s largest mortgage lender, Countrywide, says it is adding 7,500 new paid positions in the next six years.

That little detail brought a little chuckle here at the Daily Reckoning headquarters. Apparently, the only new jobs the economy creates are sales clerks and mortgage brokers. The latter help people borrow money. The former help them spend it. But where does the money come from? For that, we have to thank the long-suffering Asians. The Chinese, for example, barely have indoor plumbing. But they save 40% of what they earn; lending it out so that Americans can buy a big, new McMansions.

Meanwhile, we’ve found out over the past couple of days that last year set new records for junk bond issuance…and corporate lending. Trillions of dollars changed hands (much of it coming to rest in Wall Street’s pockets). What was all this money for? Do you see new factories springing up in Detroit or Atlanta, dear reader? We hear that young entrepreneurs are already designing successors to the iPod and discovering new ways to steal music and films without paying royalties. But where are the new industries that these trillions of dollars of capital investment are said to be spawning? Reading along, we find that the money will not likely create a single new job. Instead, the loot was mostly used for "mergers and acquisitions," which usually have the effect of reducing employment, not increasing it. M&A activity, as Kurt Richebächer has pointed out, is a sign of late, degenerate capitalism; it is a way corporate hustlers try to increase profits and generate huge financing fees quickly – without actually investing in long-term, productive projects.

This kind of economic "growth" comes at a heavy price – to be paid, eventually, with interest. People are only able to step up spending by going deeper into debt. When the bill finally has to be paid, there will tears aplenty.

But who will pay? How? When?

As always…we can’t wait to find out.

[Ed. Note: Only one thing causes a stock to pop 94%… 102%… 108%… even 250% higher in a matter of days or hours: an unexpected takeover bid. It’s the most powerful force in the market place…

Traditionally, these high-octane profits were only available to Wall St. insider types…until now! The secret formula is up for grabs…learn more about this ground-breaking system…

More news, from our team at The Rude Awakening:

Eric Fry, reporting from Wall Street…

"Meanwhile, without the benefit of a single astrological chart, Merrill Lynch’s chief U.S. strategist, Richard Bernstein, has reached a similarly bearish conclusion. Bernstein, who makes his predictions based upon phenomenon that occur right here on planet earth, believes rising interest rates and waning corporate profits will hobble the stock market in 2005."

————–

Bill Bonner, back in London:

*** "I get that question all the time," said colleague Adrian Ash in the London office. "People want to know how we can square one of the views they see in The Daily Reckoning with one of the views in an ad…or in another product we publish.

"I tell them that we don’t claim to have ‘the Truth.’ We don’t even claim to make their lives easier by giving them alternative views. In fact, what we do is to make their lives harder – by giving them a range of ideas and opinions. They’ve still got to do the hard work of figuring out which one is likely to be right. And if we do our job right – their work is harder than ever, because they will have a variety of very good ideas to choose from."

*** A reader’s letter:

Dear Folks at The Daily Reckoning.

Today I took a look at my 2004 W2, for income and taxes for the past year. Each separate line item alone is enough to make me mad, but when I added up the total I pay as a citizen to live in this weird world, it came in at 34% of my total income. I live in California, so the state income tax is high, and it has it own little social programs. 1/3 of my workday goes to support THE EMPIRE. 8% of my income goes directly to the social programs of Social Security, Medical, and Cal SDI. I don’t know how others feel, but the words that boil into my mind that describe my feelings about this are: ROME, VAMPIRES, DRACULA, PARASITES, LEACHES, BLOODWORMS, AND TICKS. And I think this has a good chance of getting worse with time. Garlic, holy water, and the cross keep vampires away, but it is not that easy when it comes to taxes.

*** So much of what goes on in life is humbug, fraud or poppycock, we keep saying.

According to an article in yesterday’s news, the average person leaves college with $18,900 in student loans. We may not know much about women, but we know a lot about the cost of college. One son has graduated. A daughter is now in college – with a year to go. Another son is putting in his applications.

The average tuition at the schools to which Jules is applying is about $40,000.

"Uh…it’s $50,000 at Columbia Law School," said a friend the other day. She’s just begun her law school career.

Why does college cost so much? We don’t know – but we have a feeling it is a hustle. Most students would learn more simply by reading a good list of books and discussing them with a sharp tutor under a maple tree. If ten students got together and hired a tutor – they might pay, what, $50,000 for a good person? That would put their tuition at $5,000 per year each – with one-tenth of a teacher all to themselves.

But, of course, there would be no billion-dollar research labs, no prestigious Nobel prize winners on the faculty, no football team, no fraternity parties, no sex on campus – okay, forget it…who would want to learn anything under those conditions!

*** "You go to war with the army you’ve got," said Donald Rumsfeld, caught in a rare moment of accidental honesty. "You do that because you don’t usually choose the hour or place of war; it’s chosen for you by the attacker. You are forced to defend yourself with the best army and best equipment on hand at the moment. But this is a different kind of war…in which the U.S. is doing the attacking…and you can’t even tell the U.S. Marines the straight skinny. Instead, you have to fudge and scam them…show compassion…show concern…hide the casualties…bring the coffins in at night…take the costs off-budget…and try to remember why the hell you got into this war in the first place."