Minding the reality gap

Minding the Gap
Matt Asher - Probability and statistics blog

Officially, unemployment in the US is declining. It’s fallen from a high of 9.1% a couple years ago, to 7.8% in recent months. This would be good news, if the official unemployment rate measured unemployment, in the everyday sense of the word. It doesn’t. The technical definition of “U3″ unemployment, the most commonly reported figure, excludes people who’ve given up looking for work, those who’ve retired early due to market conditions, and workers so part time they clock in just one hour per week.

Most critically, unemployment excludes the 14 million American on disability benefits, a number which has quadrupled over the last 30 years. If you include just this one segment of the population in the official numbers, the unemployment rate would double. On Saturday, This American Life devoted their entire hour to an exploration of this statistic. Russ Robert’s, who’s podcast I’ve recommend in the past, discussed the same topic last year. Despite the magnitude of the program and the scale of the change, these are the only outlets I know of to report on the disability number, and on the implications it has for how we interpret the decline in U3 unemployment.

Targeting the number, not the reality

Statistics, in the sense of numerical estimates, are measures which attempt to condense the complex world of millions of people into a single data point. Honest statistics come with margins of error (the most honest indicate, at least qualitatively, a margin of error for their margin of error). But even the best statistical measures are merely symptoms of some underlying reality; they reflect some aspect of the reality as accurately as possible. The danger with repeated presentations of any statistic (as in the quarterly, monthly, and even hourly reporting of GDP, unemployment, and Dow Jones averages), is that we start to focus on this number by itself, regardless of the reality it was created to represent. It’s as if the patient has a high fever and all anyone talks about is what the thermometer says. Eventually the focus becomes, “How do we get the thermometer reading down?” All manner of effort goes into reducing the reading, irrespective of the short, and certainly long-term, health of the patient. When politicians speak about targeting unemployment figures, this is what they mean, quite literally. Their goal is to bring down the rate that gets reported by the Bureau of Labor Statistics, the number discussed on television and in every mainstream source of media.

Politicians focus on high profile metrics, and not the underlying realities, because the bigger and more complicated the system, the easier it is to tweak the method of measurement or its numeric output, relative to the difficulty of fixing the system itself. Instead of creating conditions which allow for growth in employment (which would likely require a reduction in politicians’ legislative and financial powers), the US has quietly moved a huge segment of its population off welfare, which counts against unemployment, and into disability and prisons — the incarcerated also don’t count in U3, whether they are slaving away behind bars or not.

How metrics go bad

Over time, all social metrics diverge from the reality they were created to reflect. Sometimes this is the result of a natural drift in the underlying conditions; the metric no longer captures the same information it had in the past, or no longer represents the broad segment of society it once did. For example, the number of physical letters delivered by the postal service no longer tracks the level of communication between citizens.

Statistics and the reality they were designed to represent are also forced apart through deliberate manipulation. Official unemployment figures are just one example of an aggressively targeted/manipulated metric. Another widely abused figure is the official inflation rate, or core Consumer Price Index. This measure excludes food and energy prices, for the stated reason that they are highly volatile. Of course, these commodities represent a significant fraction of nearly everyone’s budget, and their prices can be a leading indicator of inflation. The CPI also uses a complex formula to calculate “hedonics,” which mark down reported prices based on how much better the new version of a product is compared to the old one (do a search for “let them eat iPads”).

I don’t see it as a coincidence that unemployment and inflation figures are among the most widely reported and the most actively manipulated. In fact, I take the following to be an empirical trend so strong I’m willing to call it a law: the greater the visibility of a metric, the more money and careers riding on it, the higher the likelihood it will be “targeted.” In this light, the great scandal related to manipulation of LIBOR, a number which serves as pivot point for trillions of dollars in contracts, is that the figure was assumed to be accurate to begin with.

Often the very credibility of the metric, built up over time by its integrity and ability to reflect an essential feature of the underlying reality, is cashed in by those who manipulate it. Such was the case with the credit ratings agencies: after a long run of prudent assessments, they relaxed their standards for evaluating mortgage bundles, cashing in on the windfall profits generated by the housing bubble.

Why we don’t see the gaps

It might seem like the disconnect between a statistic and reality would cause a dissonance that, once large enough to be clearly visible, would lead to reformulation of the statistic, bringing it back in line with the underlying fundamentals. Clearly there are natural pressures in that direction. For example, people laid off at the beginning of a recession are unlikely to believe that the recovery has begun until they themselves go back to work. Their skepticism of the unemployment figure erodes its credibility. Unfortunately, two powerful forces work against the re-alignment of metric and reality: the first related to momentum and our blindness to small changes, the second having to do with the effects of reflexivity and willful ignorance.

In terms of inertia, humans have a built-in tendency to believe that what has been will continue to be. More sharply, the longer a trend has continued, the longer we presume it will continue — if it hasn’t happened yet, how could it happen now? Laplace’s rule of succession is our best tool for estimating probabilities under the assumption of a constant generating process, one that spits out a stream of conditionally independent (exchangeable) data points. But the rule of succession fails utterly, at times spectacularly, when the underlying conditions change. And underlying conditions always change!

These changes, when they come slowly, pass under our radar. Humans are great at noticing large differences from one day to the next, but poor at detecting slow changes over long periods of time. Ever walked by an old store with an awning or sign that’s filthy and falling apart? You wonder how the store owner could fail to notice the problem, but there was never any one moment when it passed from shiny and new to old and decrepit. If you think you’d never be as blind as that shop keeper, look down at your keyboard right now. As with our environment, if the gap between statistic and reality changes slowly, over time, we may not see the changes. Meanwhile, historical use of the statistic lends weight to it’s credibility, reducing the chance that we’d notice or question the change — it has to be right, it’s what we’ve always used!

The perceived stability of slowly changing systems encourages participants to depend on or exploit it. This, in turn, can create long term instabilities as minor fluctuations trigger extreme reactions on the part of participants. Throughout the late 20th century and the first years of the 21st, a large number of investors participated in the “Carry Trade,” a scheme which depended on the long term stability of the Yen, and of the differential between borrowing rates in Japan and interest rates abroad. When conditions changed in 2008, investors “unwound” these trades at full speed, spiking volatility and encouraging even more traders to exit their positions as fast as they could.

These feedback loops are an example of reflexivity, the tendency in some complex systems for perception (everyone will panic and sell) to affect reality (everyone panics and sells). Reflexivity can turn statistical pronouncements into self-fulfilling prophecies, at least for a time. The belief that inflation is low, if widespread, can suppress inflation in and of itself! If I believe that the cash in my wallet and the deposits in my bank account will still be worth essentially the same amount tomorrow or in a year, then I’m less likely to rush out to exchange my currency for hard goods. Conversely, once it’s clear that my Bank of Zimbabwe Bearer Cheques have a steeply declining half-life of purchasing power, then I’m going to trade these paper notes for tangible goods as quickly as possible, nominal price be damned!


Don’t look down

If perception can shape reality, then does the gap between reality and statistic matter? Clearly, the people who benefit most from the status quo do their best to avoid looking down, lest they encourage others to do the same. More generally, though, can we keep going forward so long as we don’t look down, like Wile E. Coyote chasing the road runner off a cliff?

The clear empirical answer to that questions is: “Yes, at least for a while.” The key is that no one knows how long this while can last, nor is it clear what happens when the reckoning comes. Despite what ignorant commentators might have said ex post facto, by 2006 there was wide understanding that housing prices were becoming un-sustainably inflated. In 2008, US prices crashed back down to earth. North of the border, in Canada, the seemingly equally inflated housing market stumbled, shrugged, then continued along at more level, but still gravity-defying trajectory.

The high cost of maintaining the facade

Even as the pressures to close the gap grow along with its size, the larger the divergence between official numbers and reality, the greater the pressures to keep up the facade. If the fictional single entity we call “the economy” appears to be doing better, politicians get re-elected and consumers spend more money. When the music finally stops, so too will the gravy-train for a number of vested interests. So the day of reckoning just keeps getting worse and worse as more and more resources go into maintaining the illusion, into reassuring the public that nothing’s wrong, into extending, pretending, and even, if need be, shooting the messenger.

It’s not just politicians and corporations who become invested in hiding and ignoring the gap. We believe official statistics because we want to believe them, and we act as if we believe them because we believe that others believe them. We buy houses or stocks at inflated prices on the hope that someone else will buy them from us at an even more inflated price.

My (strong) belief is that most economic and political Black Swans are the result of mass delusion, based on our faith in the quality and meaning of prominently reported, endlessly repeated, officially sanctioned statistics. The illustration at the beginning of this post comes from a comic I authored about a character who makes his living off just this gap between official data and the reality on the ground, a gap that always closes, sooner or later, making some rich and toppling others.

Why Jurors Should Refuse to Convict Drug Arrestees

Jurors are not required to enforce unjust laws, and can legally refuse to convict a person, even if he or she appears to be guilty. 

By Tony Newman, AlterNet
Posted on December 22, 2011

 Should juries vote “not guilty” on low-level marijuana charges to send a message about our country’s insane marijuana arrest policy? Paul Butler, former federal prosecutor and law professor of George Washington University made a powerful argument for jury nullification in an op-ed in Tuesday’s New York Times.

Jury nullification is a constitutional doctrine that allows juries to acquit defendants who are technically guilty, but who don’t deserve punishment. Butler explained that juries have the right and power to use jury nullification to protest unjust laws.

Mr. Butler points out that nullification was credited with ending our country’s disastrous alcohol prohibition as more and more jurors refused to send their neighbors to jail for a law they didn’t believe in. Butler says we need to do the same with today’s marijuana arrests.

There is growing recognition that today's drug laws are ineffective and unfair. For the first time ever, a recent Gallup poll found that 50 percent of Americans want legalize use of marijuana. Despite half of our country wanting to end marijuana prohibition, the war on marijuana users is as vicious as ever. There were more than 750,000 arrests last year for marijuana possession alone. In New York City, marijuana possession was the #1 reason people were arrested last year; making up 15 percent of all arrests.

People hoping for change should not expect it to come from our “leaders” in Washington. While most of our elected officials know in their hearts that our drug war is an utter failure that fills our prisons while doing nothing to help people struggling with addiction, there is deafening silence when it comes to offering alternatives to the war on drugs. Democrats and Republicans are both cowardly and opportunistic and don’t want to give up their “tough on crime” credentials.

Here is where jury nullification comes in. If our leaders are going to stop the madness, maybe it is up to our peers to say enough is enough.

In Montana last year, a group of five prospective-jurors said they had a problem with someone receiving a felony for a small amount of marijuana. The prosecutors were freaked out about the “Mutiny in Montana” and were afraid they were not going to be able convince12 jurors in Montana to convict. The judge said, in a major New York Times article, “I’ve never seen this large a number of people express this large a number of reservations” and “it does raise a question about the next case.”

The highest profile group to call for jury nullification for drug offenses is from the creators of the HBO hit series The Wire. David Simon and the other creators of The Wire wrote a passionate piece in Time magazine where they called on Americans to join them in the use of jury nullification as a strategy to slow the drug war machine.

From the article:

"A long habit of not thinking a thing wrong, gives it a superficial appearance of being right," wrote Thomas Paine when he called for civil disobedience against monarchy — the flawed national policy of his day. In a similar spirit, we offer a small idea that is, perhaps, no small idea. It will not solve the drug problem, nor will it heal all civic wounds. It does not yet address questions of how the resources spent warring with our poor over drug use might be better spent on treatment or education or job training, or anything else that might begin to restore those places in America where the only economic engine remaining is the illegal drug economy. It doesn't resolve the myriad complexities that a retreat from war to sanity will require. All it does is open a range of intricate, paradoxical issues. But this is what we can do — and what we will do.

If asked to serve on a jury deliberating a violation of state or federal drug laws, we will vote to acquit, regardless of the evidence presented. Save for a prosecution in which acts of violence or intended violence are alleged, we will — to borrow Justice Harry Blackmun's manifesto against the death penalty — no longer tinker with the machinery of the drug war. No longer can we collaborate with a government that uses nonviolent drug offenses to fill prisons with its poorest, most damaged and most desperate citizens.”

Forty years after President Richard Nixon launched the “war on drugs” the casualties continue to mount with no end in sight. We need to step up our efforts to end this war at home and stop sending our loved ones to cages because they have a drug problem. We have more power than we realize. If the people lead, the leaders will follow.

The FBI and the Myth of Fingerprints

A "100 Per Cent Certainty"

by ALEXANDER COCKBURN and JEFFREY ST. CLAIR

Few law enforcement institutions have been so thoroughly discredited in recent years as the FBI’s forensic lab. In 1997 the Bureau’s inspector general of the time issued a devastating report, stigmatizing one instance after another of mishandled and contaminated evidence, inept technicians, and outright fabrication. The IG concluded that there were "serious and credible allegations of incompetence" and perjured courtroom testimony.

CounterPunch’s view is that taken as a whole, forensic evidence as used by prosecutors is inherently untrustworthy. For example, for years many people went to prison on the basis of the claims of a North Carolina anthropologist, Louise Robbins. She helped send people to prison or to Death Row with her self-proclaimed power to identify criminals through shoe prints. As an excellent recent Chicago Tribune series on forensic humbug recalled, on occasion she even said she could use the method to determine a person’s height, sex and race. Robbins died in 1987, her memory compromised by the conclusion of many Appeals Courts that her methodology was bosh. There have been similarly hollow claims for lip prints and ear prints, all of (added "of") them invoked by their supporters as "100 per cent reliable" and believed by juries too easily impressed by passionate invocations to 100 per cent reliable scientific data.

Of course the apex forensic hero of prosecutors, long promoted as the bottom line in reliability–at least until the arrival of DNA matching–has been the fingerprint.

Fingerprints entered the arsenal of police and prosecutors in the late nineteenth century, touted as "scientific" in the manner of other fashionable methods of that time in the identification of supposed criminals, such as phrenology. A prime salesman was Francis Galton, Charles Darwin’s cousin and a founding huckster for the bogus "science " of eugenics. Actually fingerprints, at least in modern times, found their original use in the efforts of a British colonial administrator to intimidate his Indian laborers (whose faces he could not distinguish) from turning up more than once to get paid. He’d make a great show of scrutinizing the fingerprints he insisted they daub on his ledger book. Then, as now, the use of the so-called "unique fingerprint" has been histrionic , not scientific. In 1995, so the Chicago Tribune series discovered, " one of the only independent proficiency tests of fingerprint examiners in U.S. crime labs found that nearly a quarter reported false positives, meaning they declared prints identical even though they were not–the sort of mistakes that can lead to wrongful convictions or arrests."

Decade after decade people have been sent to prison for years or dispatched to the death cell, solely on the basis of a single, even a partial print. So great is the resonance of the phrase "a perfect match" that defense lawyers throw in the towel, as judge and jury listen to the assured conclusions of the FBI’s analysts who virtually monopolize the fingerprint industry in the U.S.A. Overseas, in London’s Scotland Yard for example, the same mesmerizing "certainty" held sway, and still does. In the U.S.A., part of the mystique stems from the "one discrepancy rule" which has supposedly governed the FBI’s fingerprint analysis. The rule says that identifications are subject to a standard of "100 per cent certainty" where a single difference in appearance is supposed to preclude identification.

The 1997 lab scandals threw a shadow over the FBI’s forensic procedures as a whole and the criminal defense bar began to raise protests against prosecutorial use of latent fingerprint identification evidence, as produced by FBI procedures. In 2002 Judge Louis Pollak, in a case in Pennsylvania, initially ruled that the FBI’s fingerprint matching criteria fell below new standards of forensic reliability (the Daubert standards) stipulated by the U.S. Supreme Court. Ultimately he was persuaded that the FBI’s fingerprint lab had never made a mistake. In 2004, in U.S. v. Mitchell, the Third Circuit Court of Appeals upheld these same procedures.

Now at last, in 2006, the FBI’s current inspector general, Glenn Fine, has grudgingly administered what should properly be regarded as the deathblow to fingerprint evidence as used by the FBI and indeed by law enforcement generally.

The case reviewed by Inspector General Fine, at the request of U.S. Rep John Conyers and U.S. Senator Russell Feingold, concerns the false arrest by the FBI of Brandon Mayfield, a lawyer from Beaverton, Oregon.

On March 11, 2004, several bombs exploded in Madrid’s subway system with 191 killed and 1,460 injured. Shortly thereafter the Spanish police discovered a blue plastic bag filled with detonators in a van parked near the Acala de Heres train station in Madrid, whence all of the trains involved in the bombing had originated on the fatal day.

The Spanish police were able to lift a number of latent prints off the bag. On March 17 they transmitted digital images of these fingerprints to the FBI’s crime lab in Virginia. The lab ran the images through its prized IAFIS, otherwise known as the integrated, automated, fingerprint identification system, containing a database of some 20 million fingerprints.

The IAFIS computer spat out twenty "candidate prints", with the warning that these 20 candidates were "close non-match". Then the FBI examiners went to work with their magnifying glasses, assessing ridges and forks between the sample of 20 and the images from Spain. In a trice the doubts of the IAFIS computer were thrust aside, and senior fingerprint examiner Terry Green determined that he had found "a 100 per cent match" with one of the Spanish prints of the fourth-ranked print in the IAFIS batch of 20 close non-matches. Green said this fourth ranked print came from the left index finger of Brandon Mayfield. Mayfield’s prints were in the FBI’s master file, not because he had been arrested or charged with any crime, but because he was a former U.S. Army lieutenant.

Green submitted his conclusions to two other FBI examiners who duly confirmed his conclusions. But as the inspector general later noted, these examiners were not directed to inspect a set of prints without knowing that a match had already asserted by one of their colleagues. They were simple given the pair of supposedly matched prints and asked to confirm the finding. (These two examiners later refused to talk to the FBI’s inspector general.)

The FBI lost no time in alerting the U.S. Prosecutor’s office in Portland, which began surveillance of Mayfield with a request to the secret FISA court which issued a warrant for Mayfield’s phone to be tapped on the grounds, laid out in the Patriot Act, that he was a terrorist, and therefore by definition a foreign agent.

Surreptitious tapping and surveillance of Mayfield began. On April 2, 2004, the FBI sent a letter to the Spanish police informing them that they had a big break in the case, with a positive identification of the print on the bag of detonators.

Ten days later the forensic science division of the Spanish national police sent the FBI its own analysis. It held that the purported match of Mayfield’s print was "conclusively negative". (The inspector general refers to this as the "negativo Report".)

The next day, April 14, the U.S. Prosecutor in Portland became aware of the fact that the Spanish authorities were vigorously disputing the match with Mayfield’s left forefinger. But by now the Prosecutor and his team were scenting blood. Through covert surveillance they had learned that Mayfield was married to an Egyptian woman, had recently converted to Islam, was a regular attendee at the Bailal mosque in Portland, and had as one of his clients in a child custody dispute an American Muslim called Jeffrey Battle. Battle, a black man, had just been convicted of trying to go to Afghanistan to fight for the Taliban.

Armed, so they thought, with this arsenal of compromising detail, the U.S. Prosecutor and the FBI had no patience with the pettifogging negativism of the Spanish police. So confident were the Americans of the guilt of their prey that they never went back to take another look at the supposedly matching prints. Instead, on April 21, they flew a member of the FBI’s latent print unit to Spain for on-the-spot refutation of the impertinent Madrid constabulary.

The Inspector General’s report makes it clear that the FBI man returned from Spain with a false account of his reception, alleging that the Spanish fingerprint team had bowed to his superior analytic skills. The head of the Spanish team, Pedro Luis Melida-Weda, insists that his team remained entirely unconvinced. "At no time did we give our approval. We refused to validate the FBI’s conclusions. We kept working on the identification."

By now either the U.S. Attorney’s office or, more likely, the FBI was leaking to the press news of the pursuit of a U.S. suspect in the Madrid bombing. But they knew that the actual evidence they had on Mayfield was virtually non-existent, aside from the fingerprint. On May 6, the U.S. Prosecutor in Portland told U.S. District Court Judge Robert Jones that the Spanish police had ultimately accepted the FBI’s match, that Mayfield, alerted by the stories in the press about an unnamed suspect, might start destroying evidence, and that, therefore, they wanted to seize Mayfield, using the now favored charge du jour of the war on terror, claiming him to be a "material witness". Judge Jones okayed an arrest warrant.

Mayfield had no idea that the FBI had been tapping his phones and secretly rummaging through his office. The first time he became aware that he was a citizen under suspicion was on the afternoon of May 6. On that day eight FBI agents showed up at his law office, seized him, cuffed his hands behind his back, ridiculed his protestations. As they approached the door, Mayfield implored them to take the handcuffs off, saying he didn’t want his clients or staff to see him in this condition. The FBI agents said derisively, "Don’t worry about it. The media is right behind us."

Mayfield ended up with two federal public defenders, Steven Wax and Christopher Schatz. Like many such, these two were dedicated to their interest of their client, tireless and resourceful. Their first concern was to get Mayfield out on the Multnomah federal detention center in downtown Portland. Though jailed under an alias chosen for him by the U.S. Prosecutor, the feds had immediately leaked this alias–Randy Barker–to The Oregonian newspaper, and a guard at the jail had promptly roughed up Mayfield.

The two public defenders went before Judge Jones and asked that as a material witness he be kept under house arrest, there being scant apparent evidence against him. Judge Jones finally compelled the U.S. Prosecutor to say what evidence he had against Mayfield. A fingerprint, said the U.S. Prosecutor, withholding from the court the fact that this fingerprint was highly controversial and had been explicitly disqualified by the Spanish police.

The federal defenders questioned the imprisonment of their client, faced penalties of the utmost gravity, on the basis of a fingerprint. Judge Jones allowed as how he had sent people to prison for life on the basis of a single fingerprint. Mayfield’s attorneys asked to see a copy of the allegedly matched fingerprints and have them evaluated by their own expert witness. Knowing he was on thin ice the U.S. Prosecutor refused, claiming it was an issue of national security. Under pressure from Judge Jones, himself pressured by the assiduous federal defenders, the U.S. Prosecutor finally agreed he would give the prints to an independent evaluator selected by Judge Jones.

The prints were given to Kenneth R. Moses of San Francisco, an SFPD veteran who runs a company called Forensic Identification Services which, among other things, proclaims its skills in "computer enhancement of fingerprints". It was "quite difficult", Moses said, because of "blurring and some blotting out", but yes, the FBI had it right, and there was "100 per cent certainty" that one of the prints on the blue bag in Madrid derived from the left index finger of Brandon Mayfield.

Moses transmitted this confident opinion by phone to Judge Jones on the morning of May 19. Immediately following Moses’ assertion, the U.S. attorney stepped forward to confide to Judge Jones dismaying news from Madrid from the Spanish police that very morning. The news "cast some doubt on the identification". This information, he added, "was classified or potentially classified".

The prosecutors then huddled with the judge in his chambers. After 20 minutes, Judge Jones stormed back out and announced that the prosecutors needed to tell the defense lawyers what they had just told him. The prosecutor duly informed the courtroom that the Spanish police had identified the fingerprint as belonging to the right middle finger of Ouhnane Daoud, an Algerian national living in Spain. Daoud was under arrest as a suspect in the bombing. Judge Jones ordered Mayfield to be freed. The U.S. prosecutor said he should be placed under electronic monitoring, a request which the judge turned down.
Four days later, on May 24, the warrant for his detention was dismissed.

The FBI sent two of their senor fingerprint analysts to Spain on a mission to salvage the Bureau from humiliation. The two analysts did their best, returning with the claim that the fingerprint sent to the FBI by the Spanish police was of "no value for identification purposes", a claim which the inspector general later shot down by pointing that only a few weeks thereafter the FBI’s latent fingerprint unit concurred with the Spanish national police lab’s determination that the print on the bag matched the right middle finger of Ouhnane Daoud.

The FBI lab fought an increasingly desperate rearguard battle, eventually claiming that it had been the victim of an excessive reliance on technology. The inspector general points out that the only investigator in the FBI’s lab to emerge with any credit is in fact the IAFIS computer that had stated clearly, "close, no match".

The inspector general writes the bottom line on the "science" of fingerprint matching He gets the FBI’s top examiner to admit that if Mayfield had "been like the Maytag repair man" and not a Muslim convert married to an Egyptian, "the laboratory might have revisited the identification with more skepticism."

And Daoud’s fingerprint match? We don’t know, but if he was convicted on the basis of fingerprints alone, we would say there is grounds for an appeal.