Download: Kane G, Kane E. The High Reported Accuracy Of The Standardized Field Sobriety Test Is A Property Of The Statistic, Not Of The Test. Law, Probability and Risk, 29 November 2021
Or access at the journal: https://doi.org/10.1093/lpr/mgab004

At a BAC cutoff of 0.08%, the Likelihood Ratio of the SFST is 1.4.

CONCLUSION
The statistics overall accuracy and arrest accuracy to not quantify the probability that impaired driving defendants who failed the SFST had an elevated BAC or were impaired.

 

Your Best Source for SFST Science..
SFST.US is your best resource for understanding the science underlying US law enforcement's Standardized Field Sobriety Test. We'll focus on the question relevant to DUI prosecutions:

Given that our driver failed the scientific SFST, what is the probability she was impaired by alcohol?

DRE officers are trained to testify "93%." Or 91%, or 95%. Those are the SFST accuracies reported in NHTSA's scientific SFST validation studies. They all sound pretty good, which is why defense attorneys don't ask officers the accuracy question.

Here's the plot line for this entire web site: Those 90-something percent accuracies do not answer our question.

I'm not going to teach you to prove the lady's SFST doesn't count because her shoes were pointy or the officer waved the HGN pen left before he waved it right. This site isn't about how SFST measurements are made. This site is about what SFST measurements mean. The driver failed the SFST test. What does that tell us about her?

Most folks find this surprising. It is counterintuitive. Over and over scientific research discovered the accuracy of the SFST is 91 to 95%, so why isn't the accuracy of the SFST administered to our defendant 91 to 95%? The quick answer is "accuracy" is more complicated than you think. NHTSA'S "accuracy" statistic is internally valid to the studies, but not externally valid to our case. The longer answer is what this web site is about.

Why Accuracy?
The SFST is a test. No test is perfect. All tests make mistakes. Some tests make a lot of mistakes, some make hardly any. But they all make some. Any time any test gives a result, the truth is that result could be correct or it could be exactly and completely the opposite of correct.

The SFST makes mistakes. One mistake the SFST can make is to say "guilty" The SFST gave the wrong answer, doesn't mean the scientific test was tricked by driverly imperfection. The lady's HGN wasn't cause by Ménière's disease. Her WAT score wasn't caused by a gimp leg. She is a normal healthy innocent  woman, and the SFST mistakenly said she was guilty. The test failed. The SFST was wrong. when the person is innocent. The driver failed the SFST. What does that tell us about her? It tells us she is either guilty or innocent. She's guilty if the imperfect SFST gave the right answer. She's innocent if the imperfect SFST gave the wrong answer. Was our lady's SFST right or wrong? We're back to our theme question: Given that our driver failed the scientific SFST, what is the probability she was impaired by alcohol?

The simple and obvious answer is to consider the accuracy of the test. If research scientists did the SFST over and over and found that the SFST gave the correct answer 95% of the time we'll probably rely on it. If they found it was correct 5% of the time, we probably won't. Simple? Simple.

Simple, but it doesn't work. This is one of those times the simple and obvious answer is wrong. It turns out:

  • There are two different kinds of medical tests: direct and indirect.
  • Each kind of test has it's own formula for accuracy. If you mix formulas, if you use the wrong formula for your kind of test, the answer you get will be wrong.
  • For the SFST, NHTSA uses the wrong formula.
  • Using the wrong formula, the accuracy NHTSA calculates for the SFST is spectacularly wrong.

So what?
 The accuracy of the SFST is an essential element of the theory of DUI prosecutions. In DUI cases involving an SFST, the defendant generally took an SFST and failed. At trial that failure is introduced as scientific evidence she was impaired. And the way we know that scientific evidence is valid and reliable is, when NHTSA contractors did the SFST over and over, they found it gave the correct answer 91%, 93%, 95% of the time. So when our officer does the same SFST the same way the researcher did, he'll be correct the same percent of the time the researchers were—95%. The probability our lady's SFST was correct is 95%.

But if that 95% research statistic was calculated using the wrong formula, then in our case it is not valid. The accuracy of our lady's SFST is not 95%. Wow. So what is it? What is the accuracy of the SFST for our lady? The prosecution has no answer. There is no answer. For forty years every time NHTSA has studied the SFST it has used the wrong test accuracy formula. After forty years of un-peer-reviewed, in-house "scientific" projects done by hand-picked contractors with a financial interest in pleasing the agency, NHTSA has provided no scientific basis for stating any percentage as the accuracy of SFSTs done in the field.

As far as the science goes, accuracy of our lady's SFST could by anything. Or nothing. The SFSTs used by US law enforcement are accurate in the same way that astrology, phrenology and palm reading are accurate. They vaguely make sense, and everyone who relies on them seems to think they work.

Dessert first
 I'll explain indirect tests—commonly called "diagnostic tests"—at the Accuracy tab. For now let's jump to the final answer. Mathematically the accuracy of the SFST test at identifying drunkenness is the same as (actually less than ) the accuracy of an airport security detector test at identifying guns. It makes as much sense to arrest people for being uncoordinated as it does to arrest people for making a metal detector beep.

Metal detector as gun detector

Someone noticed everyone with a gun makes the metal detector beep. Hey, this thing isn't a metal detector, it's a GUN detector!

Let's arrest anyone who makes the gun detector beep!

Coordination test as alcohol detector

Somebody noticed that everyone who is drunk is uncoordinated. Hey, lets use a coordination test to arrest drunks!

Let's arrest everyone who fails our coordination test.

Measure: metal
Care about: gun

Theory of Gun Detectors
  1. Everyone with a gun sets off the metal detector.
  2. That man set off the metal detector, therefore he has a gun.

Measure: coordination
Care about: high alcohol

Theory of SFSTs
  1. Everyone who is drunk is uncoordinated.
  2. That man is uncoordinated, therefore he is drunk.

Everyone with a gun sets off the metal detector, but most people who set off the metal detector do not have a gun.

When travelers set off a metal detector, we don't arrest and convict them of gun smuggling and terrorism. Everyone with a gun sets off the metal detector, but most people who set off the metal detector do not have a gun.

Metal detectors are highly accurate at spotting people who do have a gun, but they are not especially good at telling the difference between gun and no-gun. On innocent people metal detectors often give the wrong answer. They ring for keys and phones and belt buckles, for lots of stuff.

Everybody who is drunk is uncoordinated, but most people who are uncoordinated are not drunk.

SFSTs work the same way. It is true that everyone who is drunk is uncoordinated. But it is not true that everyone who is uncoordinated is drunk. If you arrest people for being uncoordinated, you'll end up arresting, and convicting, lots of people who are innocent.

Read this web site while you can. Best I can tell, I am under ongoing threat from NHTSA contract SFST scientist Dr. Jack Stuster for exposing the scientific errors you are reading about here.

This web site is about science— NHTSA's SFST validation science. I do not know, I do not care, I do not have an opinion about Dr. Jack Stuster's knowledge or intentions at any time ever in his life. I'm not even saying he had knowledge or intentions. But if he did, this web site isn't about them. Or him.

Top

 

.