• The problem with gold standards

    In medicine, like in many things, we are always on the hunt for truth. But in medicine, like in many things, what is “true” is never really clear. People are often believe (incorrectly) that when we do tests, they are binary. They believe that we get results that tell us “positive” or “negative”. It’s rarely that clear.

    Most of the time, we just line up all the values that we get from a lot of people, and decide – a priori – that the outlying 10% are abnormal. It’s that simple. Sometimes it’s 5%. Sometimes less. But that’t not perfect. So we conduct further analyses to try and increase what we call the sensitivity and specificity of tests. We try and strike a balance against being too permissive in our diagnoses versus missing any real cases.

    In this hunt, we always have to have a “gold standard”. That’s the absolute best test that tells us whether something is true or not. Sometimes, that’s something you can only tell in an autopsy (which doesn’t work in real care). Sometimes it’s a slow and very expensive test. But it’s always our best method for getting at the “truth”.

    I bring this up because Andrew Sullivan pointed out an awesome piece in Wired* this morning that gets at the gold standard for weights. Think about it. Every scale in the world has to be calibrated against something. Every weight, every single one of them, has to be measured against something. It turns out there is one weight to rule them all:

    Familiarly known as Le Grand K and held in a vault just outside of Paris under three bell jars, it dates back to the 1880s, when it was forged by the British metallurgist George Matthey from an alloy of nine-tenths platinum and one-tenth iridium. As a metric unit, the kilogram is “equal to the mass of the international prototype,” according to the official definition. In other words, as metrologists like to point out, it has the remarkable property of never gaining or losing mass. By definition, any physical change to it alters the mass of everything in the cosmos.

    Aside from a yearly ceremonial peek inside its vault, which can be unlocked only with three keys held by three different officials, the prototype goes unmolested for decades. Yet every 40 years or so, protocol requires that it be washed with alcohol, dried with a chamois cloth, given a steam bath, allowed to air dry, and then weighed against the freshly scrubbed national standards, all transported to France. It is also compared to six témoins (witnesses), nominally identical cylinders that are stored in the vault alongside the prototype. The instruments used to make these comparisons are phenomenally precise, capable of measuring differences of 0.0000001 percent, or one part in 1 billion.

    The rest of the piece talks about the fact that Le Grand K seems to be losing mass over time. Not a lot, but some amount. Since it is the gold standard, this means that, by definition, the rest of the universe is gaining weight. Scientists are hard at work trying to figure out a new gold standard, not dependent on 19th century technology. It’s fascinating. Go read it.

    *Full disclosure: This piece was written by Jonathon Keats, who was in my class at Amherst College. Since there are only about 400 students in each class there, you get to know pretty much everyone, and I remember him pretty well. I mention this only because I try and disclose any personal connections between me and people whom I write about.

    Share
    Comments closed