What's notable about the meat of the complaint sent to the FDA is that it's less "this technology actually doesn't work" and more "this company has no culture of experimental integrity".
From the article:
"hadn’t fully assembled the proprietary machines used for the herpes study when the experiments began" ... "company underreported the rate at which the machines broke down during the study" ... "employee also alleged that some crucial parts of the devices ... were modified to improve their accuracy"
FDA is generally skeptical of validation data not generated from "production equivalent" technology, but they get even more skeptical if you tell them something is X and it turns out to be Y down the road. That (rightly) sends them on a fishing expedition.
Not necessarily to the former. Definitely to the latter.
To elaborate point 1:
- If the crucial parts of the devices that "were modified to improve accuracy" were modified early on in the study the data could be representative of the final product.
- The same applies for the devices not being fully assembled. If they weren't fully assembled, and the data were still good enough to defend a 510k, then the data should represent worst case usage, unless full assembly would somehow make the technology less performant.
Yes it is always possible their one test works, but reading between the lines it appears that Edison is flaky and hard to get to work consistently. Things like changing the pipet tips halfway through are amateur hour - issues like this should have been picked up in R not D.
> regulatory bodies do not tend to support rapid iteration of products
I disagree. 21 CFR 820 is deliberately written such that it does not prescribe any one specific development process; AAMI TIR-45 for example aligns agile methodologies with medical device software development and is an FDA recognized consensus standard.
What regulatory bodies do demand is that one document what one will do, and then, actually do what one has documented. Sounds like Theranos had trouble with that.
Thank you for pointing this out. You'd be surprised how many people even within medical device think design control necessarily means some slow-as-hell waterfall development method.
I fully understand that part, perhaps the FDA needs to adapt as well. Testing doesn't need to be static, there is room for cooperative certification and corrective actions to insure the test completes. With the rapid advances in some areas of technology you could run into a situation where you obsolete the technology you are working on as certification issues reveal improvements.
While the medical field certainly has additional concerns I do see a time where, with proper processes in place, that the final delivery may not exactly be what entered the certification phase.
>Edison machines would sometimes produce “radically different results” for the same patients, the former employee alleged. Referring to a thyroid test known as thyroid-stimulating hormone, the employee wrote that “a patient would swing between” hypothyroidism and hyperthyroidism, or too little of the hormone and too much, when the test was repeated the same day.
This sounds really bad. When you have this huge amount of uncertainty in results, it calls into question their whole finger prick blood sampling methodology. I'm starting to think that it's impossible to get an accurate blood work from a finger prick only.
>I'm starting to think that it's impossible to get an accurate blood work from a finger prick only.
It is possible, but just for certain test types and error rates. The basic problem is the blood drawn from a finger prick is affected by exactly where it goes in and how much skin tissue is picked up. This variability gets averaged out with a large venous draw and so is not quite as bad - there is still a large variability with even normal blood tests for certain test types.
The tests that work OK are things where you just want a yes/no answer (do you have HSV1) and very poorly for things where you are wanting to measuring exact levels of something.
I assume that is what they hoped to do. Lots of people with decades of experience in this area have tried to do this before without too much success. This doesn't mean it is impossible, but if you are claiming to be able do something that everybody else has tried and failed then your data needs to be impeccable. No hiding behind trade secrets and saying trust us.
It is. As someone who's been through the pain of this exact lab fuck up (though from traditional, not theranos equipment I would assume) being mis-diagnosed and starting medication for this condition incorrectly can be terrible. The cure for Hyperthyroidism is so easy, and the condition so common, that people forget what the effects of the extremes of the conditions are.
There are lots of industries you can disrupt, or fail fast at, but lab testing isn't one of them.
The options include partial thyroidectomy/ radioiodine therapy / carbimazole, propylthiouracil and a few other niche options for use use in special situations like pregnancy associated thyrotoxicosis.
It would be good if there was more clarity on whether the test was repeated on the same specimen or on another specimen collected from the same patient at a different time.
If it is the latter, case marked variations could have a number of explanations, ranging from the inevitable inaccuracy of finger prick collection, through errors due to dilution by Theranos, problems with the technology and equipment and diurnal variation of TSH levels
Compared to the other theranos article on top of HN right now, this one is far more damning. The whistle blower is claiming that even the herpes test, the only test the FDA has approved, had its results altered in some way to pass the FDA's requirements. If this is true, what does theranos even have left?
Strange. Worked for me after I was blocked, same browser. I wonder if you're using Tor, or if they're geotargeting the way they handle google referrers. Or perhaps you're coming from a google.co.XX domain instead of google.com?
The reason for taking larger samples on many tests using "normal" methods is not that the instruments are not sensitive but that one needs to draw enough blood to get a representative sample of what's actually inside your body. A little prick of blood that comes out is often not such a representative sample. I always wondered how Theranos was going to get around this fact of life. Now as things unfold it increasingly sounds like they can't... they just wanted us to think they could.
I predict Theranos will be out of business within 18 months. It's a vaporware company with a cult of personality founder. Hard to pull off with increased media exposure, even harder to pull off with regulatory agencies looking into you.
Update: too bad Theranos isn't publicly traded, I'd love to be able and bet against them.
So we have a 12 year old company here, developing a machine to test blood (which is altered for testing), has only received approval for one test, which is now being doubted again, has repeatedly lied about their technology and capability.
I don't give one sordid fuck about Holmes, let's talk about Theranos. How is this a successful business in ANY capacity? Has it made a DIME that didn't come from an investor? Does it have ANY reputation that doesn't involve a spunky CEO?
Falsifying data to obtain regulatory approval is definitely illegal, and federal regulations allow the government to hold corporate officers personally liable.
It has yet to be proven that Theranos did in fact falsify data, though.
Given how things are looking right now, Elizabeth might want to start preparing a shipment of orange turtlenecks to be delivered to her in the big house.
Not a lawyer but "the study submitted by Theranos last year to win the agency’s approval for a herpes test was tainted by breaches in research protocol" sounds iffy.
So conflicted about all of the rage and drama around Theranos. On the one hand, it smacks of the conventional backlash against solutions that revolutionize the way things have been done forever, and there needs to be space for innovators to work through kinks to really bring an idea to market. On the other hand, it sounds like this product does not work (or does not yet work well) and it's not the kind of solution space where errors can be brushed off lightly.
Well, let's see how she responds to this Federal investigation that essentially accuses her of being a fraud. If her company really did fudge FDA testing results then that's beyond bad.
From the article:
"hadn’t fully assembled the proprietary machines used for the herpes study when the experiments began" ... "company underreported the rate at which the machines broke down during the study" ... "employee also alleged that some crucial parts of the devices ... were modified to improve their accuracy"
FDA is generally skeptical of validation data not generated from "production equivalent" technology, but they get even more skeptical if you tell them something is X and it turns out to be Y down the road. That (rightly) sends them on a fishing expedition.