Saturday, August 25, 2012

Owning a Sequencer is NOT a Prerequisite for Sequencing!

Besides the little cancer genomics piece yesterday, another genomics paper getting quite a bit of popular press attention is the nice work from the NIH tracking down a deadly outbreak of drug-resistant Klebsiella pneumoniae, with 11 of 18 infected patients dying.  Rapid genome sequencing provided a much higher level of detail for tracing the outbreak than older methods, even distinguishing isolates taken from different sites on the same patient.

There's a decent piece in the Wall Street Journal on it which I'd like to focus on, because while it doesn't outright get anything grossly wrong, there's definitely a background suggestion which I think is wrong. 

The issue is whether this sort of infectious disease monitoring will become routine or not and how soon, and what would it cost:

The NIH team used sequencing devices made by drug and diagnostics maker Roche Holding AG ROG.VX +1.39% and Illumina Inc., ILMN -0.07% a gene-sequencing company, that cost between $125,000 and $500,000 each, Dr. Segre said. She said the team spent an additional $2,000 a specimen to sequence 18 samples for the study.
Until the technique is proven effective in more outbreaks, it will not be clear the benefits outweigh the time and costs that come with the genetic tools, said William Jarvis, a hospital-infection consultant and former CDC official. It's "hard to predict the value of this technology," he said.
Unsurprisingly to anyone familiar with this blog, I think this approach should be implemented, in an appropriate and staged manner, ASAP.  Small-scale outbreaks like this are bad enough; a new SARS or AIDS could be identified so much earlier on.  PacBio's Eric Schadt has been pushing this concept for a while, and it's hard to imagine the bioterrorism community raising any objections to routine high-resolution analysis of odd infections.

Clearly though, that "appropriate and staged" phrase I use could pack in a lot, and it does.  The quote above about sequencers is what set my teeth slightly on edge.  First, the paper seems to entirely use 454 sequencing and no Illumina.  But it's that cost of sequencers and sequencing that completely off.  What really worries me is if anyone reading this starts extrapolating on the lines of "if every hospital sets this up it would mean billions and billions of sequencer purchases".  That might bring smiles to the equipment makers, but it's ridiculous.

First off, the cost numbers completely failed to take into account any sort of staffing or real estate for the sequencer.  But more importantly, you don't need to purchase a sequencer, and a model in which every hospital has a center isn't (in my mind) at all plausible.

Sequencers aren't cheap (well, Illumina is giving away a few MiSeqs as Ion Torrent did with a few PGMs before them, but usually my statement holds true).  No argument there.  The cost range given is a bit silly, as the high end is a full 454 and nobody but a sales rep from 454 is going to suggest that.  Heck, they should have thrown in the price of a PacBio (somewhere north of $800K), given how much Schadt has suggested routine monitoring should be done.  Or a PGM, at $80K to round out the meaningless numbers.

Meaningless, because a model in which each hospital owns a sequencer and runs a lab is in my mind completely wrong, particularly if you think you need a PacBio.  In general these sorts of bugs are easy to assemble into a lot of big pieces, and I'm not convinced your really need to do better than this, so on my shopping list would be a PGM or MiSeq.  But that's still the wrong model.

I'm no infection control specialist, but at least initially I would presume that this sort of exercise is stochastic in frequency, even in a very large hospital.  Even if you had a case a day on average, there would be other times that more would pile up.  A PGM or MiSeq could definitely keep up with a dozen genomes a day of this size (5.2Mb.1for Klebsiella), but suppose a big outbreak hits?  Plus, even around $100K spread to every hospital is running quite a bill, plus you need some sort of skilled technician and analyst and so forth.

No, the better model would be a large degree of centralization, and perhaps even simply relying on (and maybe further funding) the existing network of service providers and core labs.  If such a network were funded, I would imagine it as a number of regional centers with an excess of rapid turnaround sequencers, technicians available seven days a week and appropriate analysts also on call.  Such centers would be deliberately overbuilt, in order to have some reserve capacity for an outbreak.  But, to keep the costs reasonable, they would be also free to sell off their reserve capacity in the sequencing market, with the key proviso that infection monitoring samples always jump ahead of anything else.

One need not build out new such centers; so many large hospitals already have sequence cores.  Indeed, the Boston area has an abundance of them, and while I have no reason to think they aren't individually all very good, it often does seem like some consolidation might make sense.  A looser network of sites might be practical with appropriate electronic coordination, but centralization is likely to have advantages -- one glitch and a sample goes to the wrong site.

Within such a model, there could be a lot of variation.  I wouldn't see a need to standardize on one sequencer model, and there is certainly value in having a few centers stocked with a PacBio in case there is a need to get very high assembly connectivity very quickly.  There should also be room for appropriate commercial centers to bid for such opportunities; most (but, alas, not quite all)  I have dealt with display a great degree of competency.

Sizing such a network would be a trick, but this is another advantage of getting away from the "a sequencer at every site" mentality; a pilot scale with a few sites scattered around the country could then grow by adding new sites or enlarging the existing ones, yet every hospital could still have access to the capability (no system of "haves" and "have nots").

There are a host of important operational issues which would need to be addressed, most of which I doubt I am capable of thinking of.  Given my own history of relying on outsourced sequencing, I can think of two.  Bacterial infections don't take weekends off, but many delivery services do.  I presume there are medical couriers that work 24x7, but wouldn't know for sure.  Placement of centers would be especially tricky in some of the wide-open spaces of the U.S., but setting some delivery time standards could help there.  I've never had a problem, but my worst nightmare is a true delivery failure of a critical sample, such as the one at the bottom.  There's also the issue of supply chain management; I once had a sample held up by several weeks because the vendor couldn't get reagents for their sequencer.  That's perhaps another reason for platform diversity; given that each manufacturer potentially (and in reality, seemingly periodically) has some reagent problem or shortage, having different centers specialize in different platforms would provide a degree of insurance.


Mick Watson said...

Don't sequencers used for clinical diagnosis have to be only used for clinical diagnosis? And there are lots of sample and data handling issues. These do mean that you can't centralise sequencing to a research sequencing lab such add Sanger, Broad etc. Not easily, anyway. There are benefits to having the machine very close to the clinic.

Keith Robison said...

You've brought up an aspect I completely ignored. It's an important issue, but not a plan killer. Regulatory isn't my strong point, so I welcome corrections to the below:

1) Being a clinical sample doesn't automatically warrant special treatment, but rather if clinical decisions are being made. Infection monitoring is arguably not clinical decision making -- but it would also be undesirable to not be able to use the information for antibiotic selection. One could also argue that deciding to quarantine or otherwise isolate a patient is a clinical decision, so it would be safest to treat these as clinical decisions and use a CLIA laboratory

2) I don't believe CLIA labs cannot legally process other samples, it's just everything going through the lab must be processed with CLIA procedures. That drives up cost, so it is unlikely that non-CLIA samples would go through the same lab

3) Still, there are lots of samples that might go through a CLIA lab but not have the same time sensitivity -- samples used in a clinical study but not for patient selection or other immediate decisions.

4) CLIA requirements would, in my mind, argue even more strongly for centralization -- there is a huge advantage to not trying to build such a heavyweight organization more than required.

5) Finally, there are already CROs that perform NGS in a CLIA setting.

I agree there are advantages to being physically close the clinical setting, but these are balanced by disadvantages to having many labs. In addition to the economic issues, there is a risk of quality issues -- a small number of high volume labs are more likely to maintain a high level of quality and to have the opportunity to learn over time.

Hisham Eldai said...

The points you mentioned Keith and the layout of your suggestion remarkably strikes chords in an exciting manner.

I am a bioinformatician who started as a microbiologist and who also worked in quality monitoring for sometime and I could relate to these ideas pretty well from all these backgrounds.

A regional facility where this centralization takes place can in fact enable gaining much deeper insights into such emergencies and supposedly raising alerts in a timely manner. At the same time, the coordination between such a facility and other prevention and health agencies would be much more streamlined. Not only that, PacBios or any light-weight sequencers can be dispatched to affected areas (hospitals) flexibly and returned back to the mothership when needed. To think about the benefits of this in terms of cost reductions for the hospitals and less idle time for the machines can justify the investment. What do you think?