Wednesday, November 30, 2016

Is PromethION a Strategic Error?


Be prepared for another burst of Oxford Nanopore coverage; their annual Community Meeting in New York City begins on Thursday (perhaps while you are reading this).  I won't be attending; only so many absences from the day job are ultimately feasible each year, and I've long since hit the limit.  Hopefully some prolific tweeters will keep everyone updated, and I can summarize from there.

As with most posts on Oxford Nanopore, my piece on the closure of the Illumina litigation captured some comments; I think it is reasonable to expect more on the piece on the opening of litigation by Pacific Biosciences.  How you perceive the ambiguity around whether they were using an MspA pore in the R6 and R7 chemistries or not tends close to a Rorschach test.  But with that behind us, attention can focus on their current state and progress. Now, I'm going to project some critical ideas around one of their platform pieces, but I have no delusions that they will affect Oxford's course.  It wouldn't shock me if Clive had a tart seven letter response, the only question being if the last three letters form a pronoun or a preposition. I don't have any inside information nor do I have any direct financial interest in the company, in case you are wondering.  As my title suggests, what I'm going to argue is the case that pursuing the launch of the PromethION instrument is an unnecessarily risky detour.



A quick recap, if somehow a reader unfamiliar with the Oxford Nanopore saga strays onto this page.  Oxford has built a handheld sequencing device called the MinION (seen above), which relies on a consumable cartridge (aka flowcell) measuring electrical changes caused by DNA being driven through protein pores in membranes.  Up to 512 such pores can be watched simultaneously, though in practice the number tends lower.  Computing power to call the reads is provided by a laptop, which connects to the device via a standard USB3 cable. Successive chemistry improvements have driven MinION's base accuracy into the low 90% range.  Reads of over 100 kilobases have been observed, though more typically they are only around 10 kilobases.  A bit of laboratory skill is required (alas, more than I seem to possess -- though high school students have successfully mastered it), but not much, and an imminent library prep kit takes only 5 minutes. At this time, up to 10 gigabases of data can be obtained from a single flowcell over the course of 48 hours.  Oxford essentially gives away the MinIONs ($1K, but the starter kit comes with about $1K of consumables), expecting to make money on the consumables.  Applications requiring long reads, portability and speed have been emerging from a small but vocal (and growing) community of early adopters. With output on the device starting to approach 10 gigabases per flowcell, multiple groups are now attempting human genomes by running multiple flowcells.  So MinION is covering a lot of sequencing space.




PromethION is a projected big sibling to MinION.  It will have 48 independently runnable flowcells; the device can run any number of them between 1 and 48 simultaneously, and each flowcell can be started or stopped without disturbing any of the others.  PromethION flowcells have a different shape than MinION flowcells and more pores per device (though this may be mostly from having a greater membrane surface area and corresponding electronics).  All of the sample preparation protocols and kits for MinION will function on PromethION.  Data processing power is supplied by a dedicated computer built into the system.  The instrument (above) fits easily on a benchtop.  Early access testers have been recruited (the PromethION Early Access Program, or PEAP).  However, Oxford is able to manufacture the instruments at about one per month (earlier this year they indicated this would increase, and perhaps it has, but production rates may remain low) and as of their last communique on the subject had not yet shipped any flowcells.  Projected capacity on a fully operational PromethION would be terabases per day, though to do so economically will require a modified flowcell design which enables reusing the expensive application-specific integrated circuits (ASICs).

PromethION is aiming at the core lab and genome center market.  These large, centralized facilities offer access to expensive, challenging instruments at many academic centers and larger companies.  In academia, many core facilities operate on a cost recovery model; grants pay for the instruments and then users pay consumables costs plus labor charges but no amortization on the instruments.  Acquisition of instruments is often a press generator for core labs and their institutions, as well as the companies supplying the instruments, and make nice things to show to visitors.  Really big instruments, such as the Illumina X series, generate lots of data and lots of headlines and really impress touring bigwigs.

MinION would not appear to be a natural fit for this model.  An instrument that fits in a shirt pocket or can be easily lost in a lab drawer isn't an easy sell.  Something with very simple logistics of use doesn't really match the model of core expertise, though that doesn't mean many MinION users won't be going to their local core groups for bioinformatics help.   Pulling out a stack of MinIONs isn't exactly the way to bowl over a visiting Congressperson, particularly as running MinIONs becomes commonplace.  I'm not trying to be cynical or dismissive here; cores are very valuable and I've used a number of them to generate critical data.  But they operate in a particular mold, and it is difficult to see MinION as working well there.

So Oxford decided to go forth with PromethION (or an earlier, similar concept called GridION) as a solution for core labs.  Even if each MinION doesn't require a dedicated laptop, there are practical considerations to trying to run 48 MinIONs in parallel, such as keeping the 48 USB3 cables orderly.

The question though is whether PromethION is a bridge too far.  PromethION requires several technologies to be successfully developed and launched, and these technologies may not port easily to MinION.  This is my first concern: PromethION represents a very large developmental fork away from MinION, which means improvements in one may not copy to the other.  It also means further proliferation of components and sub-components.  

Other companies have been caught up by this trouble.  One example is Ion Torrent.  As they developed higher density chips, Ion Torrent quietly changed the polymerase used in those chips but not the earlier ones.  This fork became public when Ion developed mutants in one of the polymerases that reduced homopolymer calling errors.  That same protein engineering had to be repeated for the other polymerase, preventing a simultaneous introduction of the newer polymerase across all parts of the platform.  So if you were developing assays on the older PGM chips, it wasn't plausible to expect them to work nearly the same on the newer instruments.  

Besides the forking of the flowcell, at least at the manufacturing stage, Oxford has brought on the challenge of developing a compute engine capable of servicing the device while fitting into a sleek case.  Not only that, but the heat from the compute engine must be successfully dumped without overly heating the flowcells above it, which also must be shielded from the electrical noise of the computer and any associated fans.  This also bumps into the happy problem of increasing productivity of the pores; each time an Oxford tweak to the chemistry improves throughput, the required power for the basecalling goes up in turn.  Oxford is already saying that the initial computers will be replaced next year with more powerful models.

Most importantly, all of these problems are to a large degree coupled; success of PromethION requires simultaneous solving of all of them.  This is very concerning, given the struggles Oxford had launching MinION.  Again, they have solved many of those problems and quite a few of those solutions will translate over, but by introducing a new consumable that can't be tested until the new hardware is also ready, Oxford has multiplied their risk.

Suppose you think the risks I've outlined are great, as evidenced by the slow delivery of PromethIONs, the lack of flowcells deliveries and the upgrade of the compute perhaps before anyone generates data.  What alternatives did Oxford have for the core facility market?

The first option would be to ignore that market.  Yes, it is an important market, particularly for going head-to-head with Illumina.  There's also the real fear that delay in getting a toehold in that market would allow Illumina to further consolidate their near monopoly.  There's also a lot of sequencing thought leaders who operate such labs; without a player in this space it might be difficult or impossible to get these individuals directly acquainted with nanopore sequencing.  

But the flip side is that MinION is making huge inroads into previously untapped markets.  Portable sequencing was at best a demonstration before, given that the lightest and smallest instrument (Ion Torrent PGM) still required at least a small car to move it and its associated hardware around.  Rapid sequencing applications are taking off.  Again, PGM was much of the previous competition but a MinION can be sequencing long before an Ion user is done running the emulsion PCR.  Long reads from MinION are demonstrating continuing success in large genome assembly problems.  If ordinary users can get 5-10Gb from a flowcell routinely, then many fungal genomes or even C.elegans or Drosophila can be run on a single flowcell.  There's also strong evidence that the world currently has a glut of capacity for existing markets, in other words for Illumina sequencing.  That would suggest a much more promising route to growth runs through novel applications, not existing ones.

The underlying assumption in my analysis is a zero sum game when it comes to clever people.  Oxford has a lot of smart people, but ultimately they are a finite resource.  While it is possible to slow a project by putting too many people on it, far more common is to under-resource a project.  Oxford is bursting with interesting concepts which could enhance the platform, but similarly it is also still burnishing the basic MinION platform.

So does it make more sense to have software engineers getting a new compute architecture running, or instead having them make the MinKNOW running software simpler to use?  Or perhaps making serious advances in correctly calling homopolymer regions, the biggest barrier to high accuracy in consensus sequences derived from MinION.  

A second option would be to take more moderate risks to drive the MinION forward.  For example, Clive Brown recently showed off the Flongle, an adapter for MinION that enables it to accept the smaller and less capable flowcells designed for SmidgION.  These flowcells lack the sensing electronics (they would be in the Flongle), lowering cost immensely.  I love the Flongle concept; while it introduces two new parts (the Flongle and the smaller flowcell), it plays into the testing and educational markets that are beginning to develop.  With the smaller, cheaper flowcells, having every student in a freshman biology course (or perhaps an advanced high school workshop) run their own sequencing experiment become economically practical.  Furthermore, successfully launching Flongle reduces risk for SmidgION; the flowcells would be heavily tested prior to the SmidgION entering field testing.

Another area which deserves attention, but hasn't yet publicly gotten any, is to perform whatever is needed to get MinION regulatory blessing to be used in diagnostic systems.  I've stayed almost purposefully ignorant of the details of this, but for the promising academic efforts using MinION to be translated to widespread clinical use will require jumping through regulatory hoops in each of the major markets.  Oxford CEO Gordon Sanghera has experience in diagnostics, but that was in a much better worked out diagnostic space (insulin testing).  MinION has a lot of potential for being used in areas that are still rapidly developing and where sequencing is a novel approach, which suggests a lot of attention will be needed to attain approval.

Another angle Oxford could have pursued would be to simplify running modest numbers of MinION flowcells simultaneously.  Instead of the 48-cell PromethION, perhaps an 8 cell MinION grid, designed to fit on a benchtop.  Emphasis here would be on making things more orderly first, but also providing a more gentle upgrade path to a unit with dedicated compute (though granted, that is inherently a step function going from external compute to dedicated).  As with PromethION, spacing to enable using multichannel pipettors would be a good idea, but not critical.  Yes, this is still developing a multi-piece sequencer, but at a much smaller scale, inviting far less risk of problems introduced by large scale such as dumping lots of heat or processing lots of data.  As Ion Torrent found with their struggles with the PII chip, trouble-free scaling of electronic systems rarely is seen in the real world.

Which brings me to a question on PromethION I haven't seen covered anywhere, nor has Oxford talked about it in their PromethION presentations (as far as I can remember).  MinION flowcells require a priming step, one I dread, which now consists of three different pipetting operations. This is required to exchange out the storage and calibration buffer for the flowcell.  Not only is this buffer different than the running buffer, but it also contains a high concentration of the DNA fragment used for calibration (Platform QC).  Pipeting requires a bit of skill, as air bubbles absolutely must not be introduced at this stage, for they kill pores.  Presumably the PromethION will require a similar priming step: can this be done successfully with a multi-channel pipettor? Otherwise, this will be a significant nuisance (even for those who are good at it) to prime many flowcells in parallel to initiate large runs.

As I stated near the beginning, I don't expect this piece to change Oxford's course.  PromethION is a big, prominent bet, not to be scuttled lightly.  Increasing numbers of users posting pictures of arriving PromethION's suggest that the production problems with the instrument may have been solved.  Perhaps the flowcell issues will be quickly solved and the compute doesn't prove a headache.  Certainly Oxford has had no difficulty recruiting a cadre of enthusiastic beta testers. All the accessories launch on time and necessary regulatory approvals for diagnostic use fall into place. Should Oxford succeed, I'll look like one more naysayer they've proved wrong.   Since I like seeing new sequencing capabilities enter the market, I'd be happy to be wrong on this.  But that's also why I'm concerned I may be right; if the path to new exciting applications lies from smaller and cheaper devices, PromethION is a step in the wrong direction.

10 comments:

brian said...

>as of their last communique on the subject had not yet shipped any flowcells

Are these the first PromethION flowcells in the wild?
https://twitter.com/AW_NGS/status/802198271926140928

Mark said...

As for the Promethion, I think it would be mainly used with 1-2 flowcells initially, otherwise would require quite a sizable datacenter linked to it for basecalling.
http://seqanswers.com/forums/showthread.php?t=72963

Hope they are wise enough to keep the data aquisition separate from the processing, or the whole thing would just overheat...

I hope that nanopore would develop some squiggle - space mapper/assembler, which does comparison on the signal (waveform) level before doing basecalling.

It may prove to be much more feasible in the long term, esp for large genome projects - just convert the reference to the waveform, and try to locate the reads signals on it (the speech/image recognition approaches may be used for it).

A few things about minion sequencing (from real user experience):

1. Why they do not mention the fuel mix/water degassing step required to avoid tons of tiny bubbles forming on their sensor array, as the buffer warms up to 34C - 37C - (I just use eppendorf concentrator for 2 minutes).
2. Different batches/versions of the flowcells have very different performance even with the same library kits - R9 chemistry works well with 2D kit and 1D (SQK-RAD001) (gives 450Mev (~Mbp) to 900Mev per 48 h run), but the R9.4 Does NOT work well with SQK-RAD001, it requires SQK-RAD002. Mev - Millions EVents.
http://seqanswers.com/forums/showthread.php?t=70359
3. The data on their website for instrument specs looks a bit like theoretical maximum, real outputs seem to be 1/20 - 1/5 of the published figures. (Maybe they've got someone from [P&G or similar] as their marketing department strategist - more optimistic by the 1 order of magnitude, than their CEO).
4. Even once you've got your 1BGp of raw data only 10%-45% of it would map to the complete reference... (using blastn -W 7)
5. While there are bits of sequence which are 90-97% accurate, they are intertwined with completely random bits 250-1500 bp long (where the basecaller had lost track) - why no mention of this in their sequencing specifications - they must get back to ground, so the users can know what to expect.
6. Offline basecaller from minknow has way worse accuracy than metrichlor as of (12/2016).
7. Lack of capability for ordinary users to train the RRN basecaller on their reference/data.

Anonymous said...

What?

I'm confused.

1. There isn no degassing step required.
2. The referenced post bears no relation to reality.
3. Name me a company that doesn't list 'ideal specs'
4. Why are you using blastn?
5. No. They aren't.
6. No. It doesn't.
7. Define 'ordinary user'.

Anonymous said...

(the commenting software blows. Trying to use G+ or blogger to be the author is not working)

A couple comments about academic cores. I wish I lived in a world where my instruments were not amortized on my books for five years as well as the service contract and everything else. Sure a grant here and there helps but most cores (and this one) pay for their instruments one way or the other, usually by amortization.

The other fallacy is that the current MinION is not a core instrument. while you are correct, a row of laptops attached to what looks like overgrown thumb drives is not sexy they really might be the sexier part of a dog and pony show than some boring large box. And cores exist to use local square footage to increase scientific effort nearby. If running MinIONs to show efficacy, to spread the knowledge throughout the local space, and maybe end up with a centralized area of expertise along with bioinformatics, sounds like a good core to me. Who cares if by doing so all the investigators eventually run off with their own MinIONS in their own lab? I hope that happens. We can then move onto something else.

In no way would I agree with Clive but I bet he would go with a preposition. :-)

Paul Morrison @MBCF
Molecular Biology Core Facilities
Dana-Farber Cancer Institute

Anonymous said...

Kroemer's lemma: the principal value of any sufficiently innovative new technology is not that it allows us to do something we can already do, just better. It is that it allows us to do something we could not do before.

This is a loose quote from a wonderful paper by a Nobel laureate. It pretty much supports your assessment (which I share) that the Promethion is a distraction. It sucks resources away from where the real novelty and power of ONT is.

In my estimate the Promethion is largely a fundraising tactic. ONT has soaked up so much capital and will likely need much more (as their recent round shows) that they need to show a believable path to a high profit/high growth market. Selling a few tens of thousands of $1k devices per year will not make the VCs happy, some of which already have had their funds tied up for a decade or so.

The MinIon has the potential to supplant all existing technologies over time. But to do so it must be content with dominating the markets where it has no competition and gradually improving upwards to decimate more profitable markets. Unfortunately that will take time (5-10 years) and ONT does not have this kind of time as a privately held cash flow negative entity.

My prediction is an IPO after another round or two of fundraising to allow the VCs to cash out at the expense of the unsuspecting lay investor, followed by severe disappointment of Mr. Market at ONT's inability to generate revenues and profits anywhere near the $2-3B valuation they will command. After that shakeup the company will get new management that will have breathing room to focus on the things that matter instead of engaging in asinine verbal attacks on the competition.

This is how things played out for PacBio. Is there a reason to believe that ONT will be different?

Anonymous said...

Anonymous

"After that shakeup the company will get new management that will have breathing room to focus on the things that matter instead of engaging in asinine verbal attacks on the competition. "

What asinine attacks on competition ? i haven't seen those - can you follow up with some URLs ?. If anything ONT are the company that dont do that, but most other people seem to attack them.

I dont think PromethION and MinION are zero sum games commercially, they seem very complementary and target very different users.

ONT doesn't have 'VC' investors.

Anonymous said...

Unsubstantiated plagiarism insinuations, a la Trump
https://twitter.com/Clive_G_Brown/statuses/480788888584470528

I am not 100% sure what the purpose of multiple retweets of this item was, so I am not going to make unsubstantiated insunuation, you make your own decisions
https://twitter.com/Clive_G_Brown/status/797743098017222656

In this story
http://www.theatlantic.com/science/archive/2016/04/this-technology-will-allow-anyone-to-sequence-dna-anywhere/479625/
Statements from a customer: "There are a lot of us who are concerned about criticizing the company because our access to the technology could be arbitrarily revoked"

In the same article, CTO speaking on the record
"“He poo-pooed everything I said,” says Brown. “So I’d go away and think: what is the simplest thing I can make that works, and that doesn’t look like an Illumina box.”

This is but one example where the CTO has, in an article nominally about ONT's success, been unable to pass on a chance to get a dig into Illumina in general and Flatley in particular. Really? Have you ever seen a corporate officer allow his personal feelings to get that entangled with the business of making a company successful?

As for ONT not having VC investors, not sure what your definition of "VC investors" is.
https://www.crunchbase.com/organization/oxford-nanopore-technologies#/entity

Anonymous said...

well the first one looks like a question. wasn't there some lawsuit around that ?

the second one looks like something people would draw their own conclusions from. Just a photograph.

the third one is weird. I wonder how many people had access revoked, i heard it was just the one (Mikheyev).

the fourth one, whats wrong with that if its true ?

you dont know whats gone on behind the scenes there either - i bet theres a back story.

the last one 'venture' doesn't describe the investor type you twit. Most of their investors are hedge funds or institutional or other.

Mark said...

Conclusions:
As the things were in summer - autumn 2016, nanopore data is very usefull for scaffolding/hybrid assembly given HQ Illumina/454/Iontorrent data. It is way better for it than just restriction digest fragments maps. For nanopore-only denovo assembly better quality/different approaches are needed in order to better resolve the ambiguous sequence motifs and reduce systematic error rate.

Comments on the reply
1. There is no degassing step required.
According to manufacturer's protocol that is true, but have a look at a flowcell after it had been running for 24-30 hours, and count the bubbles appearing from the buffers on the ASIC array due to lower solubility of the gasses in the liquid at 34 degrees, than on 0-4 degrees thaw time. And all user know, that they can easily disturb the membrane.

2. The referenced post bears no relation to reality.
Those charts were bases on the real run done in the middle of summer...

3. Name me a company that doesn't list 'ideal specs'
454/Roche, Illumina - we are able to achieve their 'ideal specs' with quite regular basis, given good sample/library prep QC... (thing can go south with any sequencing, but one think is to have 5% bad ws having 75% bad runs).

4. Why are you using blastn?
Since it is a mature and stable tool, that was used for mapping original pacBio datasets (when no other mapper could reliably map the reads with 13-15% error rate in 2009-2010), and it is way more sensitive than BLAT-like tools, also has WWW interface - wwwblast. Also you can dial blastn wordsize (minmatch) down to 4 bp on a commandline blastall utility, and provide with both pairwise and master-slave alignment outputs.
PS: Now trying to get the NaS and nanopolish pipelines to work well with my datasets.

5. No. They aren't.
But graphical overview and master/slave alignment from my www blast show clear gaps, where sequence does not align well (error rate >40%) to high quality Illumina pcr-free+nextera matepair assembly consensus (no N's in the consensus :-)... Simply there are some base combinations that are more difficult to decipher from waveform than other ones.

Also on the 2D reads from GC~71% the reverse strand has lover quality than forward one due to secondary structures (hairpins) forming on the single stranded DNA and interfering with movement of the DNA through the pore.

6. No. It doesn't.
So where is the option for doing "Catchup basecalling in MinKnow, once the run is done" - (so one can compare the actual performance - hope to see it in the future MinKnow versions).
PS: Epi2me command line client should be downloadable by the ordinary competent users without requiring a developer's license.

7. Define 'ordinary user'
For Illumina it would be somebody who knows how to run bcl2fastq to get the fastq files and assemble or map then using at least one of the following: DNASTAR, CLC-BIO, newbler, mira, velvet, map: bwa, cufflinks, etc.

PS: The any criticism must be constructive, and based on a REAL facts.

Anonymous said...

Interesting conversation. It seems the previous commentators had some experience with the ONT chemistry. Hence my question: Would you recommend a promethion to a neurobiology core lab mostly doing RNAseq, ChIPSeq and human genome assemblies? There is a hope that as the base calling algorithm matures, RNA modifications will be easily distinguished by the software. How far is it towards that goal?