tag:blogger.com,1999:blog-36768584.post4885693746171687834..comments2024-03-03T18:49:34.382-05:00Comments on Omics! Omics!: Leaky clinical metagenomics pipelines are a very serious issueKeith Robisonhttp://www.blogger.com/profile/04765318239070312590noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-36768584.post-90654012433469357902015-07-07T10:59:34.908-04:002015-07-07T10:59:34.908-04:00Hi Jonathan - we are developing a clinical version...Hi Jonathan - we are developing a clinical version of SURPI under a HIPAA-compliant framework in collaboration with a company and also actively validating the pipline in a CLIA-certified laboratory. It would be great to get in touch with you to discuss ways to work together to move this forward. -CharlesAnonymoushttps://www.blogger.com/profile/00411913282042519816noreply@blogger.comtag:blogger.com,1999:blog-36768584.post-78236960369130761962015-07-06T11:29:45.131-04:002015-07-06T11:29:45.131-04:00Please see our reply on F1000 addressing the comme...Please see our <a href="http://f1000research.com/articles/4-180/v1" rel="nofollow">reply on F1000</a> addressing the comments on the previous post by Dr. Salzberg.Anonymoushttps://www.blogger.com/profile/00411913282042519816noreply@blogger.comtag:blogger.com,1999:blog-36768584.post-31125975467649871282015-07-06T09:25:12.202-04:002015-07-06T09:25:12.202-04:00Keith - excellent points all around. The pre-NGS (...Keith - excellent points all around. The pre-NGS (wet lab dehosting) and post-NGS (in silico removal) filtering of human reads from metagenomics clinical micro Dx samples is something we are tackling with as well. It's a tough challenge - especially if you consider that h19 alone is not going to cut it. (Have you read J.Allen's paper on human <-> microbial genome cross contamination? it sort of scopes the problem pretty well). In any case, for now the approach seems to jump dump these reads. Sure-they still show up in the raw FASTQ from sequencer; but presumably those don't leave the lab.<br /><br />Also - regarding SURPI: I can tell you first hand that -- sensitivity aside -- if you were to run SURPI, GOTTCHA, KRAKEN side by side... you would likely get different "diagnostic calls". The problem with incongruence between algorithms gets worse if you start looking at trace detection as well (where your human gDNA dehosting didn't work so well; you are often 10000:1 or worse with human:pathogen reads). Lastly - metagenomics pipelines ≠ diagnostic pathogen ID pipelines. Microbial community profiling (in my mind) has a different set of requirements than a pathogen Dx, but there is a undertone in the community that existing metagenomics tools are good enough for pathogen Dx. They are not - for many reasons. First, a clinician is likely to care only about a specific set of pathogens, and the presence/absence of phenotypic predictors associated with those pathogens; not the entire milieu of microorganisms in a sample. Second, the reporting of metagenomics data can easily reach the TLDR or TMI thresholds, thus killing their utility in a clinical reporting context. Third: know of any infectious disease Dx bioinformatics pipelines that are 21 CFR Part 11 compliant? HLA7? HIPAA? OK.. how about pipelines that keep full audit trails of all data manipulations? Unless these issues are addressed; metagenomics Dx for infectious diseases will never get out of the research lab, let alone see any sort of FDA approval.<br /><br />Hopefully some smart software developers are working on solutions for this problem - and not approaching it as strictly a #metagenomics research issue. We need benchmarking - yes - and purpose built tools for the problem at hand (not a re-tooling of existing research software). ;)Jonathan Jacobshttps://www.blogger.com/profile/06133232985480734844noreply@blogger.comtag:blogger.com,1999:blog-36768584.post-54982856299733858212015-07-05T10:19:18.715-04:002015-07-05T10:19:18.715-04:00We have posted a response to the response on F1000...We have posted a response to the response on F1000, though it hasn't appeared yet. (We posted it Friday 3 July.)<br /><br />What the original paper had was a summary table (Suppl Table 3) listing the total numbers of bacterial reads in each sample. This did indeed indicate that they found bacteria in each of the two samples we focused on, but it didn't say what species. Then, in their Supplementary Table 5, they list about 75 different species, broken down by sample, showing how many reads from each species were in each sample. That table did not include any reads at all from either Haemophilus influenza or Staphylococcus aureus. So the claims in our paper are correct: from a reading of Greninger et al's published paper, there is no report that these species were found. <br /><br />The authors responded to me privately as well, and say they knew about the Haemophilus and Staph aureus, but didn't think it was significant. That may be true - it's impossible to verify - but then it is hard to explain why they include a table with 75 different species found in many other patients/samples and didn't include these.<br /><br />We stand by our results and maintain that if the authors did find these bacteria and had ruled them out, they should have at least mentioned that in their paper. And I'd add that in their response, the authors agreed that our other main finding, the unintended deposition of human reads from these patients into a public archive, was correct.Steven Salzberghttps://www.blogger.com/profile/16549957293973146438noreply@blogger.comtag:blogger.com,1999:blog-36768584.post-1091970279995382142015-07-03T07:00:17.107-04:002015-07-03T07:00:17.107-04:00Also, the original authors have written a response...Also, the original authors <a href="http://f1000research.com/articles/4-180/v1" rel="nofollow">have written a response to the F1000 criticism</a>.Keith Robisonhttps://www.blogger.com/profile/04765318239070312590noreply@blogger.comtag:blogger.com,1999:blog-36768584.post-39099032981216648822015-07-03T06:58:26.382-04:002015-07-03T06:58:26.382-04:00It has been pointed out privately to me that the o...It has been pointed out privately to me that the original paper did discuss the bacterial reads and ruled them out as a cause of the disease due to the common presence of these organisms in the upper respiratory tract (and the the H.influenzae sample was not a case of paralysis). So the criticisms of Salzberg and company on that point may have been amiss. Keith Robisonhttps://www.blogger.com/profile/04765318239070312590noreply@blogger.com