tag:blogger.com,1999:blog-36768584.post538977302856297302..comments2024-03-03T18:49:34.382-05:00Comments on Omics! Omics!: Out Damned Spot Instance! Out I say!Keith Robisonhttp://www.blogger.com/profile/04765318239070312590noreply@blogger.comBlogger7125tag:blogger.com,1999:blog-36768584.post-22905174635739099862016-12-08T01:22:30.746-05:002016-12-08T01:22:30.746-05:00We were doing great so far with “6. Put the secret...We were doing great so far with “6. Put the secret on the instance from an outside source, via SCP or SSH”, but this has to change as we move to autoscaling : upon booting from a base image, we need instances to set themselves up automatically : pull the latest app code from git (using a private ssh key) + get the secrets (credentials for other services) that we don’t want to store in git, and finally start the app.<br /><br />In this situation, can you see any added value in putting the secrets on S3, manage access to them using IAM instance roles and retrieving them upon app startup, rather than just putting them directly in user-data ?<br /><br /><a href="http://www.peridotsystems.in/aws-training-in-chennai.php" rel="nofollow">aws training institute in chennai</a><br />Anonymoushttps://www.blogger.com/profile/00968416307458995634noreply@blogger.comtag:blogger.com,1999:blog-36768584.post-87840620930085849502012-07-21T06:59:33.263-04:002012-07-21T06:59:33.263-04:00This is an interesting suggestions, I have always ...This is an interesting suggestions, I have always played around the idea of venturing into Amazon EC2 but it has not been all that friendly to me. Now that you have delved into it, what are your comments on how I should better start. <br /><br />The combination of EC2 with Star::Cluster should really be an awesome addition to the toolset, that too in a sandbox, I had to reinstall my OS on a couple of occasions, having to reinstall all the bioinformatics tools out there is not friendly at all... So thanks for this encouraging post ..Hisham Eldaihttps://www.blogger.com/profile/10803591516732164082noreply@blogger.comtag:blogger.com,1999:blog-36768584.post-3808016673105391062012-06-21T17:49:04.757-04:002012-06-21T17:49:04.757-04:00We just went through a heavy aws usage period and ...We just went through a heavy aws usage period and will likely never use it again. I/O (and network) is a huge bottleneck, even (and sometimes especially) in Raid.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-36768584.post-44749949120022876392012-06-20T18:04:24.134-04:002012-06-20T18:04:24.134-04:00For the use within bioinformatics/genomics, have y...For the use within bioinformatics/genomics, have you seen <a href="http://usecloudman.org/" rel="nofollow">CloudMan</a> as an alternative to StarCluster? It does the whole cluster management thing but also comes preconfigured with <a href="http://usegalaxy.org" rel="nofollow">Galaxy</a> and <a href="http://cloudbiolinux.org%22" rel="nofollow">CloudBioLinux</a> and thus numerous bioinformatics tools. It also allows you to customize an instance to add any tools or data that might not already be there...Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-36768584.post-16815657964241206872012-06-20T16:19:15.077-04:002012-06-20T16:19:15.077-04:00Adam: thanks for pointing out the images, which I ...Adam: thanks for pointing out the images, which I had left out. There are a number of interesting images out there for bioinformatics, and this is also how you can save a favorite configuration. For STAR::Cluster, you would need to build atop their image.<br /><br />I think images may be tied to a region, but I haven't played with that.Keith Robisonhttps://www.blogger.com/profile/04765318239070312590noreply@blogger.comtag:blogger.com,1999:blog-36768584.post-34744273946291597222012-06-20T16:11:45.500-04:002012-06-20T16:11:45.500-04:00Thanks for the tips. I've been using AWS a bit...Thanks for the tips. I've been using AWS a bit but have not gotten myself over the initial learning hump. Two items may be of interest here:<br /><br />1) Amazon offers AWS grants for researchers.<br />2) Users can share their machine images (with pre-installed software). For instance, here's one from Eric Hammond with BLAST and NMMer installed.<br />https://aws.amazon.com/amis/bioinformatics-image-based-on-ubuntu-7-10-gutsy-base-install-64-bit<br /><br />I haven't tried anyone else's image yet, but it could be useful. I could imagine building "pipeline" images for all of the bioinformatic pipelines that have been published.Adam Retchlesshttps://www.blogger.com/profile/10276008143626244411noreply@blogger.comtag:blogger.com,1999:blog-36768584.post-43004648348657756452012-06-20T09:32:52.257-04:002012-06-20T09:32:52.257-04:00StarCluster sure looks handy !
Ray supports check...StarCluster sure looks handy !<br /><br />Ray supports checkpointing (save/load your game) for unstable clusters, you may want to try that.sebhtmlhttps://www.blogger.com/profile/03840557846651053123noreply@blogger.com