Sharing science in the ‘collaboratory’
Pegasus and HUBzero: a combination written in the stars?
Image courtesy Roberto Mura, Wikimedia commons.
Ewa Deelman, principal investigator of the Pegasus Workflow Management System, and Michael McLennan, director of the HubZero platform, explain how they are working together to aid scientific research.
Over the past several years, the US National Science Foundation has been funding the development of collaborative web sites or ‘collaboratories’ and scientific workflow technologies. Many communities have adopted the HUBzero platform to create collaboratories called ‘hubs’ where they can share ideas, models, experiences, publications, and data in pursuit of research and education. Scientists have also been using the Pegasus Workflow Management System (WMS) to manage complex analyses running on their campus and on large-scale cyberinfrastructure, such as the Open Science Grid, DiaGrid, and XSEDE.
Over the last few months, a joint team of HUBzero developers (Steven Clark and Derrick Kearney of Purdue University, Indianna, US) and Pegasus developers (Mats Rynge and Karan Vahi, University of Southern California, US) together with domain experts Frank McKenna of the University of California, Berkeley, US, and Linas Mockus, also of Purdue University, have integrated the two technologies. Now, the latest HUBzero software, released at the recent HUBbub 2012 conference in September, includes Pegasus WMS as an integrated system. HUBzero supports more than 40 hubs around the world in a wide variety of scientific areas, including nanotechnology, microelectromechanical systems, manufacturing, healthcare, cancer research, biofuels, environmental modeling, volcanology, professional ethics, and STEM education. All together, these hubs have served more than 800,000 visitors during the past 12 months and the integration of Pegasus brings significant computational power to this worldwide community.
Pegasus can manage workflows that are comprised of millions of tasks, while recording all the workflow management and execution information so that the provenance of the resulting data is clear. Pegasus workflows are represented as directed acyclic graphs (DAGs) that describe the computational components, their input and output data, and parameters. These DAGs are written in XML and can be built using Java, Perl, or Python application programming interfaces (APIs). Pegasus then uses information about the available resources, data sources, and code repositories to map the high-level workflow description onto the available clusters, grids, or clouds. Through a variety of protocols, it can access Condor, GRAM, EC2, and other computational resources while retrieving data via GridFTP, Condor I/O, HTTP, scp, S3, iRods, SRM, and FDT. Not all execution environments are the same however: some have shared file systems across the head and worker nodes, whereas others have a fully distributed architecture. As a result, Pegasus needs to augment the workflow, adding tasks to appropriately manage the data. Pegasus can also perform data reuse by accessing data that was previously computed and can checkpoint the workflow, saving intermediate data products as it manages the workflow. If a failure occurs, Pegasus can retrieve the intermediate data and restart the workflow from that point. Other workflows can also make use of these intermediate products in their own computations, thus potentially saving computational time. For data-intensive applications, Pegasus can also minimize the workflow data footprint, by deleting data no longer needed from the execution sites. Equally, when tasks in a workflow are very short running (in the order of seconds), it may be advantageous to cluster them together into larger entities. This is something that Pegasus can do as well.
Pegasus can be used within each hub in a variety of ways. It seamlessly supports the execution of HUBzero’s ‘submit’ command, which sends jobs off to available grid/cloud resources. Pegasus manages the data transfers and recovers from failures should they arise. It can also be used directly by end users or simulation tool developers who want to create their own workflows and launch them within the hub. The resulting applications can be wrapped up in a convenient graphical user interface (GUI) via HUBzero’s Rappture toolkit and published as a ‘tool’ on the hub. As a result, any user can access a customized GUI interface to set up their own complex analysis and launch it with a single click onto the large-scale cyberinfrastructure.
The Community of civil engineers working on earthquake mitigation provided a compelling use case for the combined capability of HUBzero and Pegasus. Image courtesy Frank McKenna, University of California, Berkeley.
The OpenSeesLab shows a powerful example of Pegasus/HUBzero in action. McKenna used HUBzero’s Rappture toolkit to provide a suite of tools for structural and geotechnical engineers within NEEShub. These tools prompt the user for information and launch the OpenSees program as a computational engine. One of the tools, the Moment Frame Earthquake Reliability Analysis, requires a large number of OpenSees simulations and significant post-processing of the results. McKenna uses a Pegasus workflow within this tool to execute all of the simulations and the post-processing, which run on the Open Science Grid. End users, however, have no idea that all of this is happening. They simply fill in a few parameters, push a button, and view plots of the results.
Go to http://hubzero.org/getstartedfor more information about HUBzero and http://pegasus.isi.edufor more Pegasus information. For a Pegasus tutorial on HUBzero, visit http://hubzero.org/tools/pegtut.
Digital Stream
-
EVENT: iMarine Blue Hackathon, 28 June - 2 July 2013, Crete, Greece
18 hours 42 min agovia Twitter
-
ANNOUNCEMENTS: Welcome to the PRACE booth at #ISC13 http://t.co/jdMqv6UcaN & PRACE Educates Tomorrow's Researchers http://t.co/jb8gD9INLc
18 hours 44 min agovia Twitter
-
RT @ISChpc: NCAR at BSC Barcelona working together on automatic performance improvement detection of large #supercomputing codes. Kellie at…
1 day 49 min agovia Twitter
Upcoming
-
22 June 2013 - 28 June 2013,
-
25 June 2013 - 27 June 2013,
-
1 July 2013 - 6 July 2013,
-
2 July 2013 - 4 July 2013,
-
22 July 2013 - 25 July 2013,
-
29 July 2013 - 2 August 2013,
-
15 August 2013 - 16 August 2013,
-
26 August 2013 - 30 August 2013,
-
26 August 2013 - 30 August 2013,
-
16 September 2013 - 20 September 2013,
Comments
Thank you for sharing this
Thank you for sharing this information. However, is still among the leading topics of our time. I appreciate your post and look forward to more stud listings
innovative post
Ecopolitan EC is usually a 99-years leasehold Punggol EC progress situated on Punggol Wander with Section 21. It really is located right beside Punggol MRT Train station. Ecopolitan EC
it is really very good
Thanks for your post,it is very interest
This is a very well designed Coleman 9949-750 Road Trip Grill LXE, Great for outings at beaches or
parks, It pretty much comes assembled, super easy, took me under a minute, It cleans up nicely and folds up nicely, is well made and sturdy
it is really very good
Thanks for your post,it is very interest
This is a very well designed Coleman 9949-750 Road Trip Grill LXE, Great for outings at beaches or
parks, It pretty much comes assembled, super easy, took me under a minute, It cleans up nicely and folds up nicely, is well made and sturdy
You can use the interview is
You can use the interview is something that can be done to know a lot of things you're curious. This is one session a pity if you miss roofing Kettering
The scientists will discover
The scientists will discover many new things in this way.
buy wine online
Great Collaboration
This is beyond doubt a blog significant to follow. You’ve dig up a great deal to say about this topic, and so much awareness. You could my website at jual lingerie, it will show you more about lingerie babydoll
This is quite wonderful
This is quite wonderful article. Your article affects many urgent issues in our society. It is impossible to be uninvolved to these problems. This post gives the light in which we can observe our reality. I appreciate the information and the effort you put into your article.
norvasc
Excellent post i must say. It
Excellent post i must say. It touches many urgent challenges in our minds. We can not be untouched to these challenges. This article gives good ideas and concepts. This was very interesting to read.
Albenza
Wonderful article i must say.
Wonderful article i must say. The article touches many urgent challenges in our minds. It is impossible to be indifferent to these problems. Your post gives a lot of excellent information and inspiration. Very informative and practical.
amlodipine
Excellent blog post. The post
Excellent blog post. The post affects a lot of urgent problems in our minds. It is impossible to be untouched to these problems. This post gives good ideas and concepts. This is something that worth doing.
Albendazole
great {post|publish|submit|article|write-up}
Corals @ Keppel Fresh may be the most recent inclusion within the Keppel Fresh place by simply Keppel Business. Featuring unique architecture simply by internationally renowned architect Daniel Libeskind.Corals at Keppelbay
Very informative post! There
Very informative post! There is a lot of information here that can help any business get started with a successful social networking campaign! Spook SEO's service
reallly good post
Bartley Ridge is the most up-to-date future kick off along Mount Vernon Path next to Bartley MRT.BartleyRidge
really intriguing post
You've made some really good points there. I checked on the web to learn more about the issue and found most individuals will go along with your views on this website.
Very nice postI just stumbled
Very nice postI just stumbled upon your blog and wanted to say that I have truly enjoyed surfing around your blog postsIn any case I’ll be subscribing to your feed and I hope you write again soon!
We can browse interest article Sekolah Belajar Forex FBS Indonesia & don't forget please read article Konsumen Cerdas Paham Perlindungan Konsumen, & don't forget please read article ESER Unlimited Power Bank & best Cipto Junaedy dan juga Cipto Junaedy & best article Iconia PC tablet dengan Windows 8 and also nulis . Please, Love it!
Good post....thanks for
Good post....thanks for sharing.. very useful for me, I will bookmark this for my future needs.
Readymade Garment House India | Export Garment House India | Export House India | Exporters in India | Ladies Garment India
You have a very good site,
You have a very good site, well constructed and very interesting i have bookmarked you hopefully you keep posting new stuff.
Computer AMC Service Delhi | Computer Repair Service | Computer Repair Delhi | Computer AMC Delhi
Wish I had found this blog
Wish I had found this blog before. The advices in this post are very helpful and I surely will read the other posts of this series too. Thank you for posting this.
Taxi Service Faridabad | Cabs in Badarpur | Taxi Service in Badarppur | car for rent in delhi | car rental companies | Taxi service Faridabad | Taxi Service in Delhi
Wonderful post. Your article
Wonderful post. Your article touches many urgent problems in our minds. We can not be indifferent to these challenges. Your post gives the light in which we are able to observe our life. Keep it up.
http://webmedsplanet.com/
A wonderful post. It touches
A wonderful post. It touches a lot of urgent issues of our society. We can not be uninvolved to these issues. Your post gives a lot of wonderful information and inspiration. Very professional.
albendazole
Hehe
Hello i'am walman I found this post to be very educational. Thank you for broadening my knowledge of this subject. No doubt its a great piece of writing as well. Thanks by blogger boomtemplate. Visit Penting, Panas, Perlu dan Seruu and walman blog. Penting, Panas, Perlu dan Seruu, Portal Berita Friendly
Appreciation for the great
Appreciation for the great blog post Sonic Games. I am glad I have taken the time to learn this Music Games.
senior research scientist at
senior research scientist at Purdue University and director of the HUBzero Platform for Scientific Collaboration. http://www.chron.com/business/press-releases/article/Latex-Mattress-Comparison-Released-by-4266291.php
discover this info here
This post is good enough to make somebody understand this amazing thing, and I'm sure everyone will appreciate this interesting things. A good blog with exciting content, that's what I need. Thank you for keeping this site. - grosir sprei
Awesome blog post. Your post
Awesome blog post. Your post touches a lot of urgent issues of our society. We can not be uninvolved to these problems. Your article gives a lot of awesome information and inspiration. This was very interesting to read.
terbinafine
Wonderful article. Your
Wonderful article. Your article affects many urgent challenges in our minds. We can not be indifferent to these issues. This article gives good ideas and concepts. Well done.
norvasc 10
With the cuts, the various
With the cuts, the various charges and institutions is touching them get rid of their beloved official cars. This occurs in municipalities, autonomous regions and central government. Usually media New honda cars hypes, but not easy to buy one of these cars or, at least, one of the best. Again the "cronyism" is felt: one goes to the first hour on the first day of sale, direct to the car of your interest and say they are "awarded".
sharing science
Nice information, valuable and excellent design, as share good stuff with good ideas and concepts, lots of great information and inspiration, both of which I need, thanks to offer such a helpful information here hot tub works. I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful with. Thank for your time of this wonderful read!!! I definately enjoy every little bit of it!
hot tub works
thanks for great article
I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful.
konsumen cerdas paham perlindungan konsumen
cibadak
sukabumi
sewa mobil murah jakarta
This is something that
This is something that Pegasus can do as well as post abortion syndrome.
Equally, when tasks in a
Equally, when tasks in a workflow are very short running careers in business.
They simply fill in a few
They simply fill in a few parameters, push a button, and view plots of the results of hangover symptoms.
sharing science collaboratory
Inflatable hot tub or commonly known as portable hot tub is the very popular hot tub over these recent years. The portable hot tub becomes so popular because it can be used all over plus require a smaller amount equipments for installation. However , this kind of hot tub installation also has a few weaknesses. Consequently , you have to seriously contemplate about either advantages and disadvantages of inflatable hot tub
to be and not to be
If you love to enjoy your individual recreational by spending some time in your hot tub, portable hot tub seems to be something which you have to contemplate related to owning. It is cannot be dissmissed off that if this portable hot tub is usually recommended for you to private, obviously you can find quite a few features in it, which in turn cannot of course often be found in other types of hot tub, specially the one that is not portable or simply which is more common to be termed like permanently installed hot tub. Naturally , quite a few features can be found in portable hot tub can be only divided into quite a few unique categories. Here, you will be able to find not one but two examples of those people categories of features available in portable hot tub. Knowing related to those people benefits will surely make you would love to own a person even if you essentially already have a person.
I really loved reading your
I really loved reading your blog. It was very well authored and easy to understand. Unlike additional blogs I have read which are really not good.
sito incontri.
have a peek at these guys
This post has shared the information about the unique creature. I really like this post and I used to see these types of creatures on TV. I am trying to decide on a career move and this has helped me with one aspect. Took me time to read all the comments, I discovered so many interesting things inside your blog especially. Thank you so much! - tablet pc murah
any thanks for Jilbab Modern
any thanks for Jilbab Modern Ciput Kerudung giving us nice info. Fantastic walk-through. I fully grasp this post.
a cool way to improve
After I initially commented I clicked the -Notify me when new comments are added- checkbox and now every time a comment is added I get four emails with the identical comment. Is there any method you can take away me from that service? Thanks! - Hotel di Semarang
seems to have an overall
seems to have an overall organizing principle, one of a small number of ways that people create and interact with research results. best binary brokers
Like your post and look
Like your post and look forward
Perawatan Payudara to your update new article.
Perawatan Rambut when I visit! Can't wait Kosmetik Online I wish I can go to all of
Perawatan Kulit these places right now. I mean,
Perawatan Wajah you just have so good guts to go ahead and tell it like it is.
This idea is definitely worth
Reducing power consumption is definitely an area we should take a look now in 2013 for preservation of our planet. This idea is definitely worth a shot.
Produk Kecantikan
collaboratory?
The 2 person hot tub is so popular rest room setup for hot betrothed couple plus innovative married partners. It of hot tub is explained to increase closeness in your way on the path to your individual couple. However , back to the essential attributes of hot tub which are pertaining to enjoyment and health benefits, you can also use it that will elimination your stress and anxiety plus develop your healthier diet and lifestyle with your personal partners. Actually, there are many styles plus facilities made available in 2 person hot tub. They will hurt you a lot though taking spa with your partners.
Lots of thanks for this
Lots of thanks for this post.I think it is a very good post. It helps us many away. So many many thanks for this article.
ads dating.
messy
Actually, hot tubs are available in amount forms of size, model, plus types. You can have either substantial or small hot tubs according to your own demand. Talking about the assorted types of hot tub, there are some main types to consider. They are small, average, and large. From those people some main types, the small size hot bath is considered as the most economical and the a lot of practical hot bath to have, especially for the exact beginners. Despite the fact that small hot tubs are most economical and most practical, there are a lot of things to consider dealing with these types. The things to consider insure both graces plus weaknesses.
Good Content
Good website, exactly where did you arrive up with the info in this item? I’m pleased I identified it despite the fact that, ill be checking back soon to observe what other content articles you might have. Produk Kecantikan | Kado Unik | Cetak Yasin
cool laboratory?
Hot tub parts might not be the things that people previously thought in advance of. When you find out that the hot tub you own can not work accurately and you will be distressed because you simply cannot uncover where the source of the problem is, there are certain things that you will maybe do. The first thing can be giving up plus give the hot tub behind without having done nearly anything because you do not have more than enough capital to purchase a fresh one. The second thing can be squandering the money you have getting a whole new hot tub to replace the one that you think to be 100 % damaged. In this kind of circumstances, it is sure that you need to know about hot tub parts. There is of course a big quality for hot tub parts to be helpful for people.
sharing
Hot tub covers can be said to be one of many things that we can talk when we are talking about hot tub. Nowadays, people who decide to have hot tubs in their houses start to consider about the items called as hot tub covers too. Because of this fact, it can of course be concluded that those things surely have benefits that will be good not only for the hot tubs but also for people who own them. Here, we are going to talk about the benefits of hot tub covers in general. The purpose is no other but to let you know that when you have already had the hot tub, it is better for you to think about hot tub covers as well.
This is a great article. I will visit your blog regularly for so
This is a great article. I will visit your blog regularly for some latest post. Very informative and impressive post you have written,.Fiverr PR7 Backlinks
Resources like the one you mentioned here Simply put i added
Resources like the one you mentioned here Simply put i added your blog post to my favorite features and definitely will look forward for additional updates.Great Job!! text your ex back
Post new comment