Only a handful of testbeds for large-scale computer science experiments exist worldwide. Last November representatives of three prominent testbeds - Future Grid and Open Cirrus in the United States, and Grid'5000 in France - gathered for a workshop, co-located with SuperComputing 2011, to discuss the unique challenges associated with using and operating these testbeds.
The workshop organizers and attendees recently released a report based on the workshop; the report highlights some critical issues the field of computer science faces:
"...the experimental methodology for computer science so far has seen relatively little attention and, until recently, little investment."
According to the report, Grid'5000 and Open Cirrus were the first testbeds of their kind. To develop the community of testbed users and their knowledge of experimental methodologies, both testbeds have held meetings where users could share their insights.
Two main points seemed to arise in the report's conclusion:
Sound interesting? You can download the report here.