Image of the week - Matchmaking on the grid

Jobs begin in the MATCHING site at the far left in the image (click for full image). The color-coded jobs are then sent to OSG compute sites, where they are stacked. Yellow chips on the stack are queued jobs and green ones are running. Completed jobs are sent to the DONE site at the far right. The color bar for ranking OSG sites is visible in the upper right of the image.   Image courtesy of David Borland, Mats Rynge, John McGee and Ray Idaszak, RENCI

From the user’s perspective, the distributed grid computing framework known as the Open Science Grid is a seamless interface to computing system across the U.S. that allows jobs to be done more quickly and efficiently than any single computing system.

Under the virtual “hood,” however, a lot happens to make the grid environment flow smoothly. This visualization created at the Renaissance Computing Institute (RENCI), an OSG partner, helps grid engineers and programmers understand how jobs of varying types and sizes flow into the grid infrastructure of the OSG where they are matched to the appropriate resources, put in queues, run and finished, so that results can get back to the user.  

RENCI created the matchmaking layer for the OSG job flow as a way to better monitor job load, queue size, network utilization and other parameters that affect efficiency in grid environments. The matchmaking layer sits on top of the regular OSG infrastructure and makes extensive use of the Resource Selection Service developed at Fermilab. When in action, the matchmaker not only matches jobs to resources, it also dynamically ranks sites on responsiveness and job failures and automatically moves failed jobs to other sites in order to keep work flowing.