Video of the week - Monitoring with the fishes

If the fish in this week’s video of the week look familiar, it’s because iSGTW readers have seen them before, in the article entitled “Fish business.”

But unlike earlier editions of the program, in which each fish represented a node in a cluster, this version monitors the whole of Open Science Grid.

“They used Ganglia in order to get information about the cluster – the bigger the fish the more work that node was doing, and if it was dead or it was not reporting it would float to the top like a dead fish,” explained Derek Weitzel, a graduate student at the University of Nebraska, who is responsible for the last few editions of this open source program.

In the current version, if a fish is flashing, it means that the cluster that particular fish represents is overloaded, with 20 percent or more of total jobs waiting in queue. To find out more about that cluster’s status, just click on its fish.

Upgrading the program to interact with Open Science Grid may have yielded other benefits.

“In order to use the Open Science Grid, it had to query out to an external script in order to get the information,” Weitzel explained. “And that can be expanded much further in order to query all types of different information.”

A web portal where visitors could watch a given grid’s status would require a great deal of work, according to Weitzel, given that the current program is very platform dependent.

At first glance, the program may seem cute and frivolous. But the cluster-monitoring version runs around-the-clock at the Holland Computing Center.

“It’s a good way to look at, from a very high level, how the cluster’s doing,” Weitzel said. “If the entire cluster’s down and all your fish are floating at the top, you know something’s wrong. If all of them are huge and flashing, you know something’s wrong. So it’s a good way to see how your entire cluster’s doing at any one time at a very quick glance.”