9.1. Lottery simulation program
The first application simply utilizes the GSI-OpenSSH module. Many people will not consider it a true grid application, as it does not use most of the facilities provided by the Globus Toolkit, and their is little ability to manage the application and control its environment. However, it is an example of the power of using the GSI and GSI-OpenSSH packages to allow multiple systems to work together to provide a solution.
After presenting the GSI-OpenSSH version of the program we then provide a true grid-enabled version that does utilize the Globus Toolkit facilities.
9.1.1. Simulate a lottery using gsissh in a shell script
GSI-OpenSSH is a modified version of the OpenSSH client and server that adds support for GSI authentication. GSIssh can of course be used to remotely create a shell on a remote system to run shell scripts or to interactively issue shell commands but it also permits the transfer of files between systems without being prompted for a password and a user ID.
GSIssh installs the following tools in $GLOBUS_LOCATION/bin:
gsissh is used to either securely connect to a remote host or to securely execute a program on a remote host.
gsiscp is used to securely copy files or directories onto a remote host.
sftp is used to securely copy files or directories onto a remote host. sftp is not related to gsiftp. It is a different protocol that provides encryption and the same commands as the FTP protocol.
gsiscp and sftp do not perform as well as GridFTP, but they can easily be used in a shell script to transfer files from one location to another.
Note
Be careful to use host names for which certificates have been issued; otherwise, you will get the following kind of error messages if you try to use the IP address instead of the host name:
21569: gss_init_context failed (/O=Grid/O=Globus/CN=host/g3.itso-guarani.com) in the context, and the target name (/CN=host/192.168.0.203)
The purpose of this example is to simulate a large number of draws of a lottery and to check if a winning combination was drawn. Each draw consists of eight numbers between one and 60 inclusive.
The program GenerateDraws whose source code GenerateDraws.C is available in “Lottery example” on page 349 is used to generate random draws. It takes one argument, the number of draws that it needs to simulate (1000000, for example). This program is executed on n nodes by using the GSIssh tools.
[globus@m0 other]$ ./GenerateDraws 10 3 29 33 39 41 46 54 55 3 6 7 10 16 22 35 50 4 13 20 26 36 47 48 49 8 12 17 22 26 36 50 57 6 20 25 28 32 40 51 55 1 2 9 17 24 45 49 60 12 15 19 20 39 50 59 60 1 4 8 13 29 49 52 57 3 7 14 29 36 43 52 58 6 8 15 18 20 22 27 45
GenerateDraws also creates a file named Monitor on the execution node. This file is copied to the submission nodes every five seconds. Monitor contains the percentage of random draws completed and permits the monitoring of the whole application.
Figure 9-1. Lottery example
The Submit script is used to submit the jobs to the grid. We use the grep command to check the output of the GenerateDraws program and to detect if the draw we played is a winner.
Example 9-1. Submit script
#the script takes the tested draw as a parameter
#example: ./Submit 3 4 5 32 34 43
n=100000
NodesNumber=10
#temporary working directory on the execution nodes
TMP=.$HOSTNAME
i=0
#the loop variable is used is all the "for" loops
#the format is 1 2 3 4 .... n
loop=""
# use here the broker developped for the publication
# see chapter 8 (mds executable)
for node in $(mds $NodesNumber | xargs)
do
Nodes[$i]=$node
loop=${loop}" "${i}
i=$(( $i + 1 ))
done
echo The number of draws tested is $n
a=$*
#sort the numbers in the specified draw
# 2 45 23 12 32 43 becomes 2 12 23 32 43 45 so that we could use
# grep to test this draw and the ouput of the draw programs.
param=$(echo $a | tr " " "
" | sort -n | xargs )
# parrarell transfer of the draw executable
# we submit jobs in the background, get their process id
# and uses the wait command to wait for their completion
# this method is also used for the jobs submission
echo Transferring executable files
for i in $loop
do
gsissh -p 24 ${Nodes[$i]} "[ -d $TMP ] || mkdir $TMP" &
ProcessID[$i]=$!
done
for i in $loop
do
wait ${ProcessID[$i]}
gsiscp -P 24 GenerateDraws ${Nodes[$i]}:$TMP &
ProcessID[$i]=$!
done
for i in $loop
do
wait ${ProcessID[$i]}
gsissh -p 24 ${Nodes[$i]} "chmod +x ./$TMP/GenerateDraws" &
ProcessID[$i]=$!
done
#file should be made executable
#on all the execution nodes
echo Jobs submission to the grid
for i in $loop
do
wait ${ProcessID[$i]}
echo ${Nodes[$i]}
EXE="cd $TMP;./GenerateDraws $n | grep
$HOSTNAME'
gsissh -p 24 ${Nodes[$i]} "$EXE" &
ProcessID[$i]=$!
done
#for monitoring, we copy locally the Monitor files
# created on each compute nodes. This file content the
# percentage of tested draws. Each files is suffixes by
# the nodes number. $statusnum is actually the sum of all
# the percentage (Monitor files) devided by 100. When it
# equals the number of nodes, that means that we have finished
echo Monitoring
statussum=0
while (( $statussum != $NodesNumber ))
do
echo
sleep 5 #we poll every 5 seconds
statussum=0
for i in $loop
do
gsiscp -q -P 24 ${Nodes[$i]}:$TMP/Monitor Monitor.$i
status=$(cat Monitor.$i)
statussum=$(( $status + $statussum ))
echo ${Nodes[$i]}:Monitor $(cat Monitor.$i) %
done
statussum=$(( $statussum / 100 ))
done
#cleanup the tmp directory
for i in $loop
do
wait ${ProcessID[$i]}
gsissh -p 24 ${Nodes[$i]} "rm -fr ./.$TMP" &
ProcessID[$i]=$!
done
|
Submit uses the broker (mds program) described in “Broker example” on page 127 to get the host names that it will submit the jobs to by using gsissh:
for n in $(mds $NodesNumber | xargs) do Nodes[$i]=$n i=$(( $i + 1 )) done
The number of draws per node is determined by the variable n and the number of jobs by the variable NodesNumber:
n=100000 NodesNumber=10
Sandboxing
Each execution host uses a sandbox directory in each execution node. This directory is created in the local home directory of the user under which the job is executed. This directory is configured by the $TMP variable set up as .<execution hostname>:
TMP=.$HOSTNAME
gsissh ${Nodes[$i]} "[ -d $TMP ] || mkdir $TMP" &
All file copies and remote execution use the TMP variable in their relative path name to refer to the remote files. This way, each client machine can submit a job without conflicting with another client:
gsissh ${Nodes[$i]} "chmod +x ./$TMP/GenerateDraws" &
ProcessID[$i]=$!
gsiscp GenerateDraws ${Nodes[$i]}:$TMP &
ProcessID[$i]=$!
For more granularity, TMP could also use the process ID of the Submit script:
TMP=.$HOSTNAME/$$
Shell script callback
As we cannot use a callback mechanism in a shell script. We start each command in the background. Therefore, they become non-blocking operations and all commands can be submitted simultaneously.
gsiscp GenerateDraws ${Nodes[$i]}:$TMP &
gsissh ${Nodes[$i]} "chmod +x ./$TMP/GenerateDraws" &
The wait command is used to wait for their completion and acts like a (simple) callback
wait ${ProcessID[$i]}
Job submission
GenerateDraws needs to be copied to each execution node. Each execution node must have the GSIssh server up and running. gsiscp is used to transfer the files and gsissh is used to remotely execute the chmod +x command, as shown in Example 9-2.
Example 9-2. Job submission using gsissh
for i in $loop
do
gsiscp GenerateDraws ${Nodes[$i]}:$TMP &
ProcessID[$i]=$!
done
for i in $loop
do
wait ${ProcessID[$i]}
gsissh -p 24 ${Nodes[$i]} "chmod +x ./$TMP/GenerateDraws" &
ProcessID[$i]=$!
done
for i in $loop
do
EXE="cd $TMP;./GenerateDraws $n | grep "'"'$param'" && echo GOT IT on
$HOSTNAME'
gsissh -p 24 ${Nodes[$i]} "$EXE" &
ProcessID[$i]=$!
done
for i in $loop
do
wait ${ProcessID[$i]}
done
|
Monitoring
The monitoring of each job is managed by the Submit script, which reads the content of each file Monitor created by the GenerateDraws executable on each execution node:
# for monitoring, we copy locally the Monitor files
# created on each compute nodes. This file content the
# percentage of tested draws. Each files is suffixes by
# the nodes number. $statusnum is actually the sum of all
# the percentage (Monitor files) devided by 100. When it
# equals the number of nodes, that means that we have finished
echo Monitoring
statussum=0
while (( $statussum != $NodesNumber ))
do
echo
sleep 5 #we poll every 5 seconds
statussum=0
for i in $loop
do
gsiscp -q -P 24 ${Nodes[$i]}:$TMP/Monitor Monitor.$i
status=$(cat Monitor.$i)
statussum=$(( $status + $statussum ))
echo ${Nodes[$i]}:Monitor $(cat Monitor.$i) %
done
statussum=$(( $statussum / 100 ))
done
How to run it
To use this program we need a valid proxy.
echo password | grid-proxy-init -pwstdin
Your identity: /O=Grid/O=Globus/OU=itso-maya.com/CN=globus
Enter GRID pass phrase for this identity:
Creating proxy ...................................... Done
Your proxy is valid until: Thu Feb 27 06:19:17 2003
The option -pwstdin permits us to create a proxy without being prompted for a password. This way, the proxy creation can be integrated in the Submit script if needed.
You also need to compile the GenerateDraws.C program and be sure that the GSIssh server is up and running in all nodes. See “Installation” on page 211 for more information.
Note
When connecting to a host for the first time, ssh needs to retrieve its public host key. To bypass this request, you can add the following option to the configuration file $GLOBUS_LOCATION/etc/ssh/ssh_config.
StrictHostKeyChecking no
Let us perform the test on six numbers instead of eight to have a better chance to win:
[globus@m0 other]$ ./Submit 1 4 10 15 20 34
The number of draws tested is 100000
Transferring executable files
GenerateDraws 100% |*******************************| 52775 00:00
GenerateDraws 100% |*******************************| 52775 00:00
GenerateDraws 100% |*******************************| 52775 00:00
GenerateDraws 100% |*******************************| 52775 00:00
GenerateDraws 100% |*******************************| 52775 00:00
GenerateDraws 100% |*******************************| 52775 00:00
GenerateDraws 100% |*******************************| 52775 00:00
GenerateDraws 100% |*******************************| 52775 00:00
GenerateDraws 100% |*******************************| 52775 00:00
GenerateDraws 100% |*******************************| 52775 00:00
Jobs submission to the grid
d2.itso-apache.com
a1.itso-cherokee.com
c2.itso-cherokee.com
c1.itso-cherokee.com
t1.itso-tupi.com
t3.itso-tupi.com
d1.itso-apache.com
a2.itso-apache.com
b2.itso-bororos.com
t2.itso-tupi.com
Monitoring
d2.itso-dakota.com:Monitor 37 %
a1.itso-apache.com:Monitor 46 %
c2.itso-cherokee.com:Monitor 41 %
c1.itso-cherokee.com:Monitor 41 %
t1.itso-tupi.com:Monitor 51 %
t3.itso-tupi.com:Monitor 43 %
d1.itso-dakota.com:Monitor 47 %
a2.itso-apache.com:Monitor 49 %
b2.itso-bororos.com:Monitor 40 %
t2.itso-tupi.com:Monitor 55 %
1 4 10 15 20 34 47 57
GOT IT on a1.itso-apache.com
d2.itso-dakota.com:Monitor 97 %
a1.itso-apache.com:Monitor 100 %
c2.itso-cherokee.com:Monitor 100 %
c1.itso-cherokee.com:Monitor 100 %
t1.itso-tupi.com:Monitor 100 %
t3.itso-tupi.com:Monitor 99 %
d1.itso-dakota.com:Monitor 100 %
a2.itso-apache.com:Monitor 100 %
b2.itso-bororos.com:Monitor 84 %
t2.itso-tupi.com:Monitor 100 %
9.1.2. Simulate a lottery using Globus commands
We propose to implement the previous example by using Globus Toolkit 2.2 commands:
globusrun will be used to submit the job. The type of the job will be a multi-request query.
globus-url-copy will be used to copy the Monitor file from the execution nodes to the submission nodes.
globus-gass-server will start a GASS server on the execution nodes that will be used to copy the Monitor file generated by GenerateDrawsGlobus and used to monitor the status of the job.
All the programs used in the previous example are slightly modified but are still used for the same purpose. The modified versions for this example are renamed with the suffix Globus; for example, SubmitGlobus is the submission script and GenerateDrawsGlobus is the program generating random numbers. See “GenerateDrawsGlobus.C” on page 352 and “SubmitGlobus script” on page 353.
The sample broker developed in “ITSO broker” on page 327 will be used as in the previous example to obtain execution host names.
Figure 9-2. Lottery example using Globus commands
Submitting the jobs
The tag ressourceManagerContact will be used for the RSL string to specify a multi-request query. Each resourceManagerContact indicates one execution node. The structure of the query is:
+ (&(resourceManagerContact="host1")(executable= ....) ) (&(resourceManagerContact="host2")(executable= ....) ) (&(resourceManagerContact="host3")(executable= ....) )
globusrun is a blocking shell command that will wait for the completion of the jobs. Consequently, we run globusrun in the background so that the submission script (SubmitGlobus in this example) can continue running and acts as a non-blocking command. The wait command can be used to wait for the completion of this command. However, in the example it is not really required.
Example 9-3. Building a multi-request query for globusrun
echo Jobs submission to the grid
rsl="+"
for i in $loop
do
echo ${Nodes[$i]}
rsl=${rsl}"(&(resourceManagerContact="${Nodes[$i]}")"
rsl=${rsl}"(executable=$(GLOBUSRUN_GASS_URL)$PWD/GenerateDrawsGlobus.sh)
(arguments=$TMP $n "$param")
(subjobStartType=loose-barrier)
(file_stage_in=($(GLOBUSRUN_GASS_URL)$PWD/GenerateDrawsGlobus
GenerateDrawsGlobus.$TMP))
(file_clean_up=GenerateDrawsGlobus.$TMP)
(environment=(LD_LIBRARY_PATH $(GLOBUS_LOCATION)/lib)) )"
done
echo $rsl
#start the commands in the background to be non-blocking
globusrun -s -o "$rsl" &
|
Note
(subjobStartType=loose-barrier) must be used in the RSL commands to avoid premature ending of the sub-jobs that do not terminate at the same time as the others because they run slower.
Job submission
The script GenerateDrawsGlobus.sh is used to invoke GenerateDrawsGlobus and to perform the simulation on each execution node.
Example 9-4. GenerateDrawsGlobus.sh
#First argument is the hostname
#Second arguement is the number of draws to simulate
#third argument is the draw to test ("1 21 32 12 24 43 45")
chmod +x ~/GenerateDrawsGlobus.$1
~/GenerateDrawsGlobus.$1 $1 $2 | grep "$3" && echo GOT IT on $HOSTNAME
|
GenerateDrawsGlobus.sh is used as an intermediary to start the computation instead of directly invoking GenerateDrawsGlobus. It is needed because it performs three actions that cannot be described in one RSL string:
It make GenerateDrawsGlobus executable.
It invokes GenerateDrawsGlobus.
It pipes the GenerateDrawsGlobus output through grep.
We use the local GASS server that is started with globusrun and the -s option, to perform the movement of the executables GenerateDrawsGlobus.sh and GenerateDrawsGlobus. Both files are staged in the execution nodes (see the SubmitGlobus script):
(executable=$(GLOBUSRUN_GASS_URL)$PWD/GenerateDrawsGlobus.sh)
(file_stage_in=($(GLOBUSRUN_GASS_URL)$PWD/GenerateDrawsGlobus
GenerateDrawsGlobus.$TMP)
TMP is used for the same reason, as in the previous example, to avoid conflicts between jobs submitted from different nodes so that they do not work on the same files. TMP equals $HOSTNAME, actually the submission node host name. The process ID of the SubmitGlobus script could be used to add more granularity and avoid conflicts between jobs submitted from the same host and from different users or with different parameters.
If one good result is found, the stdout output is redirected locally from the execution node to the submission node, and will appear during the execution.
GenerateDrawsGlobus generates random draws. It is slightly different from the previous GenerateDraws program in that it writes the Monitor file under a different file name: Monitor.“submission node hostname”.
filename="Monitor."; filename.append(argv[1]); OutputFileMonitor.open(filename.c_str());
The “submission node hostname” is actually passed as a parameter (see “GenerateDrawsGlobus.C” on page 352) by the GenerateDrawsGlobus.sh that itself receives these parameters from the RSL command:
In the SubmitGlobus script (executable=GenerateDraws.sh)(arguments=$TMP $n "$param") In GenerateDraws.sh script: ~/GenerateDrawsGlobus.$1 $1 $2 | grep "$3" && echo GOT IT on $HOSTNAME
For a job submitted from m0.itso-maya.com to t1.itso-tupi.com, the RSL string is:
&(resourceManagerContact="c1.itso-cherokee.com") (executable=$(GLOBUSRUN_GASS_URL)/home/globus/JYCode/other/GenerateDrawsGlobus. sh) (arguments=m0.itso-maya.com 300000 "2 3 6 7 8 20 45 55") (subjobStartType=loose-barrier) (file_stage_in=($(GLOBUSRUN_GASS_URL)/home/globus/JYCode/other/GenerateDrawsGlo bus GenerateDrawsGlobus.m0.itso-maya.com)) (file_clean_up=GenerateDrawsGlobus.m0.itso-maya.com) (environment=(LD_LIBRARY_PATH $(GLOBUS_LOCATION)/lib)) )G
GASS servers on the execution nodes
A GASS server is started on all execution nodes. We use globusrun to start this server. The command globusrun is submitted in the background so that it is non-blocking in the script. The URL of the GASS server is written in the file gass-server.# in the current directory, where # is an index used in the script to refer to the execution nodes.
Example 9-5. Starting a remote GASS server using globusrun
for i in $loop
do
rsl='&(executable=$(GLOBUS_LOCATION)/bin/globus-gass-server)(arguments=-c -t
-r)(environment=(LD_LIBRARY_PATH
$(GLOBUS_LOCATION)/lib))(file_clean_up=Monitor.'"$TMP)"
globusrun -o -r ${Nodes[$i]} "$rsl" > gass-server.$i &
done
|
As the GASS server is not started immediately, we test the size of the file gass-server.# before trying to communicate with this server. This size will actually remain null as long as the GASS server started remotely, and has not returned the URL on which it will listen.
if [ -s gass-server.$i ]
then
contact=$(cat gass-server.$i)
globus-url-copy $contact/~/Monitor.$TMP file://$PWD/Monitor.$i
status=$(cat Monitor.$i)
statussum=$(( $status + $statussum ))
echo ${Nodes[$i]}:Monitor $(cat Monitor.$i) %
fi
Finally, the GASS servers are shut down at the end of the script by using globus-gass-server-shutdown, as shown in Example 9-6.
Example 9-6. Shutting down the remote GASS servers
for i in $loop do contact=$(cat gass-server.$i) globus-gass-server-shutdown $contact done |
Monitoring
globus-url-copy is used to copy the Monitor files created by GenerateDrawsGlobus. The Monitor files are not redirected to the submission node via GASS because GenerateDrawsGlobus keeps writing in the file and that would cause a lot of network traffic.
Example 9-7. Monitoring
echo Monitoring
rm -f Monitor.*
statussum=0
while (( $statussum != $NodesNumber ))
do
echo
sleep 5 #we poll every 5 seconds
statussum=0
for i in $loop
do
if [ -s gass-server.$i ]
then
contact=$(cat gass-server.$i)
globus-url-copy $contact/~/Monitor.$TMP file://$PWD/Monitor.$i
status=$(cat Monitor.$i)
statussum=$(( $status + $statussum ))
echo ${Nodes[$i]}:Monitor $(cat Monitor.$i) %
fi
done
statussum=$(( $statussum / 100 ))
done
|
Because the remote GASS server may not have started yet, we check the size of the gass-server.$i file. If empty, that means that no URL has been returned yet and therefore the GASS server has not yet started.
As in Example 9-7 on page 258, the for loop scans the content of each Monitor.$i copied from each execution host and displays them every five seconds.
Implementation
Below we discuss the implementation.
Example 9-8. SubmitGlobus script
#the script takes the tested draw as parameter
#example: ./Submit 3 4 5 32 34 43
n=300000
NodesNumber=8
#temporary filename used by by GenerateDrawsGlobus
#to monitor the job
#we can also use the process id to increase the granularity
TMP=$HOSTNAME
i=0
#the loop variable is used is all the "for" loops
#the format is 1 2 3 4 .... n
loop=""
# use here the broker developped for the publication
# see chapter 8 (mds executable)
for node in $(mds $NodesNumber | xargs)
do
Nodes[$i]=$node
loop=${loop}" "${i}
i=$(( $i + 1 ))
done
echo The number of draws tested is $n
a=$*
#sort the numbers in the specified draw
# 2 45 23 12 32 43 becomes 2 12 23 32 43 45 so that we could use
# grep to test this draw and the ouput of the draw programs.
param=$(echo $a | tr " " "
" | sort -n | xargs )
#Start the gass server on each nodes
# clean up the Monitoring file when leaving
for i in $loop
do
rsl='&(executable=$(GLOBUS_LOCATION)/bin/globus-gass-server)(arguments=-c -t
-r)(environment=(LD_LIBRARY_PATH
$(GLOBUS_LOCATION)/lib))(file_clean_up=Monitor.'"$TMP)"
globusrun -o -r ${Nodes[$i]} "$rsl" > gass-server.$i &
done
#file should be made executable
#on all the execution nodes
echo Jobs submission to the grid
rsl="+"
for i in $loop
do
echo ${Nodes[$i]}
rsl=${rsl}"(&(resourceManagerContact="${Nodes[$i]}")"
rsl=${rsl}"(executable=$(GLOBUSRUN_GASS_URL)$PWD/GenerateDrawsGlobus.sh)(argum
ents=$TMP $n
"$param")(subjobStartType=loose-barrier)(file_stage_in=($(GLOBUSRUN_GASS_URL
)$PWD/GenerateDrawsGlobus
GenerateDrawsGlobus.$TMP))(file_clean_up=GenerateDrawsGlobus.$TMP)(environment=
(LD_LIBRARY_PATH $(GLOBUS_LOCATION)/lib)) )"
done
echo $rsl
globusrun -s -o "$rsl" &
#for monitoring, we copy locally the Monitor files
# created on each compute nodes. This file content the
# percentage of tested draws. Each files is suffixes by
# the nodes number. $statusnum is actually the sum of all
# the percentage (Monitor files) devided by 100. When it
# equals the number of nodes, that means that we have finished
echo Monitoring
rm -f Monitor.*
statussum=0
while (( $statussum != $NodesNumber ))
do
echo
sleep 5 #we poll every 5 seconds
statussum=0
for i in $loop
do
if [ -s gass-server.$i ]
then
contact=$(cat gass-server.$i)
globus-url-copy $contact/~/Monitor.$TMP file://$PWD/Monitor.$i
status=$(cat Monitor.$i)
statussum=$(( $status + $statussum ))
echo $[Nodes[$i]):Monitor $(cat Monitor.$i) %
fi
done
statussum=$(( $statussum / 100 ))
done
#Stop the gassserver
for i in $loop
do
contact=$(cat gass-server.$i)
globus-gass-server-shutdown $contact
done
|
mds is the broker executable described in “Broker example” on page 127. It must be in the PATH because it is invoked by the SubmitGlobus script.
For a a short computation on three nodes the result is:
[globus@m0 other]$ ./SubmitGlobus 2 3 45 6 7 8 20 The number of draws tested is 100000 Jobs submission to the grid d2.itso-dakota.com c2.itso-cherokee.com c1.itso-cherokee.com +(&(resourceManagerContact="d2.itso-dakota.com") (executable=$(GLOBUSRUN_GASS_UR L)/home/globus/JYCode/other/GenerateDrawsGlobus.sh)(arguments=m0.itso-maya.com 100000 "2 3 6 7 8 20 45") (subjobStartType=loose-barrier) (file_stage_in=($(GLOBUSRUN_GASS_URL)/home/globus/JYCode/other/GenerateDrawsGlo bus GenerateDrawsGlobus.m0.itso-maya.com)) (file_clean_up=GenerateDrawsGlobus.m0.itso-maya.com) (environment=(LD_LIBRARY_PATH $(GLOBUS_LOCATION)/lib)) ) (&(resourceManagerContact="c2.itso-cherokee.com") (executable=$(GLOBUSRUN_GASS_URL)/home/globus/JYCode/other/GenerateDrawsGlobus. sh) (arguments=m0.itso-maya.com 100000 "2 3 6 7 8 20 45") (subjobStartType=loose-barrier) (file_stage_in=($(GLOBUSRUN_GASS_URL)/home/globus/JYCode/other/GenerateDrawsGlo bus GenerateDrawsGlobus.m0.itso-maya.com)) (file_clean_up=GenerateDrawsGlobus.m0.itso-maya.com) (environment=(LD_LIBRARY_PATH $(GLOBUS_LOCATION)/lib)) ) (&(resourceManagerContact="c1.itso-cherokee.com") (executable=$(GLOBUSRUN_GASS_URL)/home/globus/JYCode/other/GenerateDrawsGlobus. sh) (arguments=m0.itso-maya.com 100000 "2 3 6 7 8 20 45") (subjobStartType=loose-barrier) (file_stage_in=($(GLOBUSRUN_GASS_URL)/home/globus/JYCode/other/GenerateDrawsGlo bus GenerateDrawsGlobus.m0.itso-maya.com)) (file_clean_up=GenerateDrawsGlobus.m0.itso-maya.com) (environment=(LD_LIBRARY_PATH $(GLOBUS_LOCATION)/lib)) ) Monitoring d2.itso-dakota.com:Monitor 71 % c2.itso-cherokee.com:Monitor 66 % c1.itso-cherokee.com:Monitor 61 % d2.itso-dakota.com:Monitor 100 % c2.itso-cherokee.com:Monitor 100 % c1.itso-cherokee.com:Monitor 96 % d2.itso-dakota.com:Monitor 100 % c2.itso-cherokee.com:Monitor 100 % c1.itso-cherokee.com:Monitor 100 %