9.2. Small Blue example
This example shows an example of how to distribute a function across a Grid infrastructure based on the Globus Toolkit 2.2. Note that for readability reasons, not all the needed error checking is done for every Globus Toolkit 2.2 API call.
The purpose of the game (called Puissance 4, in French) is to align four chips to win. In this example, a simple artificial intelligence machine plays against a human.
The artificial intelligence is implemented in the GAME.C program available in “GAME Class” on page 337 and works in the following ways:
It evaluates the value of each position from the first column to the eighth.
For each position, it also evaluates the next positions that the adversary could possibly play and reevaluates its tested position accordingly.
When all positions are evaluated, the best move is chosen.
The standalone or non-gridified version of the game is available in “SmallBlue.C (standalone version)” on page 331. The algorithm as well as the GAME class is not studied in the publication. The GAME class provides methods to display the game, to check if someone has won, to play the position decided by the players, and to test if a player can play a specific column.
Figure 9-3. Problem suitable for Grid enablement
In the extract of the SmallBlue.C (Example 9-9 on page 264) source code, we can see that:
The evaluation of a position is performed via the Simulate() function called from the main() program.
The SmallBlue application uses the object Current of type GAME to store the game data. This data is used to perform the evaluation of a position. The evaluation of a position is performed in the function Simulate() that takes two parameters: The tested position and the game data. Simulate()returns the value of the evaluation. This function is called for each column in main().
The Value() method of the GAME class is used to evaluate the value of a position. It takes the tested column as a parameter as well as the player (BLACK/WHITE) who is playing the column. This method is called in Simulate().
Example 9-9. Standalone version
int Simulate(GAME newgame, int col) {
int l=0,s;
int start=newgame.Value(col,WHITE);
newgame.Play(col,WHITE);
l=0;
for(int k=1;k!=XSIZE+1;k++) {
s=newgame.Value(k,BLACK);
if (s>l)
l=s;
};
start-=l;
return start;
}
main() {
//*************************** Start ****************************/
GAME Current(XSIZE,YSIZE);
int s,l,k,toplay;
char c[2];
while (true) {
Current.Display();
do {
cout << "?";
cin >> c;
c[1]=' ';
l=atoi(c);
} while ((l<1) || (l>XSIZE) || !Current.CanPlay(l) );
Current.Play(l,BLACK);
if (Current.HasWon(l,BLACK)) {
Current.Display();
exit(1);
};
//*************************** Simulation *****************************/
l=-100000;
for(k=1;k!=XSIZE+1;k++) {
if (Current.CanPlay(k)) {
//*************************** call Simulate *****************************/
s=Simulate(Current,k);
if (s>l) {
l=s;
toplay=k;
};
};
};
if (l==-100000) {
cout << "NULL" << endl;
exit(1);
};
Current.Play(toplay,WHITE);
if (Current.HasWon(toplay,WHITE)) {
Current.Display();
exit(1);
};
};
};
|
The purpose of this example is to gridify the application by executing the Simulate() function on a remote host:
Each evaluation of a tested column can be executed independently.
Each evaluation modifies the game data when simulating an attempt so each job needs to have its own copy of the game. This behavior is also present in the function Simulate() where a new object GAME called newgame is created specifically for the evaluation, and used to store successive tested positions. Therefore, the game data must be replicated on all execution nodes.
9.2.1. Gridification
To gridify this application, we will use two programs:
One called SmallBlueMaster that will submit the job, gather the results, and be the interface with the human player
One called SmallBlueSlave that will perform the simulation and returns the result to SmallBlueMaster
The source code for these two programs is available in “SmallBlueMaster.C” on page 332 and “SmallBlueSlave.C” on page 336.
Figure 9-4. Gridified SmallBlue
One problem is that the simulate() function uses two variables and returns one value that needs to be passed to and retrieved from a remote host. We cannot use inter-process communications between the nodes.
A solution is to serialize objects by storing the instance value on disk and use the Globus Toolkit 2.2 data movement functions:
By using GRAM and GASS systems. The Current object will be serialized to disk and transferred to each remote host. The tested position will be passed as an argument to SmallBlueslave, which will load the current value and therefore will recreate the same environment that exists in SmallBlueMaster.
By using the GRAM and GASS subsystems, all the results of the execution nodes will be output to the same file, eval, on the master node.
Communication is accomplished:
By a local GASS server started on the master node and listening on port 10000
By the GASS servers started on each execution node by the GRAM, which will map standard input and output to remote files
The following RSL command describes this process:
&(executable=smallblueslave) (arguments=<tested column>) (stdout=https://<masternode>10000/<localdir>/eval) (stdin=https://<masternode>:10000/<localdir>/GAME) (count=1)"
Figure 9-5. How to transfer an object via GRAM and GASS
The local GASS server started by SmallBlueMaster transparently provides access to the eval and GAME file to the remote execution nodes via GRAM and the associated GASS server. It arbitrarily listens on port 10000. Two functions, StartGASSServer() and StopGASSServer(), wrap the Globus Toolkit 2.2 API calls to start and stop the local GASS server. The source code of these functions is available in “itso_gass_server.C” on page 325.
We can see in the SmallBlueSlave code source that it reads and writes only on standard input and output channels (via cin and cout standard iostream objects). It will nevertheless transparently work on remotely stored files thanks to the stdin and stdout keywords in the RSL job description language:
(stdout=https://<masternode>:10000/<localdir>/eval) (stdin=https://<masternode>:10000/<localdir>/GAME)
stdin is mapped to the GAME file located on the master node and generated by the call to the method ToDisk() of the object Current (object serialization). Another way to proceed would be to use Globus sockets to transfer the serialized object without the need of intermediate files.
stdout is mapped to the eval file located on the master node. SmallBlueSlave will write the value of a tested position to this file. All the nodes write to eval so all output will be appended to this file.
SmallBlueSlave also needs a parameter (the tested column) that will be passed as a parameter to the program via the (arguments= )expression of the RSL job submission string. Then SmallBlueSlave uses argv[1] to retrieve this parameter.
Finally, we will use the GridFTP protocol (as an example), to transfer the SmallBlueSlave executable to a remote host. The transfer is achieved via the ITSO_GLOBUS_FTP_CLIENT class and its transfer() method. The source code is available in “itso_globus_ftp_client.C” on page 313.
9.2.2. Implementation
We will use three C++ classes that wrap Globus C calls:
ITSO_CB will be used as generic callback type for all globus functions that need a callback. Note that these objects are always called from a static C function whose one argument is the object itself. ITSO_CB implements the mutex, condition variables synchronization mechanism always used with the Globus Toolkit 2 non-blocking or asynchronous functions. ITSO_GRAM_JOB and ITSO_GLOBUS_FTP_CLIENT both derive from ITSO_CB. See “ITSO_CB” on page 315 and its explanation in “Callbacks” on page 109.
ITSO_GRAM_JOB will be used to submit a job. See “ITSO_GRAM_JOB” on page 316.
ITSO_GLOBUS_FTP_CLIENT, which is a wrapper class to the C globus client ftp functions, will perform a transfer from a file stored in a storage server to a remote URL. See “ITSO_GLOBUS_FTP_CLIENT” on page 311.
The complete source code of the two programs is available in “SmallBlue example” on page 331.
StartGASSServer() and StopGASSServer() are the two functions respectively used to start and stop the local GASS server that will retrieve the result of the evaluation nodes. The source code is provided in “StartGASSServer() and StopGASSServer()” on page 324
To copy the file SmallBlue in parallel to each remote host, we use the GridFTP protocol via the ITSO_GLOBUS_FTP_CLIENT:
vector<ITSO_GLOBUS_FTP_CLIENT*> transfer;
globus_module_activate(GLOBUS_FTP_CLIENT_MODULE);
string dst;
for(i=0;i!=8;i++) {
cout << node[i] << endl;
dst="gsiftp://"+node[i]+"/~/SmallBlueSlave";
transfer.push_back(new ITSO_GLOBUS_FTP_CLIENT("SmallBlueSlave",
const_cast<char*>(dst.c_str())));
};
for(i=0;i!=8;i++)
transfer[i]->StartTransfer();
for(i=0;i!=8;i++)
transfer[i]->Wait();
globus_module_deactivate(GLOBUS_FTP_CLIENT_MODULE);
We also need to make SmallBlue executable on the remote hosts because the file copied by GridFTP is copied as a plain file and not as an executable.
for(i=0;i!=8;i++) {
rsl_req = "&(executable=/bin/chmod) (count=1) (arguments= "+x"
SmallBlueSlave)";
if ( job[i]->Submit(node[i],rsl_req))
exit(1);
}
for(i=0;i!=8;i++)
job[i]->Wait();
Example 9-10. SmallBlue Gridfication - Initialization - SmallBlueMaster.C
main() {
//Start a GASS server locally that will listen on 10000 port
//all the results of the evualtion. We will stop it at the end
//It cannot be defined as a standalone class because the static callback
//does not take any argument. So it is impossible afterwards in the
//callback to refer to the object.
StartGASSServer(10000);
//ITSO_GRAM_CLIENT does not start the module
// lets do it
if (globus_module_activate(GLOBUS_GRAM_CLIENT_MODULE) != GLOBUS_SUCCESS)
{
cerr << " Cannot start GRAM module";
exit(2);
};
// the game
GAME Current(XSIZE,YSIZE);
// used to temporary store columns positions, evaluation results
int s,l,k,toplay;
// used to store human inputs
char c[2];
// The node vector should be initialized with the value of the nodes
// what is missing here is the globus calls to the globus MDS server
// to get these values. So for the exercise, you can use grid-info-search
// to find 8 hosts on which you can submit your queries.
vector<string> node;
// ask the broker to find 8 nodes
itso_broker::GetLinuxNodes(node,8);
// variable used in all for loops
int i;
// Here we want test the existence of the file as there is
// no such checking in the ITSO_GLOBUS_FTP_CLIENT class
FILE* fd = fopen("SmallBlueSlave","r");
if(fd == NULL)
{
printf("Error opening local smallblueslave");
exit(2);
}
else {
//that fine, lets go for FTP
// we can close fd descriptor because a new one
// will be opened for each ITSO_GLOBUS_FTP_CLIENT object
fclose(fd);
// the ITSO_CB callback object is used to determine
// when the transfer has been completed
vector<ITSO_GLOBUS_FTP_CLIENT*> transfer;
//never forget to activate the Globus module you want to use
globus_module_activate(GLOBUS_FTP_CLIENT_MODULE);
// 8 transfer, let create 8 locks
string dst;
for(i=0;i!=8;i++) {
cout << node[i] << endl;
dst="gsiftp://"+node[i]+"/~/SmallBlueSlave";
transfer.push_back(new
ITSO_GLOBUS_FTP_CLIENT("SmallBlueSlave",const_cast<char*>(dst.c_str())));
};
// Let s begin the transfer in parallel (in asynchronous mode)
for(i=0;i!=8;i++)
transfer[i]->StartTransfer();
// Let wait for the end of all of them
for(i=0;i!=8;i++)
transfer[i]->Wait();
globus_module_deactivate(GLOBUS_FTP_CLIENT_MODULE);
};
// get the hostname using the globus shell function
// instead of POSIX system calls.
char hostname[MAXHOSTNAMELEN];
globus_libc_gethostname(hostname, MAXHOSTNAMELEN);
// used to store the RSL commands.
string rsl_req;
//create all the jobs objects that will be used to submit the
//requests to the nodes. We use a vector to store them.
vector<ITSO_GRAM_JOB*> job;
for(i=0;i!=8;i++) {
job.push_back(new ITSO_GRAM_JOB);
};
// By using gridftp SmallBlueSlave is copied onto the rmeote hosts
// as a plain file. needs to chmod +x to make it executable
// otherwise the job submission will fail
cout << "chmod +x on the remote hosts to make SmallBlueSlave executable" <<
endl;
for(i=0;i!=8;i++) {
rsl_req = "&(executable=/bin/chmod) (count=1) (arguments= "+x"
SmallBlueSlave)";
if ( job[i]->Submit(node[i],rsl_req))
exit(1);
}
for(i=0;i!=8;i++)
job[i]->Wait();
//finished stop everything
StopGASSServer();
}
|
The game between the two players is run in an infinite loop in which we repeatedly:
Serialize the current game:
Current.ToDisk("GAME");Submit the calculation of the value for each column:
unlink("eval"); //remove the eval file for(i=0;i!=8;i++) { cout << "submission on " << node[i] << endl;; char tmpc[2]; sprintf(tmpc,"%d",i); // build the RSL commands rsl_req = "&(executable=SmallBlueSlave) (arguments="; rsl_req+= tmpc[0]; rsl_req+= ") (stdout=https://"; rsl_req +=hostname; rsl_req +=":10000"; rsl_req +=get_current_dir_name(); rsl_req +="/eval) (stdin=https://"; rsl_req +=hostname; rsl_req +=":10000"; rsl_req +=get_current_dir_name(); rsl_req +="/GAME) (count=1)"; // submit it to the GRAM if (Current.CanPlay(i)) if (job[i]->Submit(node[i],rsl_req)) exit(1); }; // And Wait for(i=0;i!=8;i++) if (Current.CanPlay(i)) job[i]->Wait();Retrieve the results:
ifstream results("eval"); while (!results.eof()) { results >> k >> s; // get the best one if (s>l) { l=s; // store its value toplay=k; //remember the column to play }; }; results.close();
Finally, we exit the loop when one of the two players has won, by using the HasWon() method of the Current object:
if (Current.HasWon(l,BLACK)) {
Current.Display();
break;
Example 9-11. SmallBlue Gridification - SmallBlueMaster.C
..........................................
while (true) {
Current.Display();
do {
cout << "?";
cin >> c;
c[1]=' ';
l=atoi(c);
} while ((l<1) || (l>XSIZE) || !Current.CanPlay(l) );
Current.Play(l,BLACK);
if (Current.HasWon(l,BLACK)) {
Current.Display();
break;
};
// Serialize to disk the Current variable
// so that it could be used by the GRAM
// subsystem and transferred on the remote execution
// nodes
Current.ToDisk("GAME");
Current.Display();
cout << endl;
// remove eval file for each new jobs submission
// otherwise results will be appended to the same files
unlink("eval");
for(i=0;i!=8;i++) {
cout << "submission on " << node[i] << endl;;
char tmpc[2];
sprintf(tmpc,"%d",i);
// build the RSL commands
rsl_req = "&(executable=SmallBlueSlave) (arguments=";
rsl_req+= tmpc[0];
rsl_req+= ") (stdout=https://";
rsl_req +=hostname;
rsl_req +=":10001";
rsl_req +=get_current_dir_name();
rsl_req +="/eval) (stdin=https://";
rsl_req +=hostname;
rsl_req +=":10001";
rsl_req +=get_current_dir_name();
rsl_req +="/GAME) (count=1)";
// submit it to the GRAM
if (Current.CanPlay(i))
if (job[i]->Submit(node[i],rsl_req))
exit(1);
};
// And Wait
for(i=0;i!=8;i++)
if (Current.CanPlay(i))
job[i]->Wait();
// worse case :-)
l=-100000;
//Here we are reading the eval files. All the jobs
//has been completed so we should have all the results
//in the eval file
ifstream results("eval");
while (!results.eof()) {
results >> k >> s;
// get the best one
if (s>l) {
l=s; // store its value
toplay=k; //remember the column to play
};
};
results.close();
// nothing in the file, that means we cannot play
// so it is NULL
if (l==-100000) {
cout << "NULL" << endl;
break;
};
// AI plays here and checks if it won
Current.Play(toplay,WHITE);
if (Current.HasWon(toplay,WHITE)) {
Current.Display();
break;
};
};
..........................................
|
The slave code is small. GAME.C implements the game artificial intelligence.
The slave begins to read the serialized object from standard input (actually mapped to GAME file on the master node):
Current.FromDisk();
Then the slave only tests the eight positions that can be played by the adversary. It writes the result of the evaluation to standard output. The GASS server started by GRAM actually maps this standard output to the file eval on the submission node.
Example 9-12. SmallBlueSlave.C
#include <iostream>
using namespace std;
#include "GAME.C"
main(int arc, char** argv) {
GAME Current(XSIZE,YSIZE);
//load the Current object from the disk
//this object was copied from the submission node
Current.FromDisk();
//which column should we simulate ?
int col=atoi(argv[1]);
int start=Current.Value(col,WHITE);
Current.Play(col,WHITE);
int l=0,s,k;
for(k=1;k!=XSIZE+1;k++) {
s=Current.Value(k,BLACK);
if (s>l)
l=s;
};
start-=l;
// send back the information to the server
cout << col << " " << start << endl;
};
|
9.2.3. Compilation
First generate the appropriate globus makefile header that will be later included in the Makefile. Use globus-makefile-header and specify all the needed globus modules.
globus-makefile-header --flavor=gcc32 globus_io globus_gss_assist globus_ftp_client globus_ftp_control globus_gram_job globus_common globus_gram_client globus_gass_server_ez > globus_header
Compile with the following Makefile:
make -f MakefileSmallBlue
Example 9-13. MakefileSmallBlue
globus-makefile-header --flavor=gcc32 globus_io globus_gss_assist globus_ftp_client globus_ftp_control globus_gram_job globus_common globus_gram_client globus_gass_server_ez > globus_header include globus_header all: SmallBlueSlave SmallBlueMaster SmallBlue %.o: %.C g++ -c $(GLOBUS_CPPFLAGS) $< -o $@ SmallBlue:SmallBlue.o GAME.o g++ -o $@ -g $^ SmallBlueSlave:SmallBlueSlave.o GAME.o g++ -o $@ -g $^ SmallBlueMaster: GAME.o SmallBlueMaster.o itso_gram_job.o itso_cb.o itso_globus_ftp_client.o itso_gass_server.o g++ -g -o $@ $(GLOBUS_CPPFLAGS) $(GLOBUS_LDFLAGS) $^ $(GLOBUS_PKG_LIBS) |
9.2.4. Execution
Issue grid-proxy-init to acquire a valid credential in the grid.
Start SmallBlueMaster and enter the column number you want to play:
[globus@m0 JYCode]$ ./SmallBlueMaster we are listening on https://m0.itso-maya.com:10000 chmod +x on the remote hosts to make SmallBlueSlave executable Contact on the server https://t1.itso-tupi.com:34475/16083/1047519201/ Contact on the server https://t2.itso-tupi.com:33326/6614/1047519203/ Contact on the server https://t0.itso-tupi.com:58412/7839/1047519203/ Contact on the server https://t3.itso-tupi.com:55107/29288/1047519236/ Contact on the server https://t2.itso-tupi.com:33328/6615/1047519203/ Job Finished on: https://t1.itso-tupi.com:34475/16083/1047519201/ Contact on the server https://t0.itso-tupi.com:58414/7840/1047519203/ Contact on the server https://t1.itso-tupi.com:34478/16085/1047519201/ Contact on the server https://t3.itso-tupi.com:55110/29289/1047519237/ Job Finished on: https://t2.itso-tupi.com:33328/6615/1047519203/ Job Finished on: https://t2.itso-tupi.com:33326/6614/1047519203/ Job Finished on: https://t1.itso-tupi.com:34478/16085/1047519201/ Job Finished on: https://t0.itso-tupi.com:58412/7839/1047519203/ Job Finished on: https://t0.itso-tupi.com:58414/7840/1047519203/ Job Finished on: https://t3.itso-tupi.com:55107/29288/1047519236/ Job Finished on: https://t3.itso-tupi.com:55110/29289/1047519237/ ----------- | | | | | | | | | | | | | | | | | | | | ---------- 12345678?3 ----------- | | | | | | | | | | | | | | | | | | | I | ----------- 12345678 submission on t0 submission on t1 submission on t2 submission on t3 submission on t0 submission on t1 submission on t2 submission on t3 Contact on the server https://t2.itso-tupi.com:33335/6644/1047519211/ Contact on the server https://t1.itso-tupi.com:34489/16114/1047519209/ Contact on the server https://t0.itso-tupi.com:58421/7869/1047519211/ Staging file in on: https://t2.itso-tupi.com:33335/6644/1047519211/ Staging file in on: https://t1.itso-tupi.com:34489/16114/1047519209/ Staging file in on: https://t0.itso-tupi.com:58421/7869/1047519211/ Contact on the server https://t2.itso-tupi.com:33339/6645/1047519211/ Contact on the server https://t1.itso-tupi.com:34485/16113/1047519208/ Contact on the server https://t0.itso-tupi.com:58424/7870/1047519211/ Staging file in on: https://t1.itso-tupi.com:34485/16113/1047519208/ Staging file in on: https://t2.itso-tupi.com:33339/6645/1047519211/ Contact on the server https://t3.itso-tupi.com:55121/29325/1047519244/ Staging file in on: https://t0.itso-tupi.com:58424/7870/1047519211/ Staging file in on: https://t3.itso-tupi.com:55121/29325/1047519244/ Contact on the server https://t3.itso-tupi.com:55117/29324/1047519244/ Staging file in on: https://t3.itso-tupi.com:55117/29324/1047519244/ Job Finished on: https://t2.itso-tupi.com:33335/6644/1047519211/ Job Finished on: https://t1.itso-tupi.com:34485/16113/1047519208/ Job Finished on: https://t2.itso-tupi.com:33339/6645/1047519211/ Job Finished on: https://t0.itso-tupi.com:58424/7870/1047519211/ Job Finished on: https://t1.itso-tupi.com:34489/16114/1047519209/ Job Finished on: https://t0.itso-tupi.com:58421/7869/1047519211/ Job Finished on: https://t3.itso-tupi.com:55121/29325/1047519244/ Job Finished on: https://t3.itso-tupi.com:55117/29324/1047519244/ ----------- | | | | | | | | | | | | | | | | | | | IO | ----------- 12345678?