UPDATE 7-16-2010: Added a couple of lines to GPURUN.BAT that allow each GPU client to automatically copy the new DLLs required by GPU3 into their local directories.
Jason how the hell did you keep your sanity installing all those clients... since myself and a few others are either trying to get installed or in the process of receiving gtx295's can we ask what os's you're using and what flags or setup process you used???
Smoke, mirrors, batch files, and the GPU Console Client (not Windows Tray Client).
I have a network share on one of the computers, node Dagny since it was the first one I built. Looks like this:
As I add a client machine I name it and add a corresponding folder on node Dagny under C:\FAH\MACHINES\. On each of the clients I map \\DAGNY\FAH as network drive F: (including on node DAGNY where the share resides, you'll see why in a sec).
Next is a bit of secret sauce. At some point I did a basic GPU client install with -configonly for each of 8 GPUs to specify their team, machine ID, etc. But I copied the 8 directories off to a USB stick before I ever ran the clients
... This gives me 8 virgin GPU folders GPU0-GPU7 with just the basic cudart.dll, client.cfg and whatnot. I copy those virgin GPU client folders into Dagny's C:\FAH\MACHINES\NEWNODENAME\ folder. Still with me? Now I have C:\FAH\MACHINES\NEWNODENAME\GPU0 through C:\FAH\MACHINES\NEWNODENAME\GPU7 sitting there ready to fire up a client for the first time, make the WORK directory, download the fah_core, download work, etc.
In Dagny's C:\FAH folder (the "root" of the F: share) I have a simple batch file called RUN8.BAT (I also have RUN1, RUN2, RUN4, RUN6, etc.):
START CMD /C "F:\GPU\GPURUN.BAT 0"
START CMD /C "F:\GPU\GPURUN.BAT 1"
START CMD /C "F:\GPU\GPURUN.BAT 2"
START CMD /C "F:\GPU\GPURUN.BAT 3"
START CMD /C "F:\GPU\GPURUN.BAT 4"
START CMD /C "F:\GPU\GPURUN.BAT 5"
START CMD /C "F:\GPU\GPURUN.BAT 6"
START CMD /C "F:\GPU\GPURUN.BAT 7"
Now, in the GPU folder under the share I have the GPU F@H client renamed "GPU.EXE" and its DLLs along with a simple batch file called GPURUN.BAT:
TITLE %USERDOMAIN% - GPU %1
@ECHO Changing to directory F:\MACHINES\%USERDOMAIN%\GPU%1
@ECHO COPYING NEW DLL FILES (IF ANY)
XCOPY /D /Y F:\GPU\*.DLL F:\MACHINES\%USERDOMAIN%\GPU%1
@ECHO STARTING GPU%1 IN %CD%
F:\GPU\GPU.EXE -gpu %1 -verbosity 9 -advmethods -forceasm -forcegpu nvidia_g80 %2 %3 %4 %5 %6
See, that batch file uses the %USERDOMAIN% command shell macro to move to the correct MACHINE directory on the network share and then the 0-7 command line variable from the first batch file to switch to the correct GPU# directory on the network share.
Â Â becomes something like
I create a shortcut
to the F:\RUN8.BAT batch file on the desktop of the new client. Now when I click on the shortcut it runs the batch file with all the START commands in it which starts 8 shells by calling the second batch file 8 times. The second batch file changes to the correct directory for that specific GPU client, specifies all of the command line parameters for the client and adds a nifty title to the CMD window so I can easily tell the clients apart at a glance (i.e. "REARDEN - GPU 3")
Notice though that there are extra batch file command line macros %2-%6 after the standard command line. These additional command line options will be empty (null) if the RUN8.BAT batch file doesn't use them and therefore have no effect, but if you want to have some GPUs run with additional options you can copy
the RUN8.BAT file instead of creating a shortcut to it. Then in the copy of the batch file on your desktop you can add additional command line options to customize the behavior of some GPU clients to be different than the default. This gives a bit more flexibility to the system -- not every
client must run the same command line.
Since every single GPU client ultimately uses GPURUN.BAT to start, I can change every client's command line in one central location. Since every GPU client uses the same GPU.EXE to run, I can update the client software by replacing one executable file.
I arrived at this solution after much gnashing of teeth trying to keep the 34 clients in the rack coordinated. Originally I had discrete setups on each node's drive, but I was making many errors trying to manually keep them synched.
Here's the neat part. When I add a new node the only thing I have to do to the local machine is map \\DAGNY\FAH to F:, copy a shortcut to F:\RUN8.BAT to the desktop and double click it. That's IT
. There is no
F@H software on the client machines unless you want to run SMP.
In summary the entire node addition procedure is:
1. Go to node Dagny, C:\FAH\MACHINES
2. mkdir NEWNODENAME
3. Copy GPU0-GPU7 virgin dirs from USB stick to C:\FAH\MACHINES\NEWNODENAME
4. Go to new machine, map \\DAGNY\FAH to F:
5. Create shortcut to F:\RUN8.BAT on new client's desktop
6. Click shortcut and watch 8 console GPU clients spring up
Finished product looks like this (click to enlarge):