This guide covers how to integrate the FDTD Job Manager with Sun Grid Engine (SGE). After following these steps, running a simulation on a cluster will be just a seamless as running it on your local computer. You will be able to just click “Run” and the job will automatically be submitted to the SGE queue (the file will be transferred if necessary), the Lumerical Job Manager will periodically update with the simulations current state and progress, and upon completion the results will be automatically loaded in your current FDTD session.
The job manager options: Quit, Quit & Save, and Force Quit, are currently not supported.
- FDTD Solutions
- Cluster with: Sun Grid Engine (qsub), X11-Display, FDTD installed and licenses.
See: How to Use Lumerical Products With AWS-ParallelCluster
Add a new resource: The IP/hostname and Processes arguments are ignored. Set Threads=0
- Edit the advanced resource settings: Copy the configuration pictured bellow, substituting your local paths. You can find the full path to your local Python by running the command, "which python" in terminal. without the quotes.
- Optional: You can specify additional commands to be passed to qsub using the flag -qargs followed by a quoted string of arguments.
- Duplicate the Resource as many times as you want parallel jobs to run.
(Optional) Launching FDTD Jobs on a Remote Cluster From Your Local Computer
If you would like to launch jobs from your local computer for a more seamless experience, you can configure FDTD Solutions on your local computer following the same steps as above and enable ssh in qsub_submit.py:
USE_SSH = True
SSH_LOGIN = "email@example.com"
SSH_KEY = expanduser('~/lumerical-demo.pem')
CLUSTER_CWD = '~/' # NFS directory shared by all nodes
CLUSTER_FDTD_ENGINE = '/opt/lumerical/fdtd/bin/fdtd-engine-mpich2nem'
This mode will copy your .fsp file to the remote server using scp*, launch the job with ssh*/qsub, and copy all the generated files back to your local computer at the end.
Windows users will have to install Git Bash (for ssh and scp).
You can now run any FDTD simulation (single, sweeps, optimizations, etc.) directly from the FDTD Job Manager with live updates on simulation progress (quitting a job is currently not supported).