Important!

Should I Pursue a Ph.D.?

Become a Professor

Internships

Award and Fellowship Opportunities

Recommended Courses

Ph.D. Qualifying Exam Hints

Typical Conferences We Attend

Presenting

Publishing

Proposal Writing

FDTD and Debugging Notes

MATLAB Notes

Paraview (Plotting) Notes

Linux Notes

FORTRAN Compiler Notes

Lab Computer Notes

Parallelizing Notes

University of Utah Campus Supercomputer Notes

GNU Plot

Opendx

Cloth Posters

• ECE 6970/7970: If you are an RA (doing research on a funded project) you should register for 6970 or 7970 research credits. Max is 9 credits per semester of 6970/7670, but register for less in the beginning of your program and more later on because the tuition benefit will pay for courses for the first 84 credits. Talk to Megan McAllister with questions.
• If English is a Second Language: consider taking EAS-6050, Advanced Spoken English, as needed
• ECE 6340, Numerical Methods in Electromagnetics
• ECE 6322, Microwave Engineering I
• ECE 6323, Microwave Engineering II
• ECE 6324, Antenna Theory and Design
• ECE 6310, Advanced Electromagnetic Fields
• MATH 5620, Numerical Analysis 2
• CS 6230, Parallel Programming HPC
• CS 6963, Parallel Programming for GPUs
• CS 6235, Parallel Programming for Many-Core Architectures
• CS 6160, Computational Geometry
• GEO 6240, Physical Fields II: Electromagnetic Methods
• GEO 6250, Inversion Theory and Applications
• For optics, biophotonics, nano:
  • ECE 6461, Nanophotonics
  • ECE 6460, Biophotonics
  • ECE 6451, Nonlinear Optics
  • ECE 6450, Ultrafast Optics
  • ECE 6440, Integrated Optics and Optical Sensors
  • ECE 6430, Statistical Optics, Interferometry, and Detection
  • ECE 6420, Fourier Optics and Holography
  • ECE 6961, Optics for Energy
  • ECE 6960, Fabrication and Characterization Techniques for Nanostructures

• Your presentation should take about 35 min. (timed without questions from the committee). Expect many questions from the committee during your presentation.
• Keep in mind that there could be errors in the papers you are reviewing. Don't assume everything in the papers is correct.
• Do not just regurgitate what is in the papers. Instead, understand and explain how, why, the significance of the research, its relation to your research, etc.
• Do not necessarily show in your presentation every equation and figure published in the papers. Determine the main points of the paper that you want to describe, and show only those needed to support your main points.
• Understand where the equations in the papers come from, and why the authors chose to use those equations. Also understand each variable in the equations and the form of the equations.
• Understand the figures you put in your presentation. Does the behavior of the waveforms make sense? Why and how did the authors get those results? How would the results change if different values or parameters were used?
• Read through and study in detail as needed the references of the papers.
• What follow-on / future work could be performed relating to the content of the papers?

• IEEE Antennas and Propagation International Symposium and USNC/URSI National Radio Science Meeting (rotates locations, see here for list of upcoming conferences (Note the abstract / paper submission deadline is in mid-January!)
• USNC/URSI National Radio Science Meeting held annually in Boulder, CO in January (Here is the conference website. Note the abstract submission deadline is in September, including the student paper competition). Do not forget to apply for student travel support when submitting your abstract.
• Other conferences are attended by our group depending on the topic of your project, such as the AGU Fall Meeting in December annually in San Francisco or the International Conference on Electromagnetics in Advanced Applications (ICEAA).
  

• Ask yourself:
  
  • What is my main point? (What do I want the audience members to remember most if they only remember one thing.)
  • What is the fundamental purpose of my talk? Why is my work important?
  • How much time do I have to present? (typically 17 minutes at conferences, leaving 3 minutes for questions)
  • Who is the audience? What is their background? How can I tailor my talk to this specific audience?
  • Why was I asked to speak? Or, why did I decide to speak?
  • It is often a good idea to include your primary result(s) towards the beginning of your talk before diving into the details of how you obtained them.
  • What visual medium is most appropriate for this particular situation and audience? (typically PPT or posters at conferences)
• Know your material inside and out.
• Practice, practice, practice! Practice in front of someone, or at a minimum standing up in front of an empty room as if your audience is there. Also, make sure you can get through your material in the time allotted without rushing.
• Don't overwhelm your audience with equations. It will be difficult for them to absorb too many equations in one presentation.

• If you used CHPC resources to obtain your results, make sure to include an acknowledgement: "The support and resources from the Center for High Performance Computing at the University of Utah is gratefully acknowledged."
• If Prof. Simpson used research funds to support you as you performed the research, then make sure to include an acknowledgement to the funding source.
• If your first language is not English, please meet with the writing center to get the English in your paper corrected unless your English writing skills are very strong: writing center, and / or try to get a native English speaker to review your document before providing it to Prof. Simpson to review. (Ask them to keep the manuscript confidential!)
• If you want to get positive reviews, think about what the reviewers need: easy to read and understand; no typos; they need to believe and get excited about your work. Remember, they are volunteering their time to review your paper! Think about why they review proposals -- get excited about good ideas, care about the field...
• It should go without saying that you should not copy any text / figures / etc. from any sources without citing the source. Also, do not use identical text from a source even if it is cited! You need to write out what you want to say in your own words.
• Before giving Prof. Simpson a draft paper to review, make sure to check the document using a spell checker. As you progress through your studies, Prof. Simpson will likely need to make less corrections and changes to your papers.
• Don't be discouraged if you get negative reviews. Some reviewers are very crictical, even of good work. An identical paper can get both fantastic and horrid reviews, depending on who is chosen to review the article, their background, expertise, and demeanor. View their comments as constructive feedback as much as possible, and revise and resubmit!

• Grantsmanship = Salesmanship; Sell an idea to a funding agency based on a critical need within their mission.
• Personalize / tailor your biosketch for each proposal.
• Meet the reviewer's needs: easy to read, no typos, clearly point out innovation and importance of work, make them believe. Remember, they are volunteering their time to review your proposal! Think about why they review proposals -- get excited about good ideas, care about the field...
• For the budget, propose what would be reasonably needed without going over the maximum amount (if there is one). If there isn't a maximum amount stated, propose what you would reasonably need, and the funding agency will cut it down if needed.
• How will you measure success?
• Include letters of support for collaborations.
• For large projects, you could propose to have an advisory board for the project that meets every year or so.
• For NSF:
  • include what science questions you're going to address
  • usually, three objectives are clearly written out
• For NIH:
  • Comprised of 27 different institutes and centers. Submit to the one that best fits the proposed work.
  • Need detailed clinical applicability / feasibility plan.
  • Check the proposal assignment 2-3 weeks after submitting to make sure it was assigned to correct institute / center.
  • Two strikes and you're out: if a proposal is declined twice, you can't submit another proposal on the same topic. If first one is declined, it's hard to recover because reviewers on second proposal know results of first proposal.
  • Only the top 40-50% are even discussed.
  • Specific Aims -- make a promise; state problem and significance and innovation, how uniquely qualified; state aims (going to learn ** by doing **); state outcome / impact. Remember, most reviewers will only see your specific aims! Also, your aims should not be dependent on each other.
  • Research Strategy -- Deliver on that promise: Significance (1/2 to 1 pg, tailored to mission of each aim); Innovation (1/2 pg; why original and exciting, how will create paradigm shift, creative, uniquely qualified to perform the proposed work); Approach (10 pgs or 5 pgs for shorter proposals, relevant preliminary data, experimental approach, discuss strategy OR approach, aim, hypothesis, exp. design, methods, preliminary data, potential problems, expected results, impact)

• Run all codes with the flag "-O3" (or equivalent) so that the code runs faster.
• DEBUGGING:
  
  • Make sure you save different versions of your code, even as you are developing a code. This helps so that you can go back to an earlier version of your code if you suddenly get something like a segmentation fault that you can't fix (i.e. "break" your code, which sometimes happens).
  • Always use "implicit none" in your FORTRAN programs. This ensures that your program doesn't create a new varaible every time a variable is misspelled.
  • Make small changes to your code at a time and carefully test the code after each small change. If you make too many changes at once and the code does not work right or goes unstable, it can be very hard to find where the problem is and you may waste a lot of time trying to find it (or you could have multiple bugs). You can actually save a lot of time by making small changes and testing after each change than if you try to do everything all at once.
  • The best way to debug is to run as few number of time steps as possible (even after just 1 to 3 time steps if you're testing for symmetry). Note: your results may not be perfectly identical / symmetrical for the last 1-2 decimal places even for cases where you would expect them to be because of machine precision. If you aren't just testing for symmetry, you may want to run your code for just 20 or 50 time steps to make sure the source is where it should be and the generated EM wave is propagating correctly.
  • Useful flags (Note, these will slow down your code! Only use when debugging):
    Use "-check bounds" to see if trying to access an array beyond its size.
    Use "-traceback" to get more info on an error
    Use "-wall" to get runtime errors and sometimes additional information.
• Order your do loops (when using FORTRAN) in the following order for optimal computational performance: k,j,i (or 3rd, 2nd, and 1st dimension, for field_value(i,j,k))
• 64-bit processor = 8 bits / byte, 16 bytes / word (15 significant figures), 1 word = 1 real number
  32-bit processor = 8 bytes / word (7 significant figures)
• You should only use 75% of the available memory. The remainder should be reserved for running the machine.
  (# bytes on computer) X (0.75) / (# bytes / word) --> (# words) / (# stored values per cell) --> find max # grid points you should put on each processor
  If you start using virtual memory, your code will slow down substantially or crash.
  So far, the largest code our group has run was on encanto using 384 processing cores and a grid size of 2400 x 2400 x 1740.
• 1st-order accuracy --> error drops linearly
  2nd-order accuracy --> Yee grid
  4th-order accurate --> Hexagonal grid
• Try to dynamically allocate arrays, so can save on memory when arrays not needed:
  real, pointer :: temp(:,:)
  allocate(temp(1:100,1:100))
  deallocate(temp)
• DDT is a symbolic, parallel debugger that allows graphical debugging of MPI and OpneMP applications.

• "unwrap" command to unwrap phase plot that jumps by 2*pi

• Paraview.org
• ftk.org
• Tutorial

• >> man (command name)         --> access manual pages, gives description of command
• Some basic commands:
  
  • >> ls         --> list files in directory
  • >> cd (directoryname)         --> change to directory 'directoryname'
  • type first few letters of a filename and hit the TAB key to autocomplete (as long as there isn't more than one match from the letters you type)
  • >> mkdir (newdirectoryname)         --> creates a new directory having the name 'newdirectoryname'
  • >> mv (oldfilename) (newfilename)         --> move a file to have a new name (Careful: will overwrite 'newfilename' if exists already!)
  • >> mv (filename) (directory)         --> move a file into a directory (Careful: will overwrite file in 'directory' if exists already!)
  • >> cp (filename1) (filename2)         --> copies filename1 to filename2 (Careful: will overwrite 'filename2' if exists already!)
  • >> top         --> display top CPU processes
    k -9 (number)         --> kill process running on the computer corresponding to the id number
  • >> history         --> lists commands previously made
  • >> rm -r (directory name)        --> remove a directory even if files in it
  • >> diff (filename1) (filename2)         --> compares two files and writes notes to screen line numbers and text indicating where and how they are different
  • >> ps         --> see which shell running (csh, ksh, bash)
  • >> setenv (variable) (value)         --> set environment variable
  • >> reset         --> if output is garbled, will reset terminal
  • >> du -sk /directoryname         --> see how much consuming in kB
  • >> du -k /directoryname         --> see how much consuming in kB and lists all files
  • >> du -s lsort -n -r | more         --> lists directory sizes starting with largest (in kB)
• Text editors: gvim, vim, emacs, pico, nano
• Copy files using FUGU, SCP (terminal), or winscp (Windows)
  [if on computer having the file wanting to copy to other computer, and to keep filename same]:
  scp (filename) (username)@(computername):(directory)/.
  [if instead on computer where want to copy file to, and to keep filename same]:
  scp (username)@(computername):(directory)/(filename) .
  Can put a new filename instead of using "."
  Note: Pay attention to which direction between computers you copy the file. If you use the same filename on both machines, you could accidentally overwrite the wrong file.

• Always check your code to make sure no matrices are accessed out of bounds and to get information about any errors (BUT MAKE SURE TO REMOVE THESE FLAGS FOR PRODUCTION RUNS BECAUSE THEY SLOW DOWN THE CODE SIGNIFICANTLY!):
  Intel: -check bounds -tracebackk
  GNU: -fbounds-check -fbacktrace
  PGI: -C
  
• Time your code by running it 100 (or more) time steps and seeing how long just the time-stepping takes. Use this to esimate how long a longer run will take. However, note that the updates take longer when there are non-zero field components, so for larger grids, you should run the test case longer to estimate the time more time steps will take (since far away from the source there will still be non-zero components for a short number of time steps).
• If you want to use a FORTRAN compiler on your laptop before using the supercomputers, some free versions include Silverfrost (personal edition, Plato) for Windows and gfortran for Linux:
  If you are familiar with Windows but not Linux, and simulation run-time is not an important concern, then use the Silverfrost compiler since it is simple to install and use (including a integrated development environment - Plato - for both editing and compiling the code). http://www.silverfrost.com/32/ftn95/ftn95_personal_edition.aspx
  Otherwise, if you need a fast compiler, then gfortran is a good choice. The run-time is about 10 times faster than Silverfrost compiler for my codes. But you must be familiar with Linux. You can also install Cygwin - a Unix-like environment and command-line interface for Windows to use gfortran in Windows. https://gcc.gnu.org/fortran/
• ifort (program name)         --> when using the Intel compiler
• Can use flag -r8 or the like to run a code at double precision without having to go in and change each real number to double precision
  Note: For MPI codes, can use -r8, but also need to change all the send/recv commands to have MPI_REAL8 instead of MPI_REAL!
• Here is an example online tutorial you can use to learn FORTRAN (or just google FORTRAN tutorial): http://www.fortrantutorial.com/
• Unit numbers 100, 101, and 102 are permanently associated with the standard input, standard output, and standard error files, respectively. Do not use these unit numbers for opening / closing your own output files.

• You can access remotely using ssh or similar: In a terminal, type "ssh username@computername"
• ulimit -s unlimited         --> allows you to run a larger code than otherwise, just be careful?
• To log out of the computer and keep your code running:
  >> nohup ./a.out > run.dat &
  (anything that would be written to the screen will be dumped into the file run.dat)
• For Running MPI Codes on Lab Computers:
  Do this once only:
  >> $HOME
  >> touch .mpd.conf
  >> chmod 600 .mpd.conf
  >> vim .mpd.conf
  in this file, type: MPD_SECRETWORD=fdtd
  After doing the above, or at any other time you want to run an MPI code, type:
  >> mpd &
  Then you can compile and run your mpi code.
  To see if the mpd daemon is running correctly:
  >> ps -aef | grep mpd
  To see how many cores are available for processing:
  >> cat /proc/cpuinfo | grep processor | wc -l
  And finally, to run your code (example with 4 processors):
  >> mpirun -np 4 ./a.out
  -or- depending on the machine use:
  >> mpiexec -np 4 ./a.out

• If need to use fewer processors than reserving, use "set env NPROCS 128" or something similar (ex. if reserving 130 processors but using 128 in the model)
• DDT is a symbolic, parallel debugger that allows graphical debugging of MPI and OpenMP applications.
• To debug, you can use the Totalview debugger. If using a Mac, log in with "-X" to turn on tunneling.
• MPI:
  
  • The CHPC on campus has slides posted on their website and periodically runs info sessions on using MPI. Take advatage of these resources and get on the CHPC listserv to learn about upcoming events.
  • Run using
    mpif90 (program name)
    mpif90 datadefine.f subroutines.f mpiFDTD3d.f
  • Time part of a run:
    t1 = MPI_WTIME()         --> make sure to declare t1 and t2!
    t2 = MPI_WTIME()
    write(*,*)"Total time (in seconds)=", t2-t1
  • Can try to use MPI_BARRIER(MPI_COMM_WORLD,ierr) to synchronize all processes before going on, but it is known to not always work correctly. Also, if not all processes reach this command (for example, if it is in an if statement that isn't true for all processes), the code will hang.
  • Make sure to have call MPI_FINALIZE(ierr) at end of code or else you won't leave the supercomputer in a good state and the IT folks will get upset.
  • If you want to monitor the memory, try:
    qstat -m -u (username)
    ssh {top node}
    top
    ctrl-c         --> to go back to home node
    exit
• OpenMP (mostly for shared memory computers)
  pgf90: -mp
  ifort: -openmp
  gcc: -fopenmp
  For running codes on CPUs. Add simple commands (parallelizing hints) into your code as comments, but in a way that OpenMP will see them and will use them to parallelize your code for you. This is a means to avoid having to do tedious coding using MPI. Also, the same OpenMP code can run on a single processor (since the commands are added in as comments) as well as a computing cluster.
  Avoid nested "if" statements.
• OpenACC
  For running codes on GPUs (known to run FDTD simulations faster than CPUs).
  Note that moving data between the CPUs and GPUs is slow (bottleneck) and should be kept to a minimum.
  Avoid nested "if" statements.
  OpenACC Will be rolled into OpenMP 4.0
  

• Use as short of a walltime as possible without jeopordizing your job not finishing in time (otherwise the scheduler will kill your job before it completes). The reason to use a small walltime when possible is because the schedule will run jobs out of order to backfill (fill in shorter jobs when longer jobs complete early).
• Always perform initial debugging and testing on your desktop before moving your code over to the supercomputers.
• When logging on, use your UNID with a lower case "u" in front.
• Make sure to store your results in a secure location when your job completes and to remove any files you no longer need. Some disk space is cleaned regularly, and you will lose your data and codes if you don't store your files in the correct directories.
• You will need to request an account through the CHPC website to get access to any of these supercomputers. Prof. Simpson will be asked to approve the request by CHCP.
• In general, you will want to only read from or write data to files at the very beginning or end of your code. Avoid writing data to files every time step or during time stepping (unless it's an isolated event) because this will slow down your code considerably.
• Click here for more info on our campus supercomputers and how to get access.
• Ember: ssh username@ember.chpc.utah.edu
• Note ember has 12 cores per node.
• There are 6 avaialble GPU nodes on ember (em519-em524). Each has 2 GPU cards of the type M2090 (Fermi generation). The PCUs are identical to the rest of the Ember cluster. Four of the 6 GPU nodes (em519-em522) are onlly used for production runs (max walltime:72h). The remaining 2 GPU nodes (em523 and em524) can be used during office hours for compilation and testing (max wellatime: 1h). Outside of office hours, they are also production run nodes.
  CUDA version 4.2 is installed on the GPU nodes. Thus, to compile a GPU code, the GPU nodes must be accessed interactively:
  qsub -I -l walltime=01:00:00 -A general-gpu
  The OpenACC paradigm can currently only be used with the pgi compilers (pgf90).
  To use GPU's, add to your batch scripts:
  #PBS -A general-gpu
• Ember User Guide is here
• To debug, you can use the Totalview debugger. If using a Mac, log in with "-X" to turn on tunneling.