13.4 Running Simulations

13.4.1 Dongle Types and Setup

TUFLOW licences are distributed on one of three forms: via a WIBU Codemeter USB hardware lock dongle, a digital software lock licence or a cloud licence key. Licenses are available as either Local or Network types. Collectively, the full range of available hosting and licence type options includes:

  1. Local or Standalone Licence (on a USB hardware lock dongle or digital software lock licence) – In this form, TUFLOW simulations can only be run on the computer hosting the licence container. The defined licence count limits the number of TUFLOW simulations that can be run simultaneously on that computer. For example, a Local 1 licence allows a single simulation to be run at a time on the machine. A Local 4 licence allows up to 4 concurrent simulations on the machine.

  2. Network Licence on a USB hardware lock dongle or digital software lock licence – In this form, the licence container is hosted on a computer or server and other computers on the same network can access the available licences. The defined licence count limits the number of TUFLOW simulations that can be run in parallel across multiple computers. For example, a Network 5 licence allows up to 5 concurrent simulations, which could all be run on one computer, each on a different computer or any combination in between.

  3. Network Cloud Licence – This licence type has the same functionality as the Network Licence described above, but the licence server is hosted on a WIBU cloud server managed by TUFLOW. There are two available configurations for a Network Cloud Licence.

    • Cloud Direct Licence: All modelling computers link directly to the WIBU server over an internet connection.
    • Cloud Server Licence: A company server acts as the licence host and connects to the WIBU server over an internet connection. Modelling computers then connect to the company server via the company’s network.

Refer to the TUFLOW Wiki Licensing Page for further details regarding all of the above licencing options.

Note, TUFLOW transitioned to the WIBU licensing platform in 2010 as the previous provider, ‘Softlok’, did not support 64-bit systems. The older ‘Softlok’ dongles are no longer supported. If using TUFLOW versions prior to the 2010 release (2009-07, 2008-08, 2007-07 and 2006-06), the ‘DB’ release builds will need to be used to function with a WIBU Codemeter USB dongle hardware lock. The WIBU Codemeter digital software lock licence and cloud licencing were introduced in the 2016-03-AF release and are not available in earlier release versions.

13.4.1.1 Protocols for Accessing Dongles

If more than one type of dongle is available the protocols for taking and checking licences are:

  1. WIBU Codemeter licences are searched for, and if a licence is free it is taken.
  2. If no licence is available, you can optionally set for TUFLOW to continue to try and find an available WIBU dongle licence. This is achieved using the “C:\BMT\TUFLOW_Dongle_Settings.dcf” file described below. This is useful if there are no free licences and you wish to start a simulation (the simulation will start once a free licence becomes available).
  3. Once a simulation is under way, if the licence is lost, TUFLOW will try to regain a licence. For example, if a simulation is started using a local WIBU licence and that licence is disconnected, TUFLOW will search for another available licence to continue the simulation.

There are five varieties of licence type / vendor:

  • BMT physical USB lock (dongle)
  • BMT software lock (tied to a particular machine)
  • Aquaveo (SMS) USB lock
  • Jacobs (Flood Modeller) USB lock
  • Jacobs (Flood Modeller) software lock

With numerous licencing options available, setting the preferred licence type can (slightly) speed the simulation start-up. The licence search order can be set via a licence control file “TUFLOW_licence_settings.lcf”, this replaces the TUFLOW_Dongle_Settings.dcf. Note that this file can occur in several locations. When looking for a licence setting file TUFLOW searches in the following locations:

  1. A “TUFLOW_licence_settings.lcf” in the same location as the TUFLOW executable.
  2. A “TUFLOW_licence_settings.lcf” in a TUFLOW folder in the “ProgramData” environment variable location. By default this will resolve to C:\ProgramData\TUFLOW\TUFLOW_licence_settings.lcf. Entering %programdata% into the windows explorer path will take you to the location.
  3. C:\BMT\TUFLOW_Licence_Settings.lcf
  4. C:\BMT\TUFLOW_Dongle_Settings.dcf

If no licence settings files are found, TUFLOW defaults to the order listed above. WIBU Firm Code Search Order can be used to control the search order in the TUFLOW_licence_settings.lcf file.

Note: Use of C:\BMT\ above is for legacy reasons and is not recommended as read/write access to local drives is now often blocked for protection against hackers.

13.4.1.2 TUFLOW_Licence_Settings.lcf File

The TUFLOW Licence Settings (.lcf) file can be used to set WIBU settings, such as retry time interval and count. It has the same form and notation as a TUFLOW .tcf file.

This file is optional and if not found the settings below are the default:

! Use this file to set general and WIBU specific dongle parameters
! Use ! or # to comment out commands or make comments
Simulations Log Folder == C:\ProgramData\TUFLOW\<username>\log ! Path or URL to global .log file
WIBU Retry Time == 60 ! seconds. Values less than 3 are reset to 3. Default = 60.
WIBU Retry Count == 0 ! Use -1 for indefinitely.
WIBU Dongles Only == OFF ! If ON, searches for WIBU dongles only. Default is OFF.

In the above, the:

  • Simulations Log Folder sets the folder path or URL to a folder forlogging all simulations. If the keywords “DO NOT USE” occur within the folder path or URL, this feature is disabled. Also see Simulations Log Folder.
  • WIBU Retry Time sets the interval in seconds for retrying to take a licence or regain a lost licence. The default is 60 and values less than 3 are set to 3.
  • WIBU Retry Count sets the number of times to retry for a licence at the start of a simulation. By default, if a licence is lost during the simulation, TUFLOW tries indefinitely to regain a licence so as not to lose the simulation.
  • WIBU Dongles Only if set to ON will force TUFLOW to only search for WIBU dongles.

13.4.1.3 Dongle Failure during a Simulation

If TUFLOW fails to recognise a network lock during a simulation (e.g. the network dongle server computer is down) it enters a holding pattern and continues trying until a license is found.

For local, standalone locks, TUFLOW prompts with a message that the lock could not be found. If it’s a USB lock try a different USB port, and press Enter to continue.

13.4.2 Starting a Simulation

There are a number of ways TUFLOW can be started, in each case the TUFLOW executable is started with the TUFLOW control file (.tcf) as the input argument. There are a number of optional switches that can be specified when starting a simulation; these are outlined in Table 13.2.

Starting a TUFLOW simulation can be carried out in a multitude of methods:

  1. From within a text editor such as Notepad++
  2. Through a batch file
  3. From within your GIS software:
    1. QGIS via the QGIS TUFLOW Plugin
    2. ArcMap via the use of the ArcTUFLOW toolbox
    3. MapInfo (through the MiTools add-on)
  4. From a Console Window
  5. Via the right mouse button in Microsoft Explorer
  6. Via a runner such as the TUFLOW Runner or TRIM

This section of the manual only provides detailed instructions on initiating simulations through batch files as it is the most effective method to run several or many simulations. Instructions on all methods have been described on the TUFLOW Wiki Running TUFLOW page.

13.4.2.1 Batch File Example and Run Options (Switches)

A batch file is a text file that contains a series of commands or instructions that is read by the Windows Operating System. The batch file requires the extension .bat to be recognised by Windows.

The simplest format of a TUFLOW batch file is to specify a single simulation. This is of the format:

<TUFLOW Executable> <TUFLOW Control File>

For example:

C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe MR_H99_C25_Q100.tcf

The .bat file is run or opened by double clicking on it in Windows Explorer. This opens a Console Window and then executes each line of the .bat file. The above batch file will start the TUFLOW executable with the control file “MR_H99_C25_Q100.tcf” as the input file. As the control file argument does not have a file path, the batch file must be located in the same folder as the .tcf file. This could also include a relative or absolute file path, for example in the batch file line below, the batch file could be stored in any folder as the absolute file path is used. As outlined in Section 4.1.2, the relative file path is recommended for TUFLOW modelling inputs.

C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe C:\Example\MR_H99_C25_Q100.tcf

To start multiple simulations one after the other, these can be listed in succession.

C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe –b MR_H99_C25_Q100.tcf
C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe –b MR_H99_C25_Q050.tcf
C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe –b MR_H99_C25_Q020.tcf
pause

Note the use of the –b (batch) switch, which suppresses the need to press the return key at the end of a simulation. This ensures that one simulation proceeds on to the next without any need for user input. The pause at the end stops the Console window from closing automatically after completion of the last simulations so you can check the status of the simulations.

The –t (test) switch is very useful for testing the data input without running the simulation. It is good practice to use this switch before carrying out the simulations, as this will tell you whether there are any data input problems. The –t switch runs TUFLOW to just before it starts the hydrodynamic computations.

Using the example above, the recommended approach is to first run the following batch file:

C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe -b –t MR_H99_C25_Q100.tcf
C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe -b –t MR_H99_C25_Q050.tcf
C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe -b –t MR_H99_C25_Q020.tcf
pause

This will indicate any input problems (note some WARNINGS do not require a “press return key”, but they can be located in the .tlf file).

From TUFLOW version 2018-03-AA, it is possible to use the test model option without using a TUFLOW licence noting no diagnostics are reported so don’t use this option whilst you are developing/building a model. To utilise this licence free test, the -nlc (No Licence Check) input argument must be specified, e.g. the command line below would run the model in test model (-t) without needing a licence (-nlc):

C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe -b –t -nlc MR_H99_C25_Q020.tcf

To carry out the simulations the –t can be removed (and -nlc flag if used) or replaced with the –x (execute) switch. The -x switch is optional, but is useful when editing the .bat file to quickly change between –t and –x switches.

C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe -b –x MR_H99_C25_Q100.tcf
C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe -b –x MR_H99_C25_Q050.tcf
C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe -b –x MR_H99_C25_Q020.tcf
pause

Comment lines are specified in a .bat file using “REM” (remark) in the first column. Alternatively, “::” has a similar effect. The ss64 website has further guidance on using comment lines in batch files. For example, if you want to re-run only the first simulation in the examples above, edit the file as follows:

C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe -b –x MR_H99_C25_Q100.tcf
:: C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe -b –x MR_H99_C25_Q050.tcf
:: C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe -b –x MR_H99_C25_Q020.tcf
pause

Table 13.2 below describes other switches that are available. The switches are also displayed to the console if TUFLOW.exe is executed without any arguments (i.e. double click on TUFLOW.exe in Windows Explorer).

Any of the options can be prefixed by a:

  • “-” (short dash or minus sign);
  • “-” (long dash); or
  • “/” (forward slash)

For example, to start a model in batch mode, the following all perform the exact same function:

C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe -b MR_H99_C25_Q100.tcf
C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe –b MR_H99_C25_Q100.tcf
C:\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe /b MR_H99_C25_Q100.tcf

Table 13.2: TUFLOW.exe Options (Switches)
Switch Description
-acf Automatically create folders (the default). Prevents the dialog prompt from appearing when encountering non-existent folders (ie. results folders), and creates these folders automatically. If for any reason the folder can’t be created, a dialog will appear.
-b Batch mode. Used when running two or more simulations in succession from a .bat file (see Section 13.4.2.1). Suppresses display of the TUFLOW “simulation has finished” dialogue window, or a request to press Enter at the end of a simulation, so that the .bat file can continue onto the next simulation.

-c

Optional additional flags of “a” , “L”, “p” or “ncf”.

(e.g. -c, -cap, -cpncf, -cncf)

A copy of a TUFLOW model can be created as described below. Making a copy of a model is useful for transferring a model to another site or for making an archive of the input data. Also see the -pm (package model) option as an alternative.

To copy a TUFLOW model, the -c switch must be included on the TUFLOW command line, as a minimum. The -c switch copies only the files read by TUFLOW. For MapInfo users, the .mif and .mid files read by TUFLOW will be copied. To copy the remaining MapInfo format files (.tab, .id, .dat and .map) use the -ca option described further below.

Additional optional flags can be added to the base -c switch, in any combination, including:

  • “a” (all);
  • “L” (list);
  • “p” (path); and
  • “ncf” (no check files).

The addition of the “a” flag (e.g. -ca) copies all files of the same name for all input files (i.e. same name, but different extensions). This option is particularly useful if the .tab and other associated files of a GIS layer need to be archived or delivered.

The addition of the “L” flag will output the files used by TUFLOW into a .tcl (TUFLOW Copy List) file but not copy the files to a destination folder. This can be useful if scripting the copying of models. To run the copy list the character “L” needs to be specified after the -c input argument. This works for all copy options, for example the following are all valid; -cL, -caL -capL. The .tcl file produced is output in the same directory as the .tcf and takes the simulation name.

The addition of the “p” flag (e.g. -cp) allows the user to specify an alternate path in which to copy the model. Without this flag, the location defaults to the .tcf’s location. For example, specifying the following, will place a copy of the model into a folder C:\put_model_here:

“TUFLOW.exe” -cp “C:\put_model_here” “C:\TUFLOW\runs\M01_5m_003.tcf”

The addition of the “ncf” flag (e.g. -cncf) copies the essential input files and excludes all check files.

Note that these optional flags can be added in any combination to the base -c switch (e.g. -c, -ca, -cp, -cncf, -cap, -cancf, -cpncf, -capncf).

Specifying -c on the TUFLOW command line creates a folder “\(\lt\).tcf filename\(\gt\)_copy” (or “\(\lt\).tcf filename\(\gt\)_copy_all” if the “a” flag is added) in the same location as the .tcf file. Under the folder, input files are copied (including the full folder structure), and any check files and output folders created. For example, specifying:

TUFLOW.exe -c “C:\tuflow_models\my model.tcf” will make a copy of the TUFLOW model based on the file “my model.tcf” in a folder “my model.tcf_copy”, or “my model.tcf_copy_all” if using the “a” flag.

Notes:

  1. Use the full path to the .tcf file (this is generally the default if running from an editor such as UltraEdit or using the right click approach).
  2. Make sure there is sufficient disk space (no checks for sufficient disk space are made).
  3. Output folders and some output files are created but these will be empty.
  4. Unless the -ncf option is used, any check folder(s) are created and check files written (these can be deleted if wishing to minimise the size of the folder).
  5. The full path of the input files is reproduced to provide traceability and also handle inputs from other drives and URLs. Drive letters are replaced, for example, “C:” becomes a folder “C Drive”. URLs (denoted by “\” or “//” at the beginning of the path) are replaced by a folder called “URL\”.
  6. To run the copied .tcf file, it will be necessary to change any non-relative pathnames according to the point above. Alternatively you can share and then map, for example, the “D Drive” folder as “D:”.
  7. If using MapInfo formats, the Check MI Save Date will need to be set to WARNING or OFF in the .tcf file if the –ca option has not been used as the .tab and other files will not have been copied.
  8. There is a limit of 1,000 characters (including spaces) on pathnames. As very long pathnames can result due to the above approach, if the number of characters exceeds 1,000, problems may occur.
  9. The -c switch automatically invokes the -t (i.e. the simulation does not commence, only the input data is tested/checked).
  10. The -b option still applies if several models wish to be copied using a batch (.bat) file.
-e<name> -e{1-9}<name>

Specifies an event name to be used by Define Event in a .tef Event File to customise inputs for an event. There must be a space between –e and <name>. <name> may itself contain spaces, but if it does the event name must be enclosed in quotes. More than one (up to a maximum of nine) events per simulation may be specified by placing a number after –e. –e and –e1 are treated the same (don’t specify both of them otherwise indeterminate results may occur).

Also see Section 13.2.1 and Model Events.

Examples:
–e Q100 Run the Q100 event.
–e1 Q100 –e2 02h Run the Q100, 2 hour storm event.
–e Q100 –e2 02h Same as above.
–e “100 yr” Run the event name “100 yr” noting quotes required as there is a space in the name.
-et<time_in_hours> The end time for a simulation can be specified using the run option -et<time_in_hours>. Any end time specified via this run option argument is given the highest priority and overrides the End Time settings in the .tcf, event files (.tef) and override files.
-nc

The use of the -nc (no console) switch suppresses the appearance (hides) the console window (Section 14.1). You won’t be able to see the simulation running, however there will be a TUFLOW process visible in Windows Task Manager. The -nc switch automatically invokes the -nmb and -b switches.

The “start” command included in the Advanced Batch File examples in Section 13.4.2.3 should not be used with the -nc command.

It is strongly recommended to redirect standard console window output to a text file. It’s recommended that this file be given a unique name for each run, matching your simulation. “…TUFLOW.exe” –nc my_run.tcf > my_run.txt

Build 2018-03-AB included some enhancements when running with the -nc (no console option). These were designed to remove any need for user input to TUFLOW as follows:

  • If no .tcf is specified, no licence check is performed and the simulation is halted. Previously a licence check was performed and the user was prompted to hit a key to release licences and close the simulation.
  • If an invalid .tcf file is specified, the simulation stops and returns an error level of 1 to the operating system. Previously the user was prompted to enter a valid .tcf filename.
  • If the set log path (-slp) is specified in conjunction with the no console option (-nc) the simulation stops and returns an error level of 1 to the operating system. Previously the user was prompted to confirm the set log path.
  • Removed a number of locations where a simulation could pause and wait for a user input before closing.
-nlc For Build 2018-03-AA or later it is possible to use the copy model (-c option) or test model (-t option) without using a licence. To utiltise this licence free copy / test, the -nlc (No Licence Check) input argument must be specified. If running without a TUFLOW licence no diagnostic output is generated (e.g. messages layer). If these are required, the -nlc option must be removed.
-nmb No Message Boxes. Suppress use of windows message boxes to prompt the user. All prompts will be via the console window.
-nq The use of the -nq (no queries) switch prevents the termination query dialog from displaying when Ctrl+C is pressed to terminate a simulation cleanly. If –nq is specified and Ctrl-C is pressed, the simulation terminates cleanly without a query dialog to check you are certain, so be careful!
-nt Sets the number of CPU threads (cores) to use for a TUFLOW HPC simulation. Noting that the number of threads requested is limited to the maximum number of CPU cores available on the machine, and the available TUFLOW Thread licences.
-nwk Force TUFLOW to search for a network licence (i.e. skip the search for a local licence).
-od<drive> The Output Drive for a simulation can be specified with the –od command line option.  For example –odC will redirect all outputs to the C:\ drive.
-oz<name> Map output includes Output Zone <name>. For more information on Output Zones, please refer to Section 11.2.5. For example “-oz ZoneA” would include output for Zone A, similar to the .tcf command: Model Output Zones == Zone A.

-pm

Optional additional flags of “All” , “L” or “ini”.

The package model (-pm) function copies all input files for all events and scenarios defined in the model. Unlike the -c option, -pm does not read and process the data during the file copy. As such it is substantially faster than -c. 

The package model function does not require access to a TUFLOW licence.

Optional switches are:

  • “All” (e.g. -pmAll) copies all file extensions (e.g. 1d_nwk_culv_L.mif, will become 1d_nwk_culv_L.*).
  • “L” (e.g. -pmL) list the files to be copied into an output file, but doesn’t copy the files.
  • “ini” (-pmini <file.ini>) provides a .ini file with user defined options as described below.

Combinations of the above are also valid, with the order of the optional switches not being important (-pmAllL would be treated the same as -pmLAll).

Three options are also available for handling of the binary processed files created by TUFLOW to speed up the simulation start. These options are:

  • -xf0: Do not copy .xf files, only the original inputs are copied.
  • -xf1: Copy both raw input files and .xf files.
  • -xf2: Copy only .xf files, if xf files exist for an input only the xf file will be copied.

For example, the command line below will package the model but not include any .xf files.

TUFLOW_iSP_w64.exe -pm -xf0 PR_~s1~_~e1~_~e2~_001.tcf

When using package model the default destination folder is created in the same directory as the .tcf file, with the prefix “pm_”. For example, C:\Projects\Modelling\TUFLOW\runs\Run_001.tcf will create a package in the folder “C:\Projects\Modelling\TUFLOW\runs\pm_Run_001”.

A .ini file can be used to overwrite the default base and destination folders. A .ini file is specified by including the optional ini argument after -pm followed by the ini file name. For example:

TUFLOW_iSP_w64.exe -pmini package.ini PR_~s1~_~e1~_~e2~_001.tcf

Valid commands in the .ini file are:

  • Base Folder == <folder>
  • Copy Destination == <folder>
  • Model Scenario ~s<number> == <scenario a> | <scenario b> | …
  • Model Event ~e<number> == <event a> | <event b> | …
  • Max Event Name == <number> </file.ini>
-pu<id>

Used to select which processing units to direct the simulation towards. At present this only applies to the HPC GPU solver where a simulation is to be directed to a particular GPU card or cards. –pu must be specified once for each device. For example, to direct the simulation to GPU devices 0 and 2, specify –pu0 –pu2. Can be used in place of the .tcf command GPU Device IDs. Note that the numbering starts at 0 for GPU devices.

Override files (see Section 4.1.17) can be used to control the device ID’s used by a range of computers accessing the same control file.
-qcf Query the creation of a folder. If you would prefer to have the create folder query dialog to appear (rather than TUFLOW automatically creating folders - see -acf), you can specify the –qcf run time option.

-s<name>

-s{1-9}<name>

Specifies a scenario name to be used by If Scenario blocks to include/exclude commands. There must be a space between –s and <name>. <name> may itself contain spaces, but if it does the scenario name must be enclosed in quotes. More than one (maximum of nine) scenarios per simulation maybe specified by placing a number after –s. –s and –s1 are treated the same, so don’t specify both of them otherwise indeterminate results may occur.

Also see Section 13.2.2 and Model Scenarios.

Examples:
–s exg Process all exg scenario commands.
–s1 opA –s2 opB Process all opA and opB scenarios.
–s opA –s2opB Same as above.
–s “Option A” Quotes required as there is a space in the scenario name.
-slp Simulation Log Path is a legacy option for Softlok (blue) dongles to set the path to a folder on the intranet to log all simulations initiated from the lock. Refer to the 2018 manual or earlier for details. (See Section 13.4.1.2 for equivalent option for WIBU Codemeter licences, and also Simulations Log Folder.)
-st<time_in_hours> The start time for a simulation can be specified using the run option -st<time_in_hours>. Any start time specified via this run option argument is given the highest priority and overrides the Start Time settings in the .tcf, event files (.tef) and override files.
-t Test input only. Processes all input data including writing of check files, but does not start the simulation. Useful for checking that simulations in a .bat file all initialise without error, prior to carrying out the simulations (especially when the runs will take all night or all weekend and you made a change you forgot to test!).
-wibu Search for a WIBU Codemeter licence only.
-x eXecute the simulation (the default).

13.4.2.2 Copy/Package Model from Batch Files

TUFLOW can be run in copy mode, which can be useful for transferring a model to another site or for making an archive of the input data. To copy a TUFLOW model, the -c switch must be included on the TUFLOW command line. The -nlc (no licence check) flag can be used to copy a model without the need for a TUFLOW licence. Further details can be found here.

A licence free package model function that copies all input files for all events and scenarios defined in the model is also available using the -pm switch. If only a single set of events and scenarios is required, -c is preferable. Unlike the -c option, -pm does not read and process the data during the file copy, as such it is substantially faster than -c. The package model function does not require access to a TUFLOW licence. More information on the Package model function can be found here.

13.4.2.3 Advanced Batch Files

Batch files can be set up so that they are more generic and easily customised when moving from one project to another. Within the example below, a variable, TUFLOWEXE, is used to define the path to the TUFLOW executable, and a variable RUN is used to incorporate options such as starting the simulation on a low priority and to minimise the console window when the process starts (/min option).

set TUFLOWEXE=C:\ProgramFiles\TUFLOW\2023-03\TUFLOW_iSP_w64.exe
set RUN=start “TUFLOW” /low /wait /min “%TUFLOWEXE%” -b
%RUN% MR_H99_C25_Q100.tcf
%RUN% MR_H99_C25_Q050.tcf
%RUN% MR_H99_C25_Q020.tcf

The advantage of using variables in batch files is if the path to the TUFLOW executable changes, or to run a different version of TUFLOW, it is just a simple single line change in the batch file. In the above, note the use of quotes around %TUFLOWEXE% in the definition for the RUN variable – quotes are needed around file path names whenever they contain a space. Also note there is no space between the variable name and the equals sign.

For examples of batch files used for running multiple events and different scenarios from the same .tcf file, see Sections 13.2.1 and 13.2.2.

It is also possible to create control logic such as loops in a batch file. For example, this could be used to loop through a large number of events and scenarios such as the example here.

:: This sets the variables as local, another batch file can use the same variables
SetLocal
:: Set up variables
set TUFLOWEXE=C:\Program Files\TUFLOW\Releases\2023-03\TUFLOW_iSP_w64.exe
set RUN=start “TUFLOW” /low /wait /min “%TUFLOWEXE%” -b
set A=Q010 Q020 Q050 Q100 Q200
set B=10min 30min 60min 120min 270min
:: Loop through each simulation
FOR %%a in (%A%) do (
  FOR %%b in (%B%) DO (
    %RUN% -e1 %%a -e2 %%b filename_~e1~_~e2~.tcf
  )
)
pause

Further guidance on advanced batch files, including looping examples can be found on the TUFLOW Wiki Run TUFLOW from a Batch File page.

13.4.3 Running TUFLOW HPC

The front end of TUFLOW Classic and TUFLOW HPC are identical. No modification of the model input is needed to utilise the TUFLOW HPC solver (in CPU or GPU compute mode). Similarly, the output data is written by TUFLOW in the same output formats and data types irrespective of whether Classic or HPC solvers are used.

The .tcf Solution Scheme command is used to switch the solution scheme from TUFLOW Classic (the default) to TUFLOW HPC.

Hardware is used to access GPU hardware (CPU is the default). As such, to convert a model from TUFLOW Classic to TUFLOW HPC and run using GPU hardware only requires two additional TCF commands in most cases:

Solution Scheme == HPC
Hardware == GPU

With TUFLOW HPC it is possible to use multi-GPU cards to split the simulation across multiple cards to reduce run times or access more GPU memory. This is carried out either by using the GPU Device IDs command or with the processing unit command line switch “-pu”). More information can be found here. It is also possible to simulate multiple simulations on a single GPU card, should model memory requirements allow it, however, the simulations will slow down proportionately.

TUFLOW HPC simulations are started in the same manner as a standard TUFLOW simulation and explained in Section 13.4.2.

13.4.3.1 TUFLOW HPC and GPU Module Commands

The following .tcf commands (apart from Timestep) are commands specific to the TUFLOW HPC solver:

Table 13.3: TCF commands (apart from Timestep) specific to the TUFLOW HPC solver
Command Description
Control Number Factor The default HPC Courant, shallow wave celerity and diffusion control number limits can be reduced to effectively underclock the simulation. Using the above command factors all three control numbers. For example, a value of 0.8 reduces the default limits by 20%. Reducing the control number limits may be useful if the simulation is exhibiting erratic behaviour or numerical “noise”, although testing has found this is rare in real-world models, and if occurring is more likely to be a sign of poor data or poor model schematisation.
Hardware

TUFLOW HPC can be run using CPU or GPU hardware. This command defines the hardware to be used for the compute. CPU is the default and will run a simulation using the Central Processing Unit (CPU). GPU will run the simulation using the Graphics Processor Unit (GPU).

When running TUFLOW HPC using the GPU hardware module, the pre (reading of data) and post processing (writing outputs) is managed by the standard TUFLOW CPU engine. This allows the user to utilise the extensive range of GIS input functionality available in TUFLOW.

Simulation using GPU hardware requires the GPU Hardware Module in addition to a standard TUFLOW licence.
HPC DP Check

TUFLOW HPC is available in both single precision and double precision. If the simulation is started with the single precision version of TUFLOW, the HPC solver will utilise a single precision version. If the simulation is started with the double precision version of TUFLOW, CHECK 2420 will be output by default stating that the single precision version should be used.

The calculation method in TUFLOW HPC uses depth due to its explicit nature, unlike TUFLOW Classic that uses water level due to its implicit scheme. This means that precision issues when using the Classic solver, such as when applying a very small rainfall to a high elevation, are not applicable in HPC. For more information see Section 13.3.2.2.

Unless testing shows that single and double precision produce significantly different results, use the single precision version of TUFLOW for all TUFLOW HPC simulations. Note: Double precision solutions on GPU cards can be four times slower than single precision! Also, NVIDIA Cards with a compute capability of 1.2 or less are only able to run single precision versions.

The HPC DP Check command allows the user to disable this check should double precision be required, or increase it to an ERROR.
HPC Temporal Scheme

Sets the order of the temporal solution. The default is the recommended 4th order temporal solution, and this command is usually not specified.

We recommend the use of the 4th order temporal scheme as it is unconditionally stable with adaptive timestepping turned on and has been found to give accurate results and is results are not affected due to changing the timestepping. Lower order schemes save a little on memory requirements but are more prone to instability and in some cases unreliable results that are sensitive to the timestepping intervals.
GPU Device IDs Controls the GPU device or devices to be used for the simulation if multiple CUDA enabled GPU cards are available in the computer or on the GPU itself.
Solution Scheme This command is used to select the desired solution scheme for the compute. Use “HPC” to call TUFLOW HPC’s finite volume 2nd order solver (recommended over the 1st order alternative).
Timestep

If adaptive timestepping is active (the default), the timestep value is only used for the very first step. To allow the same command to be used for either a TUFLOW Classic or a HPC simulation the timestep value is divided by 10 for the initial HPC timestep, and enter a timestep value similar to that that you would use for TUFLOW Classic.

If adaptive timestepping is off, sets the fixed timestep. The timestep will always be much smaller than TUFLOW Classic’s timesteps as the scheme is explicit (TUFLOW uses an implicit scheme). As a general rule of thumb specify a timestep that is around one tenth of the TUFLOW Classic timestep you would use.

13.4.3.2 Compatible Graphics Cards

TUFLOW HPC’s GPU hardware module requires an NVIDIA CUDA enabled GPU card. A list of CUDA enabled GPUs can be found on the following website: https://developer.nvidia.com/cuda-gpus.

To check if your computer has an NVIDIA GPU and if it is CUDA enabled:

  1. Right click on the Windows desktop.
  2. If you see “NVIDIA Control Panel” or “NVIDIA Display” in the pop-up dialogue, the computer has an NVIDIA GPU.
  3. Click on “NVIDIA Control Panel” or “NVIDIA Display” in the pop-up dialogue.
  4. The GPU model should be displayed in the graphics card information.
  5. Check to see if the graphics card is listed on the following website: https://developer.nvidia.com/cuda-gpus.

The following screen images show the steps outlined above, this may vary slightly between NVIDIA GPU card models.

**Accessing NVIDIA Control Panel from the Desktop**

Figure 13.2: Accessing NVIDIA Control Panel from the Desktop

**NVIDIA GPU Model**

Figure 13.3: NVIDIA GPU Model

**Check the Website for your NVIDIA Card**

Figure 13.4: Check the Website for your NVIDIA Card

More information on the card can be found in the “System Information” section, which is accessed from the NVIDIA Control Panel. The system information contains more details on the following:

  • The number of CUDA cores.
  • Frequency of the graphics, processors and memory.
  • Available memory including dedicated graphics and shared memory.
**NVIDIA System Information**

Figure 13.5: NVIDIA System Information

On the NVIDIA website, each CUDA enabled graphics card has a “Compute Capability” listed. For cards with a compute capability of 1.2 or less, only the single precision version of the GPU Module can be utilised. However, benchmarking has indicated that the double precision version, except in rare situations, is NOT required and that for nearly all GPU simulations, TUFLOW_iSP engine versions should suffice. Refer to Section 13.3.2.

Extensive GPU hardware benchmarking has been undertaken to assist users who are upgrading hardware for TUFLOW modelling, with numerous hardware options tested for their speed performance. The results are provided on the TUFLOW Wiki Hardware Benchmarking page.

13.4.3.3 Updating NVIDIA Drivers

It is likely that the NVIDIA drivers will need to be updated periodically to the latest version as the drivers shipped with Windows Operating Systems are not always up-to-date. To update, open the NVIDIA Control Panel (by right clicking on the desktop and selecting NVIDIA Control Panel or NVIDIA Display). Once the control panel has loaded, select Help >> Updates from the menu items. For further guidance see the TUFLOW Wiki Updating NVIDIA Drivers Wiki Page page.

If new drivers are available, please download and install these by following the prompts.

NOTE: Even if not prompted by the system, a restart is recommended to ensure the new drivers are correctly detected prior to running any simulations.

**Accessing Driver Updates from the NVIDIA Control Panel**

Figure 13.6: Accessing Driver Updates from the NVIDIA Control Panel

13.4.3.5 Troubleshooting

If you receive the following error when trying to run the TUFLOW GPU model:

TUFLOW GPU: Interrogating CUDA enabled GPUs …
TUFLOW GPU: Error: Non-CUDA Success Code returned

Please try the following steps:

  1. Check the compatibility of your GPU card and whether the latest drivers are installed (see instructions in Section 13.4.3.2).
  2. Test with a user account that has administrator privileges as these may be required for running computations on the GPU.
  3. If multiple monitors are running from the video card, try running with only a single monitor.

If the above steps fail to get the simulation to run, please email the NVIDIA system information (see Section 13.5) and TUFLOW log file (.tlf) to .

13.4.4 Running TUFLOW 1D Only Simulations

TUFLOW is set up to run as either a 2D model or as a 1D/2D model. It is, however, possible to run a TUFLOW 1D only model by including a small ‘dummy’ 2D domain within the model files. The 2D domain can be set up with the .tgc file and can be small, contain few cells of uniform material type and elevation values. The .tbc file will be set up so that the 2D domain does not receive any flows and will not impact simulation runtimes. The 1D model can then be constructed within the .ecf and the simulation set up within the .tcf as per usual. An example model exists on the TUFLOW User Gitlab Page.