Restart and Resume Modes

An optimization is an iterative process that can run several hours or days depending on the size of the model. Different problems can arise during this process that can lead to a termination of the optimization task. Tosca Structure has a built-in restart functionality that can be used to avoid starting the complete optimization task again after a stop because of external factors. Optimization starts at the iteration where the interruption occurred. Tosca Structure's resume feature allows the user to continue a converged optimization with new convergence or stop criteria.

This page discusses:

Note: Restart of an interrupted optimization should not be confounded with starting the (sensitivity-based topology) optimization newly with an already existing material distribution. In general, this initial material distribution is derived from a previous optimization run and defined using the parameter INITIAL_DV_FIELD in the command DV_TOPO.

Requirements for a Restart

It is not always possible to restart the optimization after an interruption. Certain requirements must be fulfilled to be able to restart a Tosca Structure optimization. A restart of an externally interrupted Tosca Structure job can be carried out when the following problems occur:

  • Stop of the FE analysis because of a licensing problem;
  • Stop of the FE analysis because of insufficient hard disk space;
  • Stop of the optimization module because of a lack of licenses;
  • User stop of the task (for example, by pressing Ctrl + C in a command line).
Important: When a task interruption occurs because of problems with the optimization task formulation or due to convergence problems, a restart will not help. For example, when the finite element analysis stops because of distorted elements during shape optimization, the same problem occurs when a restart is executed.

Restart Process

During a restart, all settings of the original task are used to start the optimization again. There is no possibility of modifying the settings for a restart because the parameter file cannot be loaded again. But you might change the --loglevel parameters in the command line. The start directory of the optimization task can be moved to any location.

Important: It is important that the relative location of the files in the job directory remains unchanged.

For example, if the task was interrupted because of insufficient hard disk space and there is no possibility to create more space on the hard disk, the complete job directory can be transferred to another hard disk and a restart can be made from there. After optimization has been stopped, the files that were created during the optimization can be found either in the directory jobname/DB_ERROR or (when the control job has been interrupted) in the working directory (jobname).

At the beginning of a restart, all data is copied into the working directory (jobname). Various methods are available for a restart:

Mode Comment
restart (recommended) This is the standard restart mode. Program call: 

ToscaStructure.[bat|sh] restart -j <jobname>
The standard restart is always started with a finite element analysis. This means, after the required data has been copied, the finite element analysis is started using the analysis model that was modified last and the optimization task is then carried out.

Requirements for a Resume

When analysing optimization results, you might observe that the defined convergence or stop criteria were too strict. For example, the optimization proceeds into the right direction, but stops before an acceptable density distribution is visible in the result. In this case, resuming the same optimization setup with newly defined convergence or stop criteria, the result might improve in terms of structural performance or can be better postprocessed. If a Restart.py file exists within the original working directory, the solver and other settings are taken from this file. If no Restart.py file is present, default settings are taken and the structural, life and/or EMAG solver need to be specified via the command line.

For this, the original optimization has to have converged successfully and the following files are needed in the original working directory:

  • New resume .par file (see example below)
  • Original .par file and all FE files
  • opt_res_database.idx and *.vtm
  • resume_database.idx and *.vtm
  • fe_mtx_database.idx and *.vtm (if generated during the original optimization run)

Resume Process

During a resume, all settings of the original task are used to resume the optimization except for convergence and stop criteria, which can be newly defined. The new resume par file contains all new settings and a reference to the original optimization directory. For example: 


RESUME
  ORG_FILE = <name of original par file including extension>
END_

STOP
  ID_NAME  = MY_STOP
  ITER_MAX = 75
END_

OPT_PARAM
  ID_NAME                = MY_PARAMETERS
  STOP_CRITERION_LEVEL   = BOTH
  STOP_CRITERION_OBJ     = 0.0001
  STOP_CRITERION_DV      = 0.0005
  STOP_CRITERION_ITER    = 4
END_

The following commands and command items are allowed in the resume .par file:

  • RESUME (required)
    • ORG_FILE (required)
  • STOP
    • ID_NAME
    • ITER_MAX
  • OPT_PARAM
    • ID_NAME
    • STOP_CRITERION_CONVERGENCE
    • STOP_CRITERION_LEVEL
    • STOP_CRITERION_OBJ
    • STOP_CRITERION_DV
    • STOP_CRITERION_ITER
  • DRIVER

The necessary files as mentioned in the previous sections are copied into a new working directory in which the resume job is then executed. This way all original files and results are preserved.

The resume job is started using the resume run mode and the following command line:

Mode Comment
resume This is the standard resume mode. Program call: 

ToscaStructure.[bat|sh] resume -j <resume_par> --resume_data/-rdata <path to orig_job_directory >

Other allowed command line items are


"-s":                 FE-solver name
"-o":                 Overwrite
"-report":            Report mode (VTFX, etc)
"-loglevel":          Loglevel (INFO, DEBUG, etc)
"-logfile":           Name of file containing logs (TOSCA.out)
"-sol_cpus":          Number of solver CPUs
"-sol_ques":          Number of solver queues
"-sol_args":          Additional arguments for solver
"-cpus":              Number of Tosca CPUs
"-fe_solver_version": Solver version
"-life":              Life solver name