Formaline — conserve meta data about runs forever

Erik Schnetter <schnetter@aei.mpg.de>

\( \)Date\( \)

Abstract

This thorn Formaline collects and preserves meta data about the run. It can contact a database server at run time and store the meta data there. Such meta data include e.g. the parameter file, date, time, machine, and user id of the run, location of the output data, number of iterations, an efficiency summary, etc. It also writes a copy of the executable’s source code into the output directory.

1 Utility Programs provided by Formaline

Thorn Formaline provides the following utility programs:

formaline.py A python script for gdb which allows you to recover the source tarballs from the executable even if the executable can no longer be run. It writes the extracted tarballs into the current directory. Use as

  gdb -P formaline.py exe/cactus_sim

All utility programs are located in src/util.

2 Parameters




announce_to_portal
Scope: private  BOOLEAN



Description: Announce meta information to the portal



  Default: no






collect_metadata
Scope: private  BOOLEAN



Description: Collect metadata and output it in various ways



  Default: yes






create_id_files
Scope: private  BOOLEAN



Description: Create files from the build and job ids



  Default: yes






max_warn_level
Scope: private  INT



Description: Only output warnings with at most this level of severity



Range   Default: -1
-1:*






metadata_lifetime
Scope: private  INT



Description: Lifetime of meta information sent to a server (in hours)



Range   Default: 100
zero to store the metadata forever
1:*
a positive value to limit the lifetime to so many hours






nr_of_parameter_changes_to_be_logged
Scope: private  INT



Description: number of parameter changes to be logged



Range   Default: -1
-1:
log all changes
0:
no logs please
1:*
log this many changes for a given parameter, then stop logging






out_reductions
Scope: private  STRING



Description: Reduction operators that should be applied to the output variables



Range  Default: minimum maximum norm1 norm2 norm_inf sum
space separated list of reduction operators






out_vars
Scope: private  STRING



Description: Cactus variables that should be output



Range   Default: (none)
space separated list of groups or variables






output_info
Scope: private  BOOLEAN



Description: Output info messages?



  Default: no






output_source
Scope: private  BOOLEAN



Description: Output a tarball with the cactus source tree



  Default: yes






output_source_subdirectory
Scope: private  STRING



Description: Subdirectory, relative to IO::out_dir, for the Cactus source tarballs



Range   Default: cactus-source






portal_hostname
Scope: private  STRING



Description: Portal host name



Range   Default: (none)






portal_port
Scope: private  INT



Description: Portal port



Range   Default: 9296
1:65535






portal_username
Scope: private  STRING



Description: User name on the portal



Range   Default: (none)






publish_level
Scope: private  INT



Description: Importance level for meta data to be published via the Publish API



Range   Default: 1
0:10
the lower the level the more important the meta data to be published






rdf_hostname
Scope: private  STRING



Description: RDF server host name



Range   Default: (none)






rdf_port
Scope: private  INT



Description: RDF server port



Range   Default: 9296
1:65535






relay_host
Scope: private  STRING



Description: Relay host for connecting to the portal



Range   Default: (none)
.+
host name
try to determine a relay host automatically






send_as_rdf
Scope: private  BOOLEAN



Description: Send meta as RDF to a server



  Default: no






steered_parameters_log_exclusion_list
Scope: private  STRING



Description: Parameters for which runtime changes should not be logged



Range   Default: CarpetRegrid::coordinates
”whitespace-separate d list of parameter names”






storage_filename
Scope: private  STRING



Description: File name for meta information



Range   Default: formaline-jar.txt






storage_json_filename
Scope: private  STRING



Description: File name for meta information in JSON format



Range   Default: formaline-jar.json






store_into_file
Scope: private  BOOLEAN



Description: Store meta information in a file



  Default: yes






store_into_json_file
Scope: private  BOOLEAN



Description: Store meta information in a file in JSON format



  Default: yes






timeout
Scope: private  INT



Description: Timeout for sending meta information to a server (in seconds)



Range   Default: 10
1:*
a positive value






update_interval
Scope: private  REAL



Description: Update interval for the meta information (in seconds)



Range   Default: 3600.0
0:*






use_relay_host
Scope: private  BOOLEAN



Description: Use a relay host for connecting to the portal



  Default: yes






verbose
Scope: private  BOOLEAN



Description: Produce screen output



  Default: no






out_dir
Scope: shared from IO STRING






out_save_parameters
Scope: shared from IO KEYWORD



3 Interfaces

General

Implements:

formaline

Uses header:

http_Content.h

Publish.h

Provides:

UniqueConfigID to

UniqueBuildID to

UniqueSimulationID to

UniqueRunID to

4 Schedule

This section lists all the variables which are assigned storage by thorn CactusUtils/Formaline. Storage can either last for the duration of the run (Always means that if this thorn is activated storage will be assigned, Conditional means that if this thorn is activated storage will be assigned for the duration of the run if some condition is met), or can be turned on for the duration of a schedule function.

Storage

NONE

Scheduled Functions

CCTK_WRAGH (conditional)

  formaline_outputsource

  output cactus source tree

 

 Language:c
 Options: meta
 Type: function

CCTK_STARTUP (conditional)

  formaline_printids

  print the build and simulation ids

 

 Language:c
 Options: meta
 Type: function

(conditional)

  registerpublishcallbacks

  tell cactus that this group exists, but is not scheduled from here

 

 Type:group

RegisterPublishCallbacks (conditional)

  formaline_registerpublishrdf_callbacks

  register rdf callbacks for the publish api

 

 Language:c
 Options: meta
 Type: function

CCTK_WRAGH (conditional)

  formaline_registerwarnings

  register to receive warnings and info messages from the flesh

 

 Language:c
 Options: meta
 Type: function

CCTK_WRAGH (conditional)

  formaline_announceinitial

  put some meta information about the current run into permanent storage

 

 Language:c
 Options: global
 Type: function

CCTK_ANALYSIS (conditional)

  formaline_announceupdate

  put some meta information about the current run into permanent storage

 

 Language:c
 Options: global
 Type: function

CCTK_TERMINATE (conditional)

  formaline_announcefinal

  put some meta information about the current run into permanent storage

 

 Language:c
 Options: global
 Type: function

(conditional)

  unregisterpublishcallbacks

  tell cactus that this group exists, but is not scheduled from here

 

 Type:group

UnregisterPublishCallbacks (conditional)

  formaline_unregisterpublishrdf_callbacks

  unregister rdf callbacks for the publish api

 

 Language:c
 Options: meta
 Type: function