Almost start from scratch every time launch an experiment. Write down a memo for some must-have components when build a new project. Also make a TODO list for my customize lib.
Must-have factors
Reproduce
Never could make it the most solid experiment…
Must have:
- A space saves:
- network
- config file
- log file
- Module state & Optimizer state
A few more words:
-
network file should be a hard copy of defined network
-
config file should be only one file where saves all the customized config
-
log file should include all the details of what’s happening in the whole training procedure, e.g. config, peroid loss report, test results.
NOTE: should be a log module for logging and print to stdout, especially for Python 2.x
-
state
*NOTE: should be a module for state (module and optimizer) saving and reloading
- [] logging: print and log
- [] config: detectron fashion
- [] state tool: load and save all states
Monitor
TensorBoard or Visdom should be used.
- [] TensorBoard fashioned logger
Efficiency
Process the data really kills the time…
- [] dataset: from Fast R-CNN
- [] data loader: from Fast R-CNN
- [] pre-process image and related annotation
