Week 4

March 03/20/23

Plan on the Edge gateway front

1. Compare model sizes, resnet18, resnet34, resnet50, resnet101 and resnet152 and MobileNet
2. On a sample dataset, train models with these different backbones
3. Load these models on a Raspberry-pi, and note down inference times (prediction)
4. Use techniques prescribed in Knowledge Distillation to reduce the size of the these trained models
5. Load these models on a Raspberry-pi again, and note down inference times (prediction)

Alternate approach

1. Train models on the MobileNetV2, MobileNetV3 backbone
2. Load these models on a Raspberry-pi, and note down inference times (prediction)
3. Compare performance difference between the two runs

New models to train ahead

1. A model trained on the concatenation of all the PACP standard datasets, which would be DNV, COV and SD1
2. Train the model in the same way we trained our previous models.
3. A single 10-fold cross validation run.
4. Other runs with/without image augmentation, weighted loss function and training from scratch.

Ideas for refactor

1. Use of typer or click to eventually break down bigger main functions into smaller, modular, resusable functions
2. Do not put all the logic in the main function

March 03/24/23

What should a run-id look like?

Borrowing the discussion from Jake's email on how to name a GitHub repository.

info

TLDR; a repository (in broader terms a project) should be named as follows,

1. [product/project name]-[purpose]-[framework/language] e.g. myproject-api-rails
2. [product/project name]-[purpose] e.g. myproject-rest-api
3. [language/framework]-[product/project] e.g. python-security-scripts

There are a lot of parameters to be considered while writing a run-id for wandb. Paramters like dataset, backbone, training parameters, epochs, utility all need to be considered.

1. Dataset
   1. utility
   2. number of videos
   3. Continuous labelling?
2. Backbone
3. Pretrained?
4. Training method? Freeze epochs? fit_one_cycle, fit_flat_cos?
5. Epochs
6. Image transforms?

A sample run id in such a case would look like this,

1. dnv-86-C-resnet50-prT-10
2. dnv-86-C-resnet50-prT-10-resize-flip-rotate-etc
3. sd1-119-C-mobilnetv3large-prF-10

Delete models with incorrect WandbCallback

1. Make a list of those models
   1. Delete them GDrive
   2. Delete from Vannary's local drive
   3. Delete from Deven's local drive

Renaming the projects

Projects are being renamed to the following

1. cctv-sd1-multilabel
2. cctv-sd1-binary
3. cctv-dnv-multilabel
4. cctv-dnv-binary
5. cctv-cov-multilabel
6. cctv-cov-binary
7. test-runs (for testing wandb connection, multi-gpu runs, etc)

Documenting the current run-id standard

For cv_dnv486_multigpus_23_10_continuous_train_val_486_videos_from_80_20_split_v3_3_2023/03/23_07:31

1. run id: cv_dnv486_multigpus_23_10
   1. cv: cross validation
   2. dnv486: DNV utility 486 videos
   3. multigpus: run on multiple gpus
   4. 23: (indexed from 0) 24th csv file which has a 80(75-25 train/val) - 20(test) split for the entire dataset
   5. 10: → trained with fine_tune with freeze epochs 10 and 10 epochs
2. name of the csv file: continuous_train_val_486_videos_from_80_20_split_v3_3
3. time at which the execution began: 2023/03/23_07:31

Ideas for the continuously learning model

I'm not sure, how I should set this entire training pipeline up. I had 2 main thoughts on this,

1. While creating the first model which would start training, have it include all the 300 classes instead of the only available 45. Not sure if this will hamper performance as such?
2. Train the model with available 45 classes for now, in the future, when I have new class to identify, update the model accordingly and train the model again.

I've asked this question on multiple discord servers to see if I can get any input on this.