Week 4

June 06/22/23

1. AI3
   1. Tht tonic.ai account is now working.
2. Documentation

June 06/21/23

1. AI3
   1. Unable to start mqtt server, (starting it on a different port fixes the issue)
   2. Created a tonic.ai account, need a small write up for tonic.ai use
      1. Contacted support, no response yet
   3. Lookup function based on single key replacement
2. CCTV
3. Documentation
   1. Create a folder for MSI on the docs site
      1. Needs to be pushed

Steps for tonic.ai

1. Create an account
2. Go through a tutorial to see how we can generate fake IoT data.
3. Save the generated fake data a JSON file, manually test it against our jagged array of saved abbreviations.
4. Try to get an accuracy score out.

Thoughts on starting a python project

Many ways to go about it:

1. (Easiest) We should directly use the cookiecutter template.
   1. A minimal project is built by Deven, share with Sudhir
2. Python Project Template (similar to the ones that we use for React) → will require minimal setup
   1. Separate templates for Django, Streamlit and nbdev project
   2. Advantage: Customized to exact GQC needs
   3. Will be easy to maintain and develop as we go on
3. Manual setup

June 06/20/23

1. Link for example CLI we would use for model training.
2. Fix requirements.txt in the gqc-utility-notebooks repo
3. Enable dependabot in the gqc-utility-notebooks repo
4. Explore BARD alternatives as it can't be pinged multiple times reference link
   1. I am planning to try an open-source commercially usable version of open-llama-13b.
      1. Update: This does not work. The memory footprint after loading the weights as float16 is about 10.5 GB on the GPU. There is enough memory to load other batches and use it for prediction.
   2. I am trying a smaller open-llama-7b parameter model right now.
      1. Update:The 7b parameter is able to predict, but the max_token_length is too small. Hence, it is not usable.
      2. Using peft allows us to load the memory, with around 3 GB of GPU memory left for batches. Prediction is not working, which needs to be figured out.
   3. Another alternative is load one of these bigger models on a bigger GPU on Colab to do the labelling,
      1. Colab has 16GB (T4 (2 credits/hr) or V100 (5 credits/hr)) + 40GB A100 (13 credits/hr) GPUs available.
      2. Run the model for a couple of hours and get everything labelled.
      3. Download the labelled samples as csv files.
      4. Then train on the MSI server.
   4. Tried a sample mosaicml-7b paramter model on Colab, here is a sample output.
5. Fixed and improvd the create_csv CLI in the cctv-apps project.
6. [ ]

June 06/19/23