Week 4

May 05/26/23

1. AI3

May 05/25/23

1. AI3
   1. Luckily, the creator of the PR replied quickly and his fix allows me to load the OPT model with peft and lora for further processing.
   2. The next task to figure out is tokenization part of the problem.
   3. With context length of the questions and answers going above the max length of the model
      1. How to truncate the sentences and answers in that case?
      2. What is the accepted practice?

May 05/24/23

1. NKY - Presentation
2. AI3
   1. Checkout PR from huggingface to use peft for QuestionAnswering
   2. Try running the code and use the OPT model with that
      1. The code crashed and it was reported in the issue.

May 05/23/23

1. AI3
   1. Update diagram and walk Sudhir through the tokenizer stuff
2. OH
   1. 1 hr meeting for everyone, walk everyone through

May 05/22/23

1. Setup MSI Ubuntu machine with cuda
   1. Check if we can train on the GPU inside Ubuntu
   2. Document the process used to setup cuda/python/env variables and making a test run.
2. Pipe breaks app
   1. Document runs made by Vannary
   2. Get list of remaining runs
      1. Low Priority: How can we use flower (for federated learning) over here?
   3. Maybe have a simple excel sheet with the expected feature shape for base covariates and additional covariates
3. AI3
   1. Use peft + lora in Causal LM mode to train locally in Ubuntu on MSI
   2. Export out the custom data structure with all the Jaccard distance values
   3. Revise the lookup function
   4. Train model with peft + lora on custom dataset
   5. Train model for QA on the JSON data that we have created
4. (Low Priority) Colab notebooks
   1. Clean 1-2 notebooks daily and push them to their appropriate repository