Week 1

May 05/05/23

1. AI3
   1. data validation of the data from the repository
   2. dataset creation
      1. use 10 json objects, create a csv by manually entering prompts
      2. read csv into dataframe → create dataset from dataframe
      3. (optional): write to disk?
      4. Use the tokenizer to tokenize the dataset
      5. Train and val split
      6. Train the model with the Trainer object
      7. Save the model
      8. Load the model and use it for prediction.

May 05/04/23

1. Open an issue in peft for a feature request for Question Answering as a new task.

May 05/03/23

This is all for finetuning and using OPT for Question Answering on a Target dataset.

info

It would be interesting if we fine-tune the OPT on SQUAD + JSON data.

1. Loop over all the JSON data in the dataset
2. Generate sentences with the inflect
3. Create a dataset, refer the datasets library for examples of how to tokenize a dataset for QA.
   danger

   When the downstream task is different, the tokenizer needs to be used differently. The tokenizer gets invoked differently if we have CAUSAL_LM vs Question Answering

4. Create training and validation splits
5. Define Trainer with the training_args
6. Call the Trainer.train()
7. Save the model
8. Download the model, run it locally
9. Check inference speed

Report for AI3

1. Start the report, mention the submission requirements
2. Add diagrams and other charts that we have created

May 05/02/23

nbdev + colab instructions

note

These are broad instructions.

1. Create a personal access token.
2. While working inside Google Colab, checkout a new branch colab-pavan, colab-deven, or colab-user.
3. Push the changes to colab-user branch.
4. To merge the changes,
   1. Either create a pull request
   2. Or merge it manually
5. Run the nbdev commands locally (inside WSL)

May 05/01/23

Holiday