Week 4

Aug 21st 2023

• Documenting the list of notebooks used, their locations, and functions that are used as part of the flow in AI3.
  • Updating details in C:\git\gqc\AI3 Competition\phase_3\phase3_roadmap.md.
• Send Jake the task list

Aug 22nd 2023

• Running the notebooks for distance calculation to find closest match.

• List of things to be finished on AI3 front (From conversation with Deven)

  1. Classification of unknown keys needs to be implemented.
     • The idea is to look at the distribution of the values coming in and classify based on that.
  2. On the LLM training side:
     1. Label more data
     2. Create a validation dataset
     3. Train and test the model
     4. Look at the performance (output validation)
  3. MQTT side:
     1. Code clean-up
     2. Confirm with Sudhir - Where the switches (actions) are needed in the final version of the app or not.

• The unknown dataset from Google drive has been downloaded on the WD_BLACK HD. It has the video on which we ran YOLO.

• Notebooks are in 2 places - AI3-hackathon repo, AI3 folder on MSI

• Data required to run the notebooks inside AI3-hackathon are placed under /data folder.

• IoTData_EmergencyScenarios has JSON files which contain training data.

Aug 23rd 2023

• Running the subscriber.py in ai3-hackathon through debugger.
  • Created a launch.json and edited it to take the ./subscriber.yml path as config.
  • Run the ./iotgpt/mqtt/publisher.py in one terminal after editing the MQTT port in the ./publisher.yml
  • Run the debugger on ./iotgpt/mqtt/subscriber.py after editing the variables in ./subscriber.yml
• Currently, bard token hack is not working. Might have to run using LLAMA models and chatGPT.
• Created new DB with the weather data README from Phase2, such that it resembles the ai3-lookup.db with phrases and synonyms.
• TODO
  • In the link here, try to register using gqc email for Bard and see if we can access bard API.
  • Questions to ask AI3 people: either through report or email (Updated in phase3_roadmap.md)

Aug 24th 2023

• Test the weather-data database with the scripts
• Note: Do we calculate mean based on synonyms or standard keys, assuming data has synonyms.

Aug 25th 2023

• Testing on randomly assigning NK to weather data.

• TODO

  • [RP] Screenshots from the YOlO on AI3 videos they provided in the GDrive.

  • [RP] MQTT subscriber.py logic on handling at each step.

  • Use LLM to find the standardization for NK with string values, will work if we can have internet access to ask LLMs, like ChatGPT. Tested with ChatGPT and below is result.

      Question: if srujana is a value in a json, make a educated guess on what key might be? Don't give reasoning. Just give key value pair.
      Response: {
                  "Name": "Srujana"
                }

  • Use the OPENAI chatgpt implementation after sentencification. The implementation can be referred to the docs/nbs/openai-exploration.