Skip to main content

Week 1

Oct 2nd 2023​

  • Discussed with Melissa to discuss about GPT API support in ORSANCO.

    • Issue: There are multiple Access DBs with complicated and large queries associated with them, which makes it hard in our understanding of the data.

    • Solution: We want to simplify our understanding of the data using OpenAI for understanding all the queries.

    • Explained implementation using the AI3 implementation of the OPenAI API.

    • Provided a link to GPT playground: https://platform.openai.com/playground. (Check if the usage limit in the playground is up for the account used to login).

    • We decided to use SQLite DB to have the input queries and responses from the OpenAI API.

    • Each database is seperated as they are provided in Access DB, and Melissa is going to provide the databases with input queries.

    • I will work on running the databases through openAI for repsonses and provide the database with reponses back to Melissa.

    • Queries' questions that seemed to provide good responses are below. I'm choosing the #2 question.

        1. Can you give me only the structure of the tables followed by justified explanation of query, in 500 tokens?
        2. Provide only the structure of the tables followed by justified explanation of this query, in 500 tokens?

    • TODO:

      • Melissa needs to provide the SQLite database with input queries.
      • Discuss with Melissa that we need only 1 database with tables for each database, not multiple databases.
      • I need to prepare script that prepends question to all the queries and run them through API.
      • Need to add a task for this on JIRA.
      • Discuss with Jake about JIRA task for having OpenAI code in gqc-utility-python and know which project the task goes into.

    • The queries that we tested but didn't think gave good enough results are following:

      `1. Provide me the description of this query, in 250 tokens?
      2. Can you give me a discription of this query along with the structure of the tables, in 500 tokens?
      3.` Can you give me only the structure of the tables and justification of query, in 500 tokens?

  • Get latest details on all the AI projects at GQC

    • Referencing Deven's PPT and summary under general.gqc.com/Dev/AI section for this.
    • Some projects are not covered in the Summary so will be updating and cleaning the doc at the same time.

  • Will be updating LLM details like LLM playground, API etc. under General site in Dev section.

Oct 3rd 2023​

  • Created a low priority JIRA task DEV-25 for moving OpenAI implementtaion code to gqc-utility-python repo.

  • Talked with Melissa about maintaining 1 database for maintaining all the input queries, with different tables for each Access DB.

  • WIll be meeting Melissa for discussion on OpenAI playground in the afternooon.

    • OpenAI credits should be more than 0 in order to use the playground. Normally, they give free credits for 3 months but not sure if the offer is still active.
    • Will need to be verified with new accounts - but can't do it currently due to phone number restriction. (1 unique phone number per account)

  • Told Jake about the code maintenance structure discussion from yesterday -

      Yesterday there was a discussion on how we should maintain our code in order to provide them together as a product, say if we are providing SaaS services.
      It's not a priority yet, but we might have to discuss on this topic sometime later so that we can write code that aligns with this idea, like maintaining working code in python scripts, using config files, UI for each product etc.

  • Working on OpenAI code for ORSANCO DB run - Will be working on creating Python script. Will be tracking Orsanco tasks under JIRA/ODB-25

    • Currently, using the script directly from gqc-utility-python under an examples folder.

Oct 4th 2023​

  • Questions and responses - Melissa:

    If we don't have information on any of these, we might have to ask ORSANCO (Might have to confirm again with Sudhir)

    1. Do we have information on query <-> DB relation, other than us trying to understand using the access DBs?
      • We only have the Access Databases. All the databases are self contained so the Queries are part of the database. That is why I gave you the names of the Databases in the SQLite Database I sent.
    2. Do we have a list of the commonly used queries, from all the queries?
      • What do you mean by commonly used queries. We got the full databases and that is it. We have no knowledge of what they use more than others. We can see what they use as the source for reports and such but we don't know how often they use reports either. Also sources for reports can also be tables and not all databases have reports.
    3. Do we have or know any sample reports or documents that are a result of the queries? If not, we might have to ask them to provide us information on what Query outputs are being used currently.
      • Again we have some reports, Access DB reports. I would like to direct you to the documentation of this project specifically notes on databases. https://projects.gqc.com/clients/ORSANCO/DB2023/Notes%20On%20Databases That is most of the information I have I have not extensively documented the queries because there are so many of them.
      • No knowledge on how ORSANCO databases and queries/tables

  • Research more on OpenAI API, especially custom structures and functions.

  • Document Deven's implementation of OpenAI API in the general site > OpenAI.

  • Notes from Meeting from Melissa

    • Put all the questions and tasks on JIRA / Project Update
    • Updating OpenAI docs under ORSANCO on porjects site, by creating new page for this.

  • Started Run for OpenAI call to populate Result column of the OSRANCO DB provided using the git:gqc-utility-python/use-openai.py script.

Oct 5th 2023​

  • ORSANCO

    • Completed the DB run from yesterday. Refer the screenshot for the same orsanco db run
    • 3 rows were skipped due to OpenAI error. Modified code to run for those rows too. orsanco db run for errors
    • Updating docs on OpenAI usage for for the provided data.
    • Raised JIRA tickets for questions to ask and assigned to Melissa. Also talked to Jake about it.

  • Updated OpenAI docuemntation with API research me and pavan did.

  • Updated Python Based Development documentation to include the hack of using # noqa: E501 for Flake8 issue of long code line.

  • In office tomorrow

  • Will be switching back to work of AI projects.

Oct 6th 2023​

  • Done Verify ORSANCO meeting invite for the afternoon is provided.
  • Run the I-75 and other projects on local and WSL to test.
  • Ask Jake - About help on visualization using React for AI3.
  • Start the LLAMA run for the weekend (Verify with Pavan if there's any other job)
    • Label 3000 PSIAP data samples, 1000 each.
    • Merge the samples from old (4k) and new(3k) into 1 new training DB
    • Download new folder of LLAMA since original files might be overwritten - Check last edited date.
    • CLeaning the training notebook for re-ordering everything.
    • Backup copy of the base LLAMA files or last checkpoint
    • Clean the code and cells
    • Start the job
  • Research TODO
    • Image Segmentation and Augmentation - for YOLO training data
    • Resnet and LSTM for Time-series data in other AI projects - Improve or Upgrade
    • Self attention in ML
  • Started Labelling job for weekend. Will have to merge and start training job sometime in the weekend.
Last updated on by Srujana