HDF Viewer will not open large 10 year flow, velocity, and stage datasets.
It will export the data into a txt file and that can then be opened by code (it is too big for notepad and notepad++) if you allow code to reopen using more memory.
I was able to verify the 1st Stage and the 1st flow values. (Velocity is always 0 at the start of the file) But there were issues with the text export mostly to due with the automatically using word wrap and no it was not a code issue I checked. An email has been produced and it will change into an issue as well as an explanation in our documentation.
The conversation I had with Sudhir and Deven about this led them to want Steve's project and I found the link in my email. I do not think it is on our dropbox but rather shared from Steve's We may want to put it on our dropbox.
Meeting with Sudhir and Jake Office Hours for Chama
Tasks to be added to my not at work time studying
Work on my Linux now that I am not studying for Azure test.
Set up WSL on my windows 11 environment to refresh my memory on all the steps to set up WSL.
Start looking at conda and min conda.
Meeting with Deven and Sudhir about the order that I asked Deven to run.
At this point I need to make sure to cover myself. If I ask anyone to do something I need to document exactly what I want them to do. Keep Sudhir in the loop. Put it in the read me with explications why I want things done in a specific order. Then Email it to Sudhir so that he knows what is going on or create issues and loop sudhir in that way.
I need to follow up with them about doing what I asked and not just trust my colleagues to do something.
Like testing there code.
I need to start documenting every meeting with meeting min put here.
Both my tasks and others so that I have a record.
Start a folder with meetings here put all the meetings I have here.
Made sure the run I left to run last night was successful.(It was successful)
Started the daily flow history.
Checked the git hub repository for changes and found no new changes.
Sent an email to Deven about an update this morning with him about ORSANCO. Sent it right before he got in so..
Meeting with Deven Sudhir and Jake about ORSANCO after quick panic brought on by an email where Deven said he was testing on the 12 years of data.
He was not testing the project was not even ready for testing.
Deven is creating an issue in Orsanco that will explain the changes he is making.Some of which is below.
using interpolation rather than concatenation.
Code review on quires from Update_flow.py.
Making sure that folders are sorted by name which has the date in them and thus how they should be added.
I will be testing on the 2 year gap database created yesterday after Deven is done with both the coding and code review.
I will also be code reviewing really quickly before I start testing.
Questions I need to ask during the review.
Will the code delete the Local files from the machine.
Should FTP variable has been added and is located in the Settings.py and will need to be added to Local settings.py
Testing should be done as follows:
Create a clean Database
Run Migrations
Run setup_cross_sections command. To set up database.
Run insert_flow_history command. Insert 2 year gap data into database.
Run insert_flow_daily_history command. Inserts the daily flow velocity and stage statics into the database.
Go to the location of the local files downloaded from the FTP Server. Remove the files for the last few days. Say everything after 2/12/2023.
Point the Local Path at the local files folder that has all files from before 2/12/2023.
Make sure that should_FTP variable in local settings is set to False.
Run the download_hdf_ftp command. The local files should be upserted into the database in the correct order.
Change the local setting should_ftp variable in local settings to True.
Change the local path so that it is pointing at a file that is not where all the local files are this is because the files might get deleted and that would be annoying so new clean file.
Run the download_hdf_ftp command. The ftp files should downloaded and stored in the local path be upserted into the database in the correct order.
Make sure all the data is in the database and write up a testing report. (Just a quick note that this worked and then send it out in the email and also in the issue that I am going to create for this by taking this list and putting it into an issue.
There is a util function that is being referenced.