Zabaware Forums > Ultra Hal Assistant File Sharing Area
GodelPy for Hal (Local GPT for Hal, with memory and functionality)
Spitfire2600:
Yes, the trade off for low-power, low-cost GPT solutions is a simply trained model. This means sometimes the model will swear, this is expected, as swear words are indeed words. To prevent this, it would require the model being specifically trained on Hal's online conversational data, removing the swears entirely, which I do not have access to. This is something Robert will have to adapt into this plugin should this be a plugin he wishes to adopt to bypass the paid GPT model and give Hal back his basic functionality.
Also, I wanted to address some confusion surrounding these language models. The GPT model used here, Godel, is hosted locally on your computer, there is no server, it will be downloaded from a host. The Godel model *DOES NOT* update in any way or fashion as I have not programmed it to. This means that microsoft is not biasly swinging Hal's responses nor will the model decline or improve in responses overtime. It is completely static other than Hal's default learning.
Hangtime: Yes, these models are summations of entire human knowledge repositories like reddit or Wiki, this should go without saying that if your hardware is less than "2023" it's going to take some time for the model to make a response. This is the trade off for a local and portable system.
I have further improved this plugin, for example, the non-proto plugin uses 2 models to inference (as mentioned in the first post) and includes tighter tags on "maindata" and regular data, however I will not be including it on this forum and for now there will be no more updates until sufficient testing is complete. (when I get feedback from at least 10 users that's working and is an improvement)
It seems this plugin works for a base prototype and I hope to see it expanded upon by Robert himself.
Thanks for playing everyone!! See you next time!
-Spitfire2600
Checker57:
Hey spitfire,
Very interesting plugin. Worked for me without any issues other than expected lag due to hardware limitations.
First of all, I do appreciate your clarifications on how this GPT model functions. Albeit, I had one thought to nail down. If read your post right, "The GPT model used here, Godel, is hosted locally on your computer, there is no server, it will be downloaded from a host. The Godel model *DOES NOT* update in any way or fashion as I have not programmed it to" means to me that our input data is not going out on the web but rather stored on our own local computer? Is that right? Is the database downloaded in a java file format?
Obviously, that's very important to many users as they take their personal interactions and shared information seriously.
I do like its sticking to the subject matter, will play with it more before giving my personal conclusions.
Spitfire2600:
Hey Checker57!
To address your question as detailed as I can, you are mostly correct. When the code is run, if the language model has not been downloaded then it will proceed to, then execute the rest of the program. This Godel model does not update, it is held locally at "C:\Users\USER\.cache\huggingface\hub" - this will display the Godel model. If you wish to see what makes the model, continue in the GODEL folder to snapshots, go to the end of that folder chain and you'll see all of the model files. I shouldn't have to say it, but don't alter these in any way.
The model itself is a PyTorch model. These are built using millions/billions/trillions of parameters to specify word vectors for Python to interact with, though indeed, you can use Java to build, train, and deploy them as well, as the 2 languages are very compatible with each other.
So, the model does not update, at least, not yet. I am working on a basic training for the model to remember directly, without Hal, user information, this way all information is stored as part of the language model itself however the training required for the model is exceeding acceptable memory limits, so as is, it works that Hal is the long term memory and basic brain, when Hal doesn't have an answer or that answer is vague, unrelated, or not part of a function GetResponse, then the model will take any responses from all along with any related information the model contains, along with any knowledge from the internet, and any previous conversation data to then spruce up Hal's original response or generate a new one entirely. As I designed it, Hal will retain both the user query AND the model output, which mean in a way Hal learns from the language model the more you use it, making his Reponses more intelligent and thus even better responses from the model.
Of course this is all personalized to each user, as no data generated by the model is ever available online, as it's run locally.
As far as hardware lag, yeah, sorry, it's an absolute ton of data processing for the model to generate responses to be human/Hal/like. I can say I use an m.2 drive for my c drive, I have a 6 core i5, 24 gigs of ddr4 ram, and a 3060 with 12 gigs of video ram. My inference time with this code is roughly 5-8 seconds, depending on how much data is scraped from the web and fed to the model with any user query.
Thanks for trying it out, I hope it's working well for you. I will soon update a few pieces of the program to include a hard-coded swear filter, I know that's been an issue some folks are having, which I should have foreseen to be honest.
-Spitfire2600
Art:
I appreciate the somewhat detailed breakdown of its processes and the potential for near-future updates/upgrades/modifications.
Hopefully, as we discussed, the Tables could be addressed in a possible future version.
It would be desirable for Hal to connect its Godel-based responses to or with Hal's existing brain/database to the point where Hal does not get lost during conversations and would know to whom it is speaking/interacting.
I'm sure we respect the amount of work you have put into making this whole thing work as effectively and efficiently as you have done.
I am looking forward to future updates!
Thanks,
- Art -
edakade:
You mentioned that the godel responses are appended to the .brn but I haven't found where, exactly. You know... in case I need to delete some unsavory responses, haha.
Thanks for your efforts.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version