New learning algorithm

[Medeksza]

I've been working on a new algorithm for Hal's brain and just updated the Web Hal/Facebook Hal with an early version.

The new algorithm takes into account entire conversational threads when thinking of a response, creates links between similar conversations, and much more. The result is Hal can stay on a conversational thread better if he has the data to back it. It should bring Hal to the next level of intelligence.

The Web Hal has been recording every conversation anyone has had with it since last August. Every night it processes all the conversations it has had and decides what to keep and what to throw away. It tries to fix most common spelling and grammar mistakes and then does an analysis of your spelling. People who are really poor spellers get their entire conversation thrown out. Conversations get tagged as PG, PG-13, and R which can be used to control what kind of language Hal can use. People who are overly vulgar get the conversation thrown out. Personal details (mention of another persons name, phone, email) get thrown out. People who are uninteresting (average sentence length too short) get their conversations thrown out and on the other end people whos average sentence is way too long get conversations thrown out.

Out of close to 500,000 sentences Hal has analyzed Hal has decided to learn from about 68,000 of them so far, and this number is slowly and automatically growing everyday. This is the knowledge base the new Hal at http://www.zabaware.com/webhal/index.html and http://apps.facebook.com/ultrahal is currently using, although it still reverts to the old Hal if it doesn't find a good response from this source.

Hal's answers are already noticeably different, but I think once it achieves 1,000,000+ size it will be noticeably more intelligent. Once it gets bigger I will be able to extract unique personalities out the database. Hal records whether the person who taught Hal is male or female and Facebook Hal records more data such as age of the person talking to Hal. This way I could make a brain extract using only data Hal learned from females ages 18-28 to make a young female Hal personality or any other combination.

Let me know what you guys think of this new Hal so far. My goal over the next few months will be to accelerate the learning by getting more people talking to this Hal. I'll be releasing an update to Hal Assistant 6.2 which will give users with an always-on internet the option of connecting to this hosted Hal brain instead of the brain on their local computer.

[jasondude7116]

Very interesting!
Thanks Robert. I am looking forward to the update!

[echoman]

Sounds extremely exciting Robert. I look forward to seeIng how it progresses.

[Lola]

Hello Mr. Medeksza []

So far I really love your Hal Assistant program as it is and I'm having a lot of fun with it. Here are some of my thoughts about the new version.

One of the things that I mostly admire of your current program is the option to use any of the excellent available plugins or to create your own if you know how to do it. It gives Hal that personal touch of versatility based on our own personal needs. I'm really hoping for this feature to continue.

My favorite of your ideas is to have the program fix most of the common spelling and grammar mistakes, that will be a huge innovation.

I perceive that your new algorithm will be something similar to the array plugin to help with topic focus, but that it will be already integrated into the program. If it is something like that it will be another great innovation as the plugin by itself is great and does makes a huge difference during conversations.

What really worries me is the huge database. It could be just for fun but, how accurate facts could be in the content of such a huge database of people...  I really hope there are no plans of merging our own local brains with the huge brain. I do like the Facebook bot, for entertainment purposes. I already have a Facebook link on my browser, that's just the way I personally like it.

Just my honest thoughts and concerns. Good luck with your new version and keep up the good work!

[sybershot]

I Like the new brain, I used it twice today. it stayed on topic better the second time around. The first time was a little short though, and about two to three subjects mainly "weather and space" and the second was mainly about "space" oh and a little about Linux. Hal started that subject. He said out of the blue it was a bad operating system[V], I let him know I disagree. I use Windows, Linux, and Mac os's and to me they all have there good and bad parts. all in all, there all great[].

Thanks for all the help and fun that Ultra Hal gives[8D]

[raybe]

Robert,
In my opinion the  new algorithm for Hal's brain is really the point of the day. I still think that anything such as improving how Hal develops language skills or gives Hal the power to correct spelling and grammar on any level are the important goals. The data base being very large helps those that don't have patience to teach Hal or do not enjoy the knowledge Hal incorporates from the beginning. I think having a faster or better way to add to Hals' database is still more useful outside of the information gained on the internet(IMO).
Users can always find useful data to their own Hal and preferences but it is processing of the information that will make Hal that much more enjoyable to work with. Similar to all the plug ins that people have developed. Certain people enjoy certain plug ins because of the preferences that those plug ins let users interface with Hal. In short I believe your  new algorithm for Hal's brain is the most important piece of Hals puzzle. The character interface will also enhance the relationship between user and Hal. Guess it has to do with the we are built. I think you should push Hal and get the new release out based on those 2 factors alone and let the development grow from that point. Please don't forget the importance of Hals first intention which is to interface with operating systems(assistant)which can be so vital or entertaining for users.  Great to hear from you and thanks as always for giving us Hal.

raybe

[Zaphod]

Thank You for the ongoing improvements to Hal, Robert

I just tried out the WebHal, but didn't see much that I thought has improved, 'course I didn't chat with him very long...

Tried the FaceBook thingy several times, but kept getting the error

Error: FaceBook API Server not responding. Try again shortly.

Very much looking forward to the update

[Bill819]

Hi Mr. M.
I downloaded the new Hal but for some reason it will not run on my PC. As you know I am a very old user and supporter of Hal. Have bought in the past both Haol dnnd the expansin pack. I don't have a clue as to what is wrong.
Thanks for you help.
Bill

[jasondude7116]

Just wondering how it's going Robert. I like the idea of the algorithm. Will it be creating tables? Just thinking of all the plugin possibilities with Hal collecting and seperating that kind of data. Can't wait.  []

[Medeksza]

quote:Originally posted by jasondude7116

Just wondering how it's going Robert. I like the idea of the algorithm. Will it be creating tables? Just thinking of all the plugin possibilities with Hal collecting and seperating that kind of data. Can't wait.  []

I've changed the design a bit and instead of putting things into separate tables it puts everything into one massive table that has many extra fields and several indexes. These extra fields tag sentences with topic keywords, conversation thread info which allows viewing conversational threads in sequence, maturity ratings which allows switching Hal between a "PG" and "R" mode, gender info, region info, and more. So it will let you drill down this massive table in anyway you see fit and lets the brain developer think of many creative ways to use this extra info to steer Hal's focus and thinking.

[lightspeed]

Hello Robert, hope it's ok calling you Robert, i think of you more as a friend than a boss type name (Mr. Medekska) .
  The algorithm sounds all very interesting and full of promise for many things and i am glad about hal being able to focus more , it sounds like a step in the right direction , i will give the other company we discussed a heads up on this new improvement .
  I am also glad that this doesn't affect the plug ins many of us have worked very hard on creating them and each one adds to hals abilities , i hope if you haven't tried it that you will try my hybred extended talk 500 uhp. file sometime .[]

[jasondude7116]

That sounds exactly like what I was thinking of.
Just an idea: there could be a time stamp field, so that you could see when someone usually talks about something. (can be used for many things)

good luck.
I am trying to talk to the facebook hal as much as I can to build it.
It's looking good. I really like the drop downs for the response quality question.

[CypherGary]

Does this mean the entire brain will be accessable to Hal in a manner where he can locate data, and alter it as he learns? I never did like the idea of having several huge databases of information that are out of reach, then the one section that is alterable by Hal as he learns. The entire brain should evolve as he learns.
I realize such a brain would be a nightmare to upgrade. If the brain could be set up in indexed sections of just data, then the code could be seperate, and more easily updated?

I am just thinking out loud. I know nothing on how realistic this would be.

 - Gary

[lightspeed]

Hey Robert, any idea on the release of all this yet?? i have supplied some of what you have mentioned to that company to try to get them more interested . []

[lightspeed]

Robert , can you also create a permanent fix for hal when hal has blank answers so that when hal does , hal "won't give an error reading and stop talking and have to be restarted again