I've been working on a new algorithm for Hal's brain and just updated the Web Hal/Facebook Hal with an early version.
The new algorithm takes into account entire conversational threads when thinking of a response, creates links between similar conversations, and much more. The result is Hal can stay on a conversational thread better if he has the data to back it. It should bring Hal to the next level of intelligence.
The Web Hal has been recording every conversation anyone has had with it since last August. Every night it processes all the conversations it has had and decides what to keep and what to throw away. It tries to fix most common spelling and grammar mistakes and then does an analysis of your spelling. People who are really poor spellers get their entire conversation thrown out. Conversations get tagged as PG, PG-13, and R which can be used to control what kind of language Hal can use. People who are overly vulgar get the conversation thrown out. Personal details (mention of another persons name, phone, email) get thrown out. People who are uninteresting (average sentence length too short) get their conversations thrown out and on the other end people whos average sentence is way too long get conversations thrown out.
Out of close to 500,000 sentences Hal has analyzed Hal has decided to learn from about 68,000 of them so far, and this number is slowly and automatically growing everyday. This is the knowledge base the new Hal at
http://www.zabaware.com/webhal/index.html and
http://apps.facebook.com/ultrahal is currently using, although it still reverts to the old Hal if it doesn't find a good response from this source.
Hal's answers are already noticeably different, but I think once it achieves 1,000,000+ size it will be noticeably more intelligent. Once it gets bigger I will be able to extract unique personalities out the database. Hal records whether the person who taught Hal is male or female and Facebook Hal records more data such as age of the person talking to Hal. This way I could make a brain extract using only data Hal learned from females ages 18-28 to make a young female Hal personality or any other combination.
Let me know what you guys think of this new Hal so far. My goal over the next few months will be to accelerate the learning by getting more people talking to this Hal. I'll be releasing an update to Hal Assistant 6.2 which will give users with an always-on internet the option of connecting to this hosted Hal brain instead of the brain on their local computer.