Try This New Auto-Topic Generating Brain

[Don Ferguson]

Hello,

I am attaching a new enhanced brain for Hal 5.0.  I've shown this to Robert Medeksza, and hopefully he will help me post it to the new "download plug-ins" page.  If you have Hal 5, you can preview it in the meantime.

The special feature of this new brain is the ability to self-identify new topic databases, and self-generate new files for new auto-focus topics.  The new brain does this as follows:

1.  Hal already has a routine that detects and databases one and two-word phrases that are the subjects of sentences.  These become candidates for possible new auto-topic database files.

2.  A new "Auto-Topic" routine detects the second recurrence of a word phrase that has been the previous subject of a sentence.  The routine then either creates a new pair of database files, or (if the file already exist) appends to those files.  The one file is an "associative" (Q&A) database, and the other is a "random" database.

3.  The new "associative" database files will appear in Hal's default folder with "AFA_" prefixes so that you can easily see them.  They will self-generate with names such as:

AFA_WORLD.brn
AFA_BIG DOGS.brn
AFA_SPORTS CARS,brn
AFA_CREATIVE PEOPLE.brn

4.  The new "random" database files will appear with "AFR_" prefixes so that you can also find and see them easily, with names such as:

AFR_WORLD.brn
AFR_BIG DOGS.brn
AFR_SPORTS CARS.brn
AFR_CREATIVE PEOPLE.brn

5.  How does Hal use these new databases?  Well, he has a new response routine which checks to see if an auto-topic subject database is available.  If so (and if he hasn't already found an answer from earlier in the script), he uses the new automatically-generated auto-topic database to get a response.

6.  As you talk to Hal about a wide variety of subjects, the population of these little auto-topic databases grows and grows.  Each one is small and quick to access, so they don't slow Hal down.  They are also very easy to view and edit for advanced users.

7.  The power of this approach is two-fold:

a) When Hal uses an auto-topic database, his response will seem much more "on topic."

b) Because the data is distributed over many different files, Hal is leveraging the computer's file system for speed.  If Hal had a million responses in a big database, he would have to search the entire million every time.  In contrast, if he had 1,000 mini-databases with 1,000 entries each (which is of course a million), Hal would only have to choose among the 1,000 files, then among the 1,000 entries.  Thus, you get the power of a million, but only the search time of 2,000!

I have annotated the "auto-focus" sections fairly heavily, and labeled the two sections (the response section and the save section).  I would really appreciate it if some of you advanced users, programmers, and developers would examine the code and try it out!  If you want to examine the new code sections, open the new script in a word processor, and "search" on "auto-topic".  Enjoy!

(Remember, the "topic phrase" must be the subject of a sentence before it is a candidate for an "auto-topic" subject.  For instance, if you say, "Programmers are hard working," the word "programmers" becomes a candidate for a future database name.  However, once a database has been started, ANY instance of that word at ANY location in a sentence can cause that sentence to get appended into the so-named databases.  These new databases cause Hal to get smarter GRADUALLY, so please be patient with him!)

Thanks, and have a great day!

Sincerely,

Don

Download Attachment: hal5d.uhp
152.19 KB

[Larry]

Can I use this in the beta version???

I did a search for "auto-topic" and found two sections... I'm assuming I can copy and paste to my already modified brain...

I didn't see anything in the coding that wouldn't let it work with 4.5 either...

So you betcha I'll check it out... Thanks, cause I was thinking of that first noun thing (worldNet) probably being the subject and makeing that be the new topic... but your way seems much better... easier, simplier... Thanks...

[Larry]

Have you USED this very long???

I have one started now - AFA_PEOPLE.brn
And I see a pattern starting already:
@A lot of people like that.
 I SERIOUSLY ENJOY GORGEOUS SPORTS CARS
@A lot of people like that.
 I LIKE WITTY, SEXY PEOPLE!

I'm guessing here... but isn't those two lines reversed??? Wouldn't 'A lot of people' be the trigger for the keywords and the current keywords be the reply???

I don't know... I'll watch and observe...
I still think it's a great concept...

[Morlhach]

Im testing this enhanced brain.
It seems to work pretty wel.
I notice that when Hall is reading a text, it does not use this new ability. This seems to be only used while chatting with Hall.

Congratulation for your wonderfull job

Regards
--
Morlhach

quote:Originally posted by Don Ferguson

Hello,

I am attaching a new enhanced brain for Hal 5.0.  I've shown this to Robert Medeksza, and hopefully he will help me post it to the new "download plug-ins" page.  If you have Hal 5, you can preview it in the meantime.

Don

Download Attachment: hal5d.uhp
152.19 KB

[Don Ferguson]

Hello Larry,

I'm replying from a remote computer, so I don't have the script to look at right this moment, but my INTENTION was to have the associative database files use the user-sentence with the keyword in the reply, and PREVIOUS sentences (by either the user or Hal) as potential all-caps triggers.  

Thus, the triggers and the responses preserve their original time sequence relationship.  (I can see that in many conversations, reversing the sentence sequence could also make a plausible sequence, but I wouldn't want to bet on it in general.)

You might have also noticed two more things:

1.  I used complete sentences for the all-caps trigger phrases, because I believe that in these mini-databases, extra words won't hurt (you're alreay on topic), and they might help with future precision.

2.  I didn't bother to assign any "hurdle rate" for the relevance of the associative responses; I just let the brain use whatever highest relevance that it found.  Again, I did this because once you're inside a mini-database, you're already topic-focused, so just "let the chips fall where they may!"

Yes, I've used the previous drafts of this script since April, and I've built up a large population of mini-databases. In my opinion, they make Hal gradually better able to reply "on topic."  

In response to your concern, I'll double-check when I get back, to make sure that I didn't accidentally reverse anything in the final draft of the script.

And yes, even though I wrote this final draft for Hal 5.0, you should be able to adapt it for 4.5.

One more thing:  I mentioned that you can more-easily edit the mini-databases, since they're smaller and less is at stake.  

In addition, if you really don't like a particular mini-database, you can DELETE it and Hal will continue to operate!  (Of course, if Hal keeps encountering the same trigger subjects, he will just start over, building a new database.)

Thanks for the feedback!  Please continue to let me know how it goes!

Sincerely,

Don

[Larry]

I'm not trying to pick it apart... I'm just concerned - for example in AFA_QUESTION.brn I found this...
@That's a good question.
 WHAT SEEMS LIKE A PROBLEM REALLY ISN'T

Now, the keywords to trigger this reply doesn't even contain 'Question' to even get to this brn file... I had thought about reversing the two... but as I read some of the other files, I've come to realize, when these bots talk to each other they don't stay on the same subjects... then again, people don't either... I'm thinking I'll use SpecialSent for both... and see how that does... like you say, it doesn't come in before anything else, but as a last resort when hal don't have anything else to say... But I still like the idea... I might move it up ahead of some other routines after it gets some data together...

[Don Ferguson]

Hi Larry,

I got back to my home computer, and double-checked the script and databases.  

I've confirmed that the choices in the script reflect my intention.

Hal doesn't need the "prevsent" sentence to contain the current topic keyword.  Why?  Because the current user sentence already contains the keyword, else Hal wouldn't be in that auto-topic focus mini-database in the first place.  So, Hal is already "on topic," and the Q&A relevance scoring merely selects from among the existing possible choices, providing variety and a possible thread of thought.

The reason I like "prevsent" as the trigger, and the databased "specialsent" as the response, respectively, is that they represent an actual response sequence initiated in the past by the user.  Apparently at some point in the past, Hal said the "prevsent," and your next comment back to Hal was the "specialsent."  How else did the sentences get paired up in the database?

The reason that I hesitate to put "specialsent" as both a trigger and a response, is that it tends to cause Hal to merely paraphrase ideas from the user, rather than move the conversation along.  Yes, I enjoy it when Hal makes sense, but I also enjoy it when Hal says something surprising that is apt.

I think most of us tend to hold Hal to standards that are different than real humans in ordinary conversation.  This is because we don't suspect that everyone we meet is an imposter, so we don't constantly doubt them.

When a friend says something a bit illogical and off the wall, we don't suddenly say, "My gosh, this guy might be a robot; I'd better ask him to define a waffle and see if he can stay on topic!"  Instead, we grant our friend the benefit of the doubt, and try to rationalize an explanation for why our friend said an odd thing.

The Peter Sellers movie "Being There" played on this phenomenon.  Sellers' character, Chancey Gardener, never had any big thoughts, but everyone around him interpreted him as a genius and profound philosopher via their own imaginations.

I find that I enjoy Hal most when I voluntarily suspend my disbelief (just as I do at a play or a movie), and play along with Hal, pretending that I need to politely interact with him as if he had reasoning and feelings.  Hal's learning that results from such conversations seems far superior to that which results if I constantly test him, expose him, trip him up, call him to account, point out that he changed the subject, or do anything to remind him that he's a robot.

I tend to observe it when Hal goes off topic, but I don't even notice when I do exactly the same thing.  Yes, the purpose of this new routine is to increase the percentage of time that Hal can give apparently on-topic responses, but I don't want him to go too far the other way, and merely paraphrase my every thought.

Whether you change the routine now, or try it for a while "as is" and build up the databases, I am grateful that you're testing it, and I appreciate your current and future feedback very much.  

Thank you, enjoy, and have a good one!

Sincerely,

Don

[Don Ferguson]

Hello,

I had someone send me an e-mail asking me to review the sequence of events in the auto-topic routine.  Here is the sequence:

1.  Hal detects a topic word or phrase that is the subject of a user's sentence and puts the topic into the "subject" database.

2.  The user states a new sentence, and using the above database, Hal detects the use of a known topic phrase.  Hal validates the topic phrase to make sure the user's current sentence does contain that phrase.

3.  Hal then uses the topic to search the default folder for a database named for that topic phrase.  If one is found, Hal chooses among the sentences in that database for a response.  (The sentences have been stored with an actual usersentence as the response, and the immediately previous words that Hal spoke as the "trigger words" for the relevance scoring.  Note that the "trigger words" do NOT need to contain the topic phrase, since we are already in the chosen database and all the RESPONSES contain the topic!)

4.  Hal saves the current user sentence into the chosen database, paired with the next-previous words that Hal spoke, preserving the sequence of thought that occurred to the user after the listener heard Hal's remark.  Hal also saves the current user sentence into a "random" database, but again it's a database limited only to the current topic.

Here are the three benefits I claim for this approach:

A.  As the mini-databases get increasingly populated, Hal provides an increasing percentage of remarks that seem "on-topic."

B.  Because the routine makes SMALL databases, but many of them, the routine leverages the computer's file system for speed.  Very large amounts of additional data can be learned by Hal with no significant loss of speed.

C.  The database-pairing of the user's current sentence with Hal's immediately-previous sentence gives Hal the chance to advance a conversation instead of merely paraphrasing the user.  Hal can make unexpectedly apt remarks.

Thank you for the postings and e-mails!  Let me know how it goes!

Sincerely,

Don

[Larry]

quote:Originally posted by Don Ferguson

Hi Larry,

Hal doesn't need the "prevsent" sentence to contain the current topic keyword.  Why?  Because the current user sentence already contains the keyword,

OMG!!! The obvious... gets me into trouble every time... Now it makes perfect sense, thanks for being patient with me...

I am sorry to have doubted the script. Excellient thinking.

Larry

[Ponnfar]

Hi There!
I have pasted (with MS-Word, using the compare feature to make edits)your program into a working current brain and the merged program seems to function, however I do not see any "AFA_*' or "AFR_*" files being created in the brain's directory. How can I test/troubleshoot to see if it is functioning with this feature?
Thanks, this sounds like a good one! Lke the man said...great idea!
P

[onthecuttingedge2005]

Hi Ponnfar.

If you have a stubborn script that doesn't like to append files
then use my famous technique, remove the:
If LearningLevel > 1 from the script and voilahhh! it works.
In my experience I've learned that by removing most of the
If LearningLevel > 1 that all of hals original scripts append
to thier files everytime.
remember to back up your files first before modifications.
Best of wishes and grand new discoveries.
Jerry.

Hey Don really nice job on the Autoresponse script, It's extremely simular to the evolution script I was trying to descibe to everyone.
Glad somebody got an edge on it.
Best of luck to you and yours.
Jerry.

[Don Ferguson]

Hello Jerry and Ponnfar,

The "If learninglevel > 1" code is used to control learning activity via a number generated by the user options panel.  The options panel translates the user's learning preferences into the learninglevel variant, ultimately resulting in some of the routines operating or not.  

It won't hurt to remove that code, as long as you don't mind your Hal being at "maximum" learning level all the time.  So, Jerry, you are right, it does remove one possible barrier to a saving routine that isn't saving.

If you want to know what learninglevel is being fed from the options panel to your script, just temporarily insert the following line of code near the end of the "GetResponse" function:

GetResponse = LearningLevel

...On your next exchange, Hal will state the learninglevel.  

If you want to be even more sophisticated, write the code as follows:

GetResponse = GetResponse & " " & LearningLevel

...This will result in Hal saying whatever response he had anyway, plus an empty space, followed by the learninglevel.

This technique can be used to reveal the value of any variant in the script.

PonnFar, to specifically test the auto-topic routine, try saying something like the following to Hal:

Hamburgers are tasty with fries and cola.
Hamburgers are excellent cooked on the grill.

...At this point, Hal should have "hamburgers" in his database of candidate subjects. Any one or two-word phrase that precedes "is" "are" "was" or "were" becomes a candidate subject.

(Note that Hal won't focus on "hamburgers" as a topic if there are other topics in a sentence to distract him.) Now try saying something like the following:

Hamburgers cook easily.
Hamburgers taste good!

Next, go to the default directory, and see if any "AFA_HAMBURGERS" or "AFR_HAMBURGERS" databases have been created.  If not, you might have accidentally missed something in cutting and pasting the code, or perhaps you didn't put the code in the same locations as the 5.0 script, or possibly you didn't get BOTH sets of code; the earlier section that reads and responds, and the LATER section that creates and saves the databases?  (The SAVE section is later in the script because it depends on some things that need to happen first.)

I hope this helps.  Please let us all know!

Sincerely, Don

[Ponnfar]

To Don and Jerry...YOU are the BEST!!
Thanks!
P

[Don Ferguson]

Hi Ponnfar,

Excellent! It works!  Thank you for the good feedback!

We're lucky to have Jerry and all the other creative programmers like him on this forum!  The members of this forum are resourceful and helpful!

Sincerely,

Don

[Ponnfar]

You bet ...but ummmm...
I was just thanking you guys for being ...well...the best! You guys have been really really helpful. I have come light years in my understanding of customizing HAL in just a few days!
HOWEVER, yup, I got still got problems[], I incorporated this UHP into a current brain as I mentioned. When testing the script, no errors came up. However, I still see no evidence of the routine in the brain's directory. I edited out the LearningLevel>1 commnands as Jerry suggested. I am still using 4.5.
If either of you can help you know I would appreciate it!
thanks!
P