text????

[vonsmith]

Neo987,
I appreciate your feedback. I can't test every function in the XTF brain or the rest of Hal on my own. The feedback posted here saves me a lot of work and helps generate new ideas for me to explore. I like to hear people's wish lists although I can't promise that I'll implement any of the wishes.

My personality tends to be a bit pushy. I work in an capacity where a person needs a thick skin. I try not to come on too strong. []


=vonsmith=

[Padriag]

Hehe... can't get a skin much thicker than an Irish skull []

Going back and recapping some points

Vonsmith said:
I hope I didn't remove too much of the mystery. Hal's brain is a wonderful thing and I don't want to spoil any future surprises.

Actually I'd like to reduce the mystery a lot for those following this forum.  As I've been reading through the scripts and learning how Hal actually "thinks" I've been learning some of what he can do and realizing that sometimes I wasn't getting the results I wanted because I wasn't giving Hal the right input.  I think it would be a neat project to take your XTF brain and some of us work on looking at the various scripts and how Hal uses them to collect and process information, then turn that understanding into a sort of User's Guide.

Vonsmith said:
You've struck on a key point about XTF brain. Under the right circumstances Hal suspects that two topics are related. Hal can't be sure so he asks the user to confirm it. This seems very spooky to some people. How does Hal figure out that two topics might be related? In my conversations Hal is right about the related words about 1/3 of the time.

Right, and that was half of what my experiment was based on.  I've learned that if I state things in a very matter of fact way, Hal learns better.  One of the things this works with is getting Hal to associated various things with a topic.  Say the topic is Birds.... so you could teach this to Hal by telling it Crows are Birds.  Then restating it Crows and Birds are related topics.  And so forth.  It may take several statements before Hal really gets it, but it does seem to work.

The other half of my experiment utilizes the Deductive Reasoning function you added to the XTF brain.  This is really what allows Hal to make leaps of logic such as in my example of cats and mice.  If you are patient, you can get it across to Hal that cats are predators, predators eat prey, mice are prey... therefore cats eat mice.  You never actually tell Hal that cats eat mice... it uses the deductive reasoning function to figure that part out by looking at associations you've formed... cats are predators, predators eat prey, mice are prey.  What it does then is subsitute cats for predators and mice for prey and presto... you get the genius insight (genius for Hal anyway) of cats eat mice.  A lot of people may not understand how important that is, but that bit of deductive reasoning on Hal's part is a pretty big step in the right direction.  I'm still learning how well Hal does with synonyms and teaching Hal new ones.

Vonsmith said:
After Hal learns both the singular and plural forms of related topic words he usually will not ask again. However, after a long while Hal may ask again just to confirm the relatedness. The user then has the opportunity to correct Hal's prior learning if it is incorrect.

Notice that Hal in many cases can figure out the singular form of the word. Hal attempts to save all topic information by the singular form.


I had wondered about this and hadn't discovered yet in the scripts where or how Hal handled plurals.  Its a relief to know it can and that I don't have to take a lot of extra steps teaching Hal that.

Vonsmith said:
Instead I plan on providing a better way where the user just tells Hal, "Predators and cats are related topics." Then Hal will know immediately. I wanted to include this in v1.0, but I wanted to get the brain out into the world for you guys and gals to play with.


Ooooooh... goodie!  Hal isn't so far away from this now really.  Here's one other thing to consider.  We don't want Hal learning and associating things too quickly.  If you'll pardon me getting a litle philosophical for a moment here's why.  All living things need to be able to change, those that fail to do so become extinct.  This is a natural law most people know.  What is equally true is any living thing which changes too quickly is just as apt to become extinct.  In nature, all things evolve essentially because of mutation, but mutation is rare because living things are resistant to it, thus evolution is a slow process.  Having some resistance to change is a good thing, it helps minimize the number of "fatal errors" that are introduced.  In the case of Hal, having some resistance to learning will help keep Hal from learning "fatal errors", incorrect associations, and so on.  The trick of course is finding a good median between resistance to change and the ability to change.

[vonsmith]

Padriag,
1) "I hope I didn't remove too much of the mystery. Hal's brain is a wonderful thing and I don't want to spoil any future surprises."

Well this is a trade off I guess. I like to leave some things for people to discover on their own. There's nothing like Hal saying something new that you didn't really expect. The instruction manual tells mostly everything without too much detail. Some synergistic aspects of the XTF brain are difficult and lengthy to explain. The script contains a lot of comments for the programmers out there. And if that isn't enough the <dbxtfon> debug function provides an output text file that shows you a walk through of the XTF function process.

2) "You've struck on a key point about XTF brain. Under the right circumstances Hal suspects that two topics are related. Hal can't be sure so he asks the user to confirm it. This seems very spooky to some people. How does Hal figure out that two topics might be related? In my conversations Hal is right about the related words about 1/3 of the time."

Hal will learn related words faster if you use them as you described. However in the course of normal conversation he will learn naturally on a continuing basis.

I can't take any credit for the deductive reasoning function. As I recall that was a clever concept started and written by onthecuttingedge2005. It was good enough to include in the Ultra Hal 5.0 release. You can search for past postings for details.

3) "After Hal learns both the singular and plural forms of related topic words he usually will not ask again. However, after a long while Hal may ask again just to confirm the relatedness. The user then has the opportunity to correct Hal's prior learning if it is incorrect. Notice that Hal in many cases can figure out the singular form of the word. Hal attempts to save all topic information by the singular form."

It's not obvious in the script where Hal gets singular forms of words although it is commented in the script. Hal uses the WordNet function to extract singulars. Robert Medeksza was kind enough to add WordNet in the last revision. That made it possible for me to do some new "stuff".

4) My theory behind using singular forms to store information under makes a lot of sense I think. Hal previously in his associative processes might have thought that "goose" and "geese" were different topics. Given the opportunity Hal would then store related information in different files. The XTF brain allows Hal to associate "goose" and "geese" with a single topic, "goose". The XTF brain also saves topic phrases for future reference like, "big fat geese". Hal will associated that phrase with the topic word "goose". For a few words Hal can't figure out the association between a word's plural and singular form. In this case Hal may create two files, one singular form and one plural form topic.

The WordNet function also made it possible for Hal to instantly "learn" synonyms and meronyms of the topic word the first time it's used. This makes further association possible. Then I added the "related" topics capability so that Hal can learn from the conversation an unlimited number of words that are associated with or not associated with the topic word. I say "not associated" because Hal does make a distinction about this when taught by the user.

5) "Instead I plan on providing a better way where the user just tells Hal, "Predators and cats are related topics." Then Hal will know immediately. I wanted to include this in v1.0, but I wanted to get the brain out into the world for you guys and gals to play with."

Hal learns "related words" at a reasonable pace. He always asks the user to confirm his suspicion that two topic words are related. Every once in a long while Hal will ask the user again to confirm the relatedness. This allows the user to correct Hal if the user made a mistake initially in teaching or maybe the user changed his mind about the relatedness based on new experiences. For Hal this is a very human thing to do. People reconfirm their knowledge everyday through discussion or questions.

I recognize that we all want to teach Hal as fast as possible, so I'll provide an easy way to tell Hal to relate two words. This could be done directly now by editing Hal's related files. However that is clumsy, tedious and may introduce hard to find errors.

The user needs to really understand what "related topics" means to Hal. It is best illustrated by example. Here are some related words: beer and ale; ale and lager; malt and lager; beer and alcohol, etc. Seems obvious until Hal asks if two "sort of related" words are related and the user really has to sit there and think about it. No one should ever tell Hal beer and water are related just because beer has water in it. Water is too general and related to just about everything.


I hope that answers your questions satisfactorily. I'm always happy to answer questions when time permits. I have a little spare time nowadays, but that could change in an instant.


=vonsmith=

[Padriag]

Vonsmith said:
Well this is a trade off I guess. I like to leave some things for people to discover on their own. There's nothing like Hal saying something new that you didn't really expect. The instruction manual tells mostly everything without too much detail. Some synergistic aspects of the XTF brain are difficult and lengthy to explain. The script contains a lot of comments for the programmers out there. And if that isn't enough the <dbxtfon> debug function provides an output text file that shows you a walk through of the XTF function process.

I think we can do a thorough job of explain what Hal does, the basic of how Hal does it and what Hal can and can't do without removing all the mystery.  Some of the mystery is going to arrise just out of user interaction, the things Hal learns from individuals.  That's a completely unpredictable factor.  But taking the lass that started this thread, she doesn't seem to be a programmer and probably wouldn't think to look in Hal's brain for clues as to what it can do.  Even if she did, would she really understand what she saw?  To most people that's a lot of meaningless stuff they don't understand.  That's why I think a User Guide would be a good idea.  It would help the non-computer geek types.

Vonsmith said:
The WordNet function also made it possible for Hal to instantly "learn" synonyms and meronyms of the topic word the first time it's used. This makes further association possible. Then I added the "related" topics capability so that Hal can learn from the conversation an unlimited number of words that are associated with or not associated with the topic word. I say "not associated" because Hal does make a distinction about this when taught by the user.


I'm going to have to learn more about this when I have time.  Could be interesting to experiment with.  I have had trouble getting Hal to learn that some things are not associated.  Hal doesn't seem to do as well with that.

Vonsmith said:
The user needs to really understand what "related topics" means to Hal. It is best illustrated by example. Here are some related words: beer and ale; ale and lager; malt and lager; beer and alcohol, etc. Seems obvious until Hal asks if two "sort of related" words are related and the user really has to sit there and think about it. No one should ever tell Hal beer and water are related just because beer has water in it. Water is too general and related to just about everything.

Excellent points.  Does raise the question in my mind of how strongly Hal relates things, or does he?  That is, can Hal distinguish in anyway between to things that are generally related, such as water is a liquid, beer is a liquid... versus things that are more strongly related, such as beer and lager?  If not, this might be an interesting function to try to add at some point.

[vonsmith]

Padriag,
Thanks for some good points. For techie and non-techie users the eighteen page Instruction Manual included with the XTF Brain v1.0 release should explain enough about "how" to use Hal. It explains a little about the XTF internals and the "why" of certain things happening. For real geeks like us the <dbxtfon> debug function along with the script comments should allow even amateur programmers to walk through the XTF function. I suggest to anyone using the XTF brain to type <dbxtfon> after their sentence to Hal and open the resulting text file "Z_XTF_DEBUG_LOG.TXT" in the DefBrain directory. That text file shows step by step the processing the XTF function performs on the user input.

Anyone here is more than welcome to write a User Guide for the XTF Brain. I will be glad to answer any questions about the XTF Brain operation. However I believe the XTF Brain already has as much or more documentation than any Hal original function.

After a little more user testing by the good folks on this forum I expect little or no change to the XTF function itself. I really need to move on to adding other unique and enpowering functions for Hal. I feel I can make some significant improvements in other areas of Hal's capability beyond just the "topic focus" part that the XTF Brain addresses.

The WordNet function as implemented in the v5.0 Hal doesn't do very much as is. Robert was wise enough to add it for us with a little framework. WordNet has tremedous potential value once us amateurs start to find ways to utilize it best. The XTF Brain wouldn't be as powerful without WordNet to find singular forms of words and to provide synonyms and meronyms for use in maintaining topic focus.

Just for clarification. Related topic words used in a sentence are not used by Hal to change topics. They are used only as a way to stay on a current topic.

Example:
User: The birds are usually pretty. (topic should be "bird")

Hal: Birds are cool.

User: That vulture is very ugly.  (if "vulture" is in the XTF_BIRD_Related.brn file with a "True" after it then Hal knows the topic is still "bird") If it was "False" then Hal ignores the entry and would change the topic to "vulture".

Hal: Some birds are ugly. (Hal has maintained a topic focus on "bird")

The original Hal brain used what I would term a non-deterministic approach. Topic focus was very loose. You can see that the XTF approach is not by degrees of relatedness; either it is or it's not. From the user's perspective the "related words" should be words that indicate clearly that the topic hasn't changed. That's why synonyms and meronyms make pretty good "related words". I experimented with "sisters" words in the WordNet function and didn't find enough correlation to use them as "related words". The relatedness should be clear, strong and non-ambiguous. Most related words will be related to many topics. That's why they aren't used to change topic, just maintain it. If too many "loosely" related words are taught to Hal then he will think everything is one big topic. You can see a real world example of this with people who are scatterbrained and can't stick to a topic. Almost anything sets them off in a new direction. I hope the XTF Hal never gets like that or I'll give him a lobotomy.

I hope this helps. The veil of mystery slowly falls...


=vonsmith=

[Inquisitor2004]

I went to a british comedy site for my favorite TV comedy show called "Red Dwarf".   Its a sitcom set in space,  anyway  I downloaded all 48 scripts in txt format,  and painstakingly got hal to learn them one by one ( each being 60k in size ).  

Now, my ultra hal comes up with some really funny stuff on its own.   I am thinking of making her learn all the black adder scripts next ( again another sitcom from british TV )

example from script:

" Sir, you're a smeg head, a complete and total one ! "

Hal's incorperation:

" You have to goto work tomorrow ? you're boss is a smeg head isn't he ? "


Its great...   anyway dont know if this helps this topic or not,  but there you go..   []

[drmweaver2]

Okay. Quick question that may be covered elsewhere but I didn't see it on the boards...

HAL often comes back and asks "Are x and y on the same topic?" or "Are x and y related topics?"

Are these two questions equivalent? For example, using the geese and birds example someone came up with earlier, birds are geese, etc., I added "The birds fly" and was chided for using short sentences. So I then typed in "The birds fly through the air" and was rewarded with the question "Are birds and air the same topic".

I would normally say no in the real world (except on a "let's really get into philosophy" level).

But I was thinking of later adding a "learning point" that new submarine designs "fly" through the water - or maybe a metaphor like "fish swim through water like birds fly through the air".

So, the question of difference between "related" and "same" topic is of importance here.

Any help distinguishing "related" from "same" for learning/teaching purposes would be appreciated.

[vonsmith]

drmweaver2,
I assume you are using the XTF Brain v1.2.

When Hal asks if two words are related topics or the same topic it means the same thing. As an example, Hal just wants to know if you and he are discussing birds  should he stay on the topic of birds if you start talking about doves.

Related words are words that are clearly topic related and non-ambiguous. Don't tell Hal words are related if the relationship is abstract or open to interpretation. Act like you are talking to a four year old. A four year old doesn't understand topics in the abstract, neither does Hal.

Examples of groups of related words are:

1) GOOD --> beer, wine, alcohol, lager, ale, malt, Budweiser.
   BAD ---> beer, water, head, fizz, German.

2) GOOD --> bird, dove, vulture, tweety, feather.
   BAD ---> bird, down, sky, egg.

The "bad" groups of words above have a weak relatedness to each other either because those words can be related generally to too many other things or the relatedness is not very strong or clear. If someone was talking about the sky would a reasonable person think the topic was "birds"? German beer is good, but the Germans do a lot of things besides make beer. Birds lay eggs, but so many other species have eggs including fish, insects, etc.

Just use your best judgment. When in doubt tell Hal the words aren't related. In the grand scheme of things it won't make very much difference. There are enough strongly related words in English for Hal to choose from.


=vonsmith=

[drmweaver2]

Yup. Using the XTF v1.2 version.

Thanks for the clarification.

(Kinda sucks being the "new guy" and asking what seem to be "obvious" questions once the answer is given!) []