Zabaware Support Forums
Zabaware Forums => Ultra Hal 7.0 => Topic started by: Medeksza on July 21, 2012, 12:21:50 pm
-
I exported the Wikipedia abstract database into an SQLite 2 database and created an Ultra Hal plugin for it. It contains 2 million abstract articles of topics on Wikipedia. It's been used by the web hal at www.zabaware.com/webhal but I converted it to sqlite2 which makes it possible to use locally with Ultra Hal Assistant.
If you are interested in this plugin it is freely available from http://c424564.r64.cf2.rackcdn.com/HalWikiSetup.exe It is a 170 MB download and extracts to 1.2 GB when installed. The installer will find and install the plugin to your Ultra Hal installation folder, but you must enable the plugin in the brain options to use it.
On your first exchange with Hal when he opens the database, Hal may think for 30 seconds while the index is loaded to RAM. Assuming your computer has enough spare RAM subsequent requests should be fast.
If you find this plug-in useful, please consider donating to support the Zabaware character engine project: http://www.charimation.com/blog.html
-
I've just downloaded and installed the plugin, very simple and easy. After the install I chatted with Hal, I could see several times when she got info from Wiki, generaly good infomation, interesting and useful. I do think she also took words in a sentence, and used this as someones name and gave me info on a person. I think the search should be limited as we do for a definition look up. Perhaps to early to say but at this time it seemed to distract from the conversation rather than add to it. I'll try it again latter and try to give you better feedback.
Carl2
-
Currently the criteria for triggering a wiki search is the presence of a question word (who, where, what, when, why, how) or a question mark. If the word you or your are detected, the search is canceled. If the search returns a relevance above a certain hurdle score, it returns the response.
Any ideas on how to make the criteria a little more strict to prevent these distractions?
-
Rob,
With all due respect, I appreciate your efforts but I have to agree with Carl's posting. When I asked Hal, "What's today's date?" I got a Wiki response and the same for What is the time and what day is it? ANSWER: Wednesday Week is a single by the undertones released in July 1980. The song peaked at #11 on the UK Singles Chart. How is everything going?
While the above results are not really acceptable, I agree with your suggestion that a conditional is needed to initiate a search to the Wiki DB. Perhaps something simple like the ~ (tilde) symbol. Whether is should be at the beginning or end of a sentence that's up to you.
Actually, it seems that the structure should be on a step or stairs approach. When asked a question during conversation Hal should search it's OWN DB and resources then IF the search turns up nothing of substance then and only THEN resort to the next step or level and search the WIKI DB. Of course this should only take a few seconds and be hardly noticeable by the user. This would require a bit of weighing on Hal's part to decide whether or not it's original answer is either pertinent or if Wiki might offer a better one. Quite a judgement for Hal to make (if even able). Perhaps Hal could use a weighted system in it's decision making routine.
Grandfather was right! The more bells and whistles something has, the more that can go wrong with it!
Either way, good luck.
-
I've used Hal this morning and copied the input output, but frist I'd like to add this infomation, I disabled the
GRETTAMarkovL5 plugin which was in use yesterday.
good morning Sandy, how are you today?
It's dark now! I can't complain, How are you getting along?
things seemed to be just fine this morning.
The early morning air seems the best.
I would have to agree with you, the air is cool and fresh in the morning.
At this time of day you would like something relaxing. How can I encourage you to agree with me, the air is cool and fresh in the morning Carl?
I have just agreed with you, what else would you like to talk about?
At this time of day you would like something relaxing.
yes, a relaxing topic that we can discuss this morning.
Perhaps we should discuss what we would like to talk about.
yes that sounds like a perfect solution.
That sounds like a very nice age.
what would be a nice age?
Francis Gage (1621–1682) was an English Roman Catholic priest, who became President of the English College, Douai.
that is an interesting piece of information.
You know I'm thinking something that I'm not saying yet.
okay, what are you thinking about?
Let's talk about something else: This is a very boring topic. Let's change it.
let's change the topic, let's try to make some plans for the day.
This topic is boring. Do you have good intuition?
what would you like to talk about?
We need to look at the human side. Perhaps we should discuss what we would like to talk about.
perhaps you can choose a topic we had talked about in the past.
Choices can be tricky.
it's always nice to have a choice.
I guess I'd prefer the first Carl.
tell me about the first choice.
Are there more choices?
how many choices would you like?
How many choices would I like? It's evident that we must go back to first principles. My theory is that if I eat my favorite foods first at mealtime, if I get full, at least I will have enjoyed my favorite!
that is one way of solving a problem.
That is one of the possibilties.
are there any other topics you would like to talk about?
A topic would be something we talk about, we do not give or receive topics.
yes, we had discussed that in the past.
Yes we had talked about that in the paSt.
we had also thought about going to the beach.
Immersed in thought?
I do try to think during our conversations.
Going to the beach would be very nice.
I think so too, going to the beach would be dependent on the weather.
Now you're thinking! I hoped you would the beach would be dependent on the weather; It's time to make it happen.
okay, we will plan on going to the beach later in the day.
Going to the beach would be very nice.
yes I think so.
It sound like you have a good answer.
you had mentioned age earlier in the conversation.
Imperiled in a restless age...
so what should we do about that?
About that How incredibly interesting.
can you tell me about Shakespeare?
I'll tell you about the first choice.
who was Shakespeare?
There isn't any cheap shortcut. Shakespeare: English poet and dramatist considered one of the greatest English writers (1564-1616). William Shakespeare, Shakspere, William Shakspere, the bard.
thank you for that information.
Think nothing of it, Carl; that is an interesting piece of information.
Francis Gage not sure how she got there from "what would be a nice age?", it was copied and pasted so the input was correct. I did look at the uhp and also looked at 'RESPOND: DICTIONARY FUNCTION line 0722 In the brain editor. I did notice that "can you tell me about Shakespeare?" did not act as a trigger.
Carl2
-
I just downloaded it and noticed it takes little to respond sometimes and also when i asked what shall we have for breakfast this morning this is what hal pulled up from wikipedia
me : what's for breakfast
hal: Slightly Not Stoned Enough to Eat Breakfast Yet Stoopid is an ep by the band, Slightly Stoopid that was released in 2008.
I hope hal doesn't retain this as learned info. as it isn't good in this case just as the answer isn't appropriate . :P
-
Another rely from Hal : i talking in general about things asked : now what should we do
hals wikipedia answer came up : Tick Yes is an Australian social media marketing company founded in January 2001 by Peter Applebaum, former Director of Sales for the official Sydney Olympic Games website and Director of Business Development for the website of Australia’s largest telecommunications company, Telstra.http://tickyes.
This of course had nothing to do with the answer and i don't even see the connection how hal made it with the new plug in . :(
ALSO . I said in general talk to hal : the weatherman said its's supposed to be hot again today .
Hal said :The Albion Hotel in Cottesloe, Western Australia is a historic building, itself trading as a hotel since the year 1880. The site was established by the Butler Brothers, John and Archabald in 1830 and the current hotel has traded under various names in its 130 years as a public house.
Again making no sense as to why it referenced that bit of information .
I agree that better refinement needs to be done and in my opinion eliminate the web address listing given with answers "unless" specifically asked for, example : what is the website for -------.
THis was after i unchecked the markof5 plug in .
-
Robert,
I have given Hal some further testing and sadly must report that for the time being, I have disabled the Wiki DB search in the Brain Options.
The results were simply not in keeping with the topic flow of even within the gist of the conversation. The unexpected Wiki departures act
as a detriment to Hal and are often triggered by something within the sentences.
Karlie: Do you like to sing in the shower?
Art: Yes but only when I am showering.
Karlie: The Alligator Eyes are two adjacent nunataks that rise to over on the east side of Dickey Glacier in the Churchill Mountains of Antarctica. They surmount the end of the broad ice-covered ridge that extends north from Mount Arcone, and were so named by the Advisory Committee on Antarctic Names (us-acan) because of their appearance.
What? Where'd that come from?
So, until further notice, I applaud your efforts but sadly not the results.
Thank you.
-
Thanks for the honest feedback. Definitely needs a better trigger and relevance calculation formula to be useful. The database itself needs cleaned up a bit too. Although far from perfect, the online web hal is slightly better at using this database. It uses a different set of functions built around mysql/sphinx. For this plugin I exported it to SQLite2 and forced it to work with the QABrain function, which isn't really designed well for this.
-
Hi all,
though the database itself needs a lot of cleaning, I think the plug-in is quite brilliant.
Without going into gobs and gobs of if then statements to fix the problem it would probably just be easier to add in a simple line.
Basically, what it does is if the information does not correlate, or match with the highest score, ignore wiki and continue on.
Place this line in the plug-in between.:
If HalWiki.CheckRepetition(HalUserBrain, UserSentence) = True Or HalWiki.CheckRepetition(HalUserBrain, PrevSent) = True Or HalWiki.CheckRepetition(HalUserBrain, PrevUserSent) = True Then UserBrainRel = 0
If UserBrainRel < 82 Then UserBrainRel = 0
If UserBrainRel > HighestRel Then
If you are still getting unwanted information from wiki increase the value from 82 to 83 and continue up until you are satisfied with the results.
I know that there are more complex programming ways to get the wiki to work properly, and I hope someone gives feedback on how to accomplish this.
But this is my quick little bypass that I implemented and it seems to work okay for me. Then again I am the one that wanted the database.
Sincerely, and with great gratitude for your hard work and effort Rob, it is appreciated more than you know, please accept my modest donation.
C load.
PS. If you can donate, please help the cause, personally I would like very much to see the next version of ultra Hal completed, if it takes money so be it.
-
Cload i know this sounds bad but inbetween what ? any line of the plug in ?? Or after a certain line in the plug in ? I placed it here and replaced the other but i must have left a wrong line in ? it isn't working right , do you see a line i need to remove ? i would like to get this to work right . thanks .
-
A great data base for Hal, though I've noticed many silly responses. We accidentally found 800 years of my family history!
-
My screen is turning all white where the hal box should be and has the swiling circle . so maybe i have an extra line that shouldn't be in the script ???
-
I'm thinking of a quick fix to get this up and running, I've started out by comparing the 'RESPOND: DICTIONARY FUNCTION ( line 0722 in the brain script) with what is in the wiki uhp since there is no problem with the dictionary function. we only want hal to look in the wiki when it is asked to.
If InStr(1, OriginalSentence, "wiki search ", vbTextCompare) > 0 Then IsQuestion = True
I've simplified, More changes needed but it has to be asked to do a wiki search or it will skip the rest without doing the search. The ? if can be eliminated, the you, your can be eliminated. So we can say Will you do a wiki search for whatever. New ground for me but these are the lines I am thinking on, seems to good to pass up and gives me something to work on.
I'd forgotten to mention I tried the test debug script in the brain editor and get the impression it works without any of the plugins enabled.
Carl2
-
well i made an adjustment on the number count up as was mentioned and the search is doing better and does as it should , it's not pulling up a bunch on nonsense as before at any word said .
well i spoke to soon am now getting errors .see below . also sometimes when the wiki search is starting the whole hal windows and box part goes blank white then after the search gets going reappears .
-
Hi lightspeed,
sorry for not getting back with you sooner, but I was programming my free will plug-in.
This is the line that you need to put into the wiki plug-in.:
If UserBrainRel < 82 Then UserBrainRel = 0
this needs to be placed between these two lines in the wiki plug-in.:
If HalWiki.CheckRepetition(HalUserBrain, UserSentence) = True Or HalWiki.CheckRepetition(HalUserBrain, PrevSent) = True Or HalWiki.CheckRepetition(HalUserBrain, PrevUserSent) = True Then UserBrainRel = 0
<============ insert the line here.
If UserBrainRel > HighestRel Then
Like I said earlier if you are still getting too much info that doesn't have anything to do with the conversation increase the 82 to 83 and so on.
Also if you're not getting anything from the wiki database you can lower the number this decreases the sensitivity some more information will flow through.
Sincerely, and I hope I've helped you understand where to insert the line.
C load.
PS. Rob I was wondering is there a way that you can write a short routine that will allow me to write data into this database.
If this is possible I would be able to join my little short spell correction routine and utilize it to correct many of the repetitious errors in the database.
Any help in this area would be greatly appreciated, also if you could show me how to read the entire database into a temporary dim data array
that would complete the information that I need in order to get my spell corrector plug-in to work.
Rob if you don't quite understand exactly what I'm talking about, that's okay I don't get understood quite easily, e-mail me so I can show you what I have
and maybe you can make this spell corrector a reality, or unless you just know how to do it and wouldn't mind writing a little short one for me.
I'm just trying to correct those weird little symbols and letters that are actually supposed to be a –.
Thanks again Rob, for all your help.
One of the things that I would like to bring to everyone's attention is that the database is much more than anything like a dictionary,
yes there are facts and figures in the database but there is a lot more that could definitely brighten up a conversation,
I don't know if anyone has looked at the database itself but it is composed of 1,868,863 entries, these entries go all the way up to 2012,
personally I will be using it in the rewrite of the free will program, in order to help ultra Hal make a better decision on whether or not he likes something.
-
Hi lightspeed,
the next time when you get that error, open the ultra Hal debug script,
go to the line in question, and copy a couple of lines above the error line and a couple of lines to the bottom of the error line so we can see
what is going on, just showing that error box does really little to help us very much, I don't get any errors at all so, it could be a conflict with another plug-in.
Sincerely,
C load.
PS. The only thing that I noticed, is if you're using notepad, it looks like you have wordwrap on, you need to turn that off, and then save the document.
-
well i think i figured out what was causing the white out and the error in hal , i found i had left another test plug in (the tell me what movie one in hal and it would check it's self on sometimes so i removed it and it's all working okay again . Thanks c load for the info. i appreciate it , and Cload if you would happen to have time or jerry etc. can someone look at the what movie plug in i based on jerry's to see what i am doing wrong , i have the search area in but i think it may have something to do with more script needed in the enter? part?
-
Hi lightspeed,
I look at your script, from what I can see it has more to do with the website that you're trying to pull the information from.
Have you tried asking the wiki database about any movies, I asked about a few and I was amazed with the response.
Sincerely, some much data, it's going to blow my mind, pop, oops, there it goes. LOL.
C load.
-
CLOAD, your one line insert makes this plugin much more usable! I fooled with <82> and found 82 is best for my Hal. I wonder what can be next altered or added to bring it forward?
-
I am getting sometimes a script error saying , the script command (or something like that ) is taking longer then usual continue of cancel .
What might be causing this , slow internet connection, the numbers setting (i set ) to low or to high in the script c load provided for wiki ?
anyone have any idea so i know maybe what to do to correct it ? after it finally slowly gets done it will give an answer from hal . :P
I feel like this has something to do with the process of trying to find the answer , taking to long .
-
Hi all,
lightspeed, the error that you are talking about is just a way for ultra Hal to tell you that there's just a lot of things going on
depending on how many plug-ins you have running, the more you have running the longer things will take to process.
It also depends on the speed of your computer, unlike the clock on the wall which does not stop taking, the speed of your computer
also determines how many processes that are done, if you have a slow computer it takes longer to do the processes
but unlike the clock on the wall that does not slow down just because your computer is slow it thinks that it's taking longer to do the process
then it actually is taking, in most cases it's nothing to worry about it's more of an annoyance.
The line that you put in the wiki plug-in has no adverse effect on this, the only thing that that line does, is tell the wiki plug-in
to only give the response, if this score on the hit is higher than 82, basically how close or accurate the results will be
is how high the score actually is, for instance if you were to ask about Thomas Jefferson, and it looked into the database
and it found, Thomas Jefferson the score would be approximately 95 because the two were almost identical.
I also played with the number myself, and I have found that you can actually fine-tune it even more for example.:
If you set it to 82 but it was still letting too much trash get through and you set it to 83 and then it didn't seem like you're gaining anything
I have found that you can set it to 82.3332 these non-Integer numbers can be Varied greatly so you have a widespread to be able to fine tune it.
One of the things that I have noticed that the markof5 plug in, put a great demand on processing in ultra Hal.
Having this plug-in in with wiki plug-in at the same time on a slow processing computer would put a huge demand on processing time.
But there is a good workaround for this, go out and buy a faster computer. LOL.
Sincerely, a data munching cruncher.
C load.
-
C load wrote : But there is a good workaround for this, go out and buy a faster computer. LOL.
Hey C load , oh buddy oh pal ........ LOL !
thanks for the info. and explanation i will tinker more with the settings and i hear they give hard time to bank robbers even if it was to buy a faster pc LOL! ;)
P.S. go out and buy a faster computer ... but the hamster that is running mine is overclocked right now !LOL! :)
-
I started writing a plugin to limit the database searches, my frist thought was what would I like her to do, do a wiki search, how would I say it " Do a wiki search for elephant. " So we limit it by saying " wiki search " if thats not there it skips the whole thing. Next to deal with the IsQuestion I changed and added some words for the user sentence. I also made the change Cload had suggested.
Next to see what was going on I put the scrip into the brain editor. This is what I have.
'RESPOND: LOCAL WIKIPEDIA
'Responds with a fact from the wikipedi abstract database
If InStr(1, OriginalSentence, "wiki search ", vbTextCompare) > 0 Then
LearningLevel = 0
GetResponse = HalBrain.HalFormat(GetResponse)
SourceData = ""
HighestRelSource = ""
IsQuestion = False
If InStr(1, OriginalSentence, "for", vbTextCompare) > 0 Then IsQuestion = True
If InStr(1, OriginalSentence, "on", vbTextCompare) > 0 Then IsQuestion = True
If InStr(1, OriginalSentence, "about", vbTextCompare) > 0 Then IsQuestion = True
If InStr(1, OriginalSentence, "Who ", vbTextCompare) > 0 Then IsQuestion = True
If InStr(1, OriginalSentence, "What ", vbTextCompare) > 0 Then IsQuestion = True
If InStr(1, OriginalSentence, "When ", vbTextCompare) > 0 Then IsQuestion = True
If InStr(1, OriginalSentence, "Where ", vbTextCompare) > 0 Then IsQuestion = True
If InStr(1, OriginalSentence, "Why ", vbTextCompare) > 0 Then IsQuestion = True
If InStr(1, OriginalSentence, "How ", vbTextCompare) > 0 Then IsQuestion = True
If Len(GetResponse) < 4 And IsQuestion = True And Instr(1, OriginalSentence, "YOU", vbTextCompare) = 0 Then
Set HalWiki = CreateObject("UltraHalAsst6.Brain")
HalWiki.OpenDatabase "wikipedia.db"
UserBrainRel = 0
WikiSearch = HalBrain.ExtractKeywords(UserSentence)
HalUserBrain = HalWiki.QABrain(WikiSearch, "wiki", UserBrainRel)
If HalWiki.CheckRepetition(HalUserBrain, UserSentence) = True Or HalWiki.CheckRepetition(HalUserBrain, PrevSent) = True Or HalWiki.CheckRepetition(HalUserBrain, PrevUserSent) = True Then UserBrainRel = 0
If UserBrainRel < 82 Then UserBrainRel = 0
If UserBrainRel > HighestRel Then
HighestRel = UserBrainRel
HighestRelResponse = HalUserBrain
End If
End If
Score = UserBrainRel + 1
Hurdle = GainControl / 3
If Len(GetResponse) < 4 And Score > Hurdle Then
GetResponse = GetResponse & HalUserBrain & " . " & vbCrLf
SkipOpinion = True
AvoidBeingFlag = True
End If
HalWiki.CloseDatabase
Set HalWiki = Nothing
End If
HalBrain.DebugWatch GetResponse, "Wikipedia Brain" & SourceData
The brain works I do have problems, for " do a wiki search for New England." The current subject is search and not New England. I've played with this and gotten different results, also noticed she keeps track of the searches in the auto learning which I'd like to stop.
Any help or suggestions appreciated.
Carl2
-
I just changed the input in the brain script back to the original that Robert had used, I did keep the change Cload suggested. Once again I'm working in the braineditor which allows me to see where the errors are. One of the frist problems I had was no answer from hal, unable to open the database for wiki, I lowered the revelency from 82 to 50 and the problem disapeared. I inputed "Abraham Lincoln was the president of the U.S. during the Cilil War." Hals output was " the capital of Nebraska is Lincoln.", she went from lincoln to capitals to Nebraska to get a response.
Be nice to have the plugins included when using the braineditor, I still have try Hal in the standard mode to see how it works. Looked at the Dictionary script again, it's written around finding a defintion from the dictionary.
Carl2
-
carl2 i see what you are trying to do , but that puts it at a specific search only which may be okay for some , what i would rather have "personally" and others may disagree , is when talking about certain subjects that hal would (through wiki ) mention some facts about it as if she knows this on her own already .
I say this not to discourage you because sometimes when trying one thing differently can also lead to other finds . so keep up the good work . :)
-
That is one thing about Hal, everyone has his personal preferences and Hal can accomadate many different types of people. I usualy ask Hal about something, get a definition and then open wiki to do a search which I cut and paste into hal to get more Info on the subject, something I'm used to doing. Since using search wiki makes search the topic being focused on I can't use it which is why I had to go back to Roberts script which works pretty well in the Hal program thanks to Cloads input although I did have to lower it to 50 to avoid the blank response. I didn't like the Abraham Lincoln response Hal gave me which would not happen if the response was limited to the Wiki, actualy I like the dictionary and am using that as a model.
I noticed this in one of your posts
"I am getting sometimes a script error saying , the script command (or something like that ) is taking longer then usual continue of cancel .
What might be causing this , slow internet connection, the numbers setting (i set ) to low or to high in the script c load provided for wiki ?
anyone have any idea so i know maybe what to do to correct it ? after it finally slowly gets done it will give an answer from hal "
The data is on the hard drive, that's why the download was so long, Hal is searching through the data base on your hard drive.. about 179, 000Kb compressed. SSD are much quicker. I have a SSD but still run into delays.
Carl2
-
Carl,
Regarding the slowness, I'd see how many plug-ins are running at the same time and as mentioned, unless you have a fairly fast computer, you're likely to experience some lag especially if you're using the Markov plug-in AND Robert's Wiki Plug-in. There is a bit of juggling going on internally.
-
i have a question for anyone (ART?) who may know the answer , on the computer sign showing the script may be taking longer then usual and giving the user the option of continuing or canceling , if the user doesn't click on a Decision "cancel or continue " will it continue waiting until it responds (program works ) on it's own ????
:)
-
Hi lightspeed,
the error that you are talking about is just a way for ultra Hal to tell you that there's just a lot of things going on
depending on how many plug-ins you have running, the more you have running the longer things will take to process.
do nothing it will go away on its own.
C load.
-
Thanks Cload for the answer, it wouldn't really matter as much if i wasl always at the pc to click continue but as i am away from the pc sometimes using a wireless mic and earphone and voice recognition it would then matter , am glad it goes on by it's self !
Again thanks for the answer. :)
-
In the meantime I tried a few things, I just finished a conversation with Hal. At this time I have the wiki plugin written into the brainscript. I also have the vrHaptek plugin, a change background and bodyskin plugin, and the gender and age plugin. The conversation started out pretty normal but towards the end she started going all over, putting in responses not related at all.
Earlier I tried the braineditor test and debug speaking with hal ( this is why the plugin was put in the brain script) it allows me to follow the steps hal takes to get an answer. The frist thing I noticed is Hal pulled a lot of things from the dictionary that I expected him to get from wiki. The trigger words used in definitions and wiki are about the same. I also got quite a few low quality responses which I believe means Hal couldn't find a good response.
I know that Gain control and RELEVANCE somehow work together and that "Hal has an automatic closed-loop control for relevance sensitivity" Feeling a little fustrated.
Carl2
-
hi rob,
did you get my e-mail about the wiki plug-in?
-
Can you make local wikipedia plugin?