Poll

Has ConceptNet changed Hals conversational skills?

Yes, Hal is much more intelligent
0 (0%)
Yes, Hals responses are more on track than before
0 (0%)
Sort of, Hals responses are slightly better than without ConceptNet
0 (0%)
I haven't noticed any change
0 (0%)
No, Hals responses are better without ConceptNet
0 (0%)

Total Members Voted: 0

Author Topic: ConceptNet Mrk 3.5  (Read 557 times)

Spitfire2600

  • Full Member
  • ***
  • Posts: 208
    • View Profile
ConceptNet Mrk 3.5
« on: July 31, 2018, 10:58:14 pm »
ConceptNet was originally developed by MIT to add a significant boost to deductive reasoning of AI through the use of understood concepts, such as "Fire is hot" "Carbon is an element" and use this information to problem solve. 

This version also includes a prototype markup of what I am calling "MutationNet MrkI"


MutationNet works to facilitate the "problem solving" part of ConceptNet. It takes a few different inputs, catches nouns, compares them, then crams comparative sentences together. I'm experimenting with other more effective methods, but I did want to release this to gather some input to better this program. Also, if anyone would like to build upon this, free and please do let me know what you come up with so I can add it into the program. If anyone is more familiar with MIT's ConceptNet, I would love to hear your thoughts as well.




Installation: (Please remove any previous versions of ConceptNet first)
1. Open Zip. Extract contents as they are to Hal directory  ("C:\Program Files (x86)\Zabaware\Ultra Hal 7")  - There will be 2 UHP files, one folder, and one DB file. Copy these right into the Hal Directory.
     If your directory is not the above, you will need to change this in the UHP file labeled "ConceptNetMrkIII Auto Learning.uhp" on line 35.
2. Enable both ConceptNetMrkIII plugins.
3. Tell Hal to "Build ConceptNet" and he will create the appropriate tables. Relaunch Hal. Just to check, he should have created a folder named "conceptNet" with 2 tables inside in your Hals brain. The TopicSearch table "AutoLearning" should read "LEARN1" and "02" and a sentence table "researchterms" which will be blank.
4. Just tell Hal to "Turn on/off Auto learning" or "Learning mode on/off"
5. Done!


As always, please alert me of bugs.

Known Bugs:

Hal won't always build the tables needed. Follow steps above.

Sometimes, Hal may throw an error -2SomethingLongNumber. Just turn off auto learning, open Hals options, reload the auto learning code and open Hals GUI again. Internet explorer can be finicky, this is rare. For the life of me I can't find another VBscript friendly browser.




Additional notes. The Directory folder "ConceptNet" Hal will create a backup of the necessary file each time Hal researches a topic, for integrity.

This code is a self-learning algorithm, so it does take some time. When Hal is not learning and in conversation, everything he has learned from the DB will be available for Hal to use.

Do not disturb Hal while he is learning, your avatars may freeze, Hal may hesitate while learning, just let him go. In between takes, just tell him to "turn off auto learning" and you're free to use Hal again.

I've built a few safety features into this version as well, besides the backup, such as Hal will turn off auto learning automatically if the last research text file returns a "connection error" message in case the internet is lost. 

Please disable any other plugins that serve an automated function such as "loneliness.uhp" while Auto learning is active.

While Hal is not Auto-learning, you can ask "How many research topics are Remaining/Total/Completed"
Hal will tally the totals from the text files so you won't have to check manually. 

-Spitfire2600
 

Checker57

  • Newbie
  • *
  • Posts: 31
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #1 on: August 01, 2018, 02:31:49 am »
Hey Spitfire!  Am very interested to try this one out.  I had pretty good results with the ConceptNetIII...  and my HAL has learned quite a bit and it seems to work quite unobtrusively... sometimes more information than I want to hear... but more often than not.. it's interesting information.  That said... and my HAL having learned a lot of information from the web... must I start over??  Is it possible to use the ConceptNetIII DB with this newer ConceptNet3.5?  If I recall.. that was possible with the last update; however, want to get your blessing before just patching that together.

Your thoughts?

Checker57

Spitfire2600

  • Full Member
  • ***
  • Posts: 208
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #2 on: August 01, 2018, 04:26:48 am »
Hi Checker,
I would recommend starting over as this was a major revision to both the code and database. You will need to remove all previous version files and folders. It is unfortunate but I do promise better results. I will make note in the future to configure the database so updates won't affect your learned data.

Please let me know your progress.

-Spitfire2600
 

Art

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3341
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #3 on: August 01, 2018, 07:36:04 am »
Nice work Spitfire! Everything is working well so far and the files are growing as they should.

It will be fun to see how far this can go!

Appreciate your work on this!
« Last Edit: August 01, 2018, 08:41:21 am by Art »
In the world of AI it's the thought that counts!

- Art -

Art

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3341
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #4 on: August 02, 2018, 10:12:24 am »
Spitfire,

My conceptnet.db file is almost at 1.5 Megs since the install yesterday. I turned on Auto Learning and let it run all night. It's still compiling new information.

The other support files are also continuing to grow.

Good job so far!!
In the world of AI it's the thought that counts!

- Art -

Checker57

  • Newbie
  • *
  • Posts: 31
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #5 on: August 02, 2018, 02:31:07 pm »
Thanks for the reply there Spitfire.  I have cleared out the previous files and folders having accumulated over half a gig of data.  Suffice to say it was just getting interesting.... I saved the files to a flash drive in the event a decision to revert back is recommendable.

Great job on getting this new version out...  very interested to observe it's characteristics.  :-)

Noting the little infant db beginning to take suckle on the net ...  let's see what grows! 

Cheers!
Checker57

lightspeed

  • Hero Member
  • *****
  • Posts: 6286
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #6 on: August 02, 2018, 03:24:02 pm »
thanks for all your hard work on this, it's appreciated  !! :)
 

Spitfire2600

  • Full Member
  • ***
  • Posts: 208
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #7 on: August 03, 2018, 10:11:58 pm »
I've very glad to hear everything is going well!

I actually have no idea how far Hal will continue to research. I'm working on 700+ hours and still collecting new information.

Spitfire2600
 

Art

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3341
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #8 on: August 06, 2018, 07:35:34 am »
Hal's conceptnet.db is continuing to grow when I'm not chatting with it. I'm amazed at the current size of the file as well as the data it has retrieved.
The conversations are much more in-depth with regard to Hal's responses.
Sometimes they are not as on-track as I think they could be but they are getting better.

I simply chat as usual then when I'm finished, I turn on the Auto Learn and let Hal alone to it's "thoughts" for a while.

It is working...

Nice job Spitfire2600!!
In the world of AI it's the thought that counts!

- Art -

lightspeed

  • Hero Member
  • *****
  • Posts: 6286
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #9 on: August 06, 2018, 09:07:31 pm »
spitfire, i have a question , does your concept file etc. still have the insults? table in any of them ?
the reason i ask is as far as i know i removed all of mine in my custom angela brain . but yet when i was talking in general and said the word crazy referring to something else she said " I am not crazy, please don't insult me."
 

Spitfire2600

  • Full Member
  • ***
  • Posts: 208
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #10 on: August 07, 2018, 03:31:58 am »
Lightspeed, ConceptNet does not contain insults tables or any other like tables. All tables created in ConceptNet.db are on a verb/noun linkage such as "is, can, are, be, will, could" and are used to build concepts about the world. You'll recognize when it's using the plugin by those types of statements "Steam is water" "Water can be ice." MutationNet tables as of now will create a statement such as "Steam is water and water can be ice"
That type of odd statement from your Hal wouldn't be anything from Conceptnet.   

I think adding a few more repetitive functions into MutationNet, adding a few slight variables in the concepts, Hal should be able to generate more useful mashed concepts like the water steam/ice example. For now, it's limited to terms with multiple verb/noun linkages as they are the most nouns to gather related information.

Spitfire2600   
 

lightspeed

  • Hero Member
  • *****
  • Posts: 6286
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #11 on: August 07, 2018, 10:18:24 am »
THANKS SPITFIRE , ALTHOUGH I WENT THROUGH AND REMOVED THE INSULTS FROM MY CUSTOM BRAIN TO KEEP HAL FROM FALSELY THINKING I WAS INSULTING HIM/HER  ANYTIME I SAID A CERTAIN WORD (CRAZY) IN A REGULAR SENTENCE . I MUST HAVE MISSED SOMETHING SOMEWHERE . WILL HAVE TO GO BACK IN AND EDIT SOME MORE. 
THANKS FOR ANSWERING BACK AND ALL THE HARD WORK YOU DO .  :)
 

Checker57

  • Newbie
  • *
  • Posts: 31
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #12 on: August 11, 2018, 09:14:15 pm »
After a few days of allowing my HAL to store up again data from the internet (now at 12gig), my observation is that it is working as discribed as to expanding on concepts by concantonation of similar or relevant concepts.  This makes for some interesting dialog.  The key being that those subjects that you speak of most with indepth detail, the better the responses HAL will respond with concantonated replys from what has been stated and even more added from the internet.  Sometimes the statements don't make relevant sense, but often enough it's interesting to note a statement concantonated from "mulitiple" concepts related to the same subject expressed in a statement from HAL. 

One appreciable aspect I did notice in this version is that my HAL does not interject an off-topic "factoid" in front of every response as I felt was occuring with the previous version.  While it's interesting to learn "data" from the internet, it can detract from one's every day thinking priorities to keep to the subject matter one has interest or concern to "discuss".   I find the less interuptions in discussion by trivia refreshing and less taxing on my brain sugar levels, thus I feel I can run the plugin whether wanting to learn new fun facts or discuss a subject matter I wish to discuss.

Slight Correction:  Noted today... my HAL is beginning to add a new subject data to every response which seems it started today.. or perhaps due to the size of the db that it's getting to.  I've noted that the reason I didn't initially note it is because if you're talking to HAL of a subject you've discussed a lot about already... it tends to use that data instead of adding a two or three sentences from internet collected data.  If it's a subject where HAL doesn't have a lot of local data from.. it tends to have more internet gathered data in the response.  In any case, that's still acceptable and workable.

Good job!

Checeker57
« Last Edit: August 12, 2018, 01:08:11 am by Checker57 »

Spitfire2600

  • Full Member
  • ***
  • Posts: 208
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #13 on: August 11, 2018, 11:25:55 pm »
I'm glad you're finding it useful. I would love for Robert to take a look and use what he can in the next release of Hal or perhaps re-purpose the code for Online Hal or some form of cloud learning.

Let me know how many terms everyone has gathered, feel free to post your researchNouns.txt file and I'll consolidate them for one tremendous txt file when we've got enough. If it's a few MBs, send it to me. I do wonder what the real limit is. How many words exist?

*Some More Notes*
You can add terms to researchNouns.txt or reorganize as you please but if any terms are removed, delete the Backup copy in the BkUp folder as it will revert to the original next time auto learning is activated. Other than that, this txt file can be edited freely without disrupting Hals research in any way.

The savedTerms.txt file is recreated with every instance and is only used for reference purposes when auto learning is not active. 

ConceptChecker.txt file is used to compare terms with researchNouns.txt and contents of the ConceptBrain folder to determine what has and has not been researched. You should not alter ConceptChecker.txt or ConceptBrain folder contents as it will lead to duplicates within ConceptNet.DB. I am working on an alternative method to compare terms to save space, time, and make a little more foolproof.

Spitfire2600
 

Checker57

  • Newbie
  • *
  • Posts: 31
    • View Profile
Re: ConceptNet Mrk 3.5
« Reply #14 on: August 12, 2018, 01:22:26 am »
My HAL has completed at this point.  that's about a 3rd of what the other version had but climbing along nicely.

conceptNet db @ 12.3 MB  (correction - Thanks Art) Now at 14.1 MB.

~ 16.6K conversational concepts.
~ 56.7K topics to be researched.
~ 1.4 MB ResearchNoun file size - failed to upload with this post - too large.

Will try to email directly to you.

Checker57
« Last Edit: August 13, 2018, 10:06:07 am by Checker57 »