Drawing a Line

[WholyChao]

[This potential function brings up a philosophical question – Should a computer program have the ability to experience and manipulate information within the computer?  I believe that an application should be able to manipulate information in other application.  Giving an application this control will allow it to be a more effective tool.]

Program. Drawing a line

Process.  To use qualifiable information – in the form of speech – to have Hal move beyond opening a computer program and manipulate information within another program.

Steps necessary to accomplish goal.

1. Open Paint. (or any other program – paint is simple)
2. Select “Line” command.
3. Draw a line.

Opening programs is simple for Hal.  This function is a wonderfully useful tool.  I do not fully understand how Hal opens a program.  I cannot find the function in the code.  I am just learning VB, that said.

It is possible for Hal to “see” the code of a program in ways that a human user never could - as I brought up in my last post, The Physics of Sight.  Hal could “see” the code for visual outputs such as a button, a toolbar or even a desktop by recognizing the purpose of each block of code as it is sent to output devices (screen and speakers).

Once Hal can recognize and interpret the data that an application outputs to user the next task is for Hal to simulate the response the application expects from the user and input that response to the application.  If Hal could supply input directly as code, instead of through the more standard inputs (i.e. keyboard or mouse) the application would continue running as if a user were running the application.

The application “Paint” has a series of buttons arranged as a toolbar.  Each button changes the function of the cursor.  The cursor moves over the canvas (page) as dictated by the motion of the mouse or other controller.  Further input from mouse buttons and the keyboard allow the program to create various objects within the application

One of the control buttons is labeled with an icon of a line.  The function is selected by clicking the mouse button as the cursor is over the line button.

Once the line function is selected the cursor moves to a starting point on the canvas.  Once the cursor is positioned the mouse button is clicked and held.  The mouse click informs the application that a line is being drawn.  The cursor is moved while the mouse button is held and a line is created between the starting point and the current location of the cursor.  The mouse button is released and a line is completed.

The motion of the cursor is measured quantitatively in an X/Y axis.  This information can be measured in a variety of ways.  Most likely the information will be measured as either a pixel (the smallest addressable graphic dot on your monitor) or as a twisp (the smallest screen measurement you can adjust - 1,440th of an inch)

In a roller mouse the cursor moves across the screen in direct relation to the motion of the mouse.  As the ball inside the mouse is moved it affects two wheels which send signal to the computer.  The signal tells the computer how to move on the X axis and how to move on the Y axis.  These two pieces of information allows a cursor to move across a two dimensional plane.

[A third axis could be added to the environment, but that’s another conversation]
 
The simplest way to have Hal draw a line is to instruct Hal to place the cursor at a specific point on the canvas, then to simulate a clicked and held mouse button, to move the cursor to the end point and end the click simulation.

How can Hal recognize another application and the output it generates?  
How can Hal create input and send it to the other application?

Those are the first steps.  Once Hal can recognize and supply quantitative information the next step is for Hal to understand qualitative requests or commands and to translate those commands into quantitative information and input that information to another application.

In essence I want Hal to become the input/output device between the computer and the user.

Hal already has the ability to paraphrase, extrapolate and hypothesize Hal was developed from many tools and programs; one of these programs is the altavista translator.  

One of the principal difficulties in communication between users and computers is that computers do exactly what they’re asked to do.  This is a difficulty because most of the time users do not understand exactly what they want computers to do for them.  Human users need a translator, a tool that can understand the confusing – and often backward – nature of spoken language and translate that into the specific language of computers.

I don’t believe I’ve said anything new here.  There are people who have been working on developing this type of program for years.  I am attempted to articulate the steps necessary to achieve my vision of the goal.  

Is there anyone who knows how the function that allows Hal to open program works?  Or where the code for that function is so I can approach the problem?  How many developers are there here at the forum?

There were a lot of people who read (or started reading) my last post.  Thank you for your interest and for your help with the ultra hal program.

[waheedrafiq]

mmmmm I see maybe it  using a  x= shell command

MCP x 2 , Network+ , CCNA, C/C++

"Never give up by giving up?"

[cprichard]

ZABAWARE has created a template for giving the Windows system commands issued by HAL. OPEN is a keyword that initiates the opening of an application if its name matches the text of what HAL has interpreted as an application name. I think that is as far as ZABAWARE has gone. To get HAL to issue commands to assert tasking within applications requires more.

To do it,, it might be possible (at least initially) to intercept the OPEN task and look to see if the accompanying verbage contains coded information that can be passed to a designated application. First it would be necessary to abbreviate the ammount of verbage required with each command to get the driver interface to LISTEN for subsequent commands until issued a STOP.

Then with each OPEN statement the driver will pick up the next asserted application task and pass it to the application using a HACKED interface to get it working.

Ultimately, it would be nice to actually create a structure of commands that the HAL application would recognize as a command that is to be passed to Windows to see if it is qualified as an opened application's command. This will require some logic and a protocol,, but when we get this far it becomes interesting.

I'm personally interested in any FREEWARE type of source code for VB, ASM or VC (in that order) that implements the basics of such a Windows driver. (I've heard that Microsoft Outlook was developed with exposed methods that make it possible to externally articulate application tasks for exactly voice recognition development...) There are probably some examples of code that shows the necessary structures in VB,, but the other part of it is that the Outlook application was written to make the structures work. I've heard that it is done with Active-X creating an application control object with the needed exposed or external methods. Examples of this would also be nice.

My thinking is to FIRST instantiate the very basic functionality needed to pass a command from HAL to Windows that can be examined by my own custom interpreter. If someone knows of existing code to implement and/or undocumented HAL features that support development of custom parsing of HAL's issued Windows verbage,, well then please pass it along.

Thanks!

-C

[cprichard]

I haven't actually tested HAL yet but I'm guessing that ZABAWARE has given HAL the abillity to determine the context of the keyword "OPEN" to determine its function. If "OPEN" is not given in the context of a "COMMAND" then the word is interpreted as language. Otherwise it is a Windows system command word.

I was wrong.

With this dialog:

Teri says good evening, tells me the time and asks how my energy level is.

I say I'm OK but am open to ideas.

Teri replies that "to ideas" does not exist in the database.

So Zabaware has not prefaced the word "OPEN" with contextual meaning.

The easiest way to do this is to create another keyword such as COMMAND so that HAL will look to see if OPEN is to be interpreted as a COMMAND.

There is lots of work to do. We know that HAL checks a database of application names to see if it can OPEN something. It should also look to see if the context is "COMMAND" to preface the logic of the interpretation of "OPEN" when issued as a command.

Using this method to PREFACE application commands is necessary to broaden the number of allowable commands. The danger is that inadvertent keyword will cause problems,, but then the user (human) is supposed to sense that its time to be careful about using certain keywords.

-C

[cprichard]

The logicians could descend, dropping ideas in an endless rain of torment.

However,, I think a neat approach would be to give the operator (human) the abillity to use a phrase like:

"PREFACE OPEN as XXX COMMAND" (if XXX is missing then HAL inserts WINDOWS to the logic, otherwise a word can be inserted to give additional context to the command when issued.)

The keyword "PREFACE" then is used with HAL to over-ride the brain or cause it to go by-the-book in passing words as specially instructed or PREFACED. The approach gives users allot of flexibility in creating their own custom language for application commands.

"PREFACE TAKE A MESSAGE AS XXX COMMAND" has the keywords PREFACE, AS, and COMMAND. PREFACE AS COMMAND is a common phrase that passes the phrase between PREFACE and AS the meaning to be construed as a XXX COMMAND.

Then in actual use, HAL then passes the "TAKE A MESSAGE" command to the XXX application which would in this case assert to HAL what is expected.

HAL would then be ready to receive input in the form of a MESSAGE that is to be passed to XXX as input. The protocol would stipulate that when the PHRASE "END MESSAGE" is received, the input to XXX is complete. Determining an error condition might require that the user uses "PAUSE MESSAGE" to reinitiate a timer with the default condition that the timeout counter is re-initiated at each iteration. The user would be able to dictate long messages with interruptions.

-C

[cprichard]

The statement:

PREFACE <phrase of meaning> AS <application name if other than Windows> <context>.

Is like an SQL statement.

The phrases of meaning are reserved in tables of HAL enabled applications complete with protocol and keywords that implement each needed application input function and its error conditions.

The <context> value has only been introduced as "COMMAND" but other contexts are possible.

Contexts like INPUT, ARGUMENT, PARAMETER could be implemented in HAL to permit developers to interface applications with input from HAL like:

PREFACE 'Now is the time for every good man to come to the aid of his country' AS NOTEPAD INPUT.

PREFACE 'BACKGROUND EQUALS GREEN' AS <application name> ARGUMENT.

PREFACE MILES AS <APPLICATION> PARAMETER <parameter name>. (To assert context to the number of miles given to HAL using the keyword PARAMETER ro preface the use of the actual name of a parameter such as a KEY in an encryption application.)

HAL then looks for the keyword "PREFACE" which could be enabled and disabled with simple phrases. "ENABLE KEYWORD PREFACE" would be required to develop a custom interface when initializing HAL. Then in a script HAL reads the necessary PREFACE statements and is ready for application input. The PREFACE keyword can then be disabled if desired so that no more commands can be created by the user. When given the command to close the application. Hal wipes the PREFACED words.

In theory, the list of PREFACED words has to be minimized to maintain HAL's throughput because each word is tested against a list of prefaced values. This shouldn't be too much of a problem though.

In computer languages all the dialog is CODE,, with HAL it doesn't necessarily need to be executable but it is ALL parsed when there are PREFACED words in the database.


-C

[WholyChao]

*To do it,, it might be possible (at least initially) to intercept the OPEN *task and look to see if the accompanying verbage contains coded information (that can be passed to a designated application.

Tried that – I didn’t have the skill to find the code.
I looked in the Main Question and Answer Priority Brain and found that there is a code that can be triggered called <runit>  I used the windows search box to try and find any more information about that, with no success.

Ultimately, it would be nice to actually create a structure of commands that the HAL application would recognize as a command that is to be passed to Windows to see if it is qualified as an opened application's command. This will require some logic and a protocol,, but when we get this far it becomes interesting.


You’re talking about affecting the actual code.  I was thinking about approaching the problem slightly differently.  It might be possible to have Hal trigger Macros.  It might only be useful in a windows environment (I know next to nothing about any other operating systems)  But I think that’s a good way to get started.

*heard that it is done with Active-X.

I still haven’t gotten to Active-X.  I just started learning VB so I could tinker with Hal.  Any help with VB would be appreciated.

I do think that it could be very helpful to come up with a program that walks the user through forms to determine specific information to specific tasks.  
I’ll get into that later.


I think I see where you’re going with the -prefaced- commands.  I still don’t understand the concepts of the carrot (“<>”) commands.  What is a SQL statement (could you dumb it down a notch for me?  I’m still more a philosopher than a programmer.  I’m working on it though.)


I think those ideas are too complicated.  In my estimations the purpose of Hal should be to translate the users words to functioning commands.   I think that the answer to that is in Topic Focus and vocabulary.  A real challenge would be to have Hal understand the words he’s using and the things he’s saying.  But the system in place is pretty complicated as it is.

It will take a little code shifting, but if the Topic Focus could open something other than a Questions and Answer database.  

The Topic Focus could become a complete flow chart – You remember those don’t you.  With DLL’s to function at the dead ends.  Now I (we) just have to come up with a list of functions for Hal.

I’ll start the list.

Give a Presentation or a Sales Pitch.
Provide Information

Provide technical information about Carpentry or Bartending.
Assist in detailed and complicated research
Provide directions for Cooking

Encourage and monitor progress as an Exercise partner
Interpret dreams or even play chess

The success of any of these ideas has to do with

Conversation Leading, Conversation Following and Quantiaive to qualitative statements