I know this is a bit late, because my co-author has already reported it, but I am very happy to announce that some ears have been listening. The CHI ’10 Workshop, “Natural User Interfaces: The prospect and challenge of Touch and Gestural Computing” has granted an audience to the new metaphor for design.
The workshop is going to be an all day event on Saturday, where each of the authors will present and offer discussion about their position papers. The real benefit of this is that in such a narrow expertise, you really get amazing peer reviews and leave with amazing new ideas. These types of scoped interaction gatherings are a wonderful thing to foster innovation and creativity. It’s like a specialty conference inside of a specialty conference.
One of the most interesting things, as I read the position papers that were accepted, was a cite from my writings. In the paper, “Natural User Interfaces: Why We Need
Better Model-Worlds, Not Better Gestures.(PDF)” the authors argue the need for separation between “symbolic gestures” and “manipulations.”
Manipulations are not gestures.
We believe in a fundamental dichotomy of multi-touch
gestures on interactive surfaces. This dichotomy
differentiates between two classes of multi-touch
interactions: symbolic gestures and manipulations.
They go on to define each specifically.
For us, symbolic gestures are close to the keyboard
shortcuts of WIMP systems. They are not continuous
but are executed by the user at a certain point of time
to trigger an automated system procedure. There is no
user control or feedback after triggering.…
The opposite class of multi-touch interactions is
manipulations. Unlike symbolic gestures, manipulations
are continuous between manipulation initiation (e.g.
user fingers down) and completion (e.g. user fingers
up). During this time span, user actions lead to smooth
continuous changes of the system state with immediate
output.
It is a lovely way to differentiate the two and I couldn’t agree more. As the crevasse between the two interactions widens, we begin to see many more differences. I have gotten a few emails from bewildered Interaction Designers, both young and old, asking “why do we need this separation?”
The answer is not quite as apparent as it will be in the next few years. We need this distinction because designers and developers need to think about experiences differently. We need to plan and design for interactions in a fluid and responsive way. Arbitrarily throwing a manipulation on a destructive action, could have dire consequences for the user. Using a complex gesture for a simple help menu, would create a pause in your user’s experience.
Let me give you an example of the two with a nice crisp physical metaphor demonstrating OCGM
You have a pile of sticks.
Just a pile of sticks
Think of the sticks as Objects. Think of the table as a Container. Now let’s examine the relationship between the two. The container contains many objects, therefore operations executed on the container will have multiple effects on each of the singular instances of the objects.
Now, let’s give you a few things to do those operations. First, let’s give you a hand, which we will call a Manipulation. Then let’s give you a chainsaw, which we will call a Gesture.
To move the sticks, to rearrange them, to take them in and out of the container (on or off the table) we can use a hand. It might not be the most efficient, but it gets the job done and we can do these methodically. All the while, we can always undo our actions easily.
If someone comes up and talks to us, we don’t hesitate to respond because there is nothing really at jeopardy here. In fact, it might be nice for someone to come up and interact with us while we are doing these chores.
This also puts us at ease. We can do these things and if we get interrupted, we have no stress. Why? Because everything is undo-able. Things are easily moved back and forth with no real consequences. Even if we push the entire stack to the ground, we can easily pick them up and put them back. One handed.
This is the physical realization of a manipulation.
a Chainsaw "gesture"
Now, we take your chainsaw. This is a complex piece of machinery. This is going to take focus, intent, and possibly a plan to implement. The things we can do with this are destructive, irreversible, and should be taken with care because they could damage other things around the single container or multitude of objects.
We consult the manual, don safety gear, check the gasoline in the chainsaw, pull the handle, and then fire it up. Each one of these actions were simple and were not harmful, until they were combined. Combining all of these harmless manipulations results in a gesture that could do harm.
Let’s think about what we just did though. We had to do things in a certain order, we had to do each of them, and they were predetermined by something other than us.
Now could we take our gesture chainsaw and destroy the Container and all of the objects inside? Yes. Could we also affect other containers and things in the vicinity? Yes. There is no undo, when its done its done.
This is the physical realization of a Gesture.
So I digress….
Why is it important in interface design to distinguish between the two? To promote a better experience through expected interactions and results. Do you want your users concentrating on the most mundane of tasks? Let your user relax when they can. Do not make them concentrate on these details of where to move an object when they don’t have to.
Hi Ron,
Glad you’re blogging again!
Good example, although I don’t think the chainsaw is a perfect example of a gesture. I can see that adding the separate manipulations of the gas and the starter and so on lead to a larger destructive result, but I think the important part about gestures is that they communicate a meaning.
From our paper:
“Gestures are metaphors for discrete, indirect, intelligent interaction, and manipulations are metaphors for continuous, direct, environmental interaction.” I elaborate in my book that manipulations are based upon human-environmental interaction while gestures are based upon human-human communication, both verbal and non-verbal. This is based upon definitions from your original blog post.
If you want to accomplish something, you can do it yourself manually through a series of manipulations or you could communicate your desire to another person or entity who will do it for you. The gesture is a shortcut and saves you from doing manipulations.
For example, to delete a file, you could manipulate it into the trashcan, which is easily reversible, or you could perform a gesture on the file in place to cause the file to go to the trashcan itself.
With the sticks on a table example, I think an easier to understand application of gestures and manipulations would be this:
Manipulation: move each stick one by one from the table to a fire
Gesture: use a hand motion to communicate to someone or something else (a friend, a robot, or some motion-sensing mechanism built into the table) that you want it to take the sticks and put them in a fire.
In both cases the sticks end up in a fire and destroyed. With the manipulations you use human-environmental interaction to perform the actions manually yourself, and you can stop and reverse the action easily (at least until the stick is consumed by the fire.) With the gestures you use human-human communication to send a message to another intelligent being that interprets your message and takes action.
I think you are confusing the intuitiveness, meaning, communication, and the understanding of the recipient, with the actual gesture, which is purely a learned intent, and action.
Mainly just approaching it from a different point of view. Where your guys stands in the back and communicates ‘take it to the fire’, my guy actually performs multiple manipulations to get the same result.
The important difference between our examples are the use of multiple manipulations to achieve results. This demonstrates the basis of how harmless interactions can invoke, even by accident, a harmful result.
The other interesting thing is that in my example, it can always be “cancelled.” You can simply stop doing the gesture and it will stop. Such as the example of drawing your name on a surface to put your electronic signature. Its a complex series of manipulations, but if you stop before its complete, it will cancel.
On the same token, I’m not saying you are wrong. I must say that I am saddened that you would actually use a “wastebasket” in your example. I need to go hug a kitten.
PS: Using your example, what would you consider CTRL+C and CTRL+V? Manipulation or Gesture?
Well we are using examples from different points of view, but I just wanted to make sure there was a clear example of a simple gesture as well as the more involved example you gave.
I think the important part for people to understand, referring back to your first post on the definitions http://blog.rongeorge.com/design/interaction-design/terminology-the-difference-between-a-gesture-and-a-manipulation/ is for something to be gesture it must communicate or attempt to express some meaning.
Part of this is a “if a tree falls in a forest” question.
I think we would agree that if you ask me where the restroom is and I point towards it, that the action of pointing is a gesture. Now if I point towards the restroom but you happened to look away and not see it, is it still a gesture?
I’d say yes, since my intent was to communicate a meaning. Even if I were alone and I pointed at something while talking to myself or thinking, it would still be a gesture because there is some meaning associated with it. (Counting on my fingers to myself would be a gesture for the same reason.)
On the other hand, if I perform the same action without any intent to associate meaning, say while stretching, that would not be a gesture.
When you say your gesture example can be canceled, that’s not completely accurate terminology. The gesture isn’t interpreted until it is complete, so really it’s just not completing the gesture. Once you complete the gesture (saw the table in half?) you cannot cancel it. Same in my example. If I only perform half the motion that indicates “put the sticks in a fire”, the gesture isn’t completed and nothing happens. Your example just has a really long gesture made up of a sequence of manipulations (which are body motions), where my example gesture is just a single body motion.
Eh, there’s nothing wrong with the trashcan metaphor. Sure it might be a little stale, but it is based upon a real-world behavior and skill that we know and it is easily recognizable.
CTRL-C and CTRL-V are gestures, of course. If you want the computer to copy or paste the selected text, you can move the mouse to a menu or toolbar and click a button (manipulation), or use CTRL-C or CTRL-V to communicate your intent (gesture).
Those are my thoughts anyway!
Actually, I reread my reply and it didn’t make sense. Here is what I should have said.
Yes, your example is also correct but I used my example specifically to show the parallels between objects inside containers and manipulations inside gestures.
Now.. GOOD DAY SIR! haha.
There we go. 🙂
Pingback: City of Arabia | Raynesque
Pingback: City of Arabia | Raynesque
Actually, I reread my reply and it didn’t make sense. Here is what I should have said. Yes, your example is also correct but I used my example specifically to show the parallels between objects inside containers and manipulations inside gestures. Now.. GOOD DAY SIR! haha.
Hey there. When i simply noticed that your website seems like it includes a couple of code mistakes in the quite bottom of your site’s web page. I am not particular if everybody is getting this specific same bugginess when browsing your blog? I’m employing a entirely distinct browser than most individuals, known as Opera, so that’s what may be causing it? I just desired for making certain you realize. Many thanks for posting some exceptional postings and I’ll attempt to return back again getting a totally various browser to examine points out! Also, I set a link for your weblog at my web page,wish you dont mind?