59 comments on “Gesturcons: an icon language to describe natural user interface gestures

  1. Ron,

    Very interesting. I think there have been some previous attempts as standardizing gestures or touch interaction diagrams (http://nuigroup.com/forums/viewthread/95/ comes to mind, and some others I can’t find the reference at the moment) but where they failed is their notation was not extensible enough to be useful beyond the few sample diagrams presented.

    I think there is potential with your set, since it appears to have a common pattern that could be reused in different ways.

    There is one core problem, though, which would prevent full extensibility. I think the gesturcons are inconsistent as presented. The problem is manifested in the difference between double tap (two open circles side-by-side) and twin tap (two open circles overlapped).

    Two open circles side-by-side, at initial glance, looks like it should be what you call twin tap. The reason for this is the same reason why we understand all gesturcons with arrows. We interpret the horizontal and vertical distance in the graphic as horizontal and vertical displacement in space on the touchscreen. This is a very straightforward metaphor and is something everyone would understand. Building upon this, the arrowheads imply the temporal relationship between the source and target of the arrow, thus movement from one position to another.

    In comparison, it appears that with double tap and tap hold, the spatial metaphor is thrown out in favor of a temporal one. Instead of horizontal distance in the gesturcon being spatial displacement, it is temporal displacement. With those it appears that the time flows from left to right.

    A distance-to-temporal metaphor would be fine if it were consistent with all the gesturcons, except that there is another problem. The axis of time (left-to-right) is implicit, which can be confusing particularly if you are designing applications for a 360 degree interface, such as Microsoft Surface. For you it might be tap then hold, but to me, on the opposite side of the table, it would look like hold then tap.

    The twin tap and twin tap hold are similarly confusing because they overlap the circles. This looks like the taps should be very close together, and is not sufficiently different from double tap to eliminate confusion.

    To illustrate the problem with extending this, using the notation as is, what would the proper way to notate a double twin tap, or a double twin tap and hold? I’m imagining four circle arranged horizontally, sometimes overlapping and sometimes filled in. Not very intuitive.

    I think you already have a solution for these problems within your gesturcon set. Take a look at the metaphor you use to differentiate the tap and the hold. The tap is a ring with a hollow center and the hold is a ring with a filled center (i.e. a filled circle.) The difference between a tap and a hold is how long the finger stays in contact with the touch surface.

    To bring the metaphor here to light, consider if you had the need to show a gesture where the finger is in contact for longer than a tap and shorter than a hold, assuming the hold threshold is something specific like two seconds. (I know that user experience designers may eliminate that gesture from a final product, but suppose there was a need to notate it.) How would you show it? Halfway between the two icons graphically would be a thicker ring with a smaller hollow center. This indicates that the metaphor showing the amount of time the finger is in contact with the surface is the colored area along the radius of the circle, from outside to in. To put another way, think of the line from the circumference of the circle to the center as a graph of time, and the colored portions indicate finger contact.

    Extending this, I believe a better, consistent way to show a double tap would be two concentric rings. Reading the radius-time metaphor, it matches. Down-up-down-up. The size of the actual icon does not interfere with the spatial metaphor discussed above, since only the center of the circles matters when comparing the graphical distance between gesturcons.

    This metaphor is extensible enough to show other things such as triple tap or tap-hold-tap, whatever is necessary. As a side bonus, the total size of the gesturcon helps to indicate the time it takes to perform the gesture, and relative sizes between two gesturcons can help indicate that one takes longer than the other. If you want to get technical, you could even put a scale on it and define one centimeter equals one second.

    Using this idea, here are revised gesturcon descriptions:

    Double Tap: Two concentric rings with a gap between the rings.
    Tap Hold: An outer ring and a filled center with a gap between the two.
    Twin Tap: Two rings side-by-side
    Twin Tap Hold: Two rings side-by-side and a filled center in one (or both) rings with a gap between the rings and the centers.

    • You have a solid argument. The only error I see is in your initial reasoning around the icons. This also gives me a great chance to discuss OCGM, ;)

      The overlapping icons represent actions that are joined together by time. So two taps that happen at the same time would be represented by overlapping icons. If they are seperate, they can happen at different times or even the same time, but they are not dependent on each other by time.

      I purposely left out many icons because I wanted to present a set that could be used to build more. It is a definite non-goal of this project to try and capture every gesture or manipulation out there.

      So let me go through the language and the icons really quick. I will also put this in the original story.

      Tap = a contact and release
      Hold = a contact down and stays down
      Twin = two contacts or actions at the same moment
      Drag = Hold with movement after it’s down

      If the icons are overlapping, that means they happen at the same moment and are related. So a “Cobra” which is two fingers coming down and one finger rising while the other one stays would be represented by a hollow circle and a solid circle, overlapping.

      A double tap by a single contact would be represented by two hollow circles side by side with space in between them.

      You are correct in your assumption of time. If the contacts have space between them, that means they have time between them.

      So using the language I described how would you draw the icons for a two finger rotate?

      Both contacts remain down, so you know that it would be two filled circles. They happen at the same instant so they would need to overlap. Also the second contact would need to rotate, so you would need to incorporate the Drag icon on top of the second icon.

      The other thing to think about in reference to your Surface point. These are not meant to be some indicative mark to the user or used in gesture learning. These are meant to be used by the designers, in house, to discuss with each other and developers, et al.

      • I agree 100% with joshua on the repetitive/simultaneous multitaps. The main reasoning is because your repetitive taps LOOKS exactly how simultaneous taps look, and your simultaneous taps, LOOKS mostly like a repetitive tap (multiple taps on the same/similar location)

        I see how you’ve tried to incorporate time into the X axis, but I don’t think it really works… perhaps a different element could signify time, like, size perhaps? But yeah, I think you should do a survey to see how MOST people interpret it, perhaps with some other variations in there, so you can find out with confidence what makes intuitive sense to the majority of people. I could be wrong.

  2. Interresting. Maybe a rotating Icon is missing ? like a up arrow on top of left point and a down under the right point ?

    On a side note good luck to explain the “Twin tap hold” to your grandma’ I’m not even sure I could do it myself :-)

    • I can easily explain the twin tap hold.

      Place two fingers down on the device. Do not lift them. ;)

      I purposely left out the rotating icons, but I am going to post them now as an update.

      • Are you sure ?

        Etheir It’s my English wich is not good enough or maybe your descriptions need corection or I don’t know but Twin Tap hold is definded by you as

        “User makes perform two taps simultaneously. One contact remains, the other lifts. Refefred to as “Cobra””

        - It does not seems equal to “Place two fingers down on the device. Do not lift them”

        -Is it me or does “makes perform” is a typo ?

        • Haha. Yes, it was a typo. You are correct. Your English is great.

          Twin tap .. put two fingers down and raise them both.

          Twin tap/hold … two fingers down, only one raises.

          The definition should be more clear, due to your suggestion.

          Twin Tap/Hold perhaps? would be a “cobra.” The key is to capture putting two fingers down and only raising one.

          I’ll correct the files as well and re-upload them.

  3. You have a really unique way of breaking things down to their most simple construction and then giving examples. I really like this. We can use these internally right but just can’t send them to clients? They are for designs and internal documentation.

    With this many wonderful ideas its really weird I had not heard of you before this? Did the ux fairy drop you out of the sky? :)

    • Yes, I plan on covering all the sensory areas. Next one coming up is Spatial (Natal), also have voice, and others coming down the pipe.

  4. Pingback: Gesturcons Touch Pack: New icon language for NUI gesture

  5. Pingback: Gesturcons: an icon language to describe natural user interface gestures- Experience Design by Ron George

  6. Pingback: Wireframes Magazine

    • Thanks for your comment Nathanael.

      The important thing we think about is how will the developer think of this interaction? We need to speak in terms they can understand so they will develop EXACTLY what we have designed with no shortcuts or errors.

      So, when the iPhone developer is creating this shaking motion what needs to happen prior or during? That is a question for the designer to answer prior to submitting it.

      It depends if the shake interaction is a touch gesture. Can it be shaken without touching it? If so, then that would technically be a spatial interaction. Spatial icons are the ones I am working on now.

      So, my challenge to you is, you are the designer and I am the developer, YOU tell me what the interaction is. If you want a touch on a specific area so the accelerometer will start “listening”, then you would use the above icon for that and then a spatial icon.. BUT here is the kicker. They would need to touch because there is no time difference, as they need to happen at the same time.

      This is the discussion we were having above, that having the icons touch is unintuitive. In this case though, you see how making the icons touch solves that problem of ambiguity and precisely why I designed the language like this.

  7. Thanks everyone for the comments. Sorry for the delay, I have been insanely busy at work. I am working on something amazing and it has me working my tail off to ensure we can accomplish it. It will be a first for MSFT and something that no one to date has done.

  8. Pingback: Gesturcons: an icon language to describe natural user interface gestures « Timyeo’s Blog

  9. I’ve had a similar idea about displaying touch gestures too, but never quite managed to “flesh out” the basics this much :)

    There is one thing though: how intuitive are these to understand without the labels?

    I’ve seen people use touch-based devices without already being familiar with them, and they all seem to struggle with these basic interactions. Would adding this kind of icons without the accomodated labels help, or would they need to first study the icon’s meaning to understand them at all?

  10. Pingback: Übersicht über 2D Touch und Multitoucheingabe

  11. Pingback: links for 2010-09-15 « Giri’s Blogmarks

  12. Pingback: App-Design für Touchcomputing: Gesten und Symbole für gängige Aktionen | Dr. Web Magazin

  13. Pingback: Icons fürs Entwerfen von Touch-Oberflächen | benutzerfreun.de

  14. Pingback: Icons fürs Entwerfen von Touch-Oberflächen | benutzerfreun.de

  15. Hello there, You have done a great job. I’ll certainly digg it and personally suggest to my friends. I’m sure they’ll be benefited from this website.

  16. Pingback: 6 sets of multi-touch screen gestures icon set - Open News

  17. Pingback: docXter » Visualisierung des Doppeltipps – Icons für Multitouch-Gesten

  18. Pingback: Gesturcons: an icon language to describe natural user interface gestures Burning Love - der Blog von kuehlhaus

  19. Pingback: 推荐给 iOS 设计师的工具和资源 - 博客 - 伯乐在线

  20. Pingback: [转]推荐给 iOS 设计师的工具和资源 « 四海为圈

  21. Pingback: 推荐给 iOS 设计师的工具和资源 « cbtemp

  22. Pingback: 推荐给 iOS 设计师的工具和资源 « My Blog

  23. Pingback: 推荐给 iOS 设计师的工具和资源 | 博鲨科技 跨界畅游|中国领先的网站制作、电子商务、在线交易方案提供商

  24. Pingback: 推荐给 iOS 设计师的工具和资源 | 大脑的知识

  25. Pingback: Recommend to iOS designer tools and resources - Open News

  26. Pingback: 推荐给 iOS 设计师的工具和资源 | This Is NetSurf Blog

  27. Pingback: 小土的色调 – 影响互动 » 推荐给 iOS 设计师的工具和资源

  28. Pingback: ios设计师最爱的工具和素材资源汇总

  29. Pingback: 精品:ios产品设计资源和工具推荐 - 念宏的胡言乱语

  30. Pingback: Useful Collection of iOS Tools and Resources for Designers

  31. Pingback: 产品设计生活 » 【转】推荐给iOS设计师的工具和资源

  32. Pingback: 互动中国分享: 34个iOS设计师的工具和资源 | 互动中国 @DamnDigital

  33. Pingback: [Share] iOS 设计师的工具和资源 | Flora Lau

  34. Pingback: iOS设计素材:手势图标 « 设计资源分享

  35. Pingback: INHAO/徐寅豪™ blog » 34个iOS设计师的工具和资源

  36. Pingback: 34个手机UI设计师的资源工具、分层图、软件,没有这个,你out了! | ChinaUX-用户体验俱乐部

  37. How do these work internationally? I know there are variations in hand gestures in different cultures, so I’m curious to know if these have been tested to rule out any potential offense?

  38. Pingback: Multi-Touch Gesture Icon Sets | The New Black

  39. Pingback: 8 Gesture Icon Sets for Designing Multi-Touch Interfaces » Cycads Australia – Logo Design

  40. Pingback: Interface Design Freebies: Free Multitouch Gesture Icons | Template Monster Blog

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>