The Design and Evaluation of a Two-handed Drawing Program

 

 

 

 

 

 

 

by

 

 

 

 

 

 

Jade Forrest Rubick

 

 

 

 

 

 

 

 

A thesis submitted in conformity with the requirements

For the degree of Masters of Science

Graduate Department of Computer Science

University of Toronto

 

 

 

 

 

 

© Copyright by Jade Rubick (2000)

Abstract

 

This thesis describes the design of a two-handed drawing program, T4, which explores the extent to which a range of two-handed techniques can provide an alternative to the traditional graphical user interface. T4 builds upon the foundation laid by the T3 system (Kurtenbach, Fitzmaurice, Baudel, Buxton 1997), which enumerated three design goals for this interface paradigm: keep the users' attention on their work, maximise screen used for artwork, and increase the quality of input. We describe the integration of a range of techniques, new and old, that aim to fulfil these goals. We also conduct a user study, which suggests that these techniques are readily learnable, integrate smoothly, and provide a viable alternative to traditional techniques.

In particular, we find the digital-tape, A-Line, and Graffiti techniques to be compelling possibilities. The other techniques (keypad, slicing, Ligne Claire, and a selecting Magic Lens) show substantial benefits, but could benefit from further research.

To conclude, we present a possible improvement to the Toolglass, which addresses user feedback, called the On-Demand Toolglass.

Symbol Technologies Industrial Design Group

Bill Buxton

Ronald Baecker

Ravin Balakrishnan

George Fitzmaurice

Gordon Kurtenbach

Paulo Pacheco

Guillaume Fraysse

Azam Khan

Christie Johnson

1      Introduction..................................................................................... 1

1.1    Design Goals                                                                                                                2

1.2    Thesis Organisation                                                                                                        2

1.3    Terminology Used                                                                                                          3

2      Literature Review.............................................................................. 4

2.1    Two-handed Input                                                                                                           4

2.2    Novice and Expert Behaviour                                                                                            6

2.3    Marking-based Interaction: Marking Menus                                                                         7

2.4    Focus and Attention                                                                                                       8

2.5    Interaction Techniques                                                                                                   10

3      Context: T3................................................................................... 20

4      Integration and Design...................................................................... 25

4.1    Technological Environment                                                                                            25

4.2    Interaction Paradigm                                                                                                     26

4.3    Integrated Additions                                                                                                      26

4.4    Selecting and Revealing Numerical Attributes of Graphical Objects                                       27

4.5    Soft Keypad – Using a Keypad to Specify Object Attributes                                                 30

4.6    Graffiti – Using Handwriting to Specify Object Attributes                                                   33

4.7    Digital Tape Drawing                                                                                                    40

4.8    Ligne Claire Curve Modification                                                                                     43

4.9    Slicing Technique For Modifying Shapes                                                                          45

4.10  A-Line Technique                                                                                                         46

5      User Study..................................................................................... 57

5.1    Rationale for user study                                                                                                 57

5.2    Methodology                                                                                                               58

6      Results......................................................................................... 62

6.1    Selecting and Revealing Drawing Attributes                                                                      62

6.2    Using a Soft Keypad to Change Object Attributes                                                              63

6.3    Using Graffiti to Change Object Attributes                                                                       63

6.4    Digital Tape Drawing                                                                                                    65

6.5    Ligne Claire Toolglass                                                                                                  71

6.6    Slicing Technique                                                                                                         72

6.7    A-Line Technique                                                                                                         72

7      Future Work................................................................................... 74

7.1    On-Demand Toolglasses                                                                                                74

7.2    Other Future Directions                                                                                                 77

8      Conclusion.................................................................................... 79

9      Appendix A: User Study Notes............................................................ 81

9.1    User 1, Session 1, 12/16/99                                                                                           81

9.2    User 1, Session 2, 12/17/99                                                                                           83

9.3    User 1, Session 3, 12/22/99                                                                                           84

9.4    User 2, Session 1, 12/20/99                                                                                           86

9.5    User 3, Session 1, 12/20/99                                                                                           87

9.6    User 4, Session 1, 12/21/99                                                                                           87

9.7    User 4, Session 2 , 12/22/99                                                                                          87

9.8    User 5, Session 1, 12/21/99                                                                                           88

9.9    User 6, Session 1 , 12/22/99                                                                                          89

9.10  User 7, Session 1, 12/23/99                                                                                           89

10        Appendix B: Technological Environment............................................. 92

10.1  OpenGL                                                                                                                     92

10.2  Wacom Tablet                                                                                                             92

10.3  Windows NT                                                                                                               93

10.4  Visual C++ 6 and MFC                                                                                                93

11        Appendix C: Additional Design Issues................................................ 94

11.1  Drawing Space                                                                                                             94

11.2  Colouring                                                                                                                   94

11.3  Input Decisions                                                                                                            95

12        Appendix D: Selected Algorithms...................................................... 96

12.1  Ligne Claire Algorithm                                                                                                 96

12.2  A-Line Technique                                                                                                         97

13        Appendix E: User Study Findings.................................................... 100

13.1  Digital Tape Drawing                                                                                                  100

13.2  Toolglasses                                                                                                               102

13.3  Marking Menus                                                                                                          107

13.4  Sketching And Templates                                                                                             108

13.5  Line Width                                                                                                                108

13.6  Tablet / Tools                                                                                                            108

13.7  Task                                                                                                                         109

13.8  Two-Handed Interaction Issues                                                                                       110

13.9  Overall                                                                                                                     110

13.10 Feature Requests                                                                                                         113

14        Appendix F: Implementation History................................................ 114

15        Appendix G: Design Goal Evaluation Details...................................... 122

16        Bibliography.............................................................................. 124

Table 5‑1: User experience levels and computer usage. 58

Table 5‑2: Subject session length and frequency. 59

Figure 2‑1: Hierarchical marking menu, with two selection methods, (a) and (b) 7

Figure 2‑2: Toolglass interaction. 9

Figure 2‑3: Graffiti numbers are very similar to Arabic numbers. 11

Figure 2‑4: Object creation using the two-handed stretching technique. 12

Figure 2‑5: Tape artist drawing with physical tape. 13

Figure 2‑6: Tape drawing of a car interior using traditional tape 13

Figure 2‑7: Digital tape tool upon a projection wall. 14

Figure 2‑8: Editing a curve with successive editing strokes (in grey) 14

Figure 2‑9: Comparison of slicing technique and point and click technique. 15

Figure 2‑10: Rotating a graphical object using the Turntable technique. 16

Figure 2‑11: Using the turntable tool 17

Figure 2‑12: The pivot point is specified by placing the "anchor" at the base of the arrow.                                                                                           18

Figure 2‑13: The arrow is grabbed at the tip, at point A, and rotated by moving the mouse.                                                                                          18

Figure 3‑1: T3 prototypes set-up, using two pucks on two Wacom tablets 21

Figure 3‑2: A screenshot of the T3 interface. 22

Figure 3‑3: A view of the Toolglass. 23

Figure 3‑4: Object creation using the two-handed stretching technique and Toolglasses.                                                                                                   23

Figure 3‑5: The menu bar acts as a marking menu. 24

Figure 4‑1: Selecting and revealing object attributes. 28

Figure 4‑2: The Toolglass as a selection mechanism. 28

Figure 4‑3: Graffiti Toolglass, showing a screen dense with vertices. 29

Figure 4‑4: Graffiti Toolglass, showing feedback for the selected line. 30

Figure 4‑5: Early iteration of the keypad Toolglass. 31

Figure 4‑6: Sketch of one possible solution to soft-keypad obscuration problem. 31

Figure 4‑7: Revised Toolglass, with the user entering values for the side's new length.                                                                                                   32

Figure 4‑8: Revised Toolglass, showing the result after the user clicks the enter button.                                                                                          32

Figure 4‑9: a conceptual example of using Graffiti to change object attributes 33

Figure 4‑10: The Graffiti Toolglass. 34

Figure 4‑11: The strokes for backspace or erase (A) and for execute or enter (B). 34

Figure 4‑12: The Graffiti Toolglass, in length change mode. Here the value 279 is not self-revealing.                                                                                 37

Figure 4‑13: Graffiti orientation issue. 38

Figure 4‑14: An older version of the digital tape tool, with three cursors. 40

Figure 4‑15: The digital tape tool. 41

Figure 4‑16: Lines and curves being drawn in Adobe Illustrator 7.0 for the Macintosh.                                                                                                   42

Figure 4‑17: The digital tape drawing tool. 42

Figure 4‑18: This diagram shows the user making a curve. 43

Figure 4‑19: When the left button is not held down but the puck moves, a green line provides feedback.                                                                             43

Figure 4‑20: Ligne Claire Toolglass with the user making a mark. 44

Figure 4‑21: Occasionally, the algorithm produces unexpected results. Error! Bookmark not defined.

Figure 4‑22: Expected outcome after the Ligne Claire mark. 45

Figure 4‑23: Slicing technique. 45

Figure 4‑24: The initial state. 48

Figure 4‑25: The goal state 48

Figure 4‑26: Invoking the A-line tool to the square by clicking through the Toolglass.                                                                                                   48

Figure 4‑27: Positioning the hands over the points on the object to be aligned, a and b.                                                                                                 48

Figure 4‑28: Selecting points a and b. 49

Figure 4‑29: Aligning by moving hands to points A and B, respectively. 49

Figure 4‑30: Completion on release of the puck button. 50

Figure 4‑31: Rotation around a pivot which is one of the initial selection points. 51

Figure 4‑32: Rotation around a pivot which is not part of the object. 51

Figure 4‑33: Hand & Object Movement. 53

Figure 4‑34: Translation through non co-linear motion by both hands. 54

Figure 4‑35: Translation by same-direction simultaneous movement of hands. 54

Figure 4‑36: Correcting object position by constrained dragging. 55

Figure 4‑37: Constrained dragging of other objects. 56

Figure 6‑1: Examples of difficult curves with the digital tape. 66

Figure 6‑2 detailed drawings can be difficult for users at first. Scaling in helps to alleviate this problem.                                                                       67

Figure 6‑3: User 2 was able to draw pictures of this complexity after 90 minutes. 70

Figure 7‑1 Near-current version of Toolglass 75

Figure 7‑2 Concept sketch of a new Toolglass 75

Figure 7‑3: The two cursors in the default mode. In this mode, the cursors act like digital tape tools.                                                                             76

Figure 7‑4: On-demand Toolglass 76

Figure 7‑5: sketch of on-demand Toolglass's marking menus. 77

Figure 11‑1 Wacom tablet, with a puck and stylus, the setup used for T4 92

Figure 11‑2 Puck for the Wacom tablet 93

Figure 12‑1: Scaling in T4 94

Figure 13‑1: Like-This or More-Like-This Curve algorithm. 96

Figure 13‑2: A and B represent the new locations of the cursors, and C and D the previous positions. We want to move our object to a position between A and B.                                                                                                   98

Figure 13‑3: We rotate around pivot point P to match the slope of the two cursors, A and B.                                                                                           98

Figure 13‑4: We then translate along the vector from P to the line AB. 98

Figure 13‑5: the cursor is mapped to the constraint line. 99

Figure 14‑1: Narrowing the area in which cursor motion is interpreted as erase may decrease erase problems.                                                                    100

Figure 14‑2: Users tried to click through the button (labelled A) instead of seeing that as a mode, and swiping through the shape.                                             104

Figure 14‑3: Toolglass without any distinction between classes of buttons 105

Figure 14‑4 revised Toolglass, separating functionality by group and shading, and optimising layout for right-handed interaction                                         105

In this thesis, we describe the design and evaluation of a two-handed drawing program called T4. This work is a design exploration, motivated by the goal of creating drawing interaction that is more "natural", easier to learn and use, faster, and in general more effective for the artists. This design studies the integration of a range of techniques, some new, and others known but not commonly used. The goal is to explore the extent to which these techniques can work together to provide a compelling alternative to the current, ubiquitous graphical user interface (GUI).

The techniques we present are described in detail in the Literature Review chapter, on page 4. Briefly, however, the techniques we explore include the following:

•      A method for selecting and revealing the numerical attributes of graphical objects.

•      Using a keypad to specify the attributes of graphical objects. We use a soft keypad, similar to a telephone keypad, to change the attributes of drawn objects, such as the length of a line segment.

•      A similar technique using handwriting to change object attributes. We use Graffiti, an alphabet similar to English but different in that each letter can be created with one stroke, to specify attributes of graphical objects.

•      A "digital-tape" technique, which allows for the creation of perfectly straight lines and smooth curves with a seamless switch between the two.

•      A technique for editing curves by drawing over them, rather than manipulating control vertices (CVs).

•      A slicing technique for modifying shapes. We use this method for cutting objects in half, deleting objects, and for adding or removing vertices from an object.

•      A new alignment technique, called A-Line. We present an alignment tool that allows alignment and distribution along any line.

1.1      Design Goals

These techniques represent a broadening of the vocabulary with which the artist can express their goals. However, the method in which this vocabulary is presented is equally important. In the process of designing and evaluating the T4 prototype, a number of design goals were used as reference points, to determine the effectiveness of the interaction styles used. These design goals are as follows:

•      Keep the users' attention on their work

•      Maximise screen used for artwork

•      Increase the quality of input

Note these design goals are identical to the design goals for the original "T3" project, described by Kurtenbach, Fitzmaurice, Baudel, and Buxton (1997).

1.2      Thesis Organisation

Our design was affected by research on the techniques presented earlier, but it was also motivated by a collection of higher-level research on two-handed input, workflow and expert behaviour, marking based interfaces, and focus and attention. Both the research on these techniques and the higher-level research are presented in detail in a Literature Review.

T4 builds upon the work on T3. T3 was a simple "next-generation" MacDraw-like application, used for creating two-dimensional drawings. Due to its importance as a precursor to the T4 prototype, T3 is reviewed in a separate chapter, Context: T3.

We then present a description of the design, the problems faced during the integration and design process, and the choices we made in resolving these problems, in the Design and Design Rationale chapter.

We follow with a User Study that returns some tentative conclusions about the effectiveness of the techniques we explore. The results are very positive for two-handed input, but indicate that a lot of careful design and continued research are necessary.

Finally, we present our conclusions and discuss future directions for research.

1.3      Terminology Used

For simplicity, we often refer to right-handed or left-handed instead of the more correct but cumbersome terminology of dominant hand and non-dominant hand.

T4 is built upon a great body of previous research. The most important research in this area is on two-handed input, novice and expert behaviour, marking based interaction, and focus and attention. In this chapter, we will review relevant aspects of this literature. In addition, we will explore the research on techniques that have been developed in light of these ideas.

2.1      Two-handed Input

Psychology has historically focused on our two-hands as competing for dominance in our brain. This is reflected in our usage of the terms, right-handed and left-handed. The question you are asked is if which hand you use for writing, eating, and brushing your teeth.

What this viewpoint misses is that most manual activities are essentially bimanual. When we write a letter, paint a picture, play a guitar, lift a heavy object, read a book, type on the computer, eat with a knife and fork, drive a car, swing a bat or club, shoot a rifle or bow and arrow, or pull a rope, we are using both of our hands. Some of these might not be immediately obvious – for example, why writing with a pen is a two-handed task. However, when we observe people writing, they invariably use the left hand to orient and position the paper they are writing on. In fact, one informal study found that writing speed decreased by 30% when subjects were forced to use only one hand to write (Guiard 1987).

We will not cover the historical views of handedness and bimanual manipulation, which are amply covered in Guiard (1987; and Guiard and Ferrand 1995). However, research has progressed greatly in this area in the last twenty years.

We should first clarify the different forms on manipulation humans engage in. First of all, a division should be made between symmetric and asymmetric manipulation. Examples of symmetric manipulation are lifting a barbell, and pushing on a wall. In these examples, the movements are similar or the same in both hands, and they are either happening simultaneously or out of phase.

Asymmetric manipulation is by far the most common, and has come to prominence in HCI research through the work of Guiard. Generally, asymmetric bimanual manipulation can also be termed co-operative bimanual manipulation, because the two-hands take on different but complimentary roles to accomplish a single task.

Guiard introduced the Kinematic Chain (KC) model, which describes asymmetric bimanual manipulation as being like two abstract motors in a serial chain, which cooperate in a kinematic sense similar to the way the arm and hand do (Guiard 1987; Guiard and Ferrand 1995). Some principles of the model are as follows:

•      Non-dominant hand precedence – the non-dominant hand initiates a gesture that involves two hands. Thus, for example, when a person sits down to write, the movement in initiated by the left hand positioning the paper, followed by the right hand writing upon it.

•      Asymmetric scales of motion – the frequency of movement for the right hand is greater than the left; hence, the right hand tends to make finer, more detailed movements, while the left makes larger movements.

•      Dominant to Non-dominant Spatial Reference – This is shown in the example of writing on a sheet of paper, where the left hand sets up the context for the right hand. It is also demonstrated in the case of a painter's palette, where the painter dips the paintbrush in the palette, which is positioned by the left.

2.1.1     Two-handed Input in the Computer World

It wasn't until 1986 that anyone formally investigated how computers could take advantage of both hands (Buxton and Myers 1986). In this study, two experiments showed reduced time performance when the subjects used both hands as opposed to one hand. This paper opened up a new area of research in human-computer interaction.

Subsequent studies refined the understanding of two-handed interaction, and Guiard's KC theory became better known in the research community. As an example of the refinements subsequent research produced, Kabbash, Buxton, and Sellen showed that two-handed input had to be developed carefully (essentially, with an understanding of Guiard's KC model), or two-handed input could result in worse performance (1994). The same paper showed significant advantages of two-handed techniques that took advantage of asymmetrical bimanual motor and cognitive skills.

2.2      Novice and Expert Behaviour

Buxton summarises the differences between expert and novice user performance in his 1986 paper on chunking and phrasing. The level of granularity at which novice and expert users view a system is different. A novice user pays great attention to every detail of the system, and their behaviour may best be described as "problem-solving". In contrast, expert users have acquired the skills to automatically execute the low-level details, and their attention is focused on the task at hand (Buxton 1986). For example, if we imagine a novice and an expert user of a bicycle, the novice's attention is consumed with the mechanics of riding, and they are trying to "figure out" the bicycle. The expert bicyclist, however, is focusing on much higher-level details, such as where they are going, and perhaps the way in which they pedal, fast or slow. Buxton points out that a goal in designing computer systems can be to minimise the transition time between novice and expert behaviour (1986).

2.3      Marking-based Interaction: Marking Menus

Marking menus were developed in response to the problem of marks not being self-revealing with their functionality. They were originally described in 1991 [J1] (Kurtenbach and Buxton 1991) and refined in successive papers (Kurtenbach and Buxton 1993).

Figure 2‑1: Hierarchical marking menu, with two selection methods, (a) and (b)

There are several things that are remarkable about marking menus. First of all, the user has a self-revealing system, similar to normal menus, which they can browse, as shown in Figure 2‑1(a). It thus furthers recognition rather than recall. Second, as the user begins to learn the location of various items in the marking menu, they can make the stroke without waiting for the menus to come up (Kurtenbach and Buxton 1991), as in (b). This is one of the best examples of an interaction technique in which the novice behaviour trains the user for expert behaviour. This is in contrast to many other expert-level techniques, such as hot-keys, which require learning a completely different type of interaction in order to increase proficiency with the system.

Marking menus can be a simple stroke (split into n directions), or they can be hierarchical. Research suggests an upper bound of eight items in each level as a ceiling (Kurtenbach and Buxton 1993), but this depends on what error rates are acceptable and other factors.

2.4      Focus and Attention

Traditional GUIs focus on visual perception. This has allowed them to leverage many existing graphic arts skills in the design of visual interfaces; however, it may also have drawbacks (Harrison, Ishii, Vicente, Buxton 1995). As Harrison, Ishii, Vicente, and Buxton point out, an alternative is to focus on attention. This has the added benefit of looking at how layers and depth affect the user, and reflects a focus on what the user is actually noticing rather than what is there. For this reason, it may be a more user-centred approach. This concept is especially important when we consider interfaces that take advantage of transparency.

2.4.1     Toolglasses & Transparency

One such interface is the Toolglasses and Magic Lenses interface, an idea that came out of Xerox PARC, and reflected this focus on attention (Bier, Stone, Fishkin, Buxton, Baudel 1994; Bier, Stone, Pier, Buxton, DeRose 1994). Toolglasses are a two-handed technique in which the non-dominant hand controls a semi-transparent tool palette. The palette can either be a lens that reveals extra information about the underlying object (a Magic Lens) or an actual tool palette (a Toolglass). Bier, Stone, Fishkin, Buxton, and Baudel present many types of Toolglasses (1994) but generally, the mechanism is click-through. This is shown in Figure 2‑2. The user controls the Toolglass with their non-dominant hand, and the cursor with their dominant hand. Assume there is an object with an attribute "C". The user moves their non-dominant hand to position the Toolglass over the object. Next, or simultaneously, they move their dominant hand to position the cursor over the Toolglass and the object. The user it then presses a button on their dominant hand and the attribute "B " is transferred to the object, replacing the value of "C".

	(a)				(b)
	(c)				(d)

 

 

 

 

 

Figure 2‑2: Toolglass interaction.

The user positions the Toolglass over the object to be modified,

clicks through the Toolglass on to the object, and changes that attribute from C to B.

 

The original Toolglasses paper describes object creation like a cookie cutter approach. The user places the Toolglass over the location they want the new shape. When they press the button, the new object is created in that location.

In experiments with the Toolglass techniques, the Toolglass techniques proved themselves to be markedly superior speed-wise to other two-handed and one-handed techniques (Kabbash, Buxton, and Sellen 1994). This is probably due to their ability to leverage proximity and maintain focus on the artwork, requiring little time to refocus, in line with Harrison, Ishii, Vicente, and Buxton's suggestions (1995).

2.5      Interaction Techniques

Whereas the research we previously summarised represented higher-level, theoretical frameworks within which to guide design, the following techniques represent forays into possible designs. We explore researched techniques for selecting and revealing values of object attributes, using a keypad to specify those same attributes, using handwriting to specify these graphical attributes, "digital-tape" drawing, curve editing, a slicing technique, and an alignment technique.

2.5.1     Selecting and Revealing Numerical Attributes of Graphical Objects

This technique has not been previously researched, but is described in Selecting and Revealing Numerical Attributes of Graphical Objects, on page 27. It is based upon the research on Toolglasses and Magic Lenses, as described in Toolglasses & Transparency, on page 8.

2.5.2     Using a Soft-keypad to Specify Graphical Attributes

This technique has not been previously researched, but is described in Soft Keypad – Using a Keypad to Specify Object Attributes, on page 30.

2.5.3     Using Handwriting To Specify Graphical Attributes

Using handwriting to specify the attributes of drawn objects is a new technique based in research on pen-based input. Goldberg and Goodisman developed the first computer-based unistroke alphabet for text entry (1991). A unistroke alphabet is an alphabet in which each letter can be made in a single stroke. Goldberg and Goodisman's alphabet was designed for text-based entry using a stylus, and was inspired by work by Buxton, Sniderman, Reeves, Patel, and Baecker on a unistroke alphabet for scoring music (1979). The time required to learn Goldberg and Goodisman's alphabet was significant, but the speeds were much greater than a later system developed by 3Com for the Palm Computing platform. This system, Graffiti, proved to be very popular because of its similarity to the English alphabet. Mackenzie demonstrated that users with no familiarity of the Graffiti system could achieve 86% accuracy after one minute study of the system, and 97% accuracy after five minutes of practising with the system (MacKenzie and Zhang 1997). These results are for the 26 letters of the alphabet.

Figure 2‑3: Graffiti numbers are very similar to Arabic numbers.

For performance, we restrict our view to numerical entry, as shown in Figure 2‑3. Graffiti performance is about 16.7 words per minute to 17.6 words per minute, and its accuracy ranges from 87% to 94%. Of course, performance varies depending on the recogniser used, experimental conditions, and many other factors. This performance is definitely slower than on a soft keyboard (30 words per minute, with 1.2% errors) (MacKenzie, Nonnecke, Riddersma, McQueen, and Meltz 1994).

2.5.4     Digital Tape Drawing

Digital tape drawing builds upon a two-handed stretching technique first developed by Krueger (1991), and refined by Leganchuk (1996). This interaction is described in Figure 2‑4.

 

 

 

 

 

Figure 2‑4: Object creation using the two-handed stretching technique.

The user moves two cursors on the screen, as shown in (a). When they desire to make a shape, they initiate the creation, perhaps by the use of a button on one of the input devices. The interface displays a shape defined by the position of both cursors, as in (b). the user stretches out the shape to the desired location and shape, as in (c), and then lets up on the button, placing the object, as in (d).

Note this allows for translation and sizing to occur during object creation, and easily allows for creation of shapes within shapes. It becomes trivial to add an oval inside a square such that it touches all four corners, for example. This is very difficult to do with conventional techniques.

Digital tape drawing is a marriage of the two-handed stretching technique and a traditional art medium of physical, tape-based art used in the automobile industry. The tape artists in these companies use photographic tape of varying thickness to create large-scale concept-sketches of automobiles. The artist unrolls the tape with the right hand and pins it down using the left hand (note this tends to be true even of left-handed artists), as shown in Figure 2‑5. The tape affords drawing perfectly straight lines, and, because of the elasticity of the tape, smooth curves as well. As shown in Figure 2‑6, the artists can become very competent, and produce drawings equivalent to other art mediums.

Figure 2‑5: Tape artist drawing with physical tape.

 

Figure 2‑6: Tape drawing of a car interior using traditional tape

courtesy of Renault Automotive Corp., France

 

Researchers at Alias|wavefront developed a prototype digital tape drawing system which works on a projection wall, as shown in Figure 2‑7. The prototype uses Ascension Flock-of-Birds six degree-of-freedom electromagnetic trackers, one in each hand, to simulate the tape.

Figure 2‑7: Digital tape tool upon a projection wall.

They developed and evaluated this prototype by working with tape artists at several auto studios. Their evaluation measured the digital tape tool's value within the auto industry and not the usability of the tool on desktop systems or for non-tape artists (Balakrishnan, Fitzmaurice et al. 1999).

2.5.5     Curve Editing – Ligne Claire

The way users typically interact with computers in current two and three-dimensional drawing packages is by the use of control points and tangents to curves. While these provide mathematically precise control over the curvature, they do not reflect the conceptual model many users have of how to edit curves. Thomas Baudel, in both his thesis (Baudel 1995) and a paper presented for User Interface Software and Technologies (Baudel 1994), describes Ligne Claire, or "clear lines", a stroke-based method of editing curves by drawing over the curve but correcting the curve to be more like the desired result.

Figure 2‑8: Editing a curve with successive editing strokes (in grey)

As shown in Figure 2‑8, the user strokes over the curve and the curve reforms itself to the new curvature defined by the user's stroke. This technique closely matches sketching techniques in the real world, where an artist will draw successive strokes over one location to reach the desired shape and look.

2.5.6     Slicing Techniques

Direct manipulation interfaces typical adopt the point and click method of choosing an object. While this has proven to be enormously successful, other techniques may lend better to some tasks. For example, if the task is to slice a line segment in half, as shown in Figure 2‑9, using a razor-blade metaphor and making a stroke across the line is an easier task that selecting the line by clicking on it (by Fitts' law, because the target area is so small). Besides being easier, however, it mimics the user's real-world experience of using a razor blade.

 

 

 

 

 

 

 

Figure 2‑9: Comparison of slicing technique and point and click technique.

This technique is used to a limited extent in the Teddy sketching system (Igarashi, Matuoka, and Tanaka 1999). In their system, this technique is used for slicing objects. Theoretically, it could be used for other functionality as well.

2.5.7     Graphical Alignment[1]

The problem of constraining graphical objects so that they align in desired ways is not new. This was one of the main motivations for Sutherland's use of constraints in his classic Sketchpad system (Sutherland 1963) and common commercial products, such as Vellum (Ashlar 2000).

We focus on the research on rotation, because of its relevance and influence on the alignment technique, A-Line, described in A-Line Technique, on page 46.

Evans, Tanner and Wein described a rotation tool that they called The Turntable, which facilitated certain types of alignment (Evans, Tanner et al. 1981). Theirs was a two step technique, illustrated in Figure 2‑10.

Figure 2‑10: Rotating a graphical object using the Turntable technique.

(From Evans, Tanner & Wein, 1981)

 

After selecting the object to be rotated, the "F", a pivot point, or point of rotation was specified. In the figure, this appears as the "+" symbol labelled B. This point functioned like the spindle of a virtual turntable. Hence, if the mouse button was depressed at some other point, such as that labelled A, the turntable could be dragged around the spindle, thereby rotating the graphical object.

One interesting thing about this technique is that the amount of rotation is determined by the angle CBA, independent of the length of BA or BC. Hence, a vernier effect was provided, affording a fluid way for the user to control the amount of rotation resulting from a given amount of hand motion. The closer the mouse to the spindle, the higher the "gear ratio" of the rotation.

Another aspect of this, more related to our work, is how the tool could be used to easily rotate an object such that it aligned with some other point. Consider the example illustrated in Figure 2‑11, where the desire is to rotate a previously vertical "F" such that its left side was aligned with a point D on the display.

Figure 2‑11: Using the turntable tool

to align the rotation of an object with another point (B) on the display.

 

In this case, the user could specify the pivot point B as the left bottom of the "F", and grab the turntable from the initial point A, which was initially the top left corner of the "F". The alignment could then be achieved simply by dragging the mouse to point D and releasing the mouse button.

A similar, but less general approach to rotating an object such that it aligned with some other point was the rotation tool described by Bier and Stone (1986). The technique that they described is illustrated in Figure 2‑12 and Figure 2‑13, where the objective is to rotate the arrow shown such that it points to point B in the centre of the circle to the right of the arrow.

Figure 2‑12: The pivot point is specified by placing the "anchor" at the base of the arrow.

Figure 2‑13: The arrow is grabbed at the tip, at point A, and rotated by moving the mouse.

The mouse position is indicated in both (a) and (b) by the caret "L" symbol. (From Bier & Stone 1986)

 

Again, the user specifies the pivot point, in this case called the anchor. With the Snap-Dragging technique of Bier and Stone, the user must grab another point on the object to initiate rotation. In the example, the tip of the arrow, A, is grabbed.

In both the turntable and snap-dragging techniques, three points are always kept co-linear: the pivot point, the initial point "grabbed", and the mouse position. Both provide a vernier effect, and consequently both support rotation to align to a third point.

With snap-dragging, there is a gravitational pull, or "snapping to" target points, which can aid in the alignment with a third point. With the turntable, one needs not initiate the rotation from a point on the rotated object.

Both are four-step operations:

1.     Select the object to be rotated.

2.     Specify the pivot point.

3.     Specify the start point of rotation.

4.     Rotate the object by dragging.

Only steps (3) and (4) are integrated. The others are discrete (and therefore interruptible) operations. The above examples reflect the state of the art as represented in the marketplace by products such as Adobe Illustrator (2000).

This research on rotation was significant for our later work on graphical alignment.

In this chapter, we explore the history of T3, the predecessor to T4. This is useful both for historical reasons and to understand the present prototype, because much of the present work is based on the T3's paradigm of interaction.

T3 was laid out in a design specification by Buxton (1995), which described a four-state (one button in each hand), Toolglass-centred user interface paradigm. George Fitzmaurice programmed the T3 prototype, exploring several novel user interface ideas: two-handed input, transparency, and using tablets instead of mice to interact with the computer. The prototype is described in detail in Kurtenbach, Fitzmaurice, Baudel, and Buxton's paper (1997), but as a general overview, the program was based on several design goals:

•      Maximising screen used for artwork

•      Reducing amount the user interface diverts visual attention from the artwork

•      Increasing the quality of input

The T3 program used two pucks on two separate Wacom UD tablets. These tablets sensed three degrees of freedom each (position and rotation). As per the design specifications, all of the buttons were mapped to the same functionality, so despite the four buttons on each puck, the pucks acted like one-button pucks. The prototype's set-up is shown in Figure 3‑1.

Figure 3‑1: T3 prototypes set-up, using two pucks on two Wacom tablets

T3 was designed as a simple, "next-generation", MacDraw-like drawing program. The functionality provided was simple 2D graphics primitives, such as squares, circles, and lines. It also had a scribble-like pen writing tool, but it wasn't very well integrated into the program. For example, the scribbles didn't rotate with the canvas. Figure 3‑2 shows both a screenshot of the overall interface, and an example of the basic artwork that was possible with T3. The artwork was all object based, so graphical objects could be moved, scaled, and rotated.

The basic paradigm of interaction in T3 was the Toolglass technique described by Bier, Stone, Pier, Buxton and DeRose (1994). Here, the puck controls the position of a semi-transparent tool palette, and the stylus controls the position of the cursor. Central to the Toolglass technique is the notion of click-through tools. These work a bit like lettraset. One places the desired tool on the palette over the graphical object to be affected. Activating the tool with the stylus then "clicks" the action "through" the palette onto the object. Key advantages of the technique are that the eye can stay focussed on the work, since the tools come to the place of work, rather than the cursor to the tools. Furthermore, the act of selection and action are thereby integrated, thus making the interaction more fluid and connected.[2]

Figure 3‑2: A screenshot of the T3 interface.

Note the Toolglass over the "T" and the cursor to the right of it.

 

The Toolglass, as shown in Figure 3‑3, had a few types of buttons. There was a row of colour cubes that could be clicked through to change the colours of underlying shapes. Below these lay, in left to right, top to bottom order, the scribble-line tool, the triangle tool, the rubber-band line tool, the delete tool, the circle tool, and the square tool.

Figure 3‑3: A view of the Toolglass.

The checkers in the picture are an artefact and were not present in the interface.

 

The way that objects were created was inspired by Krueger's work on two-handed stretching (1991) and Leganchuk's adaptation of this technique (1996). This method is described in Figure 3‑4.

 

 

 

 

 

Figure 3‑4: Object creation using the two-handed stretching technique and Toolglasses.

The user places the Toolglass using their non-dominant hand, clicks through the desired tool (A or B in this example), and, holding down the button in the dominant hand, stretches out two corners of the object to the desired shape. During this stretching movement, the Toolglass disappears, because its presence is not necessary. Finally, the user lifts the button in their dominant hand, and the object is placed. A similar operation is done for translating and scaling the object after it has already been created.

The menu bar was a marking menu area that could be used to change the Toolglass, as shown in Figure 3‑5.

Figure 3‑5: The menu bar acts as a marking menu.

The interaction for the marking menu is as follows. The user clicks the dominant hand cursor over the Toolglass marking menu bar, which is controlled by the non-dominant hand. The user then either makes a stroke, or waits until the menu comes up and then makes the stroke. After the stroke is made, the system changes the Toolglass into whatever the user has selected.

 

This section describes the design and design rationale for T4.

The technological environment that T4 was designed on is described in detail in "Appendix B: Technological Environment", but we start by summarising the technology upon which T4 was built. Then, we describe the design decisions made while integrating the new techniques into the T4 prototype. During this process, we also describe the interaction of these techniques.

4.1      Technological Environment

All of the components for the T4 system were created on widely available, off the shelf technology. Indeed, the technology for T4 has been widely available for about six years (since the Wacom UD tablets), so the basis of this work is not technological, but interaction based.

T4 was developed on Microsoft Windows NT, in C++, using OpenGL, a standard, cross-platform graphics library. Input was handled using a Wacom Intuos tablet, a graphics tablet which supports two-handed input. For a right-handed user, the left hand holds a five-button puck, and the right hand holds a stylus. The tablet senses many degrees of freedom, but we only take advantage of position for both hands, the buttons, the thumbwheel for the puck, and azimuth (equivalent to rotation) for the stylus.

4.2      Interaction Paradigm

Interaction was directly descended from the T3 paradigm of interaction. Thus, the basic mechanism for interaction is the Toolglass, as modified for T3, and described in Context: T3, on page 20.

 

4.3      Integrated Additions

There were seven major techniques added to the T3 base to create T4. These interaction styles and techniques are as follows:

•      A method for selecting and revealing the numerical attributes of graphical objects.

•      Using a keypad to specify the attributes of graphical objects. We use a soft keypad, similar to a telephone keypad, to change the attributes of drawn objects, such as the length of a line segment.

•      A similar technique using handwriting to change object attributes. We use Graffiti, an alphabet similar to English but different in that each letter can be created with one stroke, to specify attributes of graphical objects.

•      A "digital-tape" technique, which allows for the creation of perfectly straight lines and smooth curves with a seamless switch between the two.

•      A technique for editing curves by drawing over them, rather than manipulating control vertices (CVs).

•      A slicing technique for modifying shapes. We use this method for cutting objects in half, deleting objects, and for adding or removing vertices from an object.

•      A new alignment technique, called A-Line. We present an alignment tool that allows alignment and distribution along any line.

There were several basic rationales behind the decision to include each piece of additional functionality. The first was that the technique itself seemed promising or worthy of investigation because it seemed to further one of the T3 paradigm design goals. A second was to make a "full-featured" prototype that felt like a real system, and could be evaluated as such, so that interaction between the tools, workflow, and the like could be investigated. A third reason for some of the inclusions was that users asked for that functionality.

4.4      Selecting and Revealing Numerical Attributes of Graphical Objects

The motivation for the development of this technique arose from our desire to integrate Graffiti and a soft keyboard into the Toolglass paradigm of interaction. We needed a method of selecting which object to modify, and we also needed to reveal the information about the object itself, so the user would realise the current values before they modified them. Several possibilities presented themselves. The first would be the conventional way of handling selection, which is to use the cursor to click on an object and to use feedback to show the object is selected. The second method was to use the Toolglass itself to specify which object was going to be modified.

We chose the second possibility because it seemed more in line with Guiard's model of how two-handed interaction naturally works (Guiard 1987; Guiard and Ferrand 1995). In addition, the left hand was already setting the context within which behaviour was to occur (a la Guiard) in the Toolglass paradigm.

The following describes this technique:

1.     Assume the user has a Magic Lens (see Toolglasses & Transparency, on page 8, for a review of Magic Lenses) controlled by their left hand, as in Figure 4‑1(a).

2.     The user moves the Magic Lens over the shape they want to modify, as shown in Figure 4‑1(b). A cross-hair upon the Toolglass specifies which shape will be selected for revealing its attributes.

3.     The program reveals attributes of the object in a position which makes spatial sense (for example, writes the length of a line segment next to that line, or writes the x,y values of a vertex next to the vertex), as shown in Figure 4‑1(b) and Figure 4‑1(c). This can either be done as a true Magic Lens, as in (c), revealing only the information underneath the lens, or as we chose to do, in (b).

 

(a)

(b)

(c)

 

 

 

 

Figure 4‑1: Selecting and revealing object attributes.

 

We chose to put the point of selection as the centre of the Toolglass. In the version of the software we tested in our user study, there was no explicit mark on the interface specifying that the centre was the point of selection, but users asked for it, and we quickly added it in. Algorithmically, the selected point was actually whatever line segment centre or vertex was closest to the centre of the Toolglass, as shown in Figure 4‑2. "Closest" was the shape with the minimum value for "D".

Figure 4‑2: The Toolglass as a selection mechanism.

D is the length of the line from the centre of the Toolglass to the vertex. Because this vertex is closest, it is selected, and the feedback reflects this. Note this version of the interface has cross-hairs as a clue as to the underlying mechanism for selection.

An issue related to the selection was that of feedback for the selection. As shown in Figure 4‑2, the feedback extends outside the Toolglass. Initial versions of the Graffiti tool did not do this – the feedback was restricted to be within the Toolglass. This is line with the original concept of the Magic Lens or Toolglasss, that it would act as a lens, revealing information about the drawing that is not actually in the drawing. We violated this because using the Toolglass as a selection tool in the way we did seemed to be a new method of using the Toolglass, and users complained of not being able to see when we restricted the display of attribute information to just within the Toolglass. This highlights a problem with Toolglasses and Magic Lenses: the trade-off between hiding unnecessary information and hiding information you may actually need.

Figure 4‑3: Graffiti Toolglass, showing a screen dense with vertices.

T4 gives feedback on which vertex is selected to help sort through this density.

 

A final selection issue is on what behaviour is to be expected when there is a dense cloud of vertices or edges in one location, as in Figure 4‑3. The current system merely picks whichever is closest to the centre of the Toolglass, but this may not be ideal for dense drawings. One thing we did do to help with this was to display the text of only the selected vertex or edge. Previous versions displayed everything, and this became a maze of overlapping letters and numbers in dense shapes like circles and curves. We simplified the feedback by taking out any vertices except for the one currently selected, and we also highlighted the vertex or edge to show that it is selected, as shown in Figure 4‑4.

Figure 4‑4: Graffiti Toolglass, showing feedback for the selected line.

4.5      Soft Keypad – Using a Keypad to Specify Object Attributes

The rationale for the inclusion of the soft-keypad is that, in addition to being useful as a method of comparison to Graffiti, it may provide addition benefits to those who are not familiar with Graffiti. Like Graffiti, the intention is to input numbers and take advantage of where they are being input. To do this, we put the keypad on a Toolglass. We expect that the keypad will result in faster input and less errors than Graffiti, but also that it will take more attention away from the artwork and be slightly less "natural" than using Graffiti. Writing is something we do on objects, while keyboards and sliders, even if on a Toolglass, put the focus on themselves. Thus even though Graffiti is slower and less accurate than a soft keyboard and probably sliders as well, the subjective experience may be "better". The largest caveat to this is that because recognition rates are inferior, this benefit will be easily obscured.

Because the soft keypad was conceived of as a method of comparison with the Graffiti Toolglass, most of the issues that were present for the Graffiti Toolglass were also present for the keypad Toolglass. The only area that was a little different was with selection. Selection with the soft keypad was made more complicated by the fact that the buttons were acting both as a magic lens and as functional buttons. In early versions of the prototype, the problem was so bad that we considered redesigning the Toolglasses altogether.

Figure 4‑5: Early iteration of the keypad Toolglass.

Notice how the text is obscured behind the Toolglass, and also notice that all the edges are labelled. Here, the user has entered 200 and is about to execute.

 

 

 

 

 

 

 

Figure 4‑6: Sketch of one possible solution to soft-keypad obscuration problem.

Figure 4‑7: Revised Toolglass, with the user entering values for the side's new length.

Figure 4‑8: Revised Toolglass, showing the result after the user clicks the enter button.

As shown in Figure 4‑5, the original Toolglass was 50% transparent[J2] , and was coloured orange. During the informal usability sessions in the research group at Alias|wavefront, the other researchers complained that "there are too many things going on" in the interface, and that the Toolglass was distracting and hard to look through. This led to ideas such as that presented in Figure 4‑6, with the centre of selection being somewhere other than the centre of the Toolglass. We also thought of perhaps having one text feedback area on the Toolglass, or perhaps of having a line than proceeded from the Toolglass to the object (be it a vertex or edge) being modified.

However, while experimenting with the interface, we found that by making the Toolglass completely transparent, the tool became more usable. The other researchers found the change to be good enough, and we decided to wait and see what the user study revealed about the keypad. One point to be made about this part of the design is that the keypad has a fixed layout that the user understands without looking at it, so even if the buttons weren't labelled, the user could probably use the Toolglass. This means than making the Toolglass transparent might be appropriate in this case and not in many others, because the user can just click in the button without looking at it, or even without shifting focus to the Toolglass plane of the interface.

4.6      Graffiti – Using Handwriting to Specify Object Attributes

 

The idea behind adding Graffiti into the system was to allow a user to change the attributes of a drawing by simply writing the desired values on the drawing. An example could be as follows:

Figure 4‑9: A conceptual example of using Graffiti to change object attributes

The rationale for including this functionality is that the users' focus stays on their artwork, and they can write the values they want to change where they should be changed. Thus, if the technique is found to be beneficial, with one single interaction, a user can accomplish what usually requires several separate actions.

In addition, this technique provides a more direct form of feedback, since the operations performed are so closely linked with the object. Moreover, because the writing is done right over the object, there is no device acquisition time involved with finding and using a pop-up window or another device. This difference in acquisition time should result in faster performance as well, as predicted by the Keystroke Model (Card, Moran, Newell 1980).

There were many decisions that had to be made during the course of T4's design. As shown in Figure 4‑10, the Graffiti functionality was integrated into a Toolglass. The top three buttons are used to select what attributes of the shape are to be modified: position, colour, or length.

Figure 4‑10: The Graffiti Toolglass.

The background has very light grey "notepaper"-lines. The attribute currently selected is printed near the object, with its current value, 279 pixels in this example. As the user writes new values, they appear in red underneath the old values. The user can execute an erase stroke to back up (erasing the previous value), or they can do a Southwest stroke to execute the values they've entered. See Figure 4‑11 for examples of these strokes.

 

 

 

 

Figure 4‑11: The strokes for backspace or erase (A) and for execute or enter (B).

There are two possible strokes for "execute".

Graffiti Inking

One of the first issues that had to be dealt with during the implementation of the Graffiti system was the decision of how the Graffiti marks should be laid out on the screen. Should the marks be recognised one by one, or should they be recognised all at once, upon completion?

We decided to lay down the letters one at a time for several reasons. First of all, this keeps the focus on the artwork because it de-emphasises the plane upon which the Graffiti is being written. Second, because the Graffiti is larger than the text feedback, it reduces the amount of screen space required. This can prevent problems that would arise if the user ran out of room or wrote two marks in the same location. Also, if the user is constrained to write on a Toolglass, which we chose to do, then this problem would be exacerbated, because the Toolglass is a fraction of the size of the screen.

Another problem this would prevent is with user expectations. If the mark for previous numbers was still visible, the user may feel they could correct, alter, or erase that mark by drawing or erasing the mark. This is especially likely given the eraser tip on the end of the stylus. While this could be a desirable form of interaction, it would have greatly complicated the program, and we chose a simpler approach.

The decision to constrain the Graffiti to marks initiated within the Toolglass was intended to provide a clear mechanism for separating different modes of interaction. We wanted to establish a certain class of actions that would happen inside the Toolglass and another set of actions that would happen in the periphery.

For all these reasons, then, we decided to implement the character recogniser with a letter by letter recognition.

Graffiti Termination

Another problem we encountered was the issue of deciding on an appropriate way to signal that the user was finished with their input. When a user is specifying the length of a line segment, for example, values 9, 95, 953, and 9534 are all equally valid. How does the user specify when they are done?

There were several possible solutions to this problem. We could have used a timeout, we could explicitly prompt them with instructions, or we could define a specific mark for "Enter" or "I'm done". We chose the later approach, because a user pausing can mean they are done or, just as likely, that they are thinking or confused. Also, prompting them explicitly seems to be too "messy". When we started out, we adopted this method, but users complained that the text took up too much room.

This could be solved by the use of marking menus, which would make these marks self-revealing (Kurtenbach and Buxton 1991). This version currently does not implement marking menus here, but it could be fairly easily added in.

We decided upon a Southwest mark for "Enter" because it is somewhat conventional, and a West mark for "Backspace", as shown in Figure 4‑11.

A related issue was handling movement of the Toolglass during Graffiti entry. Should entered text disappear if the user moves the Toolglass away, or should it stay the same? Should the numbers the user has entered move to the new selection, or should they be cleared? Currently, the prototype associates the entered numbers with each vertex or line edge, so moving the Toolglass will leave the values there, and visible. Feedback from the research group suggested that leaving the values in is confusing, and they were divided over whether the values should be cleared or moved along with the Toolglass. We weren't able to explore this interaction in further detail.

Ambiguity of Numbers

We also had to choose what values the numbers would represent. A value of 45 could have several alternate meanings, depending on how it was measured

Figure 4‑12: The Graffiti Toolglass, in length change mode. Here the value 279 is not self-revealing.

This is a problem that could be resolved by careful feedback, but it also has the same constraints as the Termination problem. By making the functionality obvious, it may require the display of too much information, which can be jarring for viewers.

If a preferences panel was added to the program, the user could explicitly decide what the values meant. This could also be useful for changing the measuring units for drawing to inches or centimetres, for example.

Toolglass Integration: Screen-centric or Toolglass-centric?

Is Graffiti screen-centric or Toolglass-centric? We noticed while developing the Graffiti portion of the interface that some users tended to write askew when the Toolglass was rotated. If this behaviour had been universal, we would've had to change our recogniser to take this into account. However, because this behaviour was sporadic, even for the same user, we concluded that the rotation was causing cognitive interference.

 

 

 

 

Figure 4‑13: Graffiti orientation issue.

When the user desires to make a horizontal stroke to the right,

should it follow the screen or the Toolglass?

 

As shown in Figure 4‑13, the rotation of the Toolglass led users during informal trials of the software to want to make a horizontal stroke follow the Toolglass instead of strictly in the x direction.

We saw two possible solutions to this issue. One was to not rotate the Toolglass, and the other was to add screen-centric "notepaper-lines" in the background of the Toolglass. We thought that to not rotate the Toolglass might detract from the natural, object-like feel of the puck, and decided to try notepaper lines to see if they resolved the problem.

After adding in the notepaper-lines, we did not observe the issue any further, so we thought this was a reasonable solution to the issue. It would be interesting to measure if accuracy of Graffiti input improved as a result of these notepaper-lines, but we didn't have the opportunity to test this.

Toolglass Integration: Leave Toolglass Up While Inking

In retrospect this seems obvious, but previous iterations of the program made the Toolglass disappear once the user initiated a Graffiti mark. As a result, the Toolglass seemed to flicker on and off. We noticed a marked increase in entry time and "naturalness" after we changed the code to leave the Toolglass visible during inking.

Toolglass Integration: Toolglass Size and Inking Issues

Another design issue was a consequence of our decision to limit Graffiti writing to within the Toolglass. We considered what behaviour the user would expect if the mark extended out of the Toolglass area. Our solution was to have the Graffiti mark start within the Toolglass, but extend outside until pen-up in the DH. This would have been especially important if we were inking everything and then laying it down instead of laying it down letter by letter, as we are doing currently. The reason for this choice was that the state is initiated with a kinaesthetic act of pushing down the stylus, and that motion lasts until the tip is lifted (Buxton 1990). To interrupt that gesture midway and not accept the mark seemed to be an interruption of the user's entire gesture.

Toolglass Integration: Choosing Attributes

Another Toolglass integration issue was that of how the user should switch between the different attributes of an object they wish to modify (i.e. colours, positions, and lengths). The code supported changing colours and positions of vertices, and the lengths of lines. We experimented with two possible solutions. The first was to allow a user to write in the letter "c" and have the mode switch to colour. The second option was to put a button on the Toolglass for each mode, and for the user to click on their desired mode. The first method was not very successful in our informal user tests during development. We theorise that because the interface did not reveal its functionality to the users, it would represent the same challenge as learning a command-line language. The second method was preferred, and this is the method we left in the present interface.

4.7      Digital Tape Drawing

The digital tape tool was added to test out its usefulness and usability as a new two-handed drawing technique. Specifically, we were curious how well the digital tape tool would work on desktop systems, and for non tape-artists. The possible benefits of a tape tool are as follows:

•      Easier than tangents and control vertices (CVs).

•      One tool provides functionality for straight lines and curves

•      Switching between these two modes is seamless

•      Curves produced are much "cleaner" than those produced by freehand drawing with a stylus or mouse

•      Installed base of users. While there are few artists familiar with the tape drawing techniques, those that use them in their design work would find benefits in using an interaction style similar to what they are used to.

These perceived benefits convinced us to add the tool to T4 so we could evaluate it.

Figure 4‑14: An older version of the digital tape tool, with three cursors.

The triangular cursor is the "pivot", and in this version, it is moved with the left-hand cursor but constrained to the line between its current position and the right hand.

 

The digital tape tool went through many iterations before reaching the state is at now. The initial algorithm was adapted from the prototype system described by Balakrishnan, Fitzmaurice, Kurtenbach, and Buxton (1999). As shown in Figure 4‑14, there were initially three cursors the user needed to manipulate. There were some problems when moving the tape tool from the Power Wall to the desktop. First of all, user behaviour was slightly different on the desktop, so a bug in the original code that was never activated on the Power Wall revealed itself. According to user expectations of the digital tape tool, if the left-hand was not moving, then the tape should not be laid down. However, as shown in Figure 4‑14, moving the right hand without moving the left could result in a curve in some situations. This was an artefact of using three cursors. The "pivot" point was a point tangential to the left hand, along a line set by the previous position of the pivot and the position of the right hand cursor.

The updated algorithm constantly offsets the third cursor so that it lies along the line defined by the anchor and the dominant hand. This results in the simplified interface shown in Figure 4‑15.

Figure 4‑15: The digital tape tool.

The user is constantly aware of the present state of their curve. This is in contrast to status quo systems such as Alias and Illustrator, shown in Figure 4‑16. In these tools, placing the next point can affect the shape of the curve previously laid down.

Figure 4‑16: Lines and curves being drawn in Adobe Illustrator 7.0 for the Macintosh.

Note the use of tangency lines.

 

The digital tape tool works much like the way digital tape artists work. These artists hold the tape in their right hand, and pin it down using their left thumb. To draw a straight line, they keep their right hand steady while moving their left hand towards the right. To make curves, they moved their right hand while they're moving their left hand towards the right. Similarly, as shown in Figure 4‑17 and Figure 4‑18, the user moves a pivot point, controlled by the left hand, towards a cursor, controlled by the right hand. The method of interaction for curves and straight lines is identical.

Figure 4‑17: The digital tape drawing tool.

As the left cursor moves towards the right cursor, a line is drawn, permitting perfectly straight lines when the right hand does not move, and curves when the right hand does move.

 

Figure 4‑18: This diagram shows the user making a curve.

The left hand sweeps to the right, while the right hand sweeps down, following the arrow.

 

If the user has already pinned down the left-hand cursor, but is not keep pressing the left-hand button, green feedback is given along the line between the pivot and the cursor. This is shown in Figure 4‑19.

Figure 4‑19: When the left button is not held down but the puck moves, a green line provides feedback.

If the user presses the puck button down, a line will be made from the left cursor to the end of the green line, signified by the arrow.

Initially, the pivot point was locked at the point the user clicked through the Toolglass. During informal tests of the digital tape tool, users complained about this, and compared it to the rubber band line tool. Thus, in the current version, the tape is not pinned down until the user presses down the Puck button.

4.8      Ligne Claire Curve Modification

The idea seems compelling because the interaction closely resembles those made when sketching on paper, and it seems to be a natural form of interaction for artists. In addition, the focus remains on the curve itself rather than on tangent lines and other mathematical constructs. This is an example of a "show the computer, don't tell the computer" technique.

The Ligne Claire Toolglass was developed during the first two days of user evaluation at Symbol Technologies. Because of this, it has not gone through much iteration. The interaction is exactly the same as described in the Ligne Claire paper by Thomas Baudel (1994). Essentially, the user simply makes a mark over the curve they would like to edit and the system responds by reforming the curve. This interaction is shown in Figure 4‑20 through Figure 4‑22.

Figure 4‑20: Ligne Claire Toolglass prior to making a mark.

 

Figure 4‑21: Ligne Claire Toolglass with the user making a mark.

 

Figure 4‑22: Outcome after the Ligne Claire mark.

4.9      Slicing Technique For Modifying Shapes

The slicing technique is discussed in the Slicing Techniques, on page 15, and our implementation was a straightforward implementation of this idea. The reason this technique may have value is that it not only should be faster than selecting a point on a line, but it also is useful metaphorically as well. We use this technique in several portions of T4, using the metaphor of an eraser and a razor blade. Figure 4‑23 shows an example of the slicing technique with a razor blade metaphor. The user simply moves the cursor along a path, with the stylus tip depressed. T4 leaves a trail of feedback for the user, and upon button-up, the shapes the trail crosses are sliced into pieces.

 

 

 

Figure 4‑23: Slicing technique.

We use exactly the same technique for the eraser. The stylus has an eraser at the end of its other end, just like a real pencil. If the user turns over their stylus, T4 senses the eraser, and draws an ink trail whenever the eraser tip is depressed. Any object that the ink trail crosses is then deleted.

4.10   A-Line Technique[3]

Computer graphics systems typically address functionality at the atomic task level, such as rotation, scaling, or translation. Less often is interaction structured in a way that reflects why, in the sense of workflow or intent, a particular operation is undertaken.

As an example, the reason that a graphical object is rotated or translated is often in order to align it with some other points on the screen. However, while alignment is the motivation for the transaction, there is typically little in the manipulation tools, themselves, to help. When the object cannot be "eye balled" into alignment, additional tools, such as grids or constraints, must typically be invoked to accomplish the task.

The problem in this is that generally, the more different tools and steps required to perform a task, the more complex it is to accomplish. Hence, the system is harder to learn, more prone to error, less "natural" feeling, and the tasks take longer to perform.

Since alignment is a frequent and recurring task in graphics applications, we have developed a new tool, A-line, whose affordances facilitate manipulating a graphical object such that any two points associated with it line up with any other two points on the display.

The interaction is fluid, utilises both hands, and builds upon skills developed in the everyday physical world.

We shall demonstrate the A-line tool using a number of sequences of figures. These will provide "storyboards" describing key aspects of the interaction and the technique.

Basic Alignment

In this sequence, the objective is to align the rectangle shown in Figure 4‑24 such that the vertices a and b are co-linear with points A and B, which are the centres of the circles on either side of the rectangle. The goal state is shown in Figure 4‑25.

Figure 4‑24: The initial state.

The rectangle is horizontal between the two circles.

Figure 4‑25: The goal state

The rectangle is rotated such that the corners a and b

are co-linear with the circle centres, A and B.

 

The means of getting from the initial state to the goal state are illustrated in the sequence shown in Figure 4‑26 through Figure 4‑30.

Figure 4‑26: Invoking the A-line tool to the square by clicking through the Toolglass.

Figure 4‑26 shows clicking through the A-line tool on the Toolglass sheet. Note that the tool, the cursor and the object to be aligned are at the same location.

Figure 4‑27: Positioning the hands over the points on the object to be aligned, a and b.

When the stylus tip "pushes through" the Toolglass, and as long as pressure is maintained, the Toolglass disappears, as shown in Figure 4‑27. (It makes no sense to select from it in the middle of the transaction, so its presence would simply clutter the screen.) Each hand controls a cursor. For feedback, the cursors are connected by a rubber-band line. They are positioned over the points a and b on the rectangle, which are to be aligned.

Figure 4‑28: Selecting points a and b.

When the cursors are positioned, the puck button is depressed. The effect of this is that the rectangle is now "grabbed" by the two hands, and can be manipulated. As shown in Figure 4‑28, green dots appear at the points grabbed, by way of visual feedback.

Figure 4‑29: Aligning by moving hands to points A and B, respectively.

As shown in Figure 4‑29, as the two hands move towards the target alignment points, the rectangle rotates. Points a and b remain co-linear with the cursors at all times. Notice that during the transaction, "ghost" dots are left at the original positions of a and b. These provide feedback in case, for example, the user wants to revert back to the original position of the rectangle.

Figure 4‑30: Completion on release of the puck button.

When the cursors are at positions A and B, the rectangle is in the goal position. Releasing the puck button and pressure on the stylus completes the transaction, at which the Toolglass reappears, as shown in Figure 4‑30.

The user is now ready to perform the next transaction. The entire transaction has been performed through connected flowing movement that employs motor skills similar to those used daily in the physical world.

Pivoting

Unlike the turntable and snap-dragging techniques discussed in the Literature Review, on page 16, there is not necessarily a single defined pivot point with our technique. This is one artefact of maintaining co-linearity among 4 points, rather than 3.

Another consequence of this is that in many, if not most, cases, there is more than a simple translation affecting the object being manipulated. There is some translation as well. What is important is that the interaction be such that it is smooth, reasonable, and predictable. Surprises and non-linearities are generally not a good thing!

These are things that we will now briefly discuss and illustrate.

First, if one hand is stationary and the other moves, the object pivots around the stationary hand. This is shown in Figure 4‑31 and Figure 4‑32.

Figure 4‑31: Rotation around a pivot which is one of the initial selection points.

Figure 4‑31 shows the simple case which is similar in effect (but less so in method of interaction), to the turntable and snap-dragging. This is because if one of the hands is stationary over one of the initial selection points on the object, there are effectively only 3 points to be aligned, and the pivot point is defined by the two coincident points under the stationary hand. Here, the pivot point is at a, assuming that the left hand is stationary. Point b on the box will rotate around a if the right hand, at position m moves more-or-less along the arc shown in the figure.

Figure 4‑32: Rotation around a pivot which is not part of the object.

Figure 4‑32 demonstrates that if one hand is moving around the other (stationary) hand, then the stationary hand will function as a rotational pivot regardless of any previous hand movement.

(a) L & R hands close

(b) L hand far, R hand close

 

(c ) L hand close, R hand far

 

(d) Both hands far

Figure 4‑33: Hand & Object Movement.

Other than the conditions described in the next section, the box is unaffected,

relative to the adjacent circles, by movement of either hand

co-linear to the initial points selected.

 

The pivot point can be any point where a hand can position a cursor, regardless of where it began. The hand need only be stationary. In this case, assume that the left hand is stationary at point p. If the right hand, at point m, more-or-less follows the illustrated arc, then both points a and b will rotate around the pivot point, p.

Furthermore, since the object to be rotated is selected when the initial tool is invoked via the Toolglass, as in Figure 4‑26, the two selection points, as illustrated in Figure 4‑28, need not be on the object.

Finally, the hands can move independently without causing undue or unexpected movement of the object. This is illustrated in Figure 4‑33. Each of the four examples in the figure show that as long as the motion of either hand is co-linear with the initial two points selected, the rectangle will stay stationary. (There are exceptions to this which will be discussed in the next section.)

Translation

There are conditions when the box will be beyond that which we have seen as part of the rotation. The first condition where this will occur is when both hands move to positions that are not co-linear with the points where the initial selection occurred..

Figure 4‑34: Translation through non co-linear motion by both hands.

Figure 4‑35: Translation by same-direction simultaneous movement of hands.

This is illustrated in Figure 4‑34. In this example, point a on the rectangle was selected at position I, and point b at j. Since the hands have moved out of alignment with ij, the rectangle is, as one would expect, translated up relative to its original position.

Translation also occurs whenever both hands move in the same direction at the same time, even if the motion is co-linear to the initial selection axis.

This is illustrated in Figure 4‑35. Here both hands have moved up and to the right, moving the rectangle along with them.

Correction

Since the box can be translated as part of the transaction, and the hands are not attached to fixed points relative to it, there may be cases where the final position is not what is desired once the alignment is achieved.

The tool has a mechanism to accommodate this situation. When the object is aligned, while still holding down the puck button, the pressure on the stylus can be released. The effect of this is, first, to lock the alignment line in position. Second, the now free stylus can be used to "slide" the object along the alignment line to the desired position.

Figure 4‑36: Correcting object position by constrained dragging.

This is illustrated in Figure 4‑36. Here, the right hand is dragging the box up and down the alignment line, as indicated by the arrow. The overall transaction is completed when the puck button is released.

Other Objects

Finally, rather than adjust the position of the original object being manipulated, as in the previous example, the stylus can be used to adjust the position of any other graphical object on the screen.

In this case, again, as long as the puck button is depressed, the stylus can select any other graphical object and drag it. In this case, since the object may not be on the alignment line, the selected object's movement is constrained such that it moves on a line parallel to the alignment line.

This is illustrated in Figure 4‑37, which shows the right hand circle being dragged parallel to the alignment line, as indicated by the arrow. Again, the transaction is completed when the puck button is released.

Figure 4‑37: Constrained dragging of other objects.

It is misleading to think of the steps of the development of T4 in neat categories like Implementation, Evaluation, and Results. This reflects an engineering-based methodology, which doesn't hold up to current practise in the software field.

There were multiple levels of evaluation for T4. First of all, during the design and implementation, the interface and interaction were shown to users, designers, and researchers. Comments during these sessions influenced the design and the interaction as the program evolved. We relate these design decisions out in the Implementation section of this thesis.

In addition to this, however, we also conducted a week-long user study at an industrial design group in New York. This section provides details on this study.

5.1      Rationale for user study

There are several reasons why an experiment seemed inappropriate for the evaluative portion of this thesis. First of all, experiments are used to measure a very narrow, well-defined task. They produce very good information, but on a limited range of behaviour.

The intention of the research into T4 was to explore a range of techniques and to see how well they contributed to our design goals. This range would be difficult to study in an experiment, and the results would be difficult to use. Thus, a user study seemed to be a natural choice for the evaluative portion of this thesis.

5.2      Methodology

The users studied were all industrial designers by training. We will refer to them as User 1 through User 7 for anonymity.

 

 

 

User	Years as an industrial designer	Years using computers in industrial design	Programs used in design (% of time)
1	1.5	1.5	Macromedia Director 5%, Photoshop 30%, Illustrator 30%, Alias 35%
2	13	12	Alias 10%, Illustrator 20%, Photoshop 30%, Corel 35%, Vellum 5%
3	15	13	Vellum 60%, Illustrator + Photoshop 40%
4	17	14	Illustrator 30%, Photoshop 50%, Vellum 20%
5	6	6	Alias 25%, IDEAS 30%, Vellum 10%, Photoshop 15%, Illustrator 20%
6	15	15	Alias (n/a), Photoshop (n/a)
7	8	8	Alias 70%, SDRC 10%, Illustrator 10%, Photoshop 10%

Table 5‑1: User experience levels and computer usage.

 

As shown in Table 5‑1, the users were very computer-literate, and spent a lot of time on the computer doing designs. About half had learned design using traditional methods, and had switched to design on computer later. However, all of them used computers actively in their designing.

All the users were right-handed, and they all had Palm Pilots (indeed, their group had designed a variant of the Palm Pilot), so they were experts with Graffiti. The only cross-learning effect we noticed was that the Graffiti stroke for '5' was different than the '5' recognised by T4. Otherwise, in comparison to previous users, they merely seemed more comfortable writing in Graffiti and they tended to write faster than previous users.

Many of the users could be considered expert users of Alias|wavefront software, and one user had been using Studio since the first version (15 years!).

We concentrated our study on the first user, because we wanted to see what behaviour an expert-user would have, and see how the different techniques worked for the user. Table 5‑2 shows the amount of time spent with each user, and the number of sessions for each:

User	Introductory Session	First session	Second session	Third session
1	12/16/99 22 min.	12/16/99 65 min.	12/17/99 92 min.	12/22/99 78 min.
2	12/16/99 15 min.	12/20/99 94 min.
3	12/16/99 15 min.	12/21/99 41 min.
4	12/16/99 15 min.	12/21/99 30 min.	12/22/99 10 min.
5	12/16/99 15 min.	12/21/99 82 min.
6		12/22/99 20 min.
7	12/16/99 22 min.	12/23/99 65 min.

Table 5‑2: Subject session length and frequency.

Note that User 1's introductory session and first session were merged.

What was slightly unusual about this user study is that we brought the code and development environment with us so that we could rapidly implement some of the changes the group recommended and get feedback on the changes and improvements.

This is essentially participatory development (Muller 1995), and it combines the evaluative and development stages.

This combination could complicate the interpretation of the user study results, were it not for the fact that we was looking for higher-level, integration type issues in the user study. Also, the style of interaction didn't change at all during the course of the one-week stay there, and the changes were primarily:

•      Minor interface improvements, such as adjusting the colours of a Toolglass.

•      The addition of icons for buttons, and

•      Adding addition functionality they requested, such as the Ligne Claire Toolglass.

A detailed description of the changes made to the program (a version history) appears in Appendix F: Implementation History, on page 114.

5.2.1     Procedure

We first gave a short demo to all of the users, at the beginning of the week. We acquainted them with the program, showing each of the major components, but not explaining in great detail. We also described to them that our purpose in doing this study was to find problems and issues in the design, and asked them to be critical of the program. We told them we were hoping to get as much feedback as possible on the design.

With each individual user, at the beginning of the first session, we repeated this procedure. We explained the basic functionality of the prototype, showed how the Toolglasses and marking menus worked, and asked them to give feedback. We also asked them to think aloud when possible, and during the course of their use of the program, we often asked them questions.

The users were not given a formal set of tasks to accomplish, but we asked them to experiment with the program, and we often asked them to sketch something they were working on, a concept or a design.

If the behaviour of the program was different than they seemed to expect, we asked them what they would expect the program to do, and asked them to imagine how the interaction would work. Just because they are users and making these suggestions does not mean the suggestions are definitively generalisable to all users, but it often served as insight into problems in the interaction.

5.2.2     Data Collection

Sessions with the users were videotaped, and we took notes on critical incidents and comments the user said. The notes have been transcribed in Appendix A: User Study Notes, starting on page 79. We defined critical incidents as incidents in which the user made a "mistake", paused for unusual periods of time, or in any way seemed to be confused or misled by the interface.

Video analysis was done on the videos by watching the tapes and comparing the notes with the videotapes to check for consistency. A videotape is available from us to interested parties.

This section presents an overview of the findings of the user study. We have broken down the results according to the technique the comments apply to, and summarised some higher-level issues at the end. A discussion of the implications and validity of each finding is also presented.

This section summarises the most critical findings about each of the techniques we examined. Implementation-specific, less generalisable, and minor details were relegated to Appendix E: User Study Findings starting on page 100.

6.1      Selecting and Revealing Drawing Attributes

6.1.1     Positioning Is Advantageous

The one observation that many of the users made was that being able to write the desired value for an attribute right where that attribute actually exists in the drawing "made sense". Several of the users cited this as a feature they would like to have put into Vellum, which they said currently has a laborious window-based process for specifying object attributes.

One user commented that this was a technique that was more advantageous for CAD or technical drawings, and less suited for sketching applications. So the advantages and disadvantages of this technique may become clearer if it were studied in that context.

6.1.2     Use As A Selection Tool

There some problems with the users not understanding exactly how the selection mechanism worked. Adding the crosshairs was lauded as helpful by User 1 and condemned as confusing by User 4. As such, the results on this were inconclusive. Overall, though, several users made comments such as, "how does it pick which vertex to select", indicating that the selection method is not transparent to the users.

6.2      Using a Soft Keypad to Change Object Attributes

Prior to the user study, we had concerns that the keypad might be too obtrusive for text entry. Earlier iterations of the program had had a 50% transparent orange coloured background for the keypad, and it had been very visually confusing. In response to this, we had switched the background to 100% transparent (essentially, no background colour at all). The outlines around the keys and the Toolglass were the only indicators that the Toolglass was there at all, and we suspected this design might be more successful than previous versions.

Results were mixed for this aspect of the tool. One user said that he didn't find the keypad obtrusive, while another said that it was difficult to read the text below it, presumably because there were several layers happening at the same time. Both users said that it wasn't very clear where the keypad Toolglass would be modifying. See Use As A Selection Tool on page 62 for more on this issue.

6.3      Using Graffiti to Change Object Attributes

This group has designed a handheld Palm Pilot, and they were all Palm Pilot users. This group was possibly biased towards Graffiti use, so these results could be misleading. Alternately, they could be seen as potential expert users, and the data may be more pertinent, since we are trying to evaluate expert-user performance as well.

6.3.1     Usefulness Of Graffiti Varies By Attribute

None of the users used the Graffiti tool to change the colour of any vertices, and two users made comments like, "if I wanted to change the colour, I would just do it by using the click-through tool. The numbers mean less to me." These comments suggest that some attributes, especially those that are not self-revealing like line length, are more suitable for Graffiti entry than others, such as colour, which are self-revealing, and for which alternate mechanisms for change exist.

6.3.2     Movement Before Finishing

Two of the users uncovered an issue in which they began to enter Graffiti but moved the Toolglass while doing so, thereby selecting another vertex for Graffiti entry. The old entry remains there awaiting the user's return, but the program also gives feedback on the current location. This quickly becomes messy, as shown in Figure 6‑1. The solution is either to clear the entry when a new vertex or edge is selected, or move the Graffiti value with the Toolglass.

Figure 6‑1: Graffiti becomes messy on user movement after entry.

6.3.3     Changing Length Should Change For Circle

One user, changing the length of one segment of a circle, complained that the size of the entire circle didn't change as a result. This is an understandable expectation for the user, and could be easily added in to the program.

6.3.4     Overall

The Graffiti Toolglass was overall very well received. User complained about the recognition accuracy (which in our implementation was not very good), and details of the interaction, but seemed to really like the idea of writing on their drawing. Opinions such as this were representative (User 7): "Graffiti to change values is valuable. It would be a nice addition to Vellum. I want to be able to use this in Alias."

6.4      Digital Tape Drawing

The tool that the designers seemed most interested in was the curve tool. This group spends a lot of their time drawing and manipulating curves, and some of the users spent a lot of their time experimenting with this tool and evaluating it for themselves. As a result, they uncovered a lot of possible issues with the curve tool. Of course, these results and conclusions are tentative, but if they bear out, they have strong implications for the use of this tool on desktop systems.

6.4.1     Left To Right Curve Preference

One of the easiest observations about the user's behaviour was that almost all of the curves and straight lines they drew were from the left hand to the right hand of the screen. The designers, when asked, said that this behaviour was also present in real-world drawing as well. This is fortunate, given the difficulty many of them had with other sorts of curves.

Four of the seven users had difficulties, especially in the beginning, when trying to do any curve that went from the right to the left (see Figure 6‑2). The most common problem was caused by a collision between the two hands. Although it seems this could be resolved by an offset, in reality, unless we design the input so that we completely divide the tablet into a left and right hand region, this problem will continue. We increased the offset between the two hands after first noticing this problem (after the second session of the first user), and the next users still had problems. Unless the offset it such that the hands cannot cross, there is no guarantee that the hands will not bump each other. Even if they do not bump into each other, it is also possible for the arms to interfere with each other as well. This was observed even into the third session for the first user, although it did diminish significantly.

However serious this may sound, adding the ability to easily join two curves together might provide a remedy. Because the curves could not be joined in this version of the program, the users tried to make some strokes that they may have decomposed into several curves had T4 supported it.

Figure 6‑2: Examples of difficult curves with the digital tape.

Although this problem also diminishes with experience, this could be a barrier to

adoption of this tool. If users perceive it as being unwieldy, they may choose to abandon it before they attempt to assault the learning curve.

6.4.2     Some Angles Are More Difficult

Another difficulty two users had, especially when starting out, was with curves that sharply bent more than about 40 degrees as any point. One user made the comment "that's not exactly what I wanted" after these type of curves. Accuracy improved with experience, but this was problematic for some users.

6.4.3     Scale Of Drawings

The scale of the drawings needs to be somewhat large, as doing very detailed drawings in a tight space was problematic for the users that attempted it (only a few did). In general, users attempted to draw at a scale that took up 1/2 to 3/4 of the screen area, but when User 7 tried to do smaller drawings, he complained that he needed more room. T4's dynamic scaling makes this less of a problem, but users often encountered difficulties at first.

Figure 6‑3: Detailed drawings can be difficult for users at first.

Scaling in helps to alleviate this problem.

6.4.4     Tablet Size Restrictive

The sides of the tablets became a problem for four of the seven users. They all started out by bumping the edges of the tablet on almost all of their curves, and gradually they would become more cautious and draw in the middle of the screen. Many of them voiced the desire to be able to continue a curve even after they reached the edge.

There are two reasons why this problem was so noticeable with the users. First of all, as shown by Balakrishnan and Hinckley (1999), user errors increase when the mappings between the visual realm and kinaesthetic realm do not match. This difference was caused intentionally by our use of an offset between the data received from the two hands. This offset separates the two hands, and helps to reduce collisions. Although this practise is customary in two-handed programming with input devices, the curve tool brings out a possible disadvantage to doing this offset. Also, the smaller the input space, the more this problem should occur, so small tablets would be very problematic.

Possible solutions to this problem include tablet overlays that provide reference as to where the tablet-space ends, using two tablets, or putting the screen inside the tablet, so that no mapping problem is present. Several users specifically mentioned this as something that they wanted, regardless.

6.4.5     Confusion Between Modes With Puck Button

In general, the strong point of the T3 paradigm is there are only a few rules about the interface, and once they are understood, the program is consistent. This has the obvious benefit of increasing the speed a user can learn the program, and it also reduces the number of errors.

One of the problems with blending the digital tape tool in with the Toolglass metaphor is that the digital tape tool requires the use of the puck button in order to lay down and remove "tape". This conflicts with the specifications, which leave the puck button only for rotation. One of the users made mistakes with this several times, and seemed confused by the behaviour of the program.

There are two possibilities here. One is that as other two-handed techniques emerge, like the A-Line tool, which require using the puck button for other purposes, the specification will change and the user will also become used to using the puck button for multiple uses. However, another solution is the interface described in the future work section, the On-Demand Toolglass, which offers clear contexts for the usage of the puck button.

The behaviour that beginning users exhibited is that they would hesitate before laying down tape, or not remember how to initiate tape drawing. This was cited as "confusing" by one user, but it was also something that disappeared rapidly, as the user adapted to the system.

6.4.6     Curves Suited To Sketching

These users generally use Alias tools to draw their curves, and are used to that workflow. They saw the curve tool as more of a sketching tool than anything, to be used like Illustrator or Vellum. The interaction for industrial designers is most dependent on the endpoints of the curve, and the tape tool doesn't take that into account as much as the status quo method of drawing curves. One long-term Alias user (15 years!) who was very sceptical of the curve tool was, however, very surprised with its feel, and said he liked it much more than he expected.

At the beginning of the first session, User 1 said that he wanted a little more control of the curves, and when drawing, he was unable to get the shapes he desired. However, by the second session (about 2 hours later), he was drawing sketches of products that were quite good. He said that it was "easy to get [the curves] on screen quickly". User 7 said much the same thing, saying it was "hard to trace a previously laid line" at first, but after using the system for 35 minutes total, said, "it is easy once you get it".

6.4.7     Accuracy With The Curves Improves Quickly With Practice.

The learning curve was generally very short. After about half an hour of using the program (not just curves), the users were drawing demonstrably better, as shown in Figure 6‑4. This was measured by their ability to finish lines off at the end of a looped shape in an accurate position, by their comments made while they draw, and by the amount of erasing they did.

Figure 6‑4: User 2 was able to draw pictures of this complexity after 90 minutes.

It is likely that having the erase functionality built into the curve tool eases the transition to expert user behaviour, because the transaction cost of erasing and trying again is so small.

6.5      Ligne Claire Toolglass

During the evaluation of the Ligne Claire Toolglass, the first user ran into a problem when he attempted to make Ligne Claire marks near a shape. Since their mark actually started on the shape, T4 interpreted this as a selection of the shape, and they would move the shape instead of changing its curvature. This was quickly fixed, but raises an important issue with modes in T4. See Mode Switching, on page 102 for more information on this issue.

The ability to make Ligne Claire marks was added the second day of the evaluation, and generally, the response was positive, and often surprised ("that's cool!" was a common reaction). As industrial designers, they eventually require much more precise curves, but it seemed useful to them as a "sketch tool". One user compared the difference between editing curves traditionally and with the Ligne Claire tool as, "being like the difference between using a marionette and sculpting".

It seems that integrating Ligne Claire marking in Toolglasses is fairly seamless, and can be added without much difficulty (it only took a few hours to add in the code for this). However, to flesh out the interaction and improve the algorithm will probably require more work. Thomas Baudel's Ph.D. thesis (Baudel 1995) and paper (Baudel 1994) will be insightful for further development. Unfortunately, we were not aware of this work when we added it into the T4 code-base.

There are a few issues that are unresolved by the user feedback. Users were undecided on how far the curve should move towards the new mark. Initially, User 1 suggested having the curve shift all the way to the new mark, but later, he wondered if it might be better to only go halfway. Even after implementing the code both ways, he wasn't sure which one he preferred.

6.6      Slicing Technique

Users provided a lot of feedback on the Vertex Manipulation Toolglass, which allows the users to cut lines, and add or subtract vertices using the slicing technique. In general, this tool was used a lot by users, and they quickly became proficient with it. The most common problem with this Toolglass, as noted in the Toolglass section (on page 102), was that users tried to click through the buttons that were used to select which mode they were in. However, once they understood the mechanism for using this Toolglass, they used this tool a great deal. User 1 was cutting lines smoothly and confidently by the third session, and User 4, who used this tool a great deal, was cutting curves and end points confidently within his first session. The tool that they used the most was the line slicing tool, which, with a mark, divides a line in two. Their biggest complaint with this tool, however, was that there's no difference between the feedback for the cutting tool and the vertex addition tool. Some sort of feedback should be provided in future versions of the program.

Another item to study would be to compare using the marks in the current system with a rubber band held using both cursors. One user suggested this technique as a way of taking advantage of both hands.

6.7      A-Line Technique

The A-Line tool was developed after the user evaluation at Symbol Technologies, so no formal evaluation of A-Line has been done. However, a few aspects of it stand out as worthy of mention, and we can state some expectations of what a user study would result in.

First of all, the A-Line technique is similar to the digital tape tool in that it requires the puck button to carry out rotation. Because this is the same at the other transformation functionality of the puck button, there is little cognitive dissonance, but this similarity is important to note, especially if we consider the On-Demand Toolglass. In the present interface, we might expect a few errors from confusion between the two uses of the puck button. However, until a formal user study or experiment is done, this is all just speculation.

 

 

7.1      On-Demand Toolglasses

One of the sessions with User 1 resulted in a brainstorming session on potential directions for T4. Two users suggested having round Toolglasses, and eventually, we worked out a potential refinement of the T4 paradigm. In this section, we present an overview of what this might look like.

First of all, this paradigm is marked by On-Demand Toolglasses. The Toolglasses are only present when the user needs them; otherwise, they are not presented to the user. This has several advantages. First, because many designers use curves so predominantly, we optimise access to this tool. One of the problems with the digital tape curve tool was that the puck button was used both for laying down curves and for translation. By making the Toolglass appear only on demand, the user is able to continuously draw out curves with the two-handed drawing technique. It is thus advantageous for drawing multiple curves or doing multiple alignments with the A-Line tool. A possible disadvantage is that the paradigm has states. However, the user should be able to decide whether they want On-Demand Toolglasses – the options don't necessarily have to be mutually exclusive. Also, the user could click somewhere to make the On-Demand Toolglass behave like normal Toolglasses.

 

Figure 7‑1: Recent version of Toolglass

The first item to note about the On-Demand Toolglass is the difference in form from the T4 prototype. Comparing a recent version of T4's standard Toolglass (shown in Figure 7‑1) with a concept sketch of the On-Demand Toolglass (shown in Figure 7‑2), T4's Toolglasses are squared, angular, and high-contrast, with black lines around each button. The On-Demand Toolglass is an attempt to become less visually "active", by reducing the hard edges, which draw attention, and reducing the contrast, by removing the black lines around the buttons. Another difference is that the marking menu bar is on the top of the T4's Toolglass, where the On-Demand Toolglass holds its marking menu in its centre.

Figure 7‑2: Concept sketch of a new Toolglass

Let's describe a usage scenario, which will highlight both how the paradigm works, and its advantages and disadvantages. The user starts out as in Figure 7‑3. The only elements on the screen are the two cursors. In this initial mode, the interaction can be digital-tape drawing, so the user will make curves for a while, without having to switch between using the Toolglass and not.

 

Figure 7‑3: The two cursors in the default mode.

In this mode, the cursors act like digital-tape tools.

 

When the user decides they would like to do something different, or switch to another mode, they move the right hand cursor over the left cursor. At this point, the Toolglass fades in, to appear as in Figure 7‑4. Other possible forms of feedback are possible, such as leaving the Toolglass ghosted, or leaving the outline of the Toolglass present.

Figure 7‑4: On-Demand Toolglass

When the user moves the red cursor (dominant hand) over the blue

circular cursor (non-dominant hand), the Toolglass pops up,

hence the term On-Demand Toolglass.

 

At this point, operation is identical to the regular Toolglass, except that it is possible to switch into another mode, such as the A-Line tool. The advantage here is that the user can continue to make operations, without worrying about dispelling and conjuring the Toolglass over and over.

The marking menus are also optimised for smooth and efficient transitions between the tools and Toolglasses. As shown in Figure 7‑5, the marking menu is in the centre, which is also the home position that the cursor starts in. This not only makes the marking menu an efficient operation, but it makes the buttons on the Toolglass take on kinaesthetic, marking-menu like properties. The user may want to pick the button to the left of the Toolglass, for example, and after repetition, may come to perform this operation automatically.

Figure 7‑5: Sketch of On-Demand Toolglass's marking menus.

7.2      Other Future Directions

There are many fertile areas of for future research in this vein of HCI research. First of all, the most obvious extension of this work is into 3D, which will raise new questions about the generalisability of these techniques, and perhaps will bring out some new techniques as well.

Another obvious line of extension is on algorithms for doing these techniques on real mathematical curves.

Finally, elements of this paradigm should be tested in an experimental setting, to help develop a solid, quantitative understanding of these techniques.

Integration design like this is similar to creating a new recipe from scratch. It requires trying a lot of possibilities and seeing which work best. Thus, another area of research is to continue adding new techniques that seem compatible with the current interaction paradigm, and which further our design goals.

In this thesis, we have described the design of a two-handed drawing prototype, which integrates various techniques, new and old. Our aim with this prototype was to explore how well these techniques integrated, and how well they furthered our design goals. We conclude with a summary of how well each technique contributed to our three goals:

•      Keep the users' attention on their work (Attention)

•      Maximise screen used for artwork (Screen Use)

•      Increase the quality of input (Quality of Input)

All of these techniques are compared with their status quo counterparts, and evaluates as to whether they improved in this goal area versus the status quo. Whenever there was any doubts, the status quo method was always given preference. See Appendix G: Design Goal Evaluation Details, on page 122, for more details about the way in which these scores were derived.

Techniques	Attention	Screen Use*	Quality of Input
Selection Magic Lens	yes		yes
Keypad To Specify Attributes	results vary		yes
Graffiti To Specify Attributes	yes	yes	yes
Digital-Tape Drawing	yes		yes
Ligne Claire Curve Editing	yes		yes
Slicing Technique			yes
A-Line technique (alignment)	yes		yes

Table 8‑1: Evaluation of techniques against design goals.

Note than actually all the techniques contribute to Screen Use because

extraneous interface elements are hidden. "Yes" indicates that the

technique improved in this goal area versus the status quo.

 

As shown in Table 8‑1, all the techniques increased the quality of input, and the Toolglass based design of T4 almost guarantees that the screen be wholly devoted to artwork. In addition, most of the techniques increase the user's attention on the artwork, rather than doing the opposite.

All of these techniques thus contribute to provide a compelling alternative to current GUI interaction styles.

 

 

 

 

 

This appendix contains my notes from observations during the user study. I've transcribed these notes and added explanatory comments. Thoughts we had while observing the subject are in italics and don't represent anything the user said. [Anything in brackets represents later comments.]

9.1      User 1, Session 1, 12/16/99

•      Wanted outlines around graphic objects. These are separate from the wireframe capability already present. He wanted, for example, the ability to have a solid red box with a blue outline.

•      He wanted to be able to snap an object to a grid.

•      He wanted to be able to snap a vertex to a grid, and have vertices merge and shapes join.

•      Wanted undo

•      Accidentally erased more of the curve than he wanted when using the digital tape tool.

•      Ran into the edge of the tablet when drawing a curve. This happened repeatedly. We discussed ways this could be improved, and what he would expect the program to do. We came up with these possibilities: scale out and continue drawing, translate over and continue drawing, realize that the user is at the edge and remember they are in curve mode and let them continue. Another possibility is to rely on a larger tablet.

•      He wanted to put down templates, or have a template layer.

•      He wanted to do free-hand sketching, something like a blue-pencil sketch, over which a more proper drawing could be made.

•      Had the idea of copying and pasting by "sucking up" a shape and plopping it back down. This is almost exactly like the idea in the original Toolglass video.

•      Wanted real NURBS curves instead of vertices.

•      Had the idea of sketching a line close to another line and having the line reform to that shape. [It turns out this is very similar to Thomas Baudel's Ligne Claire idea.]

•      Noticed the lag as the number of curves on the screen increased.

•      "Erase is a bit weird." He later said erasing would be okay if the eraser tip worked.

•      "I really like this. Scaling and moving around is really nice." Most impressed with this.

•      I asked him if the keypad was obtrusive [I had had reports is was with earlier iterations of the prototype]. He said it wasn't obtrusive.

•      Suggested putting a little dot in the centre of the Toolglass, as he wasn't sure where the Toolglass was going to take effect. Cross-hairs is another possibility.

•      Take out colour – transparency [unsure what this meant]

•      He wanted to look at specific colours, and didn't think their RGB values were important. Thus, he thought the Graffiti mode for colour was unnecessary.

•      The icons for the Graffiti modes in the Toolglasses were incomplete, he pointed out (P, C, and C)

•      If the Graffiti entry is not complete and the user moves the Toolglass away, he said the Graffiti entries should be cleared instead of left on the screen.

•      Wanted to remove the Toolglass instantly (perhaps with a button). He said it might be sufficient for the Toolglass sizing to be faster when using the thumbwheel on the puck.

•      Wanted feedback that he had selected something. [I had taken it out, so I put it back in.]

•      Wanted a method of specifying proportional and non-proportional scaling. Suggested perhaps using 1 hand for one, and 2 hands for the other.

•      Kept hitting shapes. He had difficulty adjusting to the click-through idea, even when he understood it consciously. He kept clicking a single time on the Toolglass, instead of holding the button down and clicking through it.

•      Not erasing

•      Suggested switching the position of the curve tool and the line tool, because he felt like he was starting in a backwards position with the curve tool. He pointed out it would be opposite for left-handed people.

•      He wanted smoother lines, so I should reduce the simplification of the curves

9.2      User 1, Session 2, 12/17/99

•      In the left-handed tool , switch the location of the curve and straight line tools.

•      Problem: select vertex tool conflicts with the more like that curves.

•      Noticed a bug when pressing that the delete template button . This sketch would become invisible for future draws .

•      Accidentally deleted the template.

•      With the curve tool, when the user deletes all the way back, should unpin the curve tool.

•      Drawing to the left with the curve tool is awkward, the hand smashed together. Most drawing is done to the opposite anyway, but it is still difficult.

•      Add and delete all.

•      (move delete tool back into the main Toolglass)

•      Accidentally colored just the vertex instead of the entire circle.

•      (simplified M L T curve before using it. ) to say

•      Grouping ideas: markets across in the same way that we cut lines, draw a circle around it.

•      Line weights are more important than colors on the main Toolglass. Move colors off of the main Toolglass?

•      Marking menu--put one on the bottom to?

•      (the angle he is holding the puck is less than 45 degrees)

•      Embed the straight line tool in the curve tool (skip forward)

•      Templates: twirl with paper, after deletion, hide template should be false.

•      Threshold for rotation?

•      If vertices are on top of each other, then vertices should merge.

•      Draw a circle to connect shapes.

•      Toolglass: Centre in lower left corner?

9.3      User 1, Session 3, 12/22/99

•      Rotate shape instead of canvas when placing object.

•      Camera manipulations moved, using both hands simultaneously.

•      Rotation done through glass-> get rid of Toolglass quickly.

•      Zooming-thinks he can't start with a Toolglass there. (inside/outside issue)

•      Fairly smooth use of cutting.

•      More like this curve-try moving towards the line instead. Take a smoother version of the curve.

•      Turn pick off for split.

•      Alternative to more like this curves-nudges, smudging. The longer this smudge , the broader the effect.

•      No difference in feedback between the add rare vertex and cut line.

•      Curve tool is not going to all the way back.

•      Curve tool-hand crossing problem

•      Likes the cross hairs.

•      Cross-hairs-move them somewhere (maybe up?)

•      Cursors should start at the middle of each tool.

•      In between paper and Vellum ,.

•      Sketch/precision mode?

•      Not running off the tablet as much as before.

•      As screen fills up, zooming, rotation screws up from accidental selection of objects-> rotate object instead of canvas on accident.

•      Problem with learning curve: confusing at first. Like going between Maya and Studio.

•      This could displace Illustrator and Vellum in my world.

•      Exploring other ways of doing this is great. Eventually want to have more control of curves.

•      Especially like more like this curve.

•      This is a new way of doing things, different than CVs and control vertices.

•      Refine these tools more, promising.

•      These ideas-> surfaces ?

•      More like this curve is like the difference between marionette and sculpture.

•      Thumbwheel-too much resistance. Free wheel better?

•      Toolglass -be able to be larger, allows larger strokes.

•      When moving points-leave vertices.maybe also leave Toolglass up.

•      Might look better to Move buttons from the top to the left hand side.

•      Aesthetic issues: hard black line-maybe take it off.

•      Look at softening Toolglass

•      Drop shadow below?

•      Remove corners?

•      Buttons with rounded edges.

•      Not lines but fields of colors.

•      Colors too dominant ?

•      Move colors to side?

•      Circle's-perfect circle?

•      Deletion is messy.

•      Thinks that purple color on Graffiti Toolglass is too dark .

•      We talked about the idea of a circular Toolglass, either on the outside where the inside having a marking menu.

•      On demand Toolglass idea-like marking menu in Alias

•      Hates the Puck.

9.4      User 2, Session 1, 12/20/99

•      Collision problem with curve tool.

•      With curved tool, pick up puck and put it back down?

•      Took several clicks to select lines .

•      Wanted to feedback on the color tool: perhaps the tool can provide feedback by coloring the shape?

•      Shakes and jitters from interference with the monitor. Filter input?

•      The sketch should move with the drawing .

•      Click through errors.

•      Mouse movements.

•      Move the modes for vertex type operations so there operable within each Toolglass.

•      Tried to click inside line objects

•      Marking menu anywhere?

•      Sketch tool draw anywhere.

•      Tate tool right to left doesn't work.

•      Picking up stylus during curve drawing is problematic. Should be OK! Currently, it goes to scaling.

•      Likes zooming , panning

•      Puck click while drawing out the shape should rotate the shape?

•      Rubber band for erasing /cutting on line-tool

•      Need feedback on the curve being cut.

•      "I feel like I'm learning this really fast. "

•      Puck is an obstacle.

•      Want shearing ability/mirroring.

•      Joining and grouping.

9.5      User 3, Session 1, 12/20/99

•      " X" on the template tool looks like the deletion tool.

•      (angle of Toolglass less than 45 degrees)

•      Perfect circle?

•      Has difficulty getting used to stylus button in tip

•      Difficulty erasing.

•      Likes the freedom to move.

•      Curves too rough.

•      Confused by feedback on Graffiti tool, second line.

•      Suggested a plus and to a square as a icon for the vertex addition tool

•      Being confined to drawing the sketches with and the Toolglass is consistent.

•      Draw objects from their centre, or mark inside circle.

•      Impression: "Accuracy most difficult" therefore, it seems like a sketch tool. It is pretty easy to get things on the screen quickly.

•      Want to snap to.

9.6      User 4, Session 1, 12/21/99

•      Same click through problems

•      Erasing in selecting with looped shapes problematic.

•      Spin during creation when puck button is down.

•      Circle: changing the length should recalculate the circle's size

•      Finds stylus hard to use.

9.7      User 4, Session 2 , 12/22/99

•      Is initiating creation of objects only at the location he wants them. User notices this, and that it is not necessary

•      Tries to click on vertices to remove them.

•      Cross -hairs are confusing.

•      To relax, wants to be able to use one hand sometimes (note: User 3 said the same thing)

9.8      User 5, Session 1, 12/21/99

•      Centre points

•      How about having an oval Toolglass? Very aware of the box.

•      Comic-book borders off.

•      Rotation on Puck: beneficial or distracting?

•      Curve tool: crossing hands is a problem.

•      Keypad: difficult to see text.

•      Keypad: highlight numbers as you go over them.

•      Keypad: click vertex to select and then enter new value.

•      Move towards having everything the curves/organic, not rectilinear.

•      Catching up with the curve tool-stops it from proceeding.

•      Curve: needs to be designed for this rubber ridge or feelers to stop or warn you.

•      Smith curves are very important, not smooth enough.

•      Likes the cut/slice/add vertices tool.

•      Double click on didn't marking a new-make Toolglass the whole screen.

•      X on the sketching tool-looks the same as delete (make red?)

•      Use colours to separate tool functionality.

•      Wants to make Toolglass disappear or be ubiquitous sometimes.

•      More like this curves: pull curves with a perpendicular stroke?

•      When the stylus is lifted, should translate without rotation.

•      Had difficulty with the curve tool-was confusing with these transformation functionality

•      Centre point should be where you start from.

•      (not used to clicking through)

•      "Could fill the surface--gooey, like putty, more life ... many things happening at a time. "

•      Hard to trace with curve tool

•      Primary hand do more?

•      Toolglass only comes when necessary, don't always need it. Want to pick a tool and use it.

•      Toolglass limited to a section of the screen.

•      Movement seems more dynamic

9.9      User 6, Session 1 , 12/22/99

•      Likes the optimised workspace

•      Sceptical about curve tool before using it.

•      Liked curve tool better than he thought.

•      Two handed trying feels odd

•      Wants to rest while drawing with his other hand

9.10   User 7, Session 1, 12/23/99

•      Snap to vertex.

•      Switching units.

•      Wants unions and subtractions (Boolean operations)

•      Wants the monitor embedded in the tablet

•      Stylus doesn't feel good.

•      Feedback is bad.

•      Curve-initially difficult to place (wanted to click)

•      Screen ending problem with curve

•      Colour vertices through colour but can't see the vertices.

•      Vellum -Graffiti to change values is valuable. Would be a nice addition.

•      A nice feature in Vellum is an x , y offset

•      Awkward to do only with and Toolglass? Maybe.

•      Toolglass-button not click through.

•      Snapped tool

•      Key point-to changed to curve

•      icons need work

•      Like to be able to join points and cut out shapes.

•      Rotating fairly intuitive

•      Rotate around locked vertex if you select it.

•      Curve-once you get it, it's really easy (35 Minutes)

•      Running off screen annoys me

•      Expects to do movement the other way (right hand-> left hand, tried to)

•      Should be able to turn to solid, oh , I can

•      Accidentally scaled object instead of the entire page

•      Undo?

•      Moving vertices is "Fun "

•      having three buttons makes him expect it to work like Alias.

•      While trying to move a vertex he accidentally cut the shape

•      Bug: when he clicked on a two vertex line with the fill tool, it disappeared.

•      Curve-difficult to do fine work, have to have room.

•      Curve-problem if you move too far with left-hand (overtakes right).

•      Grid visible all over the screen, may be snap only in Toolglass

•      Grid make it more prominent

•      Curve-automatically scroll if you reach the end of the tablet

•      Curve-likes the tool, easy to use.

•      Copy-duplicate 20 times with offset

•      Rotate doesn't feel right

•      Floating menu idea is great

•      Layers-active/inactive visible/invisible

•      Feel of both hands great

•      Wants to be able to use Graffiti

•      Wants on functional screen with alphanumeric keypad is it

•      Having delete button in multiple Toolglasses would be convenient

•      Likes marking menu

•      Likes pushing through it

•      Wants hotkeys ... or a different layer

•      Virtual keyboard-try again dropped icons to keys to set up hotkeys

•      Drag-> solids

•      What would you put in Alias? Transparency, click through, need hotkeys, Graffiti

10.1   OpenGL

OpenGL was chosen as the graphics library for T4, for historical reasons (T3 was written in GL, the precursor to OpenGL), for performance reasons, and also because the research group at Alias|wavefront had more experience with OpenGL. The version of OpenGL was 1.2, the version that comes standard on Windows NT 4.

10.2   Wacom Tablet

The Wacom Intuous tablet supports two-handed input, via any two supported devices, be it a puck, a stylus, or an airbrush. Our set-up uses a puck and a stylus.

The Wacom tablets support many degrees of freedom. The stylus detects position (x and y), tilt, rotation, and pressure. It has a button that can be depressed by the forefinger or thumb. The puck can detect position (x and y) and rotation. The puck has a self-centring thumb-wheel and five buttons. Both the stylus and the puck work as absolute as opposed to relative input devices. Thus, each position on the tablet maps to exactly one point on the screen.

Figure 10‑1: Wacom tablet, with a puck and stylus, the setup used for T4

Figure 10‑2: Puck for the Wacom tablet

Wintab is a standard library used to interface with any tablet on Microsoft Windows. On top of this, we acquired some skeletal code that acted as a wrapper between Wintab and T4. This wrapper code was originally written at Alias|wavefront by Christie Johnson. However, it was written to explore the abilities of the new Intuous tablets, and as such was not production-level code. It was on this foundation, however, that T4 was built.

10.3   Windows NT

The code was written for Windows NT, version 4.0. Specifically, it was run on machines with service packs 3 and 5 installed. Part of the reason we chose to develop on Windows NT was that the research group at Alias|wavefront had laptops that ran Windows NT, and if the program were written on Windows NT, the program could be demoed easily.

10.4   Visual C++ 6 and MFC

The development environment significantly affected the architecture of T4. The program was written as a Microsoft Foundation Classes project, with a Document-View structure. Although MFC was probably not necessary for the program, and could be easily stripped from the program to increase performance, it was used.

This appendix contains additional peripheral design issues not related to the integration of the new techniques.

11.1   Drawing Space

T4's background, or canvas, is of practically limitless size, and can be scaled, rotated, and translated with impunity. This is in many ways similar to the Pad++ system (Bederson, Stead, and Hollan 1994) in that the scaling is dynamic, smooth, and unlimited.

 

 

 

 

 

 

 

Figure 11‑1: Scaling in T4

11.2   Colouring

Early in the design process, we chose a specific colour (purple) to represent selection. This colour is used whenever objects are created, whenever the user selects an object, and to specify vertices. The colour is actually 50% transparent, which has some added benefits. First of all, it allows the user to understand the context in which the shape is being placed. In the same way that Zhai, Buxton, and Milgram's silk cursor used occlusion and semi-transparency to show what is in front of an object and what is in back of an object (1996), making the object temporarily transparent gives the user more information so that they can place or manipulate objects more easily.

For similar reasons, we chose to use the colour green to specify marks or strokes. Thus, Graffiti marks and marking menu marks are both done in the colour green.

An interesting side note about the design of the marking menus, is that we initially made the marking menus semi-transparent, but found that this left the focus on that Toolglass, when actually, the marking menu is another layer above Toolglass. While experimenting with the transparency levels, we found that opaque marking menus were much more visible and attracted the focus of the user. When the user is an "search mode", they only want to focus on the marking menu. For these reasons, we decided to implement the marking menus as opaque rather than transparent colours.

11.3   Input Decisions

In the initial implementation of T4, we tried to take advantage of the tilt of the stylus to rotate objects. Unfortunately, it seems that the stylus does not afford this type of rotation. This is partially because of the way the stylus is held. Areas where this rotation might be useful would be sculpting or air brush applications.

Instead, we used the same rotation technique as the original T3 prototype.

If the stylus is not down, the two-handed input techniques for rotation, sizing, and so on do not work. We handle this by having different behaviour when only the puck is down. We do translation and rotation based on the rotation and translation of the puck, and not using a second hand.

12.1   Ligne Claire Algorithm

Here Ligne Claire is referred to as a Like-This Curve or a More-Like-This Curve, to distinguish between the technique which re-forms the curve to the new line and the technique which moves the line closer to the new mark.

 

 

 

 

 

Figure 12‑1: Like-This or More-Like-This Curve algorithm.

Algorithm:

1.     Find closest shape – we do this by finding the object's vertex closest to the stylus cursor button-down position. Whatever that shape belongs to is the one we choose. We experimented with a best of three approach, with the beginning and ending position, and a midpoint, but this wasn't as accurate (the users often selected the wrong shape).

2.     Find the first point (where P intersects curve C). Save this point as "first point".

3.     Find the last point (where Q intersects curve C). Save this point as the "last point".

4.     Go through curve C, from first point to last.

5.     At each vertex, find the distance "d" to the MLT curve, and the point of intersection.

6.     Either move the vertex to the point of intersection, or move it a portion of the way towards the intersection, depending on whether we want the MLT curve to define a new line or pull towards a new line.

12.2   A-Line Technique

The basic algorithms to support the interaction are fairly straightforward. Our focus in this prototype was on optimizing the user interaction rather than the efficiency of the algorithms. Clearly, the transformations could be done more elegantly, but these are intended to convey a picture of one way it could be done.

Decide which object is being aligned – This is decided by the user when they click through the A-Line button on the Toolglass. The object to be edited is either the one they click on, or the closest one to their click-through.

Color object for feedback – We color it transparent purple, so the user can also see the context of the objects underneath or adjacent to the selected object.

On puck button-click – Store the location of both the cursors in reference points. Make a copy of these values as well, so we can know both the original location and the location of the reference points as they are transformed with the object.

Check for translation – if both cursors are moving in the same direction, translate the object and reference points. If one is moving faster than the other, then if the direction is collinear to the reference points, take the lesser value. If the direction is not collinear, use that amount for rotation, as described below.

Keep points collinear – Track both cursors and rotate/translate the reference points during each iteration to keep the points collinear. Perform these same transformations on the selected shape. The mechanism we followed was as follows:

Determine the distance CA and DB, as shown in Figure 12‑2.

The point of rotation is somewhere in between the two reference points. We choose the location of P by looking at the distances CA and DB, and applying that same ratio inside the reference points. See Figure 12‑2.

Rotate both the points and the object around point P, and then translate to the line AB.

 

 

 

Figure 12‑2: A and B represent the new locations of the cursors, and C and D the previous positions. We want to move our object to a position between A and B.

 

 

 

 

Figure 12‑3: We rotate around pivot point P to match the slope of the two cursors, A and B.

 

 

 

 

Figure 12‑4: We then translate along the vector from P to the line AB.

Provide feedback for points – Display the original reference points (we chose gray) in their original location, and the transformed reference points on top of the object.

Leave constraints on for adjacent objects – When the stylus tip is no longer depressed, we enter a new mode. This can be bypassed if the user lets up on the puck button before deselecting the object.

First, we change the feedback so that the line between the cursors is now an infinite line defined by the reference points.

Second, the reference points no longer move. We store the position of the stylus cursor at its point of intersection along this line, A (see Figure 12‑5). This is constantly updated. We can track how far to move any object along the line parallel to the constraint line by comparing the current value of A with the last value, A¹.

 

 

 

 

Figure 12‑5: The cursor is mapped to the constraint line.

The user can now select objects (which are then highlighted) by pressing the stylus tip down when the cursor is over any object. Objects selected are moved according to the vector defined by A and A¹ along the line defined by the reference points. The user can select and deselect multiple objects, and position them.

Return system to normal state – When the puck button comes up, the action is finished.

13.1   Digital Tape Drawing

13.1.1 Finishing Point Sometimes Wasn't Accurate

About half the users had initial difficulties with closing their curves. This was a surprising find, because the dominant hand strictly defines the endpoint of the curve or line. However, because it also must be moved in the process of creating the curve, users apparently require a little time before they understand where the curve will end up. This was one of the problems that disappeared most rapidly, in perhaps half an hour or less.

13.1.2 Erasing Problems

Several users made comments that they liked being able to pick up and redraw their curves quickly. However, they also sometimes accidentally erased parts of their curves, so a more careful erase algorithm or better feedback is probably a good idea, as shown in Figure 13‑1.

 

 

 

Figure 13‑1: Narrowing the area in which cursor motion is interpreted as erase

may decrease erase problems.

 

The problem was particularly acute because the algorithm would sometimes erase entire portions of their drawings when they didn't expect it. This happened when users would draw a tight curve that looped back on itself.

13.1.3 They Draw Almost Exclusively Curves

This may not be the case for all industrial design groups, but for this group, they rarely draw straight lines. The curve tool and the circles were the most useful to them, and the other tools less helpful. In fact, User 1 suggested delegating these tools to a less accessible Toolglass and instead putting other functionality there.

13.1.4 Quality Of Curves Extremely Important

Three of the users explicitly made comments about the quality of the curves. Because this prototype uses lists of vertices, the quality is smooth but still noticeably inferior to mathematically based curves. This is just an artefact of the prototype nature of T4.

13.1.5 Pinning Problems

Two of the users commented on the fact that the digital tape tool doesn't unpin itself even when you erase the entire line. They wanted to return to the initial state, in which the two cursors move freely. The first user said he was not sure that it would be the best way to do it, but he suggested trying and seeing if it "feels better".

13.1.6 Pick Up Stylus While Drawing

If the user picks up the stylus in the middle of laying down tape, T4 changes into the translation, scaling, and rotation mode. This is both a result of assigning multiple functionality to the puck button, and a result of not explicitly handling this case. Two of the users triggered this bug.

13.1.7 Overtaking Problem

Another problem two users triggered was an overtaking problem. The tape tool draws tape while the non-dominant hand moves towards the dominant hand. If the user's non-dominant hand overtakes the dominant hand, nothing is drawn until the dominant hand moves an equivalent amount. This might be acceptable if it matched the feedback on the screen or it was explicitly made obvious. Some sort of feedback should be given to improve this aspect of the interface.

13.2   Toolglasses

13.2.1 Mode Switching

Four of the users made some sort of mentioned about the size of the Toolglass. Generally, they wanted to move as quickly and as smoothly between using the Toolglass and not using the Toolglass. The first user, in particular, mentioned this frequently. Although he said he liked the Toolglass, he wanted to size faster, wanted it to be able to get larger or smaller than had been permitted previously, and sometimes, wanted it to disappear entirely. In the last case, he stressed that he wanted to the Toolglass to disappear and reappear instantly, like a marking menu. User 4 echoed the same comments. What User 7 mentioned that it might be difficult to restrain drawing to only within the Toolglass.

Thus, even though the users cited the Toolglasses as one of their favourite features of the program, they found it sometimes got in the way. To generalise this a bit, the problem was that different sorts of behaviour occur within the Toolglass and outside the Toolglass. The users wanted to switch rapidly between these modes, and even though Toolglasses are very quick for this, they wanted it even faster.

User 1, the user who spent the most time with the program, was particularly affected by this, and commented throughout his three sessions that he would like the Toolglass to appear and disappear quicker. We increased the speed at which the Toolglass sized itself, and in the third session, he seemed less concerned with this aspect of the interface, and made different comments about it, such as the fact that he wanted the sizes to be less constrained. This change was added, but after the user study was completed, so it has not been verified by other users.

Another problem, that was highlighted by one version of the Ligne Claire Toolglass, is that sometimes it is not obvious to the user what behaviour is acceptable within the Toolglass and outside of it. For example, with the Ligne Claire Toolglass, initially it was possible to select an object and move it around. This was fixed after users accidentally selected and moved objects while trying to make Ligne Claire marks, but it still didn't solve the problem of revealed functionality. For example, on the Line Editing Toolglass (which lets the user cut line, add vertices, and remove vertices), it makes more sense workflow-wise if the user can select a line or vertex and move it even from within the Toolglass. We previously did not allow this, and it was cumbersome to move back and forth between Toolglasses, or move the Toolglass out of the way, if we wanted to move an object we had just cut in half, for example. However, even if this is useful workflow-wise, it still needs to be implemented more elegantly, perhaps, because several users accidentally cut lines or added vertices to them, when their intention was to select that line. The On-Demand Toolglass, as mentioned earlier, may offer a possible solution to this problem.

Users offered several suggestions to deal with the mode switching issue. One user said he would like to be able to double-click on the marking menu area to have the whole screen act like the inside of the Toolglass (essentially turning the whole screen into a Toolglass). One of the research team members suggested later that this could be done by extending the grid feedback present inside the Toolglass to cover the whole screen, and then to leave the marking menu bar visible.

Most of the users had a least one incident of Toolglass mode errors. These errors also included errors of perception, such as the comment that "I can't rotate or zoom when the cursor is overlapping the Toolglass." This was not in fact true, but the perception of it was indicative of a conceptual problem or lack of communication in the interface. User 7 said it was "awkward to only act within the Toolglass, but maybe I'll get used to it"

13.2.2 Different Types Of Functionality

Another difficulty the users had was in distinguishing between the different forms of functionality in the Toolglasses. One example is demonstrated on the splitting Toolglass, shown on Figure 13‑2:

Figure 13‑2: Users tried to click through the button (labelled A)

instead of seeing that as a mode, and swiping through the shape.

 

Three of the users tried to click through the button labelled 'A' to add vertices to a line, or split the line. This was understandable, because this is the behaviour they performed in default Toolglass in order to do anything. The confusion lies from the fact that there are several different classes of buttons for T4, and the designs have few visual clues to distinguish these differences to the users. These three classes are as follows:

•      Primitives, such as squares and circles.

•      Click-through tools, such as deletion, copy, and the A-line Tool.

•      Modal buttons that change the state of the program.

In order to make the design cleaner, we should use colour and grouping to consistently display how the different areas of the Toolglass interact with the rest of the environment. We attempted to implement this when we added in the functionality for the A-Line tool.

Figure 13‑3: Toolglass without any distinction between classes of buttons

Figure 13‑4: Revised Toolglass, separating functionality by group and shading

and optimising layout for right-handed interaction

 

Clearly, it is much more evident in Figure 13‑4 than Figure 13‑3 that the deletion and A-Line tools are a separate class of buttons. This needs to be done throughout all of the Toolglass buttons, especially with the moded buttons. Other possibilities include using "3D" or raised buttons for moded buttons, or using other shapes for the mode buttons. Also, it may be useful to always have them located in a particular location on the Toolglass, so there are spatial cues as well.

13.2.3 Click-Through Problems

One of the biggest problems many of the users had when first starting out was in using the Toolglass as a click-through tool (for some this persisted for a while, others had zero problems). Three of the users tried to create a box by clicking once on the box button-area. This was exacerbated by my own inconsistencies as well, because there are a few "mode-switch" buttons on the Toolglasses that are not click-through tools. We think there needs to be a clear, graphical distinction between the two, using colours, visual grouping, and so on to show these differences. Otherwise, though, the click-through problem is just a legacy of using traditional GUI systems.

13.2.4 New Shape In Toolglass / Aesthetics And Functionality

Two users suggested having no straight lines in the Toolglass. User 4 said: "it's too boxy. The squares and lines capture your attention." They suggested a round Toolglass, or oval, and having sculpted buttons. The marking menu could be at the centre or around the edges. The centre seems compelling, because it ties into some other ideas we developed for how the curve tool could be better integrated with the Toolglass, and also at the same time the Toolglass could be an on-demand tool.

Another thing that users suggested was removing the "comic book borders" around the Toolglasses. Two separate users suggested this. The rationale behind this was that the comic book borders increased the contrast on the Toolglasses. Although this is what we desired when we added the comic-book borders, two users suggested decreasing the amount of visual attention that the Toolglass requires.

13.2.5 Object Creation

Users of programs such as Adobe Illustrator are used to having the ability to draw objects from the centre. For example, when drawing a circle, they often start the drawing in centre drawing mode. Although T4 makes this mode unnecessary because the drawing is done with both hands, several users mentioned that they wanted to know where the centre of the circle or other object was. This could be added to easily by putting a dot or circle in the middle of the object being drawn.

In addition, three users mentioned wanting to be able to make perfectly proportional circles and other shapes. This could be done in several ways. First, there could be a mode selected by a button on the Toolglass that would constrain all new objects to be perfectly proportioned. A second alternative would be to use a separate button on the puck. Another alternative would be to provide feedback when the shape was a perfect proportion. The second alternative seems unpalatable because it starts to increase the cognitive complexity of the system. Unless these types of operations are very common, the second alternative is probably not a good idea. The third alternative may be the most intuitive, but testing would be required to determine whether it was actually successful and intuitive for users.

13.3   Marking Menus

Because the users were fairly familiar with Alias products, they were all familiar with marking menus. Thus, they didn't comment on the marking menus very much. Users 7 said that he liked the marking menus, and User 1, the user that spent the most time with the program, compared them to hot keys, and wanted even faster access to his menus. He suggested trying, for example, adding a marking menu to the bottom of the Toolglass. User 2 suggested making the marking menus available all the time, by making them pop up from any point on the screen.

13.4   Sketching And Templates

Initially, when we added the sketching and template ability to the program (after the first session by the first user), we constrained the templates so that they did not rotate, scale, and translate with the rest of the objects. The next two users of the program quickly pointed out this deficiency, and said that the template layer should move in line with the rest of the objects on the screen. We quickly changed this, and the users make no further mention of it. Except for the first user, who said he "liked the addition", users did not comment on it very much at all. This may be because this functionality are status quo technologies, so they weren't any different. User 2 said he would like to be able to draw anywhere. This is related to Toolglass modes (see "Different Types Of Functionality" on page 104).

13.5   Line Width

When we installed the T4 software on the computer used for the user study at Symbol Technologies, the line widths were now working correctly, so we were not able to evaluate this aspect of the interface.

13.6   Tablet / Tools

13.6.1 Puck Size Too Large

One item of feedback that was slightly surprising was that two users said that they thought the puck felt too large for their hand. One of these users even wanted a stylus or joystick in their left hand. This user explained that the puck didn't feel very precise. This is especially surprising given Guiard's kinematic chain model.

13.6.2 Size Of Tablet

The size of the tablet was a huge problem. Part of the issue was the screen cursor didn't exactly match the tablet size, because we used offsets so the left and right hands didn't collide. This is the problem of having kinaesthetic feedback not match visual feedback. See (Balakrishnan and Hinckley 1999). Two users said they would like a monitor in the tablet.

13.6.3 Relative Vs. Absolute Mode

This is expected, but the transition from relative/mouse mode to absolute/tablet mode took time for two of the users. They kept lifting their puck and putting it down again, expecting to have the Toolglass move in a relative fashion.

13.7   Task

13.7.1 Sketch Or CAD

T4 has a number of different types of tools, and we think is a little schizophrenic in that it has both sketch and CAD-like functionality. Perhaps this is what is interesting about it, but we think that in the future, either path should be chosen.

User 1 session3: "it seems to be positioned between paper and Vellum"

13.7.2 Curves Are Most Important

For the industrial designers, as we mentioned previously, curves are everything. Circles are the most important primitive, with the line, triangle, and square being used less often. Shearing a shape is useful, and the deformations in T4 seem very welcome (selecting a vertex and moving it, scaling one shape, cutting a shape, etc)

13.7.3 Sketching

For this group, if it's 2D, it is for sketching or concepts. It is then a form of communication. Depending on the designer, the 2d drawings are done on paper (most common) or with Illustrator or Vellum, and shown to people... not used to go into modelling. User 1 said it would displace Illustrator or Vellum in his world.

13.8   Two-Handed Interaction Issues

The users also provided some feedback about two-handed input. It is difficult to determine how useful this data is, because these users are specialised in using one handed systems. Three users mentioned that they wanted to be able to relax with one hand and use the other. These users demonstrated, either putting their chin on the left hand, or leaning back and resting their left hand in the lap. However, at the same time many of these users also mentioned that they liked working with two hands. Users 7, for example, said, "the feel with both hands is great!" Sometimes, the first few minutes were a little uncomfortable while users adapted to the system. Users 6, for example, said, "it feels odd." Several users mentioned that they would prefer working on an Active Desk, where the table contains a screen embedded in it.

13.9   Overall

13.9.1 Feedback Important

Feedback and the obviousness of the functionality seem very important. Some of the Toolglasses have no explanation of their functionality, and sometimes the buttons could benefit greatly from the skills of a graphic artist or interface designer (or even just more attention from me!). we think more thought needs to be given to how the functionality of a Toolglass is revealed.

13.9.2 No Selection Mechanism A Possible Drawback

One of the weaknesses of the current T4 is it has no way to specify multiple shapes or vertices. There is no concept of a selection, except what you click on. Users all wanted to be able to do grouping, joining vertices, and so on. This doesn't have to be done in a traditional way -- several users presented some interesting ideas for new ways to select multiple objects.

13.9.3 Movement, Rotation, Scaling Are Big Wins

One of the largest advantages of the T4 system, according to users, is the fluid movement supported by the system. Most of the users commented on how much they liked this aspect of the interaction. A typical response was as follows: "I could feel the surface. It is gooey, like putty, more live. However, there are also a lot of things happening at one time."

The users wanted to have the ability to translate without rotating or scaling their objects. We added in this ability by letting the user translate without rotation or scaling if the stylus was lifted up. Thus, the puck controlled the translation of the document, just like the non-dominant hand controls the paper while drawing in the real world.

There are some differences between the original T3 prototype and the T4 implementation we've described. It might be worth experimenting using the T3's original interaction. There are some trade-offs. For example, T3 required using two buttons, one for each and. T4 requires using a stylus, and it is uncertain if pressing the left hand button and the stylus tip simultaneously will work as an interaction technique.

13.9.4 Learning

The first subject spent a lot of time with the program (4.5 hours), and his skill level was the same or better than my own at that point. His learning curve started to plateau sometime around the 2 or 3 hour mark.

I need to go back to the videotapes and double-check this, especially with some sort of quantification, but it seemed to me that users were exhibiting expert-level behaviour after about 2 hours. By this we mean they were: 1) using the marking menus without visual search, 2) were using the curve tool and making comments that indicated the drawings were what they were trying to draw, 3) were moving both hands simultaneously, 4) they would initiate drawing new objects from anywhere on the screen.

13.10   Feature Requests

The following list compiles some often requested features.

•      Need snap to grid – 2 users. maybe only snap within Toolglass or outside

•      Connect vertices to each other – 3 users. Either move on top of each other, or circle to join them

•      Grouping

•      Copy/paste like in Toolglasses video

•      Shearing and non-proportional scaling – 2 users

•      Mirroring

•      Unions/subtractions (Boolean ops)

•      Layers (visible/invisible)

•      Fixed small display problems with sketch tool (12/22/99)

•      Put standard Toolglass in display list, and made a left-handed variation of the tool that puts the curve tool on the left for lefties, reflecting feedback for righties from subject #1 (12/22/99)

•      Fixed bug: If we create a line and then go to template mode, the shape is duplicated. (12/22/99)

•      Picking up stylus + moving puck with buttondown now results in translation only (12/22/99)

•      Changed puck rotation to 30 degrees from 45 degrees (12/21/99)

•      Disabled eraser (12/21/99)

•      Added in AddVertex and Subtract vertex icons and SplitLine (12/21/99)

•      Changed colour on selected buttons on TG to green (12/21/99)

•      Changed delete templates button to X. (12/21/99)

•      Put lines around colour cubes, and thickened the selection colour (12/21/99)

•      MLTcurve now works in both directions (12/21/99)

•      Put in cross-hairs in middle of Toolglass (12/21/99)

•      Can now select looped objects from their centres (12/21/99)

•      Fixed a bug where after scaling while drawing, their cursor would go to Toolglass instead of arrow (12/21/99)

•      Increased speed of thumbwheel (12/21/99)

•      Moved delete tool back to main Toolglass (12/21/99)

•      Fixed bug where templates weren't displayed after deletion of templates (they were hidden on subsequent draws) (12/21/99)

•      Muted colours on TG in colour row (12/21/99)

•      Increased offset between cursors (12/21/99)

•      Changed colour of vertex selection tool (12/21/99)

•      Made templates rotate, size, etc.. with drawing (12/21/99) but also made them not selectable.

•      Made vertices and edges of templates not be displayed (12/21/99)

•      Took out the colour white (it was toggling templates) (12/21/99)

•      Added in Like This curve tool (12/17/99)

•      Added in a primitive template ability (sh 12/16/99)

•      Switched the curve tool over to the right hand side of the Toolglass (Sh 12/16/99)

•      Fixed a bug where the cursor wasn't reverting back to the right cursor. (12/16/99)

•      Turned back on purple highlighting when a shape is selected (Sh 12/16/99)

•      Lots of under-the-hood cleaning, changed the way the cursors are kept track of, which simplifies the management of cursors (12/10/99)

•      Cleaned up the code a bit, commented in some of the initWintab stuff (12/9/99)

•      After cutting a shape, you can now move it around (12/9/99)

•      Fixed a bug where splicing wasn't working for looped shapes. It now does. (12/7/99)

•      The curve tool now erases (12/7/99)

•      Finally(!) fixed the curve tool. I completely rewrote the code for this, and now we don't need a separate

•      Cursor for the ndh position. The feedback works much better now, and we also no longer have the curves being

•      Drawn in spirals or arcs when we move the dh cursor. It finally works! Next is to add in the eraser for it.

•      I also added in feedback which shows how far the line will be if you aren't holding down the ndh button. (12/6/99)

•      Fixed a bug in the curve tool when the user clicks but doesn't hold down ndh button. (12/6/99)

•      Added in a Toolglass that lets you move vertices through the Toolglass (12/3/99)

•      Made the backgrounds of many of the tools into a grid instead of just lines (12/3/99)

•      Took out special code for bHasEdges, so the speed of the program should go down a bit..

•      Added in Paulo's changes to wintabdevice, as recommended by Wacom, which speed up the speed of the program a lot

•      We're now at about 800-850 vertices. (12/2/99)

•      I added in a stroke erase for the eraser. It works much better now (11/29/99)

•      I took out the delete tool, and moved the opacity tool from the edge of the Toolglass to the place delete was at previously. (11/29/99)

•      I changed circles and curves so they don't have edges. This gave me a two-fold increase (700-750 vertex ceiling for rotation, scaling, etc) I also added in a T key, which displays how many vertices there are. (11/29/99)

•      Wow, just put in an optimisation that will increase the performance of move, rotate, and scale operations from O(shapes*vertices) -> O(shapes). That is (worse than n^2) to n, and it makes a huge difference. Unfortunately, I now need to squash a few minor bugs (11/27/99) Okay, this was a dead end. I would have to do this for rotation and scaling too, and these become complicated because rotation and scaling can be done around different points, so I have to composite these transformations. I am leaving in the code for the offsets, because it is more efficient for moves, but I may take it out eventually, because it complicates the code unnecessarily.

•      Changed the interface quite a bit, made the keypad completely transparent, put comic book lines around everything to increase contrast. (11/27/99)

•      Graffiti has been fixed (it wasn't working since I flipped the y coordinates on 10Nov) (11/26/99)

•      Added in the ability to add vertices and remove vertices instead of just slicing and separating vertices (11/26/99) 90 min

•      Added in ability to move to front, move forward, move backward, and move to back objects. This is for layering, and the functionality is finished, but the UI needs work (11/26/99) 20 min

•      You can now switch which hand is dominant using the 'H' key (11/26/99) 10 minutes

•      Removed curve split tool (it was redundant) (11/21/99)

•      Allow moving vertices while examining values (11/18/99)

•      We now show line widths at edges if possible (11/18/99)

•      Now recalculate edges after moving manually (11/18/99)

•      Fixed text display for shapes (verts, edges). I found a bug in the vert class code and fixed it. Now when the properties of an object are displayed, it will show only the information for the closest vertex. This can be problematic if you want more contextual information, but it is much easier to do, and I'll look at other possibilities later. (11/18/99)

•      Fixed a strange bug with the curve tool, where the first or last vertex wouldn't display. In the process, I also increased the speed of the curve tool. Now the curve won't seem to shift when the user finishes it. (11/16/99)

•      Can now select and move individual vertices. Can also colour individual vertices using colour tools. Very cool, but now have a few bugs to clean up! (11/15/99)

•      made selection easier (11/15/99)

•      Optimised and globalised the find closest edge and vertex functions. (11/15/99)

•      I've improved the curve tools a bit (very minor) (11/12/99)

•      The cutting tool now works without bugs as far as I know (11/12/99). It's slick!

•      The cutting tool now works. I need to decide whether or not to put in the concave slicer or not. (11/11/99)

•      Extensive under-the-hood overhaul. I got rid of all the different types of Shape objects and made Shape

•      A generic object. This is much cleaner, and makes more sense once we start dealing with modified objects (11/10/99)

•      We now can cut through lines. Need to debug this still, but it finally works! (11/9/99)

•      Exporting now closes paths correctly (11/4/99)

•      We now have primitive file export ability (to Studio, in illustrator format) (11/2/99)

•      Fixed two major bugs in the release version of the software (10/29/99)

•      Put in a hard-coded offset to deal with unmapped tablets. (10/28/99)

•      The curve tool now floats free before it is pinned. (10/26/99) Still need to fix feedback, but this already improves things a great deal.

•      Added in feedback for default line thickness(10/26/99)

•      Added in feedback for changing default colours (10/26/99)

•      Clicking through a line thickness when there is nothing there changes the default line thickness (10/26/99)

•      Clicking through a colour when there is nothing under it results in that being the default colour (10/26/99)

•      Fixed a bug where copying objects didn't copy filled attributes (wireframes would become not wireframes) (10/26/99)

•      Can now toggle objects between solid and wireframe (10/26/99)

•      Added in new Toolglass for splitting lines. With Paulo, figured out algorithm for using concave strokes to specify a splice AND which part to keep. Started adding in code for all this (99/10/21)

•      Added in line widths. Line widths are now displayed, and there is a new line width Toolglass (10/19/99)

•      Added a much more robust marking menu. This was necessary in order to support up to eight Toolglasses, but it also just works better. (10/19/99)

•      Fixed the rotation of pinned objects. It now feels MUCH better. Now, rotation is done only on pinned object when ndh button is down. (10/18/99)

•      Added in wireframe ability to drawings (10/15/99)

•      Added in pin rotation for objects held in the DH. Thus, the user can move everything but the object pinned down. (10/15/99)

•      Added in support for eraser. It now deletes objects (10/14/99)

•      Fixed the copy operation. When you punch through the copy tool, you now hold on to the new object that is created instead of the old one. This makes layering work in a reasonable way (10/8/99)

•      Added in anti-aliasing for lines and points. It looks much better! I would do it for polygons as well, but I get strange effects when I combine that with the blending/transparency used for the Toolglasses (10/8/99)

•      Changes all the input to doubles and added support for non-integer input from the tablet. This looks smoother (10/7/99)

•      Fixed a bug in the Graffiti which didn't allow 0 values for new colours (10/7/99)

•      Added in F command to toggle between full screen and not. (9/30/99)

•      I've finished adding in the keypad, and it works now. (9/28/99)

•      If dh is not on screen, then translate, don't scale, etc... (9/28/99)

•      make hand cursors (9/23/99)

•      switched interaction on triangle (opposite hands) (a while ago)

•      Graffiti Toolglass buttons now leave the Toolglass up (9/14/99)

•      Fixed flickering bug in display of Graffiti Toolglass (9/14/99)

•      Moved all the drawing code to the view class, where it belongs. (9/13/99)

•      Fixed this about a month ago: "Crashes if you attempt to do Graffiti and there are no objects that can be modified." (9/9/99)

•      Added in purple feedback when moving objects (9/9/99)

•      The marking menus are real now, also cleaned up Toolglass display function (9/8/99)

•      Added in a debug function to make the window smaller (9/7/99)

•      Added in a keypad. Need to add it in functionally now (9/7/99)

•      A lot of changes and clean-up to the display of Toolglass section (9/3/99)

•      Show only the currently selected values for curves and circles and show them big for everyone (9/3/99)

•      Changed colours so that Graffiti commas would preserve current colour in that digit (9/2/99)

•      Fixed some minor display bugs with property display (9/2/99)

•      Added more informative labels to the properties labels (c->colours, p->positions, l->lengths) (9/2/99)

•      Added gridlines in Graffiti mode to test orientation (9/2/99)

•      Added labels to Graffiti buttons on Toolglass (9/2/99)

•      The Toolglass stays up while writing Graffiti (9/2/99)

•      The Curve shape acts different than all the other shapes in that its Y values are flipped. This screws up everything unless you subclass every drawing method and overwrite it. I instead simplified this scheme by creating a call that returns the * or Y values for a particular vertex. This is subclasses by Curve, which solves our problem much more elegantly than before, and incidentally fixes a bug I was having with the Curve. (9/1/99)

•      Can specify colours and positions for circles and curves now, although they don't display their co-ordinates/values

•      Now restricts changing Graffiti modes to when you're not already changing something. Also, while changing a vertex or edge, the Graffiti will be displayed even if the Toolglass is not above the area. Much better from UI perspective (8/26/99)b

•      Feedback on what's closest to you, both edge and vertex info. A HUGE improvement! (8/26/99)

•      You can now use Graffiti to change vertex colours and positions (8/24/99)

•      The stencil buffer now works. This means that we can now look through a Toolglass to see the properties information. (8/19/99)

•      Changed buttons for copy/split (8/19/99)

•      Added in delete to copy Toolglass (8/19/99)

•      Added ability to generate new Graffiti strokes. The process is: go to T3View::OnKeyUp -> 'L', uncomment or add in new letter, run program, press L key, rename newstrokes.stk -> strokes.stk

•      I haven't tested this completely. (8/19/99)

•      Added in a split function for curves. It's a little crude but it works well enough. (8/18/99)

•      Added in a copy operation. Needs UI work. (8/16/99)

•      Added in "backspace" for Graffiti (8/13/99)

•      There is some sort of a layering bug with the text and the other shapes. If the text was the last thing created, it seems to occupy the whole plane. The next deletion, even if not over it, causes it to be deleted. (7/29) I'm not creating text for the user any more, so this isn't a problem

•      After I put in rotation, there is a new bug where if you go to the properties Toolglass and back, you get dramatic slowdowns in refresh rates on the screen. (7/30) This was a result of something with the text library. It seems that not displaying slows everything down. I added in a line to display a period offscreen, and it sped up considerably! (8/13/99)

•      Graffiti is working much better now. I still need to experiment with the interaction, but it's decent at least. (8/13/99)

•      Graffiti execution (changing the edge lengths) us close to working now (8/10/99)

•      I restricted Graffiti drawing to start within the Toolglass. As a side effect, this fixed the bug where the fake marking menu recognised a down stroke as a 1. (8/9/99)

•      Translation/panning now works (8/9/99)

•      Several bugs with the tape drawing tool, moving, rotating, etc fixed (8/6/99)

•      Fixed a selection+rotation bug and the interaction for the tape tool. I created a simplify function which reduces the number of vertices for the tape tool drawing, increasing speed for later operations (8/5/99)

•      I've added in the tape drawing tool (8/4/99)

•      Scaling works

•      Rotation finally works. It was actually easy to fix, but I kept putting off doing it. Anyway, it seems to work perfectly. (7/30/99)

•      Optimised code used while stretching out new objects. It was creating and deleting objects during every screen refresh. Unfortunately, the difference in speed is not noticeable. (7/29/99)

•      Put in fake marking menus in the Toolglass menu bar. They don't actually do anything, but they seem to. (7/28/99)

In this section, we give a description of the method used to determine the values in Table 8‑1. First of all, we evaluated each column according as follows:

•      Keep the users' attention on their work (Attention) – We evaluated this based on an attention-based analysis of the technique. This analysis was done informally, but in a method similar to the Keystroke Level Model (Card, Moran, and Newell 1980). An example of this analysis is as follows:

We compare the Ligne Claire curve editing method with the status quo method of manipulating control vertices. In the Ligne Claire method, the user focuses on the curve they want to change. They then draw a line where they want the curve to reform itself to. There is essentially no change of attention during the entire transaction.

Figure 15‑1: Manipulating tangency lines in Adobe Illustrator (2000)

With the status quo method, shown in Figure 15‑1, the user focuses on the curve they would like to change. They choose the vertex that they think will influence the curve, click on that vertex, and then the tangency lines and control vertices appear. They shift their attention to these CVs, select the appropriate CVs, and manipulate the curve, shifting their focus back to the curve in the process. The attention shift is minimal, but still more than Ligne Claire, so Ligne Claire performs better in this category.

•      Maximise screen used for artwork (Screen Use) – This category is easier to evaluate, because here we look at the amount of screen being taken up by the interface instead of the artwork. The main complication here is that transparency allows for layered interfaces. When in doubt, we gave the benefit of the doubt to the status quo technique.

•      Increase the quality of input (Quality of Input) – This was based on subjective comments by users, and speed increases predicted by a Keystroke Level Analysis. The analogy here is a comparison between the piano and the computer keyboard. Clearly, both are similar, and exercise similar skills, but the piano has a much greater quality of input, so much so that it takes years of practise for a "user" to perform at an expert level on it.

The techniques were compared against their status quo counterparts. These counterparts are as follows:

•      Selection Magic Lens – compared to clicking on the object to select it, and pulling up a properties window to view the properties of the shape.

•      Keypad to Specify Attributes – compared to moving from the mouse to the keyboard, typing the new values into a dialog box, and moving back to the mouse again.

•      Graffiti to Specify Attributes – compared to moving from the mouse to the keyboard, typing the new values into a dialog box, and moving back to the mouse again.

•      Digital-Tape Drawing – compared to laying down vertices and pulling tangent lines during curve creation, and either switching the hand to the keyboard and pressing backspace, or switching tools and explicitly deleting portions of the curve for going back on a curve already drawn.

•      Ligne Claire Curve Editing – compared to manipulating tangency lines and CVs.

•      Slicing Technique – compared to manually clicking on the shape to perform the same action.

•      A-Line Technique – compared to using pivot-based rotation, and scaling.

Adobe (2000). http://www.adobe.com, Adobe Systems.

Ashlar Inc. (2000). http://www.ashlar.com

Balakrishnan, R., G. Fitzmaurice, G. Kurtenbach, W. Buxton (1999). Digital Tape Drawing. User Interface Software and Technology.

Balakrishnan, R. and K. Hinckley (1999). The Role of Kinesthetic Reference Frames in Two-Handed Input Performance. User Interface Software and Technology.

Baudel, T. (1994). A Mark-Based Interaction Paradigm for Free-Hand Drawing. User Interface Software and Technology.

Baudel, T. (1995). Aspects Morphologiques de l'Interaction Humain-Ordinateur: ƒtude de Modles d'Interaction Gestuels. Paris, UniversitŽ de Paris-Sud: 209.

Bederson, B., L. Stead, and J. D. Hollan (1994). Pad++: Advances in Multiscale Interfaces. Proceedings of CHI '94.

Bier, E. A., M. C. Stone, K. Fishkin, W. Buxton, T. Baudel (1994). A Taxonomy of See-Through Tools. Proceedings of CHI '94, 358-364.

Bier, E. A., M. C. Stone, K. Pier, W. Buxton, Tony D. DeRose (1994). Toolglass and Magic Lenses the See-through Interface. Proceedings of SIGGRAPH '93: 73-80.

Buxton, W. (1986). Chunking and Phrasing and the Design of Human-Computer Dialogues. Proceedings of the IFIP World Computer Congress, Dublin, Ireland, 475-480.

Buxton, W. (1990). A Three-State Model of Graphical Input. In D. Diaper et al. (Eds), Human-Computer Interaction - INTERACT '90. Amsterdam: Elsevier Science Publishers B.V. (North-Holland), 449-456.

Buxton, W. (1995). Two-Handed UI Definition, Alias | wavefront: 1-9.

Buxton, W. and B. A. Myers (1986). A Study in Two-Handed Input. Proceedings of CHI '86.

Buxton, W., R. Sniderman, W. Reeves, S. Patel, R. Baecker (1979). The Evolution of the SSSP Score Editing Tools. Computer Music Journal 3 ( 4), 14-25.

Card, S., T. Moran, and A. Newell (1980). The Keystroke Level Model for User Performance Time with Interactive Systems, Communications of the ACM, 23(7), 396-410.

Evans, K. P. Tanner, M. Wein (1981). Tablet-Based Valuators that Provide One, Two, Or Three Degrees of Freedom. Computer Graphics (Proceedings of SIGGRAPH '81), ACM.

Goldberg, D. and A. Goodisman (1991). Stylus User Interfaces for Manipulating Text. Proceedings of the Fourth ACM SIGGRAPH Symposium on User Interface Technology (UIST'91), 127 - 135.

Guiard, Y. (1987). Asymmetric Division of Labor in Human Skilled Bimanual Action. Journal of Motor Behavior 19(4): 486-517.

Guiard, Y. and T. Ferrand (1995). Asymmetry in Bimanual Skills. Manual Asymmetries in Motor Performance. D. Elliott. Boca Raton FL USA, CRC Press.

Harrison, B. L., H. Ishii, K. J. Vicente, W. Buxton (1995). Transparent Layered User Interfaces: An Evaluation of a Display Design to Enhance Focused and Divided Attention. Proceedings of CHI '95.

Igarashi, T., S. Matuoka, H. Tanaka (1999). Teddy: A Sketching Interface for 3D Freeform Design. Computer Graphics (Proceedings of SIGGRAPH '99), ACM.

Kabbash, P., W. Buxton, A. Sellen. (1994). Two-Handed Input in a Compound Task. Proceedings of CHI '94.

Krueger, M. W. (1991). Artificial Reality II. Reading, Mass., Addison-Wesley Publishing Company.

Kurtenbach, G. and W. Buxton (1991). Issues in Combining Marking and Direct Manipulation Techniques. User Interface Software Technology.

Kurtenbach, G. and W. Buxton (1993). The Limits of Expert Performance Using Hierarchic Marking Menus. Proceedings of CHI '93.

Kurtenbach, G., G. Fitzmaurice, T. Baudel, W. Buxton (1997). The Design and Evaluation of a GUI Paradigm based on Tablets, Two-hands, and Transparency. Proceedings of CHI '97.

Leganchuk, A. (1996). Manual and Cognitive Benefits of Bimanual Manipulation: Two Experimental Studies. Department of Computer Science. Toronto, University of Toronto.

MacKenzie, I. S., and L. Chang (1999). A Performance Comparison of Two Handwriting Recognizers. Interacting with Computers, 11, 283-297.

MacKenzie, I. S., B. Nonnecke, S. Riddersma, C. McQueen, M. Meltz (1994). Alphanumeric Entry on Pen-Based Computers. International Journal of Human-Computer Studies, 41, 775-792.

MacKenzie, I. S. and S. Zhang (1997). The Immediate Usability of Graffiti. Proceedings of Graphics Interface '97, 129-137. Toronto: Canadian Information Processing Society.

Muller, M. J. (1995). Diversity and Depth in Participatory Design: Working With A Mosaic Of Users And Other Stakeholders In The Software Development Lifecycle. Conference Companion On Human Factors in Computing Systems. 385-6.

Sachs, E., A. Roberts, D. Stoop (1990). 3-Draw: a tool for designing 3D shapes, IEEE Computer Graphics and Applications, Nov. 1991, 18-26.

Sachs, E., D. Stoop, A. Roberts, (1989). 3-Draw: a three dimensional computer aided design tool, Proceedings of the 1989 IEEE International Conference on Systems, Man and Cybernetics, 1194-1196.

Sutherland, I. (1963). SKETCHPAD: A Man-Machine Graphical Communication System. Spring Joint Computer Conference.

Zeleznik, R., K. Herndon, J. Hughes (1996). Sketch: An Interface for Sketching 3D Scenes. Computer Graphics (Proceedings of SIGGRAPH '96), 163-170.

Zhai, S., W. Buxton, P. Milgram (1996). The Partial-Occlusion Effect: Utilizing Semitransparency in 3D Human-Computer Interaction. ACM Transactions on Computer-Human Interaction 3(3): 254-284.