Eyewear displays allow users to interact with virtual content displayed over real-world vision, in active situations like standing and walking. Pointing techniques for eyewear displays have been proposed, but their social acceptability, efficiency, and situation awareness remain to be assessed. Using a novel street-walking simulator, we conducted an empirical study of target acquisition while standing and walking under different levels of street crowdedness. We evaluated three phone-based eyewear pointing techniques: indirect touch on a touchscreen, and two in-air techniques using relative device rotations around forward and a downward axes. Direct touch on a phone, without eyewear, was used as a control condition. Results showed that indirect touch was the most efficient and socially acceptable technique, and that in-air pointing was inefficient when walking. Interestingly, the eyewear displays did not improve situation awareness compared to the control condition. We discuss implications for eyewear interaction design.

Document de travail utilisé dans la rédaction de l’annexe H “Historique et méthodologie” de la norme AFNOR Z 71-300 : “Dispositions de clavier bureautique français”.

Document de travail utilisé dans la rédaction de l'annexe F “Élaboration de la disposition AZERTY modernisée” de la norme AFNOR Z 71-300 : “Dispositions de clavier bureautique français”.

Despite the ubiquity of touch-based input and the availability of increasingly computationally powerful touchscreen devices, there has been comparatively little work on enhancing basic canonical gestures such as swipe-to-pan and pinch-to-zoom. In this paper, we introduce transient pan and zoom, i.e. pan and zoom manipulation gestures that temporarily alter the view and can be rapidly undone. Leveraging typical touchscreen support for additional contact points, we design our transient gestures such that they co-exist with traditional pan and zoom interaction. We show that our transient pan-and-zoom reduces repetition in multi-level navigation and facilitates rapid movement between document states. We conclude with a discussion of user feedback, and directions for future research.

Large displays are becoming commonplace at work, at home, or in public areas. However, interaction at a distance -- anything greater than arms-length -- remains cumbersome, restricts simultaneous use, and requires specific hardware augmentations of the display: touch layers, cameras, or dedicated input devices. Yet a rapidly increasing number of people carry smartphones and smartwatches, devices with rich input capabilities that can easily be used as input devices to control interactive systems. We contribute (1) the results of a survey on possession and use of smart devices, and (2) the results of a controlled experiment comparing seven distal pointing techniques on phone or watch, one- and two-handed, and using different input channels and mappings. Our results favor using a smartphone as a trackpad, but also explore performance tradeoffs that can inform the choice and design of distal pointing techniques for different contexts of use.

In 1997, Accot and Zhai presented seminal work analyzing the temporal cost and instantaneous speed profiles associated with movement along constrained paths. Their work posited and validated the steering law, which described the relationship between path constraint, path length and the temporal cost of path traversal using a computer input device (e.g. a mouse). In this paper, we argue that the steering law fails to correctly model constrained paths of varying, arbitrary curvature, propose a new form of the law that accommodates these curved paths, and empirically validate our model.

A well-designed control-to-display (CD) gain function can improve pointing performance with an indirect pointing device such as a trackpad. However, the design of gain functions has been challenging and mostly based on trial and error. AutoGain is an unobtrusive method to obtain a gain function for an indirect pointing device in contexts where cursor trajectories can be tracked. It gradually improves pointing efficiency by using a novel submovement-level tracking+optimization technique. In a study, we show that AutoGain can produce gain functions with performance comparable to commercial designs in less than a half hour of active use. This is attributable to reductions in aiming error (undershooting/overshooting) for each submovement. Our second study shows that AutoGain can be used to obtain gain functions for emerging input devices (here, a Leap Motion controller) for which no good gain function may exist yet. Finally, we discuss deployment in a real interactive system.

Advances in markerless and un-instrumented hand tracking allow us to make full use of the hands' dexterity for interaction with computers. However, the biomechanics of hand movements remain to be thoroughly studied in HCI. The large number of degrees of freedom of the hand (25) presents us with a huge design space of possible gestures, which is hard to fully explore with traditional methods like elicitation studies or design heuristics. We propose an approach to develop a model of fatigue and stress of manual mid-air input, inspired by prior work on the ergonomics of arm movements and on the performance of multi-finger gestures. Along with our vision of the incoming challenges in mid-air interaction, we describe a design framework for mid-air input that, given such models, can be used to automatically evaluate any given gesture set, or propose an optimal gesture vocabulary for a given set of tasks.

The development of robust methods to identify which finger is causing each touch point, called “finger identification,” will open up a new input space where interaction designers can associate system actions to different fingers. However, relatively little is known about the performance of specific fingers as single touch points or when used together in a “chord.” We present empirical results for accuracy, throughput, and subjective preference gathered in five experiments with 48 participants exploring all 10 fingers and 7 two-finger chords. Based on these results, we develop design guidelines for reasonable target sizes for specific fingers and two-finger chords, and a relative ranking of the suitability of fingers and two-finger chords for common multi-touch tasks. Our work contributes new knowledge regarding specific finger and chord performance and can inform the design of future interaction techniques and interfaces utilizing finger identification.

We describe Gunslinger, a mid-air interaction technique using barehand postures and gestures. Unlike past work, we explore a relaxed arms-down position with both hands interacting at the sides of the body. It features novel ‘hand-cursor’ feedback to communicate recognized hand posture, command mode and tracking quality; and a simple, but flexible hand posture recognizer. Although Gunslinger is suitable for many usage contexts, we focus on integrating mid-air gestures with large display touch input. We show how the Gunslinger form factor enables an interaction language that is equivalent, coherent, and compatible with large display touch input. A four-part study evaluates Midas Touch, posture recognition feedback, fundamental pointing and clicking, and general usability.

We describe a mid-air, barehand pointing and clicking interaction technique using electromyographic (EMG) and inertial measurement unit (IMU) input from a consumer armband device. The technique uses enhanced pointer feedback to convey state, a custom pointer acceleration function tuned for angular inertial motion, and correction and filtering techniques to minimize side-effects when combining EMG and IMU input. By replicating a previous large display study using a motion capture pointing technique, we show the EMG and IMU technique is only 430 to 790 ms slower and has acceptable error rates for targets greater than 48 mm. Our work demonstrates that consumer-level EMG and IMU sensing is practical for distant pointing and clicking on large displays.

We introduce BodyScape, a body-centric design space for both analyzing existing multi-surface interaction techniques and suggesting new ones. We examine the relationship between users and their environment, specifically how different body parts enhance or restrict movement in particular interaction techniques. We illustrate the use of BodyScape by comparing two free-hand techniques, on-body touch and mid- air pointing, separately and in combination. We found that touching the torso is faster than touching the lower legs, since it affects the user’s balance; individual techniques outperform compound ones; and touching the dominant arm is slower than other body parts because the user must compensate for the applied force. The latter is surprising, given that most recent on-body touch techniques focus on touching the dominant arm.

Rich interaction with high-resolution wall displays is not limited to remotely pointing at targets. Other relevant forms of interaction include virtual navigation, text entry, and direct manipulation of control widgets. However, most techniques for remotely acquiring targets with high precision have studied remote pointing in isolation, focusing on pointing efficiency, and ignoring the need to support these other forms of interaction. We investigate high-precision pointing techniques capable of acquiring targets as small as 4 millimeters on a 5.5 meters wide display while leaving up to 93 % of a typical tablet device's screen space available for task-specific widgets. We compare these techniques to state-of-the-art distant pointing techniques and show that two of our techniques, a purely relative one and one that uses head orientation, perform as well or better than the best pointing-only input techniques while using a fraction of the interaction resources.

Large display environments (LDEs) are interactive physical workspaces featuring one or more static large displays as well as rich interaction capabilities, and are meant to visualize and manipulate very large datasets. Research about mid-air interactions in such environments has emerged over the past decade, and a number of interaction techniques are now available for most elementary tasks such as pointing, navigating and command selection. However these techniques are often designed and evaluated separately on specific platforms and for specific use-cases or operationalizations, which makes it hard to choose, compare and combine them. In this dissertation I propose a framework and a set of guidelines for analyzing and combining the input and output channels available in LDEs. I analyze the characteristics of LDEs in terms of (1) visual output and how it affects usability and collaboration and (2) input channels and how to combine them in rich sets of mid-air interaction techniques. These analyses lead to four design requirements intended to ensure that a set of interaction techniques can be used (i) at a distance, (ii) together with other interaction techniques and (iii) when collaborating with other users. In accordance with these requirements, I designed and evaluated a set of mid-air interaction techniques for panning and zooming, for invoking commands while pointing and for performing difficult pointing tasks with lim- ited input requirements. For the latter I also developed two methods, one for calibrating high-precision techniques with two levels of precision and one for tuning velocity-based transfer functions. Finally, I introduce two higher-level design considerations for combining interaction techniques in input-constrained environments. Designers should take into account (1) the trade-off between minimizing limb usage and performing actions in parallel that affects overall performance, and (2) the decision and adaptation costs incurred by changing the resolution function of a pointing technique during a pointing task.

The WILD room (wall-sized interaction with large datasets) serves as a testbed for exploring the next generation of interactive systems by distributing interaction across diverse computing devices, enabling multiple users to easily and seamlessly create, share, and manipulate digital content.

Research on cluster-driven wall displays has mostly focused on techniques for parallel rendering of complex 3D models. There has been comparatively little research effort dedicated to other types of graphics and to the software engineering issues that arise when prototyping novel interaction techniques or developing full-featured applications for such displays. We present jBricks, a Java toolkit that integrates a high-quality 2D graphics rendering engine and a versatile input configuration module into a coherent framework, enabling the exploratory prototyping of interaction techniques and rapid development of post-WIMP applications running on cluster-driven interactive visualization platforms.

Ultra-high-resolution wall displays have proven useful for displaying large quantities of information, but lack appropriate interaction techniques to manipulate the data efficiently. We explore the limits of existing modeless remote pointing techniques, originally designed for lower resolution displays, and show that they do not support high-precision pointing on such walls. We then consider techniques that combine a coarse positioning mode to approach the target's area with a precise pointing mode for acquiring the target. We compare both new and existing techniques through a controlled experiment, and find that techniques combining ray casting with relative positioning or angular movements enable the selection of targets as small as 4 millimeters while standing 2 meters away from the display.

Several approaches have been proposed to increase the breadth of standard Marking Menus over the 8 item limit, most of which have focused on the use of the standard 2D input space (x-y). We present Push Menu, an extension of Marking Menu that takes advantage of pressure input as a third input dimension to increase menu breadth. We present the results of a preliminary experiment that validates our design and shows that Push Menu users who are neither familiar with pen-based interfaces nor continuous pressure control can handle up to 20 items reliably. We also discuss the implications of these results for using Push Menu in user interfaces and for improving its design.

Imaginez un mur d’écrans qui affiche des images en haute définition. Imaginez que par des gestes simples, vous puissiez interagir avec lui… Nous ne sommes pas dans le film «Minority Report », mais face à la concrétisation d’un projet unique d’interaction homme-machine (IHM), la plate-forme Wild, qui permet d’interagir avec des masses de données complexes.

Cet article présente un espace de conception basé sur une analyse morphologique des mécanismes de structuration des menus et de sélection des items. Son but est de faciliter l'exploration de nouveaux types de menus afin notamment d'augmenter leur capacité sans détériorer leurs performances. L'article démontre l'aspect génératif de cet espace de conception grâce à quatre nouveaux designs de menus, basés sur des combinaisons de dimensions pas ou peu explorées. Pour deux d'entre eux, des expérimentations contrôlées montrent qu'ils offrent des performances comparables aux menus de la littérature.

Marking menus have many benefits, including fast selection time, low error rate and fast transition to expert mode, but these are mitigated by a practical limit of 8 items per menu. Adding hierarchical levels increases capacity, but at the expense of longer selection times and higher error rates. In this paper we introduce Extended Marking Menus, a variant of marking menus that increases their width without sacrificing performance. Extended marking menus organize the items in several rings or layers. Selection is achieved by simultaneous control of direction, as in traditional marking menus, and another dimension such as distance, speed or pressure. We examine the design space of these new menus and study the Distance Extended Marking Menu, or Dartboard Menu, in more detail. We report on two experiments, one to calibrate the sizes of the rings, the other showing that it performs faster than the Zone and Flower menus but is less accurate than the Zone menu.