We believe that computer science and mechanical engineering are about to unite. In the future, users will solve mechanical problems by digitizing the involved objects using 3D scanners, solving the problem in the digital domain using the means of computer science, and converting the result back to the mechanical domain using a 3D printer. This will allow solving mechanical problems with the effectiveness and efficiency of computer science, including the ability to scale massively.
This will not only change mechanical engineering, but also allow computing to reach its next phase, which is to merge into matter itself, where the physical matter of objects will also perform the computation, rather than separate micro controllers.
The role of our group is to drive this unification process, in particular by creating and re-purposing fabrication machines and haptic machinery.
(Previously: we worked on unifying the virtual world of the computer with the physical world of the user into a single space. We argue that this is the key to intuitive or “natural” user interfaces. Most of our research takes on an engineering perspective. We create new types of mobile devices, touch devices and interactive floors, and gesture input devices. In addition, we are trying to gain a deeper understanding of touch input through scientific experiments).
2015+ Keynote: The Six challenges in Personal Fabrication
How computer science and mechanical engineering will unite, the resulting opportunities of having millions of people build machines, and the resulting five grand challenges that we as researchers in the fields of human-computer interaction and graphics have to solve in order to get there.
2012-2013 Keynote: Natural User Interface Hardware
"What makes an interface natural" and "why can 1-year olds use touch screens"? I discuss NUI at the example of prototype devices that unify the virtual world of the computer with the physical world of the user into a single "Euclidean" space. PPT 54.6MB given at British HCI 2012 and 3DUI 2013
2010-2011 Keynote: My new PC is a mobile phone
I argue that the computational device for the world is already here--it is mobile phones. 4 billion of them. But to make them useful, we need to create new software that turns phones into stand-alone computers. PPT+Video 116.3M given at Mobile HCI 2010 and USAB 2010
Scenograph(fullpaper at UIST 2018) allows real-walking VR experiences to run in arbitrary tracking volumes.
VirtualSpace(fullpaper at CHI 2018) allows multiple users in virtual reality to use the same tracking space at the same time. The users are immersed in different virtual worlds and not aware of one another.
Mobile Fabrication(full paper at UIST 2016) we studied mobile 3D-printing in order to find out what people would make on the go and how technology can enable true mobile fabrication
TrussFormer(full paper at UIST'18) end-to-end system that allows users to 3D print large-scale kinetic structures, i.e., structures that involve motion and deal with dynamic forces.
TrussFab(full paper at CHI 2017) an integrated end-to-end system that allows users to fabri-cate large scale structures that are sturdy enough to carry human weight. TrussFab achieves the large scale by complementing 3D print with plastic bottles.
Protopiper(full paper at UIST 2015) Physically Sketching Room-Sized Objects at Actual Scale. The key idea behind protopiper is that it forms adhesive tape into tubes as its main building material, rather than extruded plastic or photopolymer lines.
Haptic interfaces for visually impaired users
DualPanto(fullpaper at UIST'18) is a new haptic device that enables blind users to continuously track the absolute position of moving objects in spatial virtual environments, as is the case in sports or shooter games.
Linespace(fullpaper at CHI'16) is an interactive system that consists of hardware and software and that allows visually impaired users to interact with spatial contents.
Metamaterial Mechanisms (best paper nominee at UIST'16) integrate mechanical functionality within the material structure. Such Mechanisms consist of a single block of 3D printed material with cells that play together to redirect forces, thereby implementing mechanisms.
Digital Mechanical Metamaterials(CHI '17) proposes integrating computational abilities into the structure of 3D printed objects. While analog metamaterial mechanisms are subject to damping, causing the mechanical "signal" to decay, digital mechanical metamaterials allow an arbitrary number of cells.
Metamaterial Textures (CHI'18) Alexandra Ion, Robert Kovacs, Oliver Schneider, Pedro Lopes and Patrick Baudisch
Haptics using electrical muscle stimulation
Feel what you touch in AR (fullpaper at CHI 2018) utilizes electrical muscle stimulation to providing haptics to Mixed and Augmented Reality (AR & MR) without ever covering the users hands. This allows users to still be able to seemlessly interact with their real suroundings.
VR walls (fullpaper at CHI 2017) utilizes electrical muscle stimulation to providing haptics to walls and other heavy objects in virtual reality. The resulting system is wearable and fits in a backpack.
Ad Infinitum(exhibition at ARS Electronica 2017, Science Gallery Dublin 2017) is a parasitical entity that uses electrical muscle stimulation to lives off human energy. This parasite reverses the dominant role that mankind has with respect to technologies: the parasite shifts humans from "users" to "used".
Muscle-plotter is an interactive system based on EMS that produces spatial output. It achieves this by persisting EMS output by actuating the user's wrist as to plot with an Anoto pen. Muscle-plotter is the first EMS-based sensemaking system.
Impacto (full paper at UIST 2015) is a wearable device that simulates the sensation of physical impact in virtual reality. It does so by decomposing the stimuli into (1) the tactile sensation, which is rendered by tapping the user's skin with a solenoid, and (2) the impulse, which is rendered by actuating the user's body with electrical muscle stimulation.
Affordance++ (best paper, fullpaper at CHI 2015) allows everyday objects to communicate dynamic use: motion (spray can shakes when touched), multi-step processes (peeling an avocado), and behaviors that change over time (don't grab the hot cup by its body).
Proprioceptive Interaction (fullpaper at CHI 2015) allows for eyes-free interaction based on proprioception alone. We devised a bracelet which allows to input and output to happen exclusively through the user's muscles.
Muscle-Propelled Force-Feedback (note at CHI 2013) provides force feedback via electrical muscle stimulation, e.g., in interactive games. This saves the exoskeleton and motors, making the approach mobile.
Gesture Output(fullpaper at CHI 2013) allows mobile touch devices to send eyes-free messages to the user as a sequence of marks of graffiti characters via our force-feedback touch screen (PocketOuija).
Haptics using human actuation
iTurk (fullpaper at CHI 2018) makes the user reconfigure otherwise passive haptics to convey a sense of a living, animate virtual world with haptic feedback, while in reality the user is the only animate entity present in the system.
Mutual Human Actuation(full paper at UIST 2017) leverages human actuation but eliminates the need of dedicated human actuators by making them as players. The players are fully immersed into their own virtual experiences while providing the other haptic feedback.
TurkDeck(full paper at UIST 2015) is a prop based VR system which leverages human actuation to move and re-organise the props in the scene. In combination with portals to move between rooms this results in scalable prop based VR.
Haptic Turk (fullpaper at CHI 2014) offers the functionality of a motion platform--but is based on people. Allows bringing motion the concept of immersive haptic experiences to millions of users.
Level-Ups(note at CHI 2015) are computer-controlled stilts that allow virtual reality users to experience walking up and down steps. Unlike traditional solutions that are integrated with locomotion devices, Level-Ups allow users to walk around freely.
RoMA: Interactive Fabrication with a Robotic Arm 3D Printer Huaishu Peng, Cheng-Yao Wang, James Briggs, Kevin Guo, Joseph Kider, Stefanie Mueller, Patrick Baudisch and Francois Grumbetière In Proceedings of CHI'18.
Destructive Games (note at CHI 2016) are games that result in valuable physical objects being damaged or destroyed. Surprisingly, 8 out of 12 participants would play again.
Patching Physical Objects(full paper at UIST 2015) Instead of re-printing the entire object from scratch during design iteration, we suggest patching the existing object and replacing only the unsatisfactory parts.
LaserStacker(full paper at UIST 2015) LaserStacker allows users to fabricate 3D objects with an ordinary laser cutter through a cut-weld-heal-release process. The key idea is to use the laser cutter to not only cut but also to weld.
Platener (best paper nominee at CHI 2015) speeds up design iterations of 3D models by extracting straight and curved plates from the 3D model and substituting them with laser cut parts.
Scotty (full paper at TEI 2015) is a simple self-contained appliance that allows relocating inanimate physical objects across distance.
WirePrint(full paper at UIST 2014) prints 3D objects as wireframe previews. By extruding filament directly into 3D space instead of printing layer-wise, it achieves a speed-up of up to a factor of 10, allowing designers to iterate more quickly in the early stages of design.
FaBrickation(best paper nominee at CHI 2014) speeds up fabrication time of functional 3D printed objects by integrating construction kit building blocks.
LaserOrigami(best paper award at CHI 2013) allows users to produce fully assembled 3D objects using a laser-cutter--two orders of magnitude faster than with a 3D printer.
Constructable (fullpaper at UIST 2012) allows users to create physical objects by drawing directly on the workpiece while in the laser cutter
gravity + multitouch = 3D tracking (since 2010)
Ergonomic Interaction on Touch Floors (fullpaper at CHI 2015). The main appeal of touch floors is that they are the only direct touch form factor that scales to arbitrary size. In this paper, we argue that the price for this benefit is bad physical ergonomics, and propose addressing this issue by allowing users to operate touch floors in any pose they like.
Kickables (fullpaper at CHI 2014) are tangibles for feet, which provide tangible experiences for large scale installations with interactive floors.
Fiberio (fullpaper UIST 2013) is a multitouch table that senses fingerprints and identifies users biometrically during each touch.
GravitySpace (best paper nominee at CHI 2013)tracks users and their poses in smart rooms based on the latest 8m2 version of our pressure-sensing multitoe floor.
Bootstrapper(note at CHI 2012) recognizes tabletop users by their shoes. Bootstrapper uses a Kinect attached to a Microsoft Surface table pointed at users' shoes.
Multitoe (fullpaper UIST 2010) is an interactive floor. Based on FTIR sensing, it can identify users based on their soles, track users' foot and body postures, and enable high-precision interaction-using feet.
Lumino (best paper award CHI 2010,demos at Siggraph 2010 and Tabletop 2010)is a system of building blocks that allows microsoft surface to sense building elements arranged in three-dimensional structures.
RidgePad was our first project to explore 3D reconstruction from contact area.
Imaginary Reality Games (paper UIST 2013) are games that mimic real-world sport, such as basketball or soccer, except that there is no visible ball.
Understanding palm-based imaginary interfaces (fullpaper CHI 2013) allow users to interact with unfamiliar imaginary interfaces. We also explore revisit the fundamentals of imaginary interfaces, i.e. the role of visual and tactile cues.
BodyScape(best paper nominee CHI 2013) a body-centric design space for exploring how users may interact across the surfaces of their body.
Imaginary Phone (fullpaper UIST 2011) allows users to operate their phone without taking it out of the pocket. Instead users type on their hand. Users learn the device automatically by transferring spatial knowledge from the use of the physical device.
Data Miming(fullpaper CHI 2011). Users retrieve 3D objects from a database by describing their shape through gesture in 3-space. Tracked using a Kinect/PrimeSense depth camera.
Imaginary Interfaces (fullpaper UIST 2010) are devices that allow users to interact spatially as they normally would with a touch screen--yet without the screen.
Understanding touch (since 2010)
Understanding touch (fullpaper at CHI 2011). In order for a touch device to be highly effective it needs to match users' mental model of touch. But what is that model?
Touch on Curved Surfaces (fullpaper CHI 2011). Technology is emerging that allow touch-enabling non-planar surfaces, e.g., on mobile devices. We are modeling how users interact with such surfaces.
Nenya (note at CHI 2011). This input device is a passive magnetic ring. We obtain this tiny device form-factor by offloading all electronics into a bracelet that tracks the ring using a magnetometer.
Touch Projector (fullpaper CHI 2010) allows users to manipulate the contents located on distant public displays by touch manipulating its video image on their mobile device.
Disappearing Mobile Devices (fullpaper UIST 2009) are devices so small, that they only allow certain types of gesture interactions: a thought experiment about the ultimate future of mobile miniaturization.
Nanotouch (best paper nomineeCHI 2009) is a prototype device that users operate via the device backside. Allows for making very small mobile devices.
Let's kick it(note at CHI 2014) makes the unreachable bottom of vertical displays fully interactive by expanding direct touch to the feet.
360° Panoramic Overviews (note CHI 2012) allow users to get an overview of a augmented reality scene without losing spatial orientation.
Relaxed Selection Techniques (fullpaper UIST 2009) allow users to search time series data using a pen stroke that not only specifies the shape of the sought segment, but also specifies tolerances.