Gierad Laput | EM-Sense
Most everyday electrical and electromechanical objects emit small amounts of electromagnetic (EM) noise during regular operation. When a user makes physical contact with such an object, this EM signal propagates through the user, owing to the conductivity of the human body. We can detect and classify these signals in real-time, enabling robust on-touch object detection. Unlike prior work, our approach requires no instrumentation of objects or the environment. We call our technique EM-Sense and built a proof-of-concept smartwatch implementation. Our studies show that discrimination between dozens of objects is feasible, independent of wearer, time and local environment.
Source: www.gierad.com/projects/emsense/
Smartwatches and wearables are unique in that they reside on the body, presenting great potential for always-available input and interaction. Their position on the wrist makes them ideal for capturing bio-acoustic signals. We developed a custom smartwatch kernel that boosts the sampling rate of a smartwatch’s existing accelerometer to 4 kHz. Using this new source of high-fidelity data, we uncovered a wide range of applications. For example, we can use bio-acoustic data to classify hand gestures such as flicks, claps, scratches, and taps, which combine with on-device motion tracking to create a wide range of expressive input modalities. Bio-acoustic sensing can also detect the vibrations of grasped mechanical or motor-powered objects, enabling passive object recognition that can augment everyday experiences with context-aware functionality. Finally, we can generate structured vibrations using a transducer, and show that data can be transmitted through the human body. Overall, our contributions unlock user interface techniques that previously relied on special-purpose and/or cumbersome instrumentation, making such interactions considerably more feasible for inclusion in future consumer devices.
Source: www.gierad.com/projects/viband/
ViBand
Gierad Laput | Synthetic Sensors
Our high-speed sensors, namely the accelerometer, microphone, and EMI, powers a lot of the events that we’ve seen so far. These reults surprised us! Some take-aways: 1) high-speed sensing is a big win for instantaneous environmental events; accel wouldn’t be in this list if running at ~50Hz. 2) Manufacturers/researchers can use this knowledge to target their hardware for most “bang for the buck.“ 3) Infrequently used sensors are critical for some events (e.g. kettle boiling); to be truly general purpose, you need to support the long tail of exotic/edge cases. 4) Most of the events we studied were seconds-scale, and thus were biased towards our faster sensors. In contrast, hour/day/week-scale events (e.g. house insulation issue) are better suited to low-sampling sensors, which we didn’t focus.