Schematic of the textile-based tactile learning platform.
a, Scalable manufacturing of tactile sensing textiles using a customized coaxial piezoresistive fiber fabrication
system and digital machine knitting. b, Digitally designed and automatically knitted full-sized tactile sensing wearables:
i, artificial robot skin, ii, vest, iii, sock, and iv, glove. c, Examples of tactile frames collected during human-environment
interactions and their applications explored using machine learning techniques.
Abstract
Recording, modelling, and understanding tactile interactions is important in the study of human behaviour
and in the development of applications in healthcare and robotics. However, such studies remain challenging
because existing wearable sensory interfaces are limited in terms of performance, flexibility, scalability,
and cost. Here we report a textile-based tactile learning platform that enables recording, monitoring, and
learning of human–environment interactions. The tactile textiles are created via digital machine knitting of
inexpensive piezoresistive fibres, and can conform to arbitrary three-dimensional geometries. To ensure that
our system is robust against variations in individual sensors, we use machine learning techniques for sensing
correction and calibration. Using the platform, we capture diverse human–environment interactions (more than
a million tactile frames), and show that the artificial intelligence-powered sensing textiles can classify
humans' sitting poses, motions, and other interactions with the environment. We also show that the platform
can recover dynamic whole-body poses, reveal environmental spatial information, and discover biomechanical
signatures.
Paper
Learning Human-environment Interactions using Conformal Tactile Textiles
Y.Luo, Y.Li, P. Sharma, W. Shou, K. Wu, M. Foshey, B. Li, T. Palacios, A. Torralba, W. Matusik
Nature Electronics, 4, 193–201(2021)
[Paper (External link)]
[Dataset]
[Code]
Selected as Nature Electronics, Volume 4 Issue 3 (March 2021) Cover Image [Link].
Video
Press
Related Project
IntelligentCarpet: Inferring 3D Human Pose from Tactile Signals
Y.Luo, Y.Li, M. Foshey, W. Shou, P. Sharma, T. Palacios, A. Torralba, W. Matusik
CVPR 2021
[Project page]
KnitUI: Fabricating Interactive and Sensing Textiles with Machine Knitting
Y.Luo*, K. Wu*, T. Palacios, W. Matusik
ACM CHI 2021
[Project page]
Learning the Signatures of the Human Grasp Using a Scalable Tactile Glove
S.Sundaram, P. Kellnhofer, Y.Li, J. Zhu, A. Torralba, W. Matusik
Nature, 569 (7758), 2019
[Project page]