Using a novel manufacturing process, MIT researchers have created smart textiles that closely conform to the body, allowing them to sense the wearer’s posture and movements.
By incorporating a special type of plastic yarn and using heat to melt it slightly – a process called thermoforming – the researchers were able to greatly improve the precision of pressure sensors woven into multi-layer knitted textiles, which they call 3DKnITS.
They used this process to create a “smart” shoe and mat, and then built a hardware and software system to measure and interpret data from the pressure sensors in real time. The machine learning system predicted movements and yoga poses performed by a person standing on the smart textile mat with about 99 percent accuracy.
Their manufacturing process, which takes advantage of digital knitting technology, allows for rapid prototyping and can be easily scaled up for high-volume production, says Irmandy Wicaksono, research associate at MIT Media Lab and lead author of a paper showcasing 3DKnITS.
The technique could have many applications, particularly in healthcare and rehabilitation. For example, smart shoes could be made that track the gait of someone who is learning to walk again after an injury, or socks that monitor the pressure on a diabetic’s foot to prevent ulcers from forming.
“With digital knitting, you have this freedom to design your own patterns and also incorporate sensors into the structure itself to make it seamless and comfortable, and you can engineer it based on the shape of your body,” says Wicaksono.
He wrote the work with MIT students Peter G. Hwang, Samir Droubi, and Allison N. Serio as part of the Undergraduate Research Opportunities Program; Franny Xi Wu, graduate of Wellesley College; Wei Yan, assistant professor at Nanyang Technological University; and senior author Joseph A. Paradiso, Alexander W. Dreyfoos Professor and leader of the Responsive Environments group at the Media Lab. The research results will be presented at the IEEE Engineering in Medicine and Biology Society Conference.
“Some of the early pioneering work on smart fabrics happened in the Media Lab in the late 1990s. The materials, the electronics that can be integrated and the production machines have developed enormously since then,” says Paradiso. “It’s a great time to see our research returning to this field, for example through projects like Irmandy’s – they point to an exciting future where perception and functions diffuse more fluidly into materials, opening up enormous possibilities.”
To create a smart textile, the researchers use a digital knitting machine that weaves layers of fabric together with rows of standard and functional yarns. The multi-layer knitted textile consists of two layers of conductive yarn knit sandwiched around a piezoresistive knit that changes resistance when compressed. Following a pattern, the machine sews this functional yarn through the textile in horizontal and vertical rows. A pressure sensor is created where the functional fibers cross, explains Wicaksono.
But yarn is soft and pliable, so the layers shift and rub against each other as the wearer moves. This creates noise and causes fluctuations that make the pressure sensors much less accurate.
Wicaksono found a solution to this problem while working at a knitting factory in Shenzhen, China, where he spent a month learning how to program and maintain digital knitting machines. He observed workers making sneakers from thermoplastic yarns, which when heated above 70 degrees Celsius, began to melt, making the textile slightly harder so it could hold a precise shape.
He decided to try to integrate fusible fibers and thermoforming into the manufacturing process of smart textiles.
“Thermoforming really solves the noise problem because it hardens the multi-layer textile into one layer by essentially compressing and fusing the entire fabric together, which improves accuracy. Thermoforming also allows us to create 3D shapes like a sock or shoe that actually fit the user’s exact size and shape,” he says.
After perfecting the manufacturing process, Wicaksono needed a system to accurately process pressure sensor data. Because the fabric is knit in a grid, he made a wireless circuit that scans rows and columns on the fabric and measures the resistance at each point. He designed this circuit to overcome artifacts caused by “ghosting” ambiguities that occur when the user applies pressure to two or more separate points at the same time.
Inspired by deep learning image classification techniques, Wicaksono developed a system that displays pressure sensor data as a heatmap. These images are fed into a machine learning model that is trained to recognize the user’s posture, pose, or movement based on the heat map image.
Once trained, the model was able to classify the user’s activity on the smart mat (walk, run, push-up, etc.) with 99.6 percent accuracy and recognize seven yoga poses with 98.7 percent accuracy.
They also used a circular knitting machine to create a form-fitting smart textile shoe with 96 pressure measurement points distributed throughout the 3D textile. They used the shoe to measure the pressure exerted on different parts of the foot when the wearer kicked a soccer ball.
The high accuracy of 3DKnITS could make them useful for applications in prosthetics where precision is important. A smart textile lining could measure the pressure a prosthesis is exerting on the socket, allowing an orthotist to easily see how well the device is fitting, Wicaksono says.
He and his colleagues are also exploring more creative applications. In collaboration with a sound designer and a contemporary dancer, they developed an intelligent textile carpet that drives musical notes and soundscapes based on the dancer’s steps to explore the bi-directional relationship between music and choreography. This research was recently presented at the ACM Creativity and Cognition Conference.
“I’ve learned that interdisciplinary collaboration can create some really unique applications,” he says.
After the researchers demonstrate the success of their fabrication technique, Wicaksono plans to refine the circuit and machine learning model. Currently, the model needs to be calibrated to each individual before it can classify actions, which is a time-consuming process. Removing this calibration step would make 3DKnITS easier to use. Researchers also want to run tests on smart shoes outside of the lab to see how environmental conditions like temperature and humidity affect the accuracy of sensors.
“It’s always amazing to see technological advances in such a meaningful way. It’s incredible to imagine that the clothing we wear, an arm sleeve or a sock, can be made in such a way that its three-dimensional structure can be used to capture it,” says Eric Berkson, assistant professor of orthopedic surgery at Harvard Medical School and a sports medicine orthopedic surgeon at Massachusetts General Hospital, who was not involved in this study. “In the medical field and in particular in orthopedic sports medicine, this technology offers the possibility to better recognize and classify movements and to recognize force distribution patterns in real situations (outside the laboratory). This is the type of thinking that will enhance injury prevention and detection techniques and help evaluate and guide rehabilitation.”
This research was supported in part by the MIT Media Lab Consortium.