The Japanese Professor Who's Spent Three Decades Perfectin
Source: Emiko Jozuka

Professor Susumu Tachi with TELESAR V inside the Cyber Living Lab. Image: Emiko Jozuka

When Susumu Tachi made his first prototype telexistence machine in 1981, he was amazed at what he saw. As he peered through the contraption, he glimpsed another version of himself from behind, looking through the very same prototype device. The effect induced a curious out-of-body sensation. Excited, he called his lab mates, who experienced similar feelings of self-displacement when trying the tech.

“It was different to looking at yourself in a mirror, or looking at your image in a video recording. I saw my image moving as I was moving myself. At that moment, I wondered where I was,” said the virtual reality and robotics professor at the University of Tokyo with a boyish grin. “I really felt that this was telexistence.”

For the uninitiated, “telexistence” is the real-time feeling of being in another location (real or virtual) that is different to one’s current location. Telexistence is different to “teleoperation,” which sees the operator electronically control another machine using a remote control. With the latter, there is no sense of being in the place.

Tachi said he came up with the concept of “telexistence” in 1980 and corroborated his discovery with a publication in Japanese in 1982, and then in English in 1984. Over in the US, “telepresence,” which is similar to “telexistence,” was coined by Marvin Minsky, a cognitive scientist at the Massachusetts Institute of Technology in 1980. Tachi has written that while they are similar, telexistence has more focus on make a user feel like they actually inhabit the virtual space they're working in.

Susumu Tachi tests a prototype telexistence contraption in 1982. Image: Tachi Laboratory, The University of Tokyo

Tachi’s initial brush with the novelty of telexistence over 30 years ago has since morphed into a lifelong quest to discover what it means to mentally inhabit another operable robot or vehicle. Over the decades, he has created a series of anthropomorphic robots, dubbed TELESAR, which all induce the same out-of-body experience while they are controlled remotely by human operators.

I visited Tachi’s Cyber Living research laboratory in Tokyo’s Miraikan (National Museum of Emerging Science and Innovation), where he explained how remotely operating machines is becoming more commonplace, and how effective this will be once the person using the machine feels like they are mentally inhabiting it. He also demonstrated how the physical manifestation of over 30 years' worth of research and development―the TELESAR V―worked.

Charith Fernando helps Kevin Fan get rigged up to control TELESAR V. Image: Emiko Jozuka

The TELESAR V is a flexible white robot with stereo cameras for eyes. It has 15 joints in its hand and can pour marbles from one cup to another using its hands and give handshakes. It’s propped up on a pedestal while its predecessors, the chunky red TELESAR (the godfather of the series) and the globular white TELESAR II, stand dormant nearby, peering out of the lab window at any passing visitors.

Immediately to TELESAR V’s left, Kevin Fan, a doctoral researcher at the lab, was seated on a pedestal wearing a 3D VR headset and haptic gloves. The headset communicates sounds and images captured by TELESAR V back to Fan. The gloves are lined with sensors and transmit back what the robot "feels" in its fingertips to Fan’s hands.

Kevin Fan virtually goes through the motions of pouring marbles into a cup while TELESAR V performs the task for him. Image: Emiko Jozuka

“The robot moves according to the human motions,” explained Tachi as Fan bashed out some moves. “The human hand motions are measured by a data graph, and that information is sent to the robot. Right now the two are very close, but they could also be at greater distances from each other.”

Tachi beckoned me forward, chuckling. “Say hello to the robot. Try shaking its hand.”

I stepped tentatively in front of TELESAR V.

“I can see you, I’m trying to say hello,” Fan said to my left, while I looked up at TELESAR V as it gave me a creaky wave. I attempted to shake the robot’s hand without damaging its wiring, transmitting my grasp over to Fan through TELESAR V. Fan told me that he could feel my handshake.

Moments later, Kevin demonstrated TELESAR V’s dexterity by going through the motions of pouring some marbles from one cup to the other.

“This kind of complex work can be done easily as Kevin is just doing it directly,” explained Tachi. “Kevin is not a seasoned operator, but he can still do this.”

The point of all this is to demonstrate just how accurately and easily telexistence-controlled robots, rigged up to human operators, can perform tasks. After all, this level of dexterity could be beneficial in a wide range of applications from industry to healthcare.

The different uses envisioned for telexistence under the Japanese National Large Scale Project. Image: Tachi Laboratory, The University of Tokyo

According to the researchers at the Tachi lab, telexistence allows people to perform tasks more effectively than teleoperation. The latter requires people to learn how to use controllers to manipulate a machine that they don’t feel physically connected to. Inhabiting either a physical or virtual machine or robot through telexistence allows people to feel like they are the thing they control.

Since inventing the concept, Tachi’s manifestations of telexistence have evolved. Back in 1983, he participated in an eight-year government project dubbed the Japanese National Large Scale Project that tested out the feasibility of using telexistence systems in everything from industry to disaster missions.

Highlights of the project included a larger, chunkier ancestor to Honda’s lightweight Asimo clambering up and down stairs with a researcher telexisting in it, and a small garbage car that let a researcher feel like he or she is riding on top of it. Currently, Charith Fernando, an assistant professor at Keio Media Design who works in Tachi's lab, is conducting research on how telexistence can be applied in the construction industry. The idea here is to let construction workers control large vehicles remotely, and avoid having to be in the cockpit.

TELESAR II and TELESAR peer out of the Cyber Living Lab’s window at passing visitors. Image: Emiko Jozuka

Independently, Tachi has applied his ideas of telexistence to easing the strain on long-distance relationships and augmenting remote social interactions. Tachi said that, as technology develops, humans won’t be content with keeping in touch with people solely through phone calls and Skype. They’ll crave the physicality, or in this case, the illusion of human presence, which can be communicated through temperature, pressure, and vibrations.

TELESAR IV, for example, isn’t just a plain and simple telexistence robot: it can also display the operators face on its body by using retro-reflective projection technology (RPT), which covers a given surface with microscopic beads, roughly 50 micrometers in diameter, that reflect back projected light. The goal is to show the face of the person who controls the robot on it.

“When someone operates the robot at a party, they feel like they’re in the room, but everyone else can’t see who that is,” said Tachi, noting that RTP allowed the human operating the telexistence robot’s image to be superimposed on top of the robot, when someone with head mounted screen looked in their direction.

This, said Tachi, improves social interactions between people and people attending a social settings as a telexistence robots. While someone attending a party as a telexistence robot might feel fully present, to others he or she is just a robot. When you see a human operator’s face on the robot you’re chatting with, the robot immediately gains a more friendly human-like presence.

In the past, head-mounted displays used in telexistence were definitely a lot chunkier. Image: Tachi Laboratory, The University of Tokyo

Telexistence itself offers seemingly limitless uses going forward. Now, more than three decades since Tachi first pioneered the idea, the world is finally catching up. Increases in high-speed communication networks have, for example, made telesurgery possible. Over in Sweden, researchers at the Karolinska Institute are investigating ideas of selfhood and bodily attachment when people are teleported into bodies other than their own, and ESA researchers are testing out the idea of using telexistence in space missions of the future.

Tachi himself described the idea of a “anywhere door” (doko demo door), which is a magical portal that Doraemon, a fictional blue robotic animation cat from the 22nd century, pulls out of his stomach pouch in order to let he and his band of friends travel anywhere they want. Tachi envisions the telexistence robots offering a similar experience.

Students brainstorming ideas at the Miraikan’s Cyber Living Lab. Image: Emiko Jozuka

Japan also boasts an increasing silver population and Tachi said that in the near future, telexistence robots could provide both the elderly and young people who have to care for their children or elderly parents with a way of continuing to work without leaving the comfort of their homes.

For the moment, TELESAR V is still limited to shaking hands and pouring marbles from cup to cup, but Tachi has big plans for TELESAR VI. He envisions a more integrated machine that can move, and that is equipped with tactile sensors all over its body.

“There are many challenges until we reach the perfect avatar body,” Tachi told me.

“Ultimately, I’d like [to be able to perform telexistence] with a pair of glasses. But it’ll take a long time until we get there.”

Cool Japan is a column about the quirky and serious happenings in the Japanese scientific, technological and cultural realms. It covers the unknown, the mainstream, and the otherwise interesting developments in Japan.