Development of a new interface to realize “Stepping outdoors while being confined indoors”Emi Tamaki
＜Series 1 / complete＞
The fields of medical care and welfare are becoming more and more diversified. To cope up with the needs, utilizing cutting-edge technologies such as AI and AR/VR will undoubtedly be the key that will open up the future of medical care and welfare.
Well, what exactly is the current state of cutting-edge technology? Our guest this time is Ms. Emi Tamaki, Associate Professor, Faculty of Science and Engineering, Waseda University. Ms. Tamaki is a pioneer in the research on Body Sharing that involves the transmission of the sense of touch and physical sensations to human beings through machines. The device “PossessedHand” developed independently by Ms. Tamaki in which the computer controls human hand movements was selected as the “World’s 50 Best Inventions” by American “TIME” magazine in 2011. She is one of the researchers who has attracted attention not only in Japan but from across the globe.
The writer interviewed Ms. Tamaki about “The future of medical care and welfare where PossessedHand and VR technology are put to use”. How can VR technology, including “PossessedHand” be utilized in the medical and welfare fields such that it will lead to solving of problems? Considering that Ms. Tamaki is the founder of a company (H2L Co., Ltd.), the kind of ideas and concepts required for SMEs in the future will be introduced in this four-part series.
Driving Factor Behind the Development of PossessedHand
The driving factor was that “I wanted to stay confined to a room” and was an “Indoor person”. I was even hospitalized many times during my childhood.
If you are hospitalized, you cannot experience the outside world, neither can you interact with the outside world (or impact the outside world). Though information from the outside world is available, hospitalized patients cannot interact with the outside world. As of now, it is difficult to be socially active while being confined to a hospital room. Participating in a video conference is possible, however, physical involvement is not possible.
Many times, I have not been able to go on family trips or not been able to participate in a camp when all my friends went, and though I kept saying that I am an indoor person, I hated those situations. Thanks to the assistance of medical personnel, my stay in the hospital was comfortable, on the other hand, I felt that there are many things that I cannot experience unless I step out of the hospital.
That was when I realized that this is because there is no interface to input and output physical movements. This realization came to me when I was hospitalized while studying in high-school. An interface is something that you use for input and output of information, for example, keyboards, trap pads, mouse and displays. Though you can input and output visual, acoustic and verbal information, you cannot input and output your body movements through a computer. I thought that if this is possible, then one can interact with the outside world while being confined to a room.
I started planning about what I should study at the university to make this idea a reality. I thought about “What is required to create an interactive system and interface” and researched on how to communicate body movements from a computer to humans. One of the results of that research is the device called “PossessedHand” in which a computer controls human hand movements.
According to Ms. Tamaki, “The thought of wanting to experience the outside world even if I do not go out or not able to go out” proved to be the driving factor behind the development of PossessedHand. The path of research and development that started with such a thought could never have been smooth. What kind of plan did you consider for beginning the development?
Development Background and Her Thoughts
At first, I thought that “Such tools must be available” and wanted to buy one. When I checked, such tools were not sold anywhere. Though many robots had been created around 2000, when I was a high school student, they were at a research stage, and there was no way of buying them. Also, robots are not an interface. Then I started thinking of developing one by myself.
This was not just my problem, if the development is delayed by ten years because no one works on such a device, then during those ten years, many people in hospitals will die without experiencing the outside world. When I thought about such people, I wanted to start the development at the earliest. It would be nice if everyone can interact with the outside world and share experiences as much as possible. I thought that even if I do not complete the development, because of my initiative, if I could contribute even a little to the research, then it would translate into shorter lead time for realization.
After graduating from high school, I studied at the Faculty of Engineering, University of the Ryukyus. During that time, while I was interacting with people who were engaged in research on robots, and while coming up with a plan, I realized that “This seems to be realizable in my lifetime”.
Since both the interface and robot have to be studied separately, I moved to the Graduate School of Tsukuba University and started studying about robots. Although many researchers were working on robot development, I realized that there was hardly any research on interfaces.
Ms. Tamaki says, “At that time, I felt that even if I did not develop robots, other researchers would advance the development”. She realized that for further research and development, a new interface is essential. The idea that she thought about while in a hospital room during her high-school days, was gradually taking shape.
New Interface that Conveys Proprioception
I realized that the interface is not adequate at all for research and development, and I started working on interface research. For stepping out, I thought that the sense “Proprioception”, which is also called “bathyesthesia”, was lacking.
Sensations such as “Smooth” and “Soft” or “Warm” and “Cold” are called “Superficial sensations” or “Surface sensations”. In contrast to these, “Sense of position” or “Sense of heaviness” are well-known “Proprioception” senses. Without these sensations, position information such as “Extent to which the hand is bent” or “The amount of force exerted” cannot be communicated.
For example, when paddling a kayak, even if someone says, “The paddle is so heavy”, you cannot understand that sensation merely with words. Information such as, “No matter how much you try to paddle through the water, there is so much resistance”, cannot be conveyed with the sensations of “Smooth” and “Soft”. I thought, “Something must be done to solve this problem”.
Until now, there has been little research on the interface that can convey “Proprioception”. There was no way for a computer to output the sense of resistance of a hand or information such as “Bending angle of the finger”. There were even doubts as to “Whether such information can be conveyed in the first place?”
Since we cannot progress without experimentation, I tried creating a prototype of PossessedHand when I was at the University of Tokyo Graduate School.
PossessedHand was initially created as an experimental tool.
According to Ms. Tamaki, the background to the development of PossessedHand was that she knew beforehand what she wanted to develop and then she gained knowledge of various domains to carry out the research and development. (Refer to her profile below). She worked on soldering when creating the prototype of PossessedHand, and this too was only a step to advance the development. Clarity of purpose seems to be reflected in the technique and speed of research and development.
In the next part, the writer will interview Ms. Tamaki to find about the mechanism of PossessedHand and application examples of VR technology.
Interview Date: March 14, 2019
Development of a new interface to realize “Stepping outdoors while being confined indoors”
Ms. Tamaki was born in 1984 in Okinawa Prefecture. She graduated in 2006 from Department of Information Engineering, Faculty of Engineering, University of the Ryukyus. She completed the master’s course in 2008 from the Graduate School of Systems and Information Engineering, University of Tsukuba. Worked as a seeds search intern at the University of Tokyo Edge Capital (UTEC), in 2009. She was engaged in research at Disney Research, Pittsburgh 2010.
In 2011, announced “PossessedHand”, a device in which human hand movements are controlled by a computer, which attracted attention in several conferences. “PossessedHand” was reported by CNN and ABC of the United States and was selected as one of the 50 Best Inventions by the “TIME” magazine. In the same year, received a doctorate from the University of Tokyo. Awarded “University of Tokyo President’s Award” and served as a valedictorian at the same time.
She founded H2L Co., Ltd. in 2012 to provide research equipment to researchers and share it with consumers. Moved to Waseda University in 2013, and continued research activities. Announced the world's first tactile-type game controller “UnlimitedHand” through the Kickstarter Website in 2015, which achieved its target in 22 hours. Awarded the runner-up prize for the “Woman of the Year Award by Nikkei Woman”. In 2016, received “WIRED Audi Innovation Award 2016”, “Nikkei Business One hundred people creating the next generation”, and “National Institute of Science and Technology Policy (NISTEP) Researcher (Noteworthy work in science and technology) Award”. From the same year, she served as a member of the Council for Science, Technology and Innovation of the Cabinet Office and Expert Panel on Basic Policy for Science, Technology and Innovation, for the comprehensive strategy for the 5th Science and Technology Basic Plan. Appointed as an ambassador of WINDS initiative (Women's Initiative in Developing STEM Career) of the Ministry of Foreign Affairs in 2017. As of 2019, she is working as an associate professor at the Faculty of Science and Engineering, Waseda University.
BreakThrough Study Idea＜Series 2 / complete＞
It’s important to clarify required output when developing new plastics
BreakThrough Study IdeaStudy Idea
Automation of agriculture, unmanned brings "A society where you can live without working."
Masayuki Hirafuji＜Series 4 / complete＞
BreakThrough Study IdeaStudy Idea
Data acquisition is barrier of agricultural development. Overseas that strength is intellectual property
Masayuki Hirafuji＜Series 2 / complete＞