Harold Soh
Description: Harold Soh is an Assistant Professor of Computer Science and Collaborative Learning and Adaptive Robots (CLeAR) group at the National University of Singapore. His research looks at decision making, human-robot interactions, and trustworthiness in artificial intelligence. In this episode we talk about the current state of robotics, the difficulties of aligning robot capabilities with human trust and the challenges of encoding behavioral values in robots given our ever-changing social norms.
Websites:
Publications:
Resources:
Show Notes:
[0:00:00] Introduction
[0:01:05] Research Experience in the Private Sector
[0:01:29] Transition to Robotics Research
[0:02:22] Transitioning to Robotics and Mechanical Engineering
[0:03:23] Current State of Robotics
[0:04:30] Driving Innovation in Robotics
[0:05:29] Advancements in Sensor Technology
[0:06:43] Gaps in Robotics Innovations
[0:07:40] Impact of Machine Learning on Robotics
[0:09:18] Future of Robotics and Sensor Technology
[0:10:35] Building Trust in Human-Robot Collaboration
[0:11:49] Psychology and Trust in Robots
[0:14:21] Vision of Humanoid Robots in the Future
[0:16:18] Viability of Humanoid Robots in Today's Market
[0:18:05] Specific Research Goals
[0:20:11] Children's Perception of Robots
[0:21:45] Understanding Trust and Trustworthiness
[0:22:46] Assigning Values to Robot Decisions
[0:24:22] Challenges in Aligning Robot Values
[0:25:32] Teaching Humans to Collaborate with Robots
[0:26:35] Physical vs. Functional Trust in Robots
[0:26:59] The Role of Robot Embodiment in Trust
[0:29:44] Balancing Trust and Overreliance in Technology
[0:31:27] Robots in Mental Health and Wellness
[0:33:21] Impact of Robotics on Job Market
[0:35:21] The Balance Between Innovation and Human Happiness
[0:37:59] Enhancing Lives Through Robotics in Disability Care
[0:39:01] Advancements in Assistive Daily Living Tasks
[0:40:02] Visual Design of Assistive Feeding Robots
[0:43:45] Funding and Business Models in Robotics Research
[0:45:30] Defining What Qualifies as a Robot
[0:48:59] Ways for Students to Get Involved in Robotics
[0:50:28]Entry-level Kits and Simulators in Robotics
[0:52:05] Embracing Computational Thinking
[0:54:01] Utilizing Abstraction in Problem-Solving
[0:54:08] Encouragement for Curiosity and Focus in Learning
Unedited AI Generated Transcript:
Brent:
[0:01] Welcome, Professor Harold Soh. Thank you for coming on today.
Harold:
[0:04] Thanks. Thanks for having me. It's great to be here with the both of you.
Keller:
[0:07] We'd love to start off by hearing a little bit more about your story, how you got to NUS, and what got you interested in computer science and robotics.
Harold:
[0:14] Well, I've always been interested in computer science ever since I was a kid.
I was fortunate because computers were, you know, the thing to look at when I was a kid.
And my school had a computer class, and that really got me on this path.
I liked looking at the problems from a computational perspective And that suited with my way of thinking I think that's how I got started into this But I didn't know that I wanted to doresearch That came much, much later I did not go through a very straightforward path From grade school to PhD and to university In fact, I ended up working for a while I worked in anaudit firm for three months And discovered it was not for me I was a terrible auditor editor.
And then I went for a master's program, and it was there that I discovered research.
And then I worked for a little while in research, and then I went to grad school and ended up here in NUS. Yeah.
Research Experience in the Private Sector
Brent:
[1:05] Did you work for research in the private sector?
Harold:
[1:08] I worked for research in a research institute, actually here in Singapore. It's called ASTAR.
And it was a great way and a great experience for me, because I got to see how different different source of research was done.
I collaborated in a project involving genomics as well as computer science.
And that was a lot of fun as well as an eye-opening experience for me.
Transition to Robotics Research
Brent:
[1:29] Yeah.
Keller:
[1:29] Was the research you did there different? I guess the process different than how you do research at NUS?
Harold:
[1:35] Not quite. It's fairly much similar. We looked at, I would say the problems that we looked at were a little bit different.
So there, the main project I was involved in was looking at dengue.
Dengue, it's a disease in this region of the world, and in the US as well, I think, right?
We're spread by mosquitoes. So we were investigating the genomics between the disease spread and trying to uncover the epidemiology of dengue.
So that was a fairly interesting project.
But the work I look at now is more robotics. So the techniques that we employed were machine learning oriented in both that past research and what we do now.
But I think that the outcomes and the questions that we're looking at were quite different.
Transitioning to Robotics and Mechanical Engineering
Brent:
[2:22] And then as you transitioned more to robotics, did you have to refresh your memory on engineering and the more mechanical side of robotics?
Harold:
[2:31] I would say so. So when I started grad school at Imperial, I went in knowing very, very little about the mechanical aspects of robotics.
I did not know much about electronics, for example. I remember having to solder my electronics board and just making a huge mess out of things.
So it was really a steep learning curve.
I think robotics is one of those fascinating interdisciplinary fields, right? I bring together lots of different parts from different like separated disciplines and I'm coming together towardscreating this unique entity and artifact.
So that was definitely a learning experience. I made a lot of mistakes along the way and learned from them. Yeah.
Keller:
[3:15] So where is the field of robotics today? Like I know a lot of people hear robot and very different images might come to mind. Could you give a little insight on that?
Current State of Robotics
Harold:
[3:23] Sure. Sure. So I think that depending where you come from, you might, like you said, you have a different idea of where robotics is.
I think from my perspective in robotics, we are much further along than I thought we would be when I was a grad student.
That said, we are not at the realm of science fiction yet.
So I think if you watch Star Trek or you see Star Wars, you know, you have this idea of robots being able to walk around, do everything, you know, communicate fluently with people.
No, we're not there yet. I think there's a long way to go before we get to robots that adapt.
In navigating the real world. That said, I was just at ICRA, which is the largest conference in robotics in the world.
And we had robots just walking around, moving around, legged robots, and moving around, interacting with one another.
They are constraints, so a lot of them were teleoperated. So people were driving the robots.
There were some robots which would get moving around, but a lot of them were teleoperated.
So yeah, there are some aspects which we still need to work on.
That's what makes this whole field very exciting but I think it's moving along at a very fast pace which is which is what you know I'm really excited about nowadays.
Driving Innovation in Robotics
Brent:
[4:31] And then who do you think drives most of the innovation in robotics is it like the military is it more manufacturing and industry or who like do you think has, the most advancedrobots right now?
Harold:
[4:44] Oh, so that's a complex question. I think that advanced robots, the question might be advanced in what way.
Brent:
[4:51] Right?
Harold:
[4:51] Because as you said, it's an interdisciplinary field. We were talking about how different disciplines come together for robotics.
And there are advancements being made along all these different axes, okay?
So there are advancements being made in the morphology of the robots, in mechanics, the motors.
There are advancements being made in the AI, the intelligence of the robots, okay? And the advancements are made in different sectors.
So the military might have some uses for robots, which may be different from maybe commercial usage, like delivery robots, which may be different from robots that might be deployedin care homes, for example.
So I would say the advancements are being made on all fronts by different sorts of entities, but on different aspects of the robots.
Advancements in Sensor Technology
Keller:
[5:30] And is there like a reduction in cost in sensors that's helping make some of these robots, I guess, more efficient?
Because I know like earlier, it's paying most of the money on just that, and then you still have all the problems.
Involved with matching the sensor to the function.
Harold:
[5:43] That's right. So most definitely. So we're seeing advancements made in kind of all parts of the robotic pipeline, if you will.
So you imagine robots as a kind of like entity that takes in some information about the world from using the sensors and then doing perception on this.
So understanding what the sensors are giving them and then making some decisions based on their perceptions and then taking the actions in the world, actuating that.
So we're seeing advancements along all these different components, right?
So the cost of the parts of them is also moving in the right direction.
So we are seeing sensors getting better and also cheaper, which allows us to drive down the cost of the world.
We're seeing cheaper motors as well on the side and better manufacturing processes, as well as cheaper computational units, right?
So we're seeing GPUs nowadays that we wouldn't have years ago and able to process information, be able to run these deep learning models, which allow, give us all this power in termsof the intelligence.
Gaps in Robotics Innovations
Keller:
[6:43] Are there any gaps within that chain that aren't being met with these innovations?
Harold:
[6:49] Whoa. Okay, so there are a lot of innovations coming across the board.
Integrating all of them together still remains a very big challenge, I think.
Getting all the way from sensing to actuation is not easy. and operating the multiple scales is also hard, right?
So you have, like, for example, just imagine us as we're interacting with the world.
We are, you know, part of our responses are happening at different frequencies, right?
As I'm holding a cup, for example, I'm enacting control in the cup to hold it stably and make sure it doesn't spill.
But at the same time, I'm thinking about, oh, you know, maybe I want to hand a cup to somebody. I want to get a drink of water, right?
And that's happening at a different timescale. So all these things have to be integrated together in a cohesive manner for a robot to interact well in the world with objects.
Impact of Machine Learning on Robotics
Brent:
[7:41] And then you talked about how you're using machine learning for a long time.
How have the recent advancements in AI machine learning, just general processing power, been able to advance robotics?
Harold:
[7:53] Right. So in the past, we mainly used machine learning models on low-dimensional data.
So these are things where we have extracted the problem down to what we call features that then we can process using machine learning models.
So what has changed dramatically is with, you know, the rise of compute, as well as with the advancements made in, you know, like neural network architectures, is that we're doing reallywell in processing low level data.
So what we call like raw sensory data that's coming in from the cameras of the robot, from audio signals, from tactile sensors.
And we're able to process this in a fairly, I would say, streamlined fashion.
Okay, using these modern architectures in order to make predictions about how the world might evolve or to make appropriate decisions about how to act in the world.
Right so these these advancements that are made right now and i would have supercharged robotics to some extent yeah.
Brent:
[8:52] I can imagine that's pretty critical in getting more complex robots because you just talked about holding the cup and how many like sensory inputs and if we were sensors and robotslike how many data points would have to be coming in so like do you think that that's just going to become even more and more necessary like every single robot's going to have to haveany robot that's going to be doing more advanced tasks that have to have an insane amount of sensors and like a large computing process?
Future of Robotics and Sensor Technology
Harold:
[9:19] Ah, so I think that we have to be careful there. So that comes from a paradigm where we want to control everything and, you know, have very precise control.
That's one way of achieving things. I think that's another exciting kind of field of research, looking at soft robots, looking at the mechanical aspects of robots.
So in this kind of way of thinking, you can outsource some of your computation to the environment.
And that's what human beings do. We leverage biomechanics, we leverage physics to operate efficiently in the world, and we don't really compute everything, right?
We don't actually move our fingers in very precise ways all the time, right?
We are able to leverage the environment, we are able to leverage the physics of the world in order to achieve what we want to do in the world.
Keller:
[10:08] And we're going to steal a question from your website because we're looking at questions that came up and it seemed like a good one.
How do we collaborate with robots fluently to solve the problems of the future?
Harold:
[10:17] Okay. So I think that right now human-robot collaboration is still in its infancy.
So there's a lot of work that needs to be done. So if you think about what is it that a robot needs in order to collaborate with a person, I think the robot needs to have some sense of howpeople work.
Building Trust in Human-Robot Collaboration
[10:35] Some sense of intuitive psychology.
So we talk about intuitive physics where human beings, you know, intuitively you understand how things work, you know, how things move in the world.
When you push something over, it collapses. You know, we have this idea, right? That's how we play Jenga and work with Legos, right?
Similarly, we have a form of intuitive psychology when we work with people.
So when I say certain things, I kind of know how that's going to make you feel, how you're going to react as a result.
As I do things, you're observing me. You have an idea of what I'm feeling, what I'm doing, what I'm trying to achieve.
My beliefs, my hopes, my desires. And I think that robots need to understand these things fairly well for them to interact with people.
And we are building these things, but that's where I think the science of psychology and cognitive science come together with engineering and computational thinking and algorithms andcomputer science in order to operationalize some of these ideas into robots and get them to have this intuitive psychology.
Brent:
[11:38] Yeah. Yeah. And for the field of psychology, it's already so complex just dealing between human to human.
How do you think we're going to be able to make that, like bridge that gap between robot and human?
Psychology and Trust in Robots
Harold:
[11:49] So this is challenging. I think that we need to already build some common sense aspects.
What I just mentioned about intuitive psychology into the robots.
This is already quite challenging because, but, and so that's to start.
And then to get the rest of the way, I think the robot has to learn because as it interacts with, no two people are exactly the same, right?
We are similar, but we're not exactly the same.
So if it just uses some rules, that's not going to work.
It's going to start off by saying, okay, I have some idea of how people are, but to learn about a specific person, I'm going to have to interact with them, okay?
What we want to do is to make sure that this kind of interactions happen in a good way, right?
We want to make sure that the robot doesn't, you know, unnecessarily violate social norms or put people at risk.
And we want to make sure that they end up to be trustworthy entities, in a sense, when they're interacting with people.
Keller:
[12:46] Is there like a program that's used as like the psychology of the robot?
Like what's the input given to the robot to help have it understand these social norms?
Harold:
[12:56] So some of these social norms will have to be baked in, I think, initially, or they maybe can be learned by observing how people operate.
So these are the two principal ways that we're going to put them into robots, at least initially.
And then over time, these norms may change. And then the robots may have to adapt their behavior accordingly.
So I think that if you just look at how people work with one another, how technology has shaped how we interact with the world and one another, I think that can be quite telling abouthow we might change our own behavior as we interact with robots.
Robots so not too long ago we did not have you know smartphones smartphones were not that widespread, right and look at how the behavior has changed over time with the eventinvention of smartphones and the fact that all of us have one of these devices in our pockets right now i mean right before this interview i did i purposefully set my phone on do not disturbright put it away so that i don't get distracted um i don't think i would have had to do that in the past right so this is this has changed our behavior and you can see people walking along thestreets uh just staring at their phones, I mean, that's something pretty new as well.
So this is quite exciting, I think, but also tells us that people's behaviors will change with the advent of technology like robots, and they will adapt accordingly.
And robots have to adapt as well in response.
Vision of Humanoid Robots in the Future
Brent:
[14:22] And I think a lot of people have this vision of the far-out future of a humanoid robot that they're interacting with, just like we interact with a smartphone now.
Now, could you speak on how difficult it is to actually create that droid or human-like robot that's capable of being your general like butler assistant, capable of many different tasks?
Harold:
[14:41] Oh, okay. So this is a hard problem. I know that there's a lot of excitement now because some companies are proposing these humanoid robots that can move around and do all thethings that we want, like the Tesla bot.
And I think Unitree from China has just released a new humanoid robot, right?
So there's problems on all fronts.
There's problems on getting the mechanics right, okay? Just making a stable robot that walks around, doesn't fall over, is robust enough.
[15:08] In a home, okay, in an unstructured domain. Trying to get the robots to have enough power, battery power just to last a day is already a principal challenge.
And the stuff that I'm most interested in is about really the intelligence of the robots, right?
So previously, robots were mainly confined to factories where they kind of performed these repetitive tasks.
Where do we want our human errors? We don't want them in factories.
We want the factories of the future maybe, but I personally would like to see them in our homes, right?
And helping us in our daily tasks. Now, our homes are very different from factories where we can kind of organize things, put things everywhere.
I don't know if you see my home, it's a mess. I have two small kids and toys everywhere.
You know, the clothes are everywhere. And a robot has to navigate in this kind of environments and kind of do tasks for us in the settings.
This is really challenging, right? It has to perceive the world.
It has to understand what the objects are. It has to understand the physics of the world.
It has to understand people, as we talked about before, what if my kids are tormenting the robot?
How is a robot going to respond you know um all these things that has to have to be addressed uh before we can get a humanoid robot that we can you know trust this to deploy in realworld settings yeah do.
Viability of Humanoid Robots in Today's Market
Keller:
[16:18] You think there is a function for the humanoid type robot the multi-purpose robot in the future is like i guess i guess a lot of the companies that are coming out now are trying to youknow gain a lot of hype for it do you think that's solely like a marketing thing or is it more so like there might be viability now despite all those issues?
Harold:
[16:36] So I think that just like most technologies, robotics will be deployed in stages.
I think that right now we're already seeing deployment. I mean, a lot of us have Roombas or similar type of vacuum cleaners in our homes.
There are limited deployments of delivery robots.
They can deliver food to you as well as other items.
We see some deployment of autonomous vehicles in some settings, but there have been some news about that recently.
So about safety and things like that. Likewise, I think for humanoid robots, we're going to see deployment in stages.
I think the first level of deployment will probably be in more controlled settings like factories where we can ensure safety to some degree.
And as robots gain in their abilities, then we're going to see deployments in less constrained settings, maybe in maybe in more let's say what control settings do we have we have, maybein environments like like I would say some aspects of logistics right you see them there in warehouses we might see them and then moving on to places like in the logistic settings inhospitals, or maybe even in how to say when we're talking about delivery of items right in supermarkets and things like that so we might see those kind of deployments and eventually,you know, arriving in our homes. Yeah.
Brent:
[18:01] And then...
Specific Research Goals
[18:05] Moving into your specific research, because we've been pretty general right now. What are you aiming to do?
What problems are you aiming to tackle?
Harold:
[18:14] Right. So I spoke a little bit earlier about this intuitive psychology that I would like to have, that I think robots need.
And my research, to some degree, aims to develop that intuitive psychology.
First to have the basic level of understanding that a robot needs, and then enabling the robot to learn from observations of human behavior, from the interactions with people, in order toupdate this knowledge.
That's one core aspect of this research.
Essentially, I'm trying to work towards robots that are trustworthy, that we can trust to, that I can trust to look after my kids.
Kids i think that would be great i mean right now there is no robot out there that i would trust to to look after my my two kids my kids would kill that robot so i think uh i think that that'suh something that i would love to have so my research is geared in that direction and there's a lot of different parts that have to come into play for that to happen but i think getting robotsto understand people and interact with people safely is one very important piece of that.
Keller:
[19:25] Have you talked to your kids about robots in that way? Do they have a similar interest in interacting with robots?
Harold:
[19:32] So my kids are still quite young. My oldest is seven years old, and the younger one is five.
The older one has seen robots, so he has a fairly good understanding now of what robots, current day robots, can and cannot do.
So he's been to my lab, so he's seen the robots. He has been to the lab at Imperial because we had a visit to London last year, and he played around with the spot robot.
So he kind of knows, oh, the robot crashed into the wall.
And so he has an idea of what robots can and cannot do, which is great.
He has interest in it, so he's mechanically inclined, I think.
Children's Perception of Robots
[20:11] He likes things, so that's good. um i think whether he understood he's too too young to understand all the you know the technical details of course uh but he doesn't he definitelydoesn't have a sci-fi uh understand he doesn't think that robots are you know like in the sci-fi movies partially because he maybe hasn't seen that yet and.
Keller:
[20:33] What are the different facets of trust that you have to build like there's the i guess the psychological aspect but also like i think it's the physical aspect of trust too can the robot, youknow, if you're going to be caring with children, like if it's going to help the children move somewhere, is it going to hurt the child?
Like how do you gauge the sensory aspects of it?
Harold:
[20:53] So I think it's important first to understand what we mean when we talk about trust, right?
So I think I delineate between trustworthiness and trust over here.
So trustworthiness is some property of the robot, right?
So like you want to think about the guarantees of what the robot can do, right?
Maybe some performance guarantee or some safety guarantee of the robot.
This is a property of the robot.
So human trust is a little bit different. friend. This is our willingness to put ourselves at risk with respect to the actions of the robot.
So it's something on the side of human, okay?
[21:28] So it's important to, when we discuss trust, it's not to confuse these things because sometimes our trust may not align with the trustworthiness of the robot.
The robot may be fully capable of doing something and we just don't trust the robot to do that.
Or it might be the other way around, okay? Sometimes the robot can't do something, but we trust the robot to do it and that can lead to problems as well.
Understanding Trust and Trustworthiness
[21:46] So I think that there are different elements of this that come into the research.
So when we were looking at human trust in robots, two kind of important facets of it was one, the capability of the robot.
You know, whether we believe the robot can do it or not, right?
The other is intention, because now robots will make decisions on their own, right?
We'll tell them to do stuff, but there are a lot of little decisions that they have to do on their own. and whether we think that those decisions are aligned to our values, okay?
And our trust in robots really hinges on these two aspects along others, but these two are very important.
And what we need to do, I think, is to make sure that our trust in robots are well calibrated with the trustworthiness of the robot, that the robot is not only capable of doing what we think itcan do, but also aligned in terms of the values that it's trying to optimize.
Assigning Values to Robot Decisions
Brent:
[22:47] Could you describe some of those examples of a robot making its own decision and also how you go about assigning value to those decisions and whether or not they align with theobjectives?
Harold:
[23:00] Oh, sure. So let's just say I want the robot to look after my kids for an afternoon, okay? Now, I may have some particular values in mind.
So, for example, I would like my kids to be well-fed and entertained and not get hurt.
Let's say I have these values in mind. The robot, however, may align with me and kind of have knowledge that I want these things, but also may place values in other aspects.
So, for example, it may have placed a higher value in the kids being entertained than being well-fed.
Then it might say okay I'm going to entertain the kids for like 2-3 hours and not feed them well enough okay or it might feed them different things then I would feed the kids right maybeyou'll feed, popcorn and candy because that makes them very happy uh but may not make me happy because i would prefer my kids get a you know nutritious nutritious meal right so ithink that you can see here that there are different values that come to bear and values of different people might might be different right so my kids might have a different value systemfrom me uh and so the robot might will have to take these things in consideration when making decisions about what what to do in the world.
Brent:
[24:15] And then between, do you guys like refer to it as alignment?
Like, okay. So how do you calibrate alignment then?
Challenges in Aligning Robot Values
Harold:
[24:23] So this is a very tricky problem because alignment, there are different aspects of the problem which are challenging.
One, as I said before, different people have different values.
So what should a robot align to?
Who should the robot align to? I can see, you know, if you just imagine in the world right now, different cultures have different value system.
Then you go down to the individual differences as well.
So it's going to be extremely challenging just knowing who you should calibrate to. The other is from a technical perspective, how to perform the calibration.
So, you know, the people are notorious for saying one thing and then doing another.
So what is the robot going to, what should the robot do there?
What should it pay attention to? Should you pay attention to what you say? or what you do?
How is it going to know what is it that you're trying to achieve and what your goals are?
How do you know what constraints exist in the world? How can it learn that just from observations or from instruction?
This remains a very challenging research topic.
Teaching Humans to Collaborate with Robots
Brent:
[25:33] Do you think you'd have more success teaching and aligning the human to work with the robot than actually training the robot?
Harold:
[25:41] Yeah i think that it probably has it will probably be both right we need to one develop better techniques uh for the robot to understand people and to align itself uh but also we needto teach people how to interact with robots uh and this is an educational process will happen over time but yes uh it's just like how we have learned to use our smartphones very effectivelywe'll learn how to use robots effectively as well yeah.
Brent:
[26:06] I think with ai people are already starting to learn a a little bit more.
Just you don't have to talk to it like a human. You can be cryptic and just straight to the point.
And so I think hopefully that interaction will start to get a little bit easier.
Harold:
[26:17] Yeah, yeah, that's absolutely right. So people are coming up with creative ways to prompt these large language models to coax it to do what we want.
And that's really fascinating. And people, you can see how adaptive people are, right, when confronted with the new technology.
You see a lot of innovations taking place, which is very exciting with these new technologies. Yes.
Physical vs. Functional Trust in Robots
Keller:
[26:36] You think part of the gap with the trust is from the fact that the robots have to be physical units that are distinct?
Because I think a lot of people are blindly trusting a lot of technologies in general life.
And I don't think necessarily the function of the robot would be that much different, but it's more just the fact that it's so clearly a separate entity from you.
Is the physical aspect more of an issue or is it the function that they're performing?
The Role of Robot Embodiment in Trust
Harold:
[27:00] Both turn out to be important. So the physical embodiment of the robot actually plays a role in trust.
So we have seen in prior research that if the robot is somewhat anthropomorphic, so it appears somewhat human-like in its form, people may trust it more or certain people may trust itmore.
That said, there is this notion of the uncanny valley.
So the uncanny valley where the robot seems too really similar to human, but not quite.
Then we get all freaked out. okay and we've seen that in in some movies uh animated movies where the you know the animations are close to human-like but not quite there something'smissing and then we get freaked out right we look at it and say that's weird um and we might have that similar uh feeling when we come across these robots as well i think if you have uhpay attention to news maybe you've seen uh some some robots like sofia i'm not sure the dog yeah uh sorry sofia the the human life.
Brent:
[27:57] Okay it's like like the girl okay yeah.
Harold:
[27:59] Yeah so we have robots i wear them you know they may look human at first glance but when they start to move then then things start to get awry so as i said in your mind you'rejust like oh that's a little strange um so i think that uh all these factors come together when uh when when we're talking about humans trusting robots and then when the human the robotsstart behaving in ways that maybe don't align well with what we expect then that can uh Trust can take a hit there as well.
Keller:
[28:30] Do you think the negative effects of trust are a lot stronger than any kind of positive?
Harold:
[28:35] Again, it depends. Okay. So, for example, you want to say, let's look at Tesla.
Not Tesla. Let's not name any companies or anything like that.
So, let's say autonomous vehicle deployments, right?
So, we see sometimes people are overtrusting these machines.
So they're not supposed to drive autonomously in all conditions, unsupervised.
So you should always pay attention to what the car is doing.
Yet we see people who are overtrusting the robots and kind of falling asleep at the wheel. That's not good.
Yet when then we see accidents happen and then the general perception is that, whoa, these things aren't reliable.
And then trust takes a massive hit. right so usually the truth is somewhere in between uh that you know autonomous vehicles are useful they can hit several levels of autonomy not fullautonomy yet uh but they can be useful in lots of different situations and just that we need to use them appropriately uh and we need to get appropriate experience with them uh in order touse them appropriately do.
Balancing Trust and Overreliance in Technology
Brent:
[29:45] You think there's innovation in robotics like like the electric vehicles driving themselves if you have to sit they're paying attention to the road why not just drive it yourself do youever feel like some of these innovations are doing it for the innovation's sake.
Harold:
[30:00] Some are i think that there is some aspects of it i don't think that in the context that you get the example you gave that's necessarily true so there's for example i could be you knowpaying attention but not driving this is lower cognitive load on the user, on the driver.
So there's some benefit to be had there, I'm saying. So we've had cruise control for some time now, right?
And that can be very helpful because it saves you some effort, right?
Now you can have a conversation and not have to manage that aspect too much.
With ontology, that brings down the cognitive load even more.
So I think it does provide some benefit in some circumstances.
Brent:
[30:39] And then do you ever feel like there's roles that robots shouldn't play, especially given the value of a human-to-human interaction?
Harold:
[30:47] That's right. So I'm hoping that robots will take away some of the boring, repetitive, dull tasks that we don't want to do so that we can pursue more creative, more engagingexperiences in life.
That's my hope for robotics. I would be dismayed, I think, if robots started taking on a lot of this role, this replacing interaction that we have between one another.
I think that's what enriches life to a large degree.
And if like the smartphones, they come in between people sometimes, then that becomes a problem.
So I think this is a process that we will have to manage as this technology become more and more advanced.
Robots in Mental Health and Wellness
Keller:
[31:28] Advanced have you had an experience with a robot that felt similarly i guess fulfilling to a human in terms of like maybe it was when they had like a language model like do youthink there's an aspect that robots could fill some part of that connection need there.
Harold:
[31:43] It could be uh i think that sometimes i i just found myself the other day just chatting with chat jpt uh just like a friend i.
Keller:
[31:50] Say hey how you doing you know uh.
Harold:
[31:53] Just for fun um i think that there is some avenue for that so So, people are looking into the potential for robots and AI for mental health, wellness, just to give you a sounding board.And I think there's no harm in that.
I think the harm comes into play when they become barriers to communication rather than being conduits for it.
I think that is what we have to watch out for.
I think that there's definitely use. There's a lot of use for it.
For example, we are trying to trial ChatGPT as 24-hour TAs for our students.
I mean, it's great because some students are studying at like 1, 2 in the morning, and they need a question answered. And I mean, my TA is not going to be awake at 1 at 12.
Sometimes they are, but you can't expect them to answer a question at that hour.
But it'd be nice if you have a 24-hour teaching assistant that can guide you along through the process. And I think robots and AI can fulfill that role to a degree.
Keller:
[32:53] Yeah, someone at our school that was working on live feedback grading using AI, instead of you get your grade back and the class doesn't change, the professor could get generalgauges on where the class is heading and what gaps there are, and then actively change the class to address those needs.
Harold:
[33:10] Exactly. So that's what I mean about AI playing a role of a conduit.
Really, it can improve our interaction, improve the communication between us.
And that's where I hope that this technology will take us.
Impact of Robotics on Job Market
Brent:
[33:21] Are you ever worried about less jobs by more and more advanced robots filling more and more roles, especially given the importance of feeling like you have a purpose as ahuman?
Harold:
[33:35] Yep. That worries me a little bit. So there are, I guess, some people who say that, you know, we don't need to worry because new jobs will be created and things like that. But whatworries me is the pace.
If there is not enough time for people to adapt to gain new skills for those new jobs, then this is a problem because we will have people who are unemployed and may lose their purpose orbecome very dissatisfied with how things are going.
And this is a very natural thing for them to feel, I think.
So we should manage the process carefully.
I think the technology is getting better, but not perfect.
Not superb yet. It's not flawless, right? I think that we, if through our interactions, with large language models as with robotics, we can see their failings.
And so as it becomes more, as people use these technologies more and more, these failings become more and more apparent, right?
That said, we do see some new jobs popping up, like Prom Engineer is a new job that didn't exist until these large language models made appearance on the scene.
[34:35] And I think that to some degree, they will allow us to really expand upon our creative impulses.
You know, I've been using, for example, when I'm creating my slides for teaching, there are some times where I want to, you know, create a nice image or things like that, and I use, youknow, MidJourney or one of these tools to help me create these images.
Sometimes when I'm thinking of a right way to describe something to a student, I sometimes use ChatGPD, you know, to help me to do this, right?
So I think that having these tools in play can really enhance our ability to do our jobs.
Brent:
[35:10] Right?
Harold:
[35:11] I do worry a little bit about those jobs being lost and I do think we need to pay attention to those.
But at the same time, we also should look at the flip side about the enhancements that are coming.
The Balance Between Innovation and Human Happiness
Brent:
[35:21] Do you ever think that like a more pensive, thoughtful, creative world is actually detrimental to a lot of like human happiness?
Because oftentimes you see artists being among the most unhappy people in societies societies and sometimes just there's something so simple and like nice about doing a simple manualtask and if that's what robots are replacing how do we deal with becoming more like creative okay.
Harold:
[35:49] So i think that i'm thinking more further down the line here so i think there's a difference between doing something manual for the joy of it than having to repetitively do, I thinkthose are very, very different tasks. So I would like robots to do the repetitive tasks in the factory so that we can pursue these more creative engagements where we are creating things forthe joy of it.
And this is further down the road. I mean, yes, there will be speed bumps along the way. There will be hurdles while we transition over.
But this will take some time, right? It's not like tomorrow we're going to have robots going to take a while creative endeavor.
I don't think that's the case. And we shouldn't engineer a society like that.
Keller:
[36:37] Yeah do you think despite the fact that you don't need to have any computer science or any background knowledge to use you know chat gbt or these ai models do you think there'sa value for general students whether they might be studying a literature to just know the functional aspect of how these programs are running like how they're doing what they're doingbecause a lot of people i think are using them relatively blindly they don't really have a care of why it's doing what Yeah.
Harold:
[37:04] So I think that one, having a good foundation in computer science, or at least an understanding of computer science will help you to understand the pitfalls or the limitations ofthese systems.
If you know how these systems are trained, then you have an idea of what they can and cannot do, and then you wouldn't trust them inappropriately.
Two, if you have a good understanding, you can also kind of fix things when things go wrong.
Okay if your you know chat gpt isn't giving you the answer that you like maybe you can revise your prompt in a suitable way and you if you you know how these systems are trained youmight have an idea of how to do that um and if you were even more inclined towards computer science then you may come up with ideas on how to improve these systems for the futureright so yeah i think that having a you know understanding across all levels does help and allows you to do different things with these new technologies.
Enhancing Lives Through Robotics in Disability Care
Brent:
[37:59] Yeah. And then could you maybe describe in a little bit more detail some of your work in disability care?
Harold:
[38:05] So I was working with children with disabilities when I was a PhD student in London.
There we were trying to create smart wheelchairs for children with cognitive and physical disabilities and help them get around safely so that they can explore their world.
And we managed to kind of get the system working with two children with disabilities.
And that was a really great experience for me as a young PhD student to see the children happy, you know, as they try out these things. That was really fun.
Nowadays, I'm looking at elder care. As we know, our population demographics are changing.
And we need to make sure there's sufficient support for people as they grow older and while people are living longer.
And so we would like to make sure that they have, you know, enjoyable lives, right?
Brent:
[38:54] Yeah.
Harold:
[38:54] So yeah, we are tuning our attention towards those types of tasks that we think that robots may be able to help with.
Advancements in Assistive Daily Living Tasks
[39:02] For example, assistive daily living tasks.
Brent:
[39:06] Where are those robots currently in their abilities to help?
Harold:
[39:10] So our robots are still very much at the infancy level.
They can perform well in lab situations, I would say, but there's a far cry from being deployed in real-world situations.
So our robots, there's still a lot, lot of work to be done.
So for example, we see some elements of, there's some great work being done on assisted feeding.
So getting robots to help feed people who cannot feed themselves. so there's some technology some nice advancements being held there and there are trials being conducted now um on onthis task uh but having a full-fledged system that we can deploy in a that is low cost uh that is effective enough for a wide variety of people that remains a principal challenge.
Brent:
[39:53] Yeah and then that feeding arm what that feeding like robot would it look like an arm attached to the side of the bed just helping feed like what does it look like visually So.
Visual Design of Assistive Feeding Robots
Harold:
[40:03] The robot that we have now is an arm that can perform this task.
It looks like a robot arm that can hold a fork and then feed somebody.
Whether that is the eventual morphology, the eventual embodiment of that system as it is deployed, that remains to be seen.
Because right now, our robot arm is a general purpose arm. It can do more than feeding.
And if we want to get drive the cost lower, we may need to make modifications to that. that yeah.
Brent:
[40:29] Because what would be like some other goals with like helping flip them over helping them stand up could like that type of arm do those roles.
Harold:
[40:36] So the arm that we have now in my lab at least uh is not able to do that role principally because of payload so it can't lift uh people yeah right it can't handle the weight right so it'snot strong enough yet uh whether we can engineer arms they can do that um down the road yes it's but the problem is weight um and the the cost of those arms tends to go up.
So that's gonna be a challenge, okay?
And then safety becomes a major issue there.
If you have these huge motors, you have to make sure that they don't injure the person in the process of picking them up or rolling them over.
Keller:
[41:11] And then you mentioned success in the lab. Could you explain how those labs are built? I think it's like the human-robot interaction simulations.
How do you set the stage for testing?
And how do you try to mimic the real world?
Harold:
[41:23] Right. So at NUS, we have a nice lab called the Robot Living Studio.
It's designed like an apartment, actually. So it has a kitchen, a kitchenette.
It has a dining table. It has a living room with a TV.
And we conduct experiments in this kind of space in order to simulate what the experience might be like in a real-world setting, where the robot might go and how it would interact withpeople and things in this environment.
So that's how we can push our lab experiment closer to real-world.
I would like to say that there's nothing, it's not a replacement for in-the-world studies.
So what we really like to do is to deploy robots in real homes, in real hospitals, in real shopping malls, doing these tasks.
And that's where the field is headed to a degree. We see more and more in the world studies.
These are very hard to do, but very, very useful, both from testing out the technical advances that we're making, but also understanding how human beings act and react to robots in thesekinds of situations.
Brent:
[42:23] Yeah do you think so let's take the apartment for example do you think a smart home with a lot of different sensors like an internet of things connectivity with a robot integratedwould allow like that uh robot to operate in that type of environment a lot smoother versus having to have one robot have all of its sensors like internal yeah.
Harold:
[42:45] So if you have more information to the robot in this sort of setting, in that context, I think it would help, yes.
There are other issues that come into play. So you have to make it well integrated.
And that may be a challenge, okay.
The other is robustness. So if you have things on the robot and you design it so they can operate on its own without having to rely on these external sensors, then the robot is self-contained.
And if it had access to the sensor, it could use that. If the sensor goes down, it can still work on its own.
That's what we mean by robustness, right? If one sensor goes down and the whole system ceases to operate, we don't like that, right?
So yes, we would like to have this internet of things, have this connected kind of home, smart home, as you mentioned, with a robot. That would be great.
At the same time, I think that we need to cater for things failing, the robustness of the overall system.
Keller:
[43:39] And you also mentioned like trying to drive the cost down. I guess, who are the agencies that are funding a lot of these research?
Funding and Business Models in Robotics Research
[43:45] And is there any talks about like with the childcare or the elder care, like healthcare providers, insurance providers covering robots?
Is that a discussion that's been had?
Harold:
[43:54] Yeah. So I think that there are companies looking at various sort of business models in order to make sure that these technologies can be deployed in a cost-effective manner.
So, for example, instead of selling robots, one option may be to rent out the robots to corporations, hospitals, or companies.
So, there are companies in the U.S. which are doing this, are trialing out this at the moment.
In Singapore, not yet, but we're hoping down the road to be able to do something similar.
Brent:
[44:27] Where in the world is leading most of the robotics?
Harold:
[44:30] Oh, there's pockets of robotics all over the place.
You know I think principally you see a lot of development happening in the US as was in China and Japan some in Singapore as well so the major players tend to be from what I can see atleast the US Japan China Is Boston.
Brent:
[44:49] Like a hub because the Boston Dynamics are they leading or are there different areas in the US?
Harold:
[44:55] Yeah, it's the different areas in the US and also in different areas in geographically but also in terms of the technology development development right so uh we boston recently wasacquired by hyundai okay um and hyundai has set up shop here so we've been talking to to some of them as to how we can utilize technologies uh for for human interaction yeah.
Keller:
[45:19] And that kind of reminded me like boston dynamics you know they did a lot of bionic stuff and like i guess what distinguishes a robot from another other type of technology like isthere a strict term.
Defining What Qualifies as a Robot
Harold:
[45:30] There is no strict term um there there is some consensus about what a robot is um so it should be an agent to some degree uh it needs to make decisions on its own act in the worldum act in the world meaning that some sort of physical embodiment i think that's what um some consensus is uh that said i would like to say that the term robot is very general right it'sused a lot in different settings and for example there are software robots where there is no physical embodiment, but when I talk about robots typically I do mean I take it in a broad sensebut when I talk about the robots research we do typically it's an embodied robot so it inhabits some sort of physical presence in.
Brent:
[46:09] The world would you say like the quote unquote iron man suit that people have seen where like the military is like using to like carry like 10 times the load that they otherwisewould would that suit be a robot.
Harold:
[46:20] Yeah yeah I would regard it as a robot it's an exoskeleton yeah so I would regard it as a robot okay It's a control robot, but yes, it's a robot.
Keller:
[46:29] And then in our intro call, you said that you thought that everyone would have a robot in the future.
Could you expand on that vision and kind of look out towards the future and what, you know, our interactions might be?
Brent:
[46:41] And also maybe add some time scales on that.
Harold:
[46:44] Oh, I wouldn't know how to add time scales. This is like crystal ball gazing.
Very, very precarious exercise, crystal ball gazing. I don't know.
I think that instead of looking forward, maybe we look back.
Look back to history. so when was when was the first car invented i think it was 1880s that we had uh we had car bands had patented the the car um and that that car looks nothing like thecar we have today how many people had cars back then very very few nowadays cars are in you know almost everyone has a car not everyone but you know a lot of people have cars okayuh how about planes right The first powered air flight with a pilot was the Kitty Hawk, right? That was in 1903.
That's about a century, a little more than a century ago.
[47:33] Today we have, I think that flight was like 250 meters or something like that, right?
I just took a flight to New York and back. The flight to New York was like 18 hours long and 15,000 kilometers, something like that.
So look at the change that's happened in 100 years. Okay, it's crazy.
Now that budget airlines almost everyone can fly both of you have flew all the way from yeah from from california to singapore to you know do an exchange i mean could you imaginedoing that in the 1900s you hear that's that will be unthinkable i think um likewise i think that uh we're gonna see some changes in robotics is just this useful technology that is just it's souseful that i I cannot, if it does what we think it can do, there's no way that it would not be widespread.
[48:28] So I think that right now we see limited, you know, like limited uses of it in factories or in labs around the world.
But what we want to do is we kind of take that out of the lab, right, into home.
And I think that's going to happen because the technologies are just too useful for them not to happen, right? Just like cars, just like planes, just like smartphones, right? It's going tohappen.
And that's what I mean by everybody's just going to have a robot eventually, just like how most of us have.
Ways for Students to Get Involved in Robotics
Brent:
[48:59] And then how can, like looking forward, how do you want, or how would you suggest students start getting involved in robotics if they're interested, whether they're like highschool or like in college or getting involved in like a lab like yours?
Harold:
[49:12] Yeah. So there are many, many different ways. We live in a, it's a great time for learning.
I mean, if you want to learn something, just go online and you can Google robotics, you know, and classes or lectures and immediately things pop up.
Right. And there are lots of these. Nowadays, a lot of universities like NUS and schools like MIT and Stanford, they all offer courses for free that you can take online. And many of themare accessible.
And so I would say that's never been a greater time for learning a new field.
And if you're interested in these things, your first internet is your first stop.
The second stop might be your local university, your local institute of higher learning.
You might want to reach out to people like myself who are working in the field.
And very often we can accommodate some amount of interns.
Not always, depending on the level of business and the activity level of the lab.
But I would like to encourage students of all ages to start exploring.
Ask questions. I think that's the first thing you want to do.
Ask questions about the field.
Like what you're doing here today, you're asking great questions about how robotics is changing and what the future might be like.
And asking questions is the way to get started. And then start searching for those answers.
Entry-level Kits and Simulators in Robotics
Keller:
[50:29] And with the robotics, are there any small kits that people come by?
Because I know a lot of people that do electrical engineering, they like to buy circuit boards and tinker with those.
And it's like a cheap way to get involved where you don't need an institution.
Is there anything similar with robotics?
Harold:
[50:40] Oh, yeah, plenty. Again, Google is your friend here.
You can find kits that are kind of fairly easy to use. So my kid at home has his Lego Mindstorms kind of thing. So, you know, he got that from his cousin.
So, yeah, that's one way to get kids interested in this or start looking at programming robots.
Robots um there are you know more advanced kits if you are in you know college or you're in high school uh there are plenty of these you can start with arduino boards yeah right gettingthem starting hooking up the electronics together uh there are lots of different tools which are pretty low cost nowadays uh and can get you uh gaining the skills that you need to be aroboticist.
Brent:
[51:18] Yeah and i'm sure there's still a lot of simulators as well to make the barrier to entry even.
Harold:
[51:22] Lower right Right. So there's a simulator. When I was a PhD student, we, okay, not me, maybe my supervisor, when he was a PhD student, he had to build everything from scratch,right, to get a robot. It was hard.
Then when I was a PhD student, we already had some of the tools in place and the software simulators, they were already in there. There was something called Player Stage at that time.
So Player Stage has morphed and now we have something called the Robot Operating System or ROS.
This is open source online anyone can download this and start playing there's a simulator in there called gazebo nvidia has their own simulator called isaac and this you can you candownload this from the internet start installing this on your computer and start playing around robots and simulators and this is also another way a great way to get started uh in this field.
Embracing Computational Thinking
Brent:
[52:06] Yeah and then with the mindset of asking questions getting involved is that what you would define find is like thinking like a computer scientist because we want to hear yourthoughts on what is it to mean to you to think like a computer scientist and how can people learn from this mental model right.
Harold:
[52:24] So i think asking questions is just a generally good thing to do so uh it's not limited computer science any any field you're in i think you should ask questions uh and seek outanswers okay uh computational thinking is perhaps more narrow uh in the scope so the usually Usually in that way of thinking, of the mode of thinking, we try to think of problems in acomputational sense.
How might we frame this problem and its solution such that it can be automated or executed in a computer?
So this is one way of thinking. It's not the only mode of thinking.
I don't want to say that, you know, it is the principal mode of thinking also.
It is the way that it is a mode of thinking that is useful and is tremendously useful nowadays because we have computers and we have robots that we can start to, when we think of asolution, we can then automate that solution using one of these machines.
The way of thinking also allows us to frame problems and their solutions in a way where is algorithmic so it's step by step right which is also appealing to us because then we have arecipe for how we would solve a particular problem.
Keller:
[53:36] Yeah i feel like there's something very common about the logic based behind when i first started trying to do it i was like okay i can kind of get out of the minutiae of generalthought and just try to think okay what's the simplest way to get from a to b.
Harold:
[53:48] Yeah so one of the key aspects of this problem solving technique is abstraction so to get away from all the minutiae all the little details that don't matter what is what are theelements the key key elements of the problem, right?
And then we can start to manipulate those in our mind.
Utilizing Abstraction in Problem-Solving
Keller:
[54:02] Yeah. And then do you have any other parting advice for students, whether they'd be interested in computer science or robotics or just generally?
Encouragement for Curiosity and Focus in Learning
Harold:
[54:09] Oh, stay interested, ask great questions. You know, you live in a fantastic time.
I mean, you have access to all, so all the world's information at your fingertips.
So stop looking at TikTok all the time, unless they're TikTok videos of like computer science or something like that. Like, you know, there's some information, but I think sometimes weget distracted.
So managing your attention is very important. I think that is your most important resource at the moment.
And it's too easy to get distracted nowadays with lots of things that don't matter, all right?
So you got to think about what matters to you. What do you really like to do?
And then seek for information that helps you to achieve what you want, right?
Don't just get, you know, don't just be receptacle for people to pour stuff in, and be active in your inquiry of the world.
And if you can do that, I think you'll go far.
Brent:
[55:02] Perfect.
Keller:
[55:02] Thank you.
Brent:
[55:03] Thank you.