Cautious Cars, Cranky Kitchens, Demanding Devices — transcript
Machine-generated transcript — not human-reviewed
About this transcript
The original audio is authoritative. This draft may mishear words or names, omit speech, and merge questions with answers. Changing voices have not been identified, and meaningful non-speech sounds have not been described. Unflagged text may also be inaccurate.
The full recording was submitted to local speech recognition. Generated text is preserved in its original order, including suspected repetitions. Machine-flagged uncertainty identifies possible errors, not verified corrections. Gap notes may indicate silence or omitted speech; they do not establish which.
Download the original recording (MP3 audio, 1:14:55). Timestamp links open that recording at the indicated time where the browser supports audio fragments; the displayed times can also be used to seek manually.
Send a transcript correction to Joseph, including the timestamp and the words you heard.
Generated transcript
0:00–5:00
Good afternoon, everybody. I'm Bob Gloschko, in the Information School. It's a real pleasure to invite you all to our first distinguished lecture of the spring semester. And it is a great personal and professional privilege to invite and introduce Don Norman today. You've often heard a distinguished speaker introduced quickly as someone who needs no introduction. And that's certainly true here. But that wouldn't be fair to either Don or to you. So please let me have two minutes to introduce him properly, and then we'll let him talk. All of you know Don Norman for some important and provocative ideas, but most of you, especially
those who say at this school have studied human-computer interaction and user interface design, think of Don only as a usability and design guru. That's about one third of the Don Norman that I know. Don has done foundational theoretical and applied work in innovative ways in every field he's ever worked in, and in nearly 50 years he's worked on a lot of them. After studying engineering at MIT and Penn, about 40 years ago Don Norman was innovative
in using engineering methods to study human perception. This work led to new ways of thinking about memory and attention, and within a few years he'd written a couple very influential books, Memory and Attention and Human Information Processing, that laid the foundation for the human information processing perspective of on cognition, which led to the field of cognitive psychology. About 30 years ago, Don Romer and his colleagues and students at UC San Diego led the expansion of cognitive psychology
out of the narrow confines of the university laboratory and into deep ideas of semantic representation to study language, memory, and learning in the real world. Don went from the university laboratory for the world of complex systems like power plants, operating rooms, control rooms, airplane cockpits. And while today it's fairly vogue to talk about ethnomethodology and systems development, Don was one of the founders of this area 25 or 30 years ago.
About 25 years ago, when the PC was being invented, Don was one of the first to study it as a piece of software, as a hardware, and as a tool in the world. This was a time when he essentially was one of the founders of two separate and related disciplines of cognitive science and human-computer interaction. Now you all know the phrase, user-centered system design, a phrase that Don coined, but you may not realize that these are the initial letters of the academic institution where he did the work.
Now it's been almost 20 years, within the last third of Don's career, since he wrote The Design of Everyday Things, which is a landmark book that was a brilliant synthesis of theory and practice about design. This is about the time he left Apple Research, and what left UCSD for Apple Research at elsewhere. And the next decade he wrote several other influential books, including Things That Make Us Smart, The Invisible Computer, and most recently, Emotional Design. We are truly fortunate to have such an extraordinarily productive and insightful thinker as Don Norman here today, so please let me welcome Don Norman.
Untranscribed interval, 3:13–3:23: possible silence or omitted speech. No description of this interval has been verified.
Thanks, Bob. Of course, when Bob asked if I was interested in working with the startup, I said no. And that was so funny. When I met two young graduate students at Stanford who said, could I help them with their startup, What was the startup? Another search engine. I said, yeah. There's still time, Don. There's still time, right.
So I've been bothered recently by automation that's entering everyday life. As Bob's history kind of pointed out, many years ago I did look at automation in nuclear power plants, in aviation, both commercial and military, and in other situations. And there we learned that automation was introduced in ways that was actually quite dangerous.
People made assumptions. The people using the systems didn't always understand the assumptions. people using the systems often had one idea in mind and the automation had another idea in mind but of course it didn't have a mind and there were many conflicts in the industrial world you have very highly trained and skilled operators and usually quite a bit of time about a three mile island incident nuclear powers took days if you're in an airplane and everything goes wrong
5:00–10:00
and the airplane starts crashing to the earth, you have several minutes with highly skilled pilots. If you're in an automobile and something goes wrong, you have an ill-skilled person who's not paying attention, who doesn't want to pay attention, and who may have a half second. The automobile is far more dangerous than the airplane. As you can just look at the accident statistics, over 40,000 people killed every year in the automobile in the United States alone.
Six million people injured. And in many years, we have zero fatalities in commercial aviation. So, that's what I'm worried about, and that's what I'll be working on. So let me tell you a story. Gee, what did my story go? I'll have to make it up.
I didn't take it. What's happened with this talk, actually, is that in theory I'm writing a book, and as usual, my books take a long time to get them into my head, and nothing comes out until they're ready. When I'm ready, I can write a book in a month or two. People say, wow, that's fast. They don't realize it takes two or three years of struggling with the ideas. So when I agreed to give this talk and a whole series of talks that follow it, wow, this is March, and I agreed, I don't know, many, many months ago, and I said, I'll be ready then.
Ah. But I think I am ready now, and this is the very first presentation I will give on this matter. it'll be pretty good three or four times from now. But I think I now understand the points I want to make. I'm still learning how to make them. But imagine this. I'm driving my car through the winding mountain roads from Palo Alto to the ocean. And I'm enjoying it.
Beautiful. The car responding wonderfully. And then I look at my wife and I see she's sitting there tense. Her feet jammed against the floor, her hands jammed against the dashboard. And I say, well, what's the matter? It's okay, it's okay, I know what I'm doing. Now imagine another scenario. I'm doing the very same drive, but I notice my car is tense. The seats straighten up, the seat belts tighten,
the dashboard starts beeping at me, the brakes start to be applied, Oops, I say I had better slow down. I was once a member of a panel of consultants at an automobile company, and I told this story, and one of the other consultants was Sherry Turkle, an old friend at MIT who studies these very matters, and who's a good friend of my wife's. And she said, hey, Don, what is this? How come you trust your car more than you trust your wife?
well how come it's actually interesting to think about why I might respond differently to the two I think the answer comes down in the end to a rather simple point it's not that I trust my car more than my wife it's that I can't do anything about it when my car starts complaining there's no communication with my wife I can ask her what the real issue is try to reassure her or change my driving. With a car, I have no idea what's going on.
All I know is that the car isn't letting me do it. And therefore, I could change my driving. I mean, this happened, a similar story. I was in a friend's car. He was driving me to the airport. And I noticed he had a fancy new navigation system. And he said, do you like your new car? I asked him. And Tom, who was driving me, said, oh, I love the car, but I never use a navigation system. I don't like them. I'd like to decide what course I take. And this doesn't give me any say.
Notice Tom's predicament, that he doesn't have any say. Now, that's the observation that really gets to the heart of the matter, is that when we communicate with our machines, we aren't communicating. We are getting more and more intelligent machines but put quote marks around the word intelligent. They aren't intelligent. The designers will tell you they communicate with you. There's a whole field of human-machine interaction
which talks about human-machine interaction, human-machine communication. But you know what? It's a monologue. Two monologues. And two monologues don't make a dialogue. We command our machines, and the machines command us. and it goes back and forth. I command it, it commands me. There's no give and take, there's no discussion, there's no common ground. There's none of the essentials for a real communication. And that's the real problem.
10:00–15:00
Now this was a complaint that Socrates had 2,000 years ago about books. Socrates said, a book is a horrible thing because with a book, what, with a person, you can talk back, you can reason, you can argue, but with a book the words just sit there on the page because there's no way of arguing with it. Well, that's the way it is with our machines. So Sokofi was right about our machines today.
Well, I listen to my car more than my wife because I have no choice. More and more of the devices that we are getting in front of us present us with this. They give us no choice. So what I'm going to do today is show you something. Well, let me go on a little bit first. Tell you about my friend, Jim.
You probably know all these people, Tom and Jim and so on. You just have to guess which ones I need. My car almost got me into an accident, Jim told me. Your car? How could that be, I said. Well, I was driving down the highway using adaptive cruise control. You know about cruise control? You set it and the car keeps a constant speed. Do you know about adaptive cruise control? It's only on the expensive cars, the luxury cars. But it has a front end, it has a radar in the front, or sometimes a laser.
And what it does is it looks to see if anything's in front of you. And if a car darts in front of you, it will slow down. and it will slow down to keep a constant distance in seconds between you and your car in front of you. So in other words, this distance depends upon the speed. The ones that are out today only work to about 30 miles an hour and they turn off at speeds lower than that. But however, the first, they're now coming out where they'll work all the way down to zero. Mercedes-Benz just announced it on its S-Class.
As I say, they start at the top and work down, which means that if you're on rush hour traffic, You don't have to do anything, right? You just follow the cars in front of you. If they stop, you will stop. Well, everybody's worried about that one. For example, the car doesn't know when there's a pedestrian in front of it, doesn't know there's a stop sign, doesn't know that there's a red traffic light. So, Mercedes has announced that it will stop, but to start it up again, you have to push on your gas pedal.
Notice, however, that if the car in front of you doesn't stop, when it's the last one through the traffic light, you will follow it. So, that's adaptive cruise control. Now, Jim is driving with adaptive cruise control on the highways near San Diego, and he's been doing it for quite a while, and he's actually going rather slow, because it's a crowded highway, and the cars in front of him are going slow, so his car is properly and safely going slow. Now he's nearing his exit. So he moves over to the right lane,
and when the exit comes, he gets off the exit. And the car says, hey, there's nobody in front of me. And he takes off the highway speed. So I've told this story now to engineers from several automobile companies. And their responses have been all similar. They have two components. The first one is to blame the driver. Well, why didn't he turn off the cruise control before he exited? That's what you're supposed to do. And I explain he forgot.
And they say, for God, stupid driver. Well, you know, you can't design things for the ideal person. You have to design things for real people. And guess what? All of us forget. You're driving along for a long time. If you actually look at the dashboards of these cars, there's no indicator. You don't know what's going on. Those of you who have cruise control know that sometimes there's a light that comes along when you arm the cruise control. Not when it's working. when you've told it, get ready to work.
So cruise control is bizarre. You tell it, get ready, and then you can turn it on, but there's no indicator that says the difference. You hit the brakes, cruise control turns off, but the light stays on because it's still on. You don't know what mode turns in. It's a major problem, these kinds of mode problems. So forgetting, of course you forgot. I can predict lots of people will forget. So I explain this to them, and they say, well, that's okay, don't worry. We'll fix it. Why, let's see. No car that has adapted cruiser dog
15:00–20:00
doesn't have a navigation system. They all come together as a package. So, hey, the navigation system will know that he's on an exit, so obviously we will slow down appropriately. Sure. We trust that. The navigation system doesn't always know that you're on an exit. Try it. Trust it. Let it pop. On top of that, unexpected things always happen. And there are two things I can guarantee about unexpected things. First, that they will always happen.
And second, that the unexpected thing will not be expected. Which means that the system will know how to respond. So, in the automobiles, it's getting worse and worse. And this is now coming into other things besides the automobiles. It's going to come in... I haven't been using you, so therefore, obviously, I will turn you off for protection.
Now I have to get somebody in the room who knows the magic password so I can continue. Dilbert? Username is Dilbert.
Untranscribed interval, 16:19–16:31: possible silence or omitted speech. No description of this interval has been verified.
Adaptive computer control. Here's something that Toyota is now doing. You're driving along and it's adaptive cruise control. Notice there's a car in front, but notice it's getting closer and closer and closer, and it predicts that you're going to have an accident. And so what the Lexus does is it looks at the driver. See the TV camera there in the steering wheel? It looks at the driver, and then it has a nice little algorithm.
And the green on the left is normal driving, and then driver not facing straight ahead, plus the possibility of collision. So the next thing it does is a warning light and buzzer. And then if nobody responds, well, then there's a high probability of collision. And so it warns about the brakes. And then when the system concludes collision is imminent, it just puts on the brakes. But then notice that the engineers carefully said, collision occurs.
To the point being, these cars are not only driving for you, they're watching you. And they're trying to figure out what you're doing. and they don't have actually much evidence about what you're doing. In this case, all it knows is which way the head is facing it. It doesn't do anything nearly as sophisticated as look at where the eyes are directed. It's just how the head is facing. But even if you knew where the eyes were, it wouldn't do it. And I can imagine this happening. Mr. Nexus.
What was the defendant doing just before the crash? Correct. He was looking to his right the whole time, even after I signal and warn them of the danger. So this is coming. The conversation won't happen, but let me assure you that the contents of the hard drive are going to be subpoenaed. They already have been. It's already a black box in these systems. They already have been used in accidents.
Now it's interesting to contrast the way that different automobile manufacturers have cautiously introduced their automation. They're basically at kind of four levels. You start off with full automation, go down to what's called collaborative automation, and then what I call mind reading technologies, the Toyota Lexus was an example, and then what I'm calling cautious automation, 80% solution. if any of you know about supervisory control theory that's sort of what people in supervisory control are worried about
different levels of automation I'm going to conclude by the end of this talk that there should be two levels of automation there should be full automation that does the job well effortlessly and you don't have to worry about it and zero level of automation where it doesn't try I believe it's the in between ones that get you in trouble The full automation includes lots of support services. Basically, when it's really full automation, it's part of the infrastructure and you shouldn't have to worry about it. In the automobile, it's fuel injection
and fuel-air ratio and anti-skid brakes and stability control. So they work very, very well. You don't have to think about it. In the home, the thermostat that controls your room temperature is full automation. It's a pretty simple one. Even the advanced systems that try to predict what the temperature will be. And you don't have to worry about it. Full automation, when it works, is the correct way. It's the in-between where the trouble comes. But let me tell you that this is happening faster than you might realize in many different ways.
20:00–25:00
So as I was leaving my house this afternoon to come here, I did one last quick check of email, and there was an email from a friend with this movie. And I thought I would show it to you. So I've only seen the movie once. once which is enough to say I should bring it with me. So, let's see if I can do it. It has sound. So, this is BMW. And BMW now has a car that does parallel parking.
So you will see an example of BMW doing parallel parking. It's a serious movie. It's not a joke. It's not a caricature. It's not a cartoon. It's real. It will show you your video and you can see that with a single time they are very fast and very clear. That in modern cars the driver can't do anything, is still a vision, but the trend is clearly behind. What the latest stand of the electric car is, that we will show you now.
The traffic is a bit too much. The traffic is a bit too much. The sad balance is due to the failure of the traffic.
Untranscribed interval, 21:42–22:03: possible silence or omitted speech. No description of this interval has been verified.
Prius actually has done this now for a year or two. So the Prius will automatically parallel park itself, but only in Japan. The very same car the United States will not. The difference between Japan and the United States is the legal system. now I don't know very much about the BMW system yet I repeat, I just got the movie that's now the second time I've seen it
it didn't look like the driver was doing anything, in the Prius the driver has to do a few things in the Prius you were shown on the navigation system a diagram of the parking spot and a rectangle that represents the automobile and the rectangle is in theory you're supposed to park, and the driver has to adjust it and certify the rectangle is placed properly. And then the driver pushes the button and I believe the driver must always have the foot on the brake pedal or it won't work. So it's a partial automation
if you like, it's a collaborative automation. And it's hard to tell from the BMW diagram just what is going on, how much of that the driver must be doing. Collaborative include systems are basically where the person and the automatic system are collaboration. So your navigation system is a kind of collaborative system. You say where you want to go and then it issues recommendations. And it doesn't actually take over so that you can ignore the recommendations. Sometimes it would be
a nuisance as it's reminding you at the next possible opportunity please make a legal U-turn. I've heard that over and over again. On the whole, these do actually quite well. The roots are interesting. Sometimes they're silly, but quite often, even if you know the area well, you discover roots that are better than the ones you've used all your life. And if you bypass a term, it doesn't correct you, but rather it does the proper thing. It simply calculates the new root for where you happen
to be at the moment. So when these work well, they do work well. And people have love-hate relationships with them, they can be very useful, but they can also be very irritating. Now, the directional systems, navigation systems, are not a safety critical item. So they're not going to be the major concern I have. There's a lot to be said about them, but I'm going to skip over. The difficulty with most so-called collaborative systems is that there's no real collaboration.
So even in this navigation system, I'm not really collaborating. I tell it where I want to go and it tells me how to get there. I don't get to discuss with it whether, well, actually, for the first part of the trip, I'd like the scenic part. But then I know it's going to be rush hour traffic, so I want you to avoid the freeways in the second part, etc. I don't get to say that. What I get is a route. And sometimes I'm allowed to say whether I want a fast route or the non-freeway route or the shortest route or sometimes even the scenic route, but it's all or none.
25:00–30:00
there's no real collaboration no real discussion mind reading is another approach which is basically more like brake assist it's been widely observed that if you have an accident coming you put on the brakes most people don't put on the brakes fully interestingly enough whether it's because you're afraid of jostling passengers or whether it's because you hardly ever have a reason to put on the brakes full, so when you do need them, you're not well practiced. So what most cars do today,
again, the fancier one, is they look to see how fast you've applied the brakes. If you applied the brakes fast, they assume you want it on full. Whether or not that's true. Anti-skid brakes are another kind of mind reading, if you like, and for that matter, and so are stability controllers. Now, professional drivers hate ampi-skid brakes and hate stability controllers because they believe, and probably correctly,
that they can control the car better without it. For example, they deliberately wish to skid around the curb, and the stability controllers will fight you all along the way. Now, a lot of the sportier cars let you turn them off. But not completely. So Porsche. Porsche knows a lot of its drivers are really skilled drivers, and the stability control gets in the way. So they allow you to turn off the stability control
when it watches over you. And if ever you break too quickly, it turns it back on again. It doesn't tell you. There's no way of knowing it's back on again. It just silently goes back on again. Because some engineer in Stuttgart has decided that, Well, see, the person put on the brakes that quickly, not as good a driver as they think they are. There's more. Many of the manufacturers are working on lane-keeping systems. So they detect the driving lane,
ensure the car is within the lane, and if not, move you back in. Well, that's a problem. So here's a Honda steering wheel. Notice, by the way, how complicated the steering wheels are becoming. There's stuff all around it, including the stalks, which are sort of hidden by all the other buttons here. But on the bottom right, you see four things next to the ignition key. That's the lane-keeping control. But what Honda does, if you're driving along,
and you start veering out of the lane, it detects this. And so it starts to steer you back. but they're very concerned that you might notice that you might take advantage of this and not even pay attention to the driving so they only apply 80% of the force required to steer you back so you have to keep alert and they watch to see whether you are turning the steering wheel and if you are fine and if you aren't then they beep at you
now what if you're drifting into the wrong lane and they're beeping at you and you haven't done anything what should they do call your lawyers they could they could let you drift they could give you all of the control, get you back into the lane which is the safe thing to do but what Honda does is just disconnect the lane keeping control turn it off, which at first glance sounds crazy, that's the most dangerous
time to suddenly turn off but Honda is clearly very concerned that if they would get you back in your lane, you would simply learn you would ignore the warning signs because the car would automatically take you in and therefore you could do things like this. So this is a picture taken by the journalist who was writing up the Honda lane keeping system in England, hence the right hand drive, although Japan uses right hand drive too. But they assured you in the caption of the article
This is a fake picture. The car was actually parked. When I first discovered this, I talked to some of the engineers at Toyota about this, and Toyota's experimenting with it. And they all told me that I was being very silly and that these systems were extremely relaxing and that they made the job of driving much, much easier. You could really relax.
I'm not convinced we want relapsed drivers. Look at what else is coming. Swarms. Swarm control. Each car has an ad hoc wireless radio communication. It sets up a network with all the cars around it, and it carefully keeps itself a certain distance. It keeps itself close to the center of mass of the pack, but avoids being collisions, avoids colliding with any of the other items
30:00–35:00
and any other object that comes in. Now, what's kind of neat when you have swarm behavior, there's no leader. It's like birds flocking or fish schooling. There's no leader. They do wonderful maneuvers. They do not collide with each other. They do not collide with other objects. So swarm around an object. You don't have a need for lanes. You only have traffic lanes because that helps people coordinate, but with swarm technology, you wouldn't need traffic lights. You wouldn't need stop signs. You wouldn't need traffic lights.
Two different streams of traffic coming through, they would simply talk to one another and the appropriate cars would slow up or speed up, so it would go through without any perception. And I've seen some wonderful simulations of this going on. A pedestrian could just cross the street because it would detect you and all the cars would just maneuver around. Now, of course, one of the problems is that some of us nasty people could take advantage of that. I'm driving behind the swarm, and I want to go 80 miles an hour,
and the swarm is keeping to the speed limit. Well, I'll simply do it. Boom. Go all the way through, because I'll be confident that the other cars will get out of my way. When I mention this to the engineers, they look astonished. Why would anybody ever do that? And that's the problem. There's a long history of automation failure. You have to take into account real human behavior, but in fact, it isn't being taken into account.
So let me tell you about the cruise ship, The Royal Majesty. A modern cruise ship carrying 1,200 passengers is going from St. George Bermuda to Boston. It's about 700 miles. It takes, I don't know, about 40 hours. looks like a pretty simple cruise, and it is, and with all the automatic equipment we have, it's pretty trivial. And so here's this thing going along and along and along, and suddenly, it hits the ground. It ran aground on the shoals of Nantucket.
Nantucket, if you look just to the bottom right of Boston, there's this little hook coming out. That's Cape Cod, and just below that, you can see some small islands. That's where it went aground. why well actually it's been a wonderful accident for accident investigators I've now read three complete reports on it and all my friends keep sending me more it's a very complex story the problem is this
it had a very fancy navigation system controlled by GPS the geophysical global positioning system. Thank you. The only problem was that the antenna from the GPS system to the navigation system was disconnected. Now when they started the cruise, it was fine. They checked it all out and everything was working. So when they started the cruise, it was
fine. But roughly an hour into the cruise, it got disconnected. But nobody knew that. It's only when we went back after the accident we discovered that. But there's no problem. When this happens, the navigation system switches to a system called dead reckoning, where it's estimating the ship's heading speed through the water and computing how far and what direction it's going. And after 30 hours of dead reckoning, the ship was about 17 miles off course. Now that's actually not bad. After 300 miles
30 hours, only 17 miles off. But it was enough so it went aground, and nobody was injured, but the ship was stuck for 24 hours, and it cost $7 million to repair the damage. Now, the complete story is very complex. If you look at this display, you will notice that it's giving latitude and longitude 40 minutes, 19, I'm sorry, 40 degrees, 19.60 minutes north, and 68 degrees, 36.08 minutes west.
0.08 minutes. A one-hundredth of a minute is approximately 60 feet. One degree is approximately 69 miles. One degree of latitude is 69 miles. one degree of longitude near Nantucket's about 50 miles. The point is it's given accuracy within up to 60 feet, 30 hours after the satellite antenna had been disconnected. Can you tell from this display that it had been disconnected?
35:00–40:00
Well, yes you can. This display is actually hidden away because they just look at the big numbers now and then. But if you look at the little letters S-O-L, that means it's computing a solution. But if you look at the little letters R is doing dead wrecking.
And it nicely plots a map for you on the radar screen and there's a little dot in the center, which is where you are relative to everything else. And it's a really complicated story. There were two buoys that they had to look for while they were going, and lo and behold, they found one just at the right time. It turned out to be the wrong one, but if you're in a ship, and this is a buoy, very far away, actually. You can see it on the radar. It's at the right angle, the right time. Everything was right.
How are you supposed to know that's the one on the left, not the one on the right? So you went to the left or the one on the left, and centered between them. But lots of things happened. Yeah, so there's actually a different map. This is a Google Earth map of the trip from Bermuda to Boston. And just to show you where they went aground, they went aground roughly there. I actually can't, I don't know the exact spec
of where they went aground. But the radius of that circle is 17 miles. So you can see that 17 miles, although not much of an error after a 300 or 400 mile trip, is quite large compared to the size of Cape Cod and the islands. Now take a look at Southwest Airlines accident in Midway Field
in Chicago in December last year. The plane, this one, ran off the edge of the runway. Failed to stop in time. Nobody in the plane was injured, but the plane ran off the runway. So the runway is this one that's going from bottom right to top left and slightly that rectangle in the upper left is my attempt to show you where it ran off. It ran off the edge of the runway, which you can see abuts the streets. Ran into the street, hit a car, and killed a six-year-old child in the car.
Now, why did that happen? Again, this story is very complicated. If you actually look, though, at the history of Midway Field, these are very short runways and they are very close to the city streets and this has been a known problem for a very long time in fact the FAA and a few others have tried very hard to put in a safety barrier there have been several proposals it should be at least a thousand feet separating the end of a runway from a street and maybe there should be
heavy gravel so that when the plane got into it automatically it slowed up none of that had ever been instituted there's another important thing to look in the picture runways have a certain distance a certain length and they're computed to make sure the plane can stop in time but where does the plane sit down on the runway planes never land at the very beginning of the runway because that would be dangerous if you actually hit the very start of the runway any slight error you'd hit before the runway
if you look at this runway here notice it's white and then it starts getting dark. That dark area is where planes hit. Because that dark area is actually the parts of the tires that have been burned off as the planes hit. So you can see on the runway exactly where planes hit. There are two runways, this one and this one. And again, on both runways, you can tell roughly where planes hit and how far up they go. When this accident happened, it had been snowing.
the runways were wet and therefore slippery the pilots decided therefore to use full thrust thrust reversers by the brakes automatically and they used a calculator now this is not the calculator because I can't find a picture of it but this is one that's very similar it's a calculator in their laptop that's FAA approved for this purpose this is a different calculator that's FAA approved for other kinds of airplanes.
40:00–45:00
This one is the UltraNav landing computer. It's used a lot by general aviation. This particular one is not suited for the 737, which is the plane. But you can see what is in there. Field elevation in feet, parametric pressure in inches of mercury, the pressure altitude, the temperature, the degree centigrade the wind direction and degrees and in the wind speed and knots the runway heading
the landing gross weight in pounds all sorts of thrust information the speed you're going to be landing at how far the landing is etc etc etc and this particular one on the bottom right you see that it computes that you will take 2,809 feet to stop, and since the runway length is 4,682 feet, it sounds like it should be safe. When they did the computation,
the field told them that it was wet. They were told that the runway is wet. So they entered wet into their calculator. And they put in all the rest of their numbers, and they computed they had 500 feet to spare. If you actually put very wet into the calculator, it would compute you had 30 feet to spare. But take a look at the precision. This is giving you precision to the foot
of how long it will take to stop. When all the stuff going in is gas, we're wet versus very wet. where are you actually landing the plane so how long really is that runway if you landed a thousand feet at the beginning they actually landed two thousand feet after the start of the runway how heavy is a plane you don't really know that you know roughly how heavy it was
when you took off you don't know the real weight you know the empty weight of the plane you know the weight of the fuel that's all you know you know how many passengers and so you make a guess there's a formula you use so there's an approved legal guess which is how much a passenger and the passenger's baggage weighs on average and you just multiply them you don't really know so when you're landing you roughly know how much fuel you have and you compute how much you weigh how fast you're going to be
when you land well the instant you land you know but when you do the computation you're not landing you do the computation 15 minutes before landing. So you don't know how fast it is that you're going to land. There's a whole bunch of variables in there you just don't know the answer to. Yet this thing gives you the answer precise to the foot. Why is that? If, in fact, instead of a precision like that, you've got a range, you might actually realize you couldn't stop. And in this particular case, when does the antithrust,
when does the thrust reverser come on? when you land? No. It always takes 5 to 10 seconds before you can actually get the thrust reversers on. In part because you can't turn them on when you land. There's a switch in there that says you can't turn on the thrust reverser unless the landing gear certifies that the plane is on the ground. It would be dangerous to have them turn on while you're flying. So how long does it take for the planes to be heavy enough on the ground for that switch to say it's okay?
But we don't know the answer yet. The National Transportation Safety Board is looking into that. But we do know that thrust reversers came on 18 seconds after landing. They landed late in the field. The thrust reversers came on 18 seconds afterwards. So when you do the computation, you can see that you couldn't have landed in time. Couldn't have stopped in time. In fact, this is the only plane that's allowed to assume thrust reversers. All the other aircraft are not allowed to use thrust reverses in the computation.
If you don't put in thrust reverses in the computation, you can't land on that airport. Not when it's wet. There's not enough room. So whose fault is it? I don't know whose fault it is, and the NTSB is looking at it, and it usually takes about two years, but they realized this was dangerous enough, they issued an advisory warning. Overprecision. It doesn't go on and on, but I want to get to my conclusion. So let me pass quickly through a few other things.
This is the Mosier home, Mike Mosier, for those who know him, University of Colorado, controlled by a neural network, five miles of pavement, umpteen sensors and actuators. It watches over his behavior and predicts what he's going to do. So in the morning when he gets up and walks into his bathroom and then into the kitchen, it turns on the lights and turns on the this and the that. In the evening when he stands up and says, oh, he's going to bed, and it turns on the lights in the bedroom and turns off the lights in the other places.
45:00–50:00
Turns off the heat when he's not using the house. Mike says it sometimes is a problem, but he's at work late at 7 p.m. and he's still working, and he realizes his house is expecting him and is turning on the heat and getting ready, so he better get home. So this is the mind reading. Now it's fine for Mike because he put it all in and he loves neural networks and that's his field and he has published several papers on his own house.
But he also lives alone. It's not obvious how this mind reader would work with more than one occupant at a time. And it's not obvious that it's really livable in. He recounts a whole bunch of irritating issues that come up with a house all the time. that for him, every irritating issue is another part of his paper that he's going to publish. So it's fine, but not the rest of it. But here's an approach I particularly like. Mike was actually a student of mine,
so it's interesting that here's another student of mine, Abby Sellin, who's at Microsoft Research in Cambridge, England, who has also done a smart house, but just the opposite approach. What she and her group did was try to figure out where the problems were in living in homes. And one of them is keeping track of all the people. When you want to be picked up, and who's late, and who's not, and who's coming to dinner. And so they set up a little bulletin board, and here it is. But the point is, it doesn't try to read your mind.
It doesn't try to do things automatically. It doesn't do what the average smart home people do. The average project doing a smart house says, oh, we'll look into the outlook calendar of everybody, and we'll know just where they are and what they're doing and when they're expected home and who needs a ride and all that. And they try to do all this wonderful AI, artificial intelligence and try to figure it all out. It never works. But the technologies don't care. They always say, oh, yeah, it didn't work because of X. I'll fix X.
And it doesn't work. Oh, but that's because of Y. And they fix Y. But it doesn't work because there's always something unexpected. What's nice about this one is that it's yours. It just helps you. And there's a little nice display. So Richard sends a note from his cell phone to the display at home saying, can someone get me from the train station? And then whoever it was decided, okay, this writes on it saying,
I've gone to pick up bad so that the synchronization is complete. Because you don't want three different people to receive the note and rush off to get bad. You just want one person. and so they have actually tried hard to think through the different issues of synchronization but again it's under the people's control they also have some really nice refrigerator magnets that are labeled by the day of the week so that when you want to remember something on a day you stick the days one up there and when the day comes it glows reminding you to look at it because they
pointed out people put lots of reminders on the refrigerator door but somebody reminders you can no longer find any of them. So they simply said, okay, we'll just make it glow at the time that you preset. So it's a very nice way of doing things. Robots are coming to the home. This is the room of a vacuum cleaner, which is a very stupid robot that has the same intelligence as your pool cleaner. If you really want a robot in your home, the most intelligent ones
are probably your dishwasher or your clothes washer. which have actuators, motors, processors. Some of them have displays. They have sensors. They sense how dirty the water is, determine how long to do the wash. The dryers sense how much water is in the atmosphere and decide when to turn off. Microwave ovens today do the same thing. I've actually discovered mine works pretty well. You just tell it to cook, and it actually stops at the right time. You have to tell it what kind of food it is,
You don't have to tell it what temperature setting or how long to cook. It takes care of that. These are getting more and more intelligent. And my coffee machine is the same. This is probably the most complex thing in my house. A motor or microprocessor is played, and it heats the water and grinds the beans and disposes of the residue afterwards and commands me periodically that it's time to empty the residue and time to give it more beans
50:00–55:00
and time to decalcify it. So, there's a tension going on here is that we see that automation is coming in smarter and smarter automation. We see that in the automatic parking of the BMW and in the automatic cruise control and lane-keeping controls, with automobiles. We certainly hear that about smart homes. I've been in smart home laboratories all over the world, in Japan, in Holland, in the United States, in England. Everybody's
proud to show me that, look, the lights turn on to your mood. We guess exactly how you feel. We play the music that we know suits you at this moment. They control the temperature, the lighting, the music, everything. I mean, even your TiVo record to TV programmer that TV programs is certainly you are going to like. More and more of this is being done for us. At the moment, it's still optional, but not for long. Even the lights are not completely optional.
The lights set up, not only to an amount, but to a hue. It'll change the color of the lights if you're sad or if you're happy. Yeah, you can change it back, but it's a nuisance to have to change it back. Some of these may someday be actually safety tracking. more and more of these systems are being deployed for the elderly. And it turns out the able-minded people don't want it at all. People have discovered oh, I know what, everybody's worried about watching over their elderly parents. So, robots.
What are robots good for anyway? Besides vacuum cleaners and grass cutters. Nobody can figure it out, it turns out. I'm actually with a startup robot company and we gave up. We couldn't figure out what to do with it. because what you want a robot to do is go and pick up things around the house and put things away and so on and you know the dishwasher is a robot but the hardest part about using a dishwasher is putting the dishes into it and taking it out but it's really difficult to make a robot that can do that
like you make a robot that can go and get you a beer and bring it back to you ha it couldn't open the refrigerator that's hard so you have to make a special beer dispenser to make this work. So it's really hard. So, oh, I know it. We'll make the robot go around the TV camera and make sure your old parents haven't fallen down or that they're eating well. We'll monitor everything they eat or we'll watch this and we'll watch that. And so it's really going to be very abusive.
Turns out there's a lot of research money in it. I think that the approach is being done by the Microsoft folks in Cambridge, England. By the way, not to be confused with the Microsoft folks in Redmond, Virginia, who have another smart house, which is all that I'm arguing against. It's really what you want. What you want... See, machines can't communicate with us, and I think it's a fundamental issue,
and nobody seems to understand the fundamental problem. The fundamental issue is that real communication is difficult. If you look at any of the people who have studied psycholinguistics, the nature of interaction and communication, there has to be a common ground, a shared set of assumptions, a shared body of knowledge. There has to be some give and take. There has to be error correction mechanisms so that we can interrupt somebody, ask for clarification, so that we can even change our own mind. We often say things that aren't what we meant or aren't precise to be meant, so we have
all sorts of ways of stopping our utterances and backing up and then continuing correcting. We often do this without even being aware of that and correcting ourselves, and your listeners isn't even aware of it. That's why you can listen to a wonderful, fluid speech, and then go read the transcript and discover the transcript is unintelligible. There's a big difference. Common ground. There is no common ground. Common sense knowledge has been sort of the holy grail of artificial intelligence projects for a long time.
We don't know how to give artificial systems common sense. It's very complex, very difficult. But without having this common ground, without having some way of negotiation, without having some way of dealing with the unexpected, we're not going to ever have automatic systems that can handle the real eventuality. Remember, we have systems today that are very, very powerful, but they're not a little or a heuristic. So even our best chess playing programs, which are very, very powerful,
that's all they can do. If you were to say, hey, I have an idea. Instead of an 8x8 board, let's play chess with a 7x7 board. You know, we'll invent a new game. It'd be kind of like chess. We'll just use much of the same rules. Wouldn't that be interesting? Well, people might like it or not like it, but you could figure it out. you probably could learn to play that game in a few minutes. And it might take you years to play it well, but you can learn it in a few minutes. But our chess playing program,
55:00–1:00:00
you'd have to start all over again and reprogram the whole thing from scratch. There is no generality. So as long as that's going to be the case, and I believe it's fundamental, we really do not know enough, and it's going to be a long time. If it's fundamental that we cannot have real communication and collaboration, then let's not try. If you look at all of the major accidents that have happened, it's because of this halfway automation. The full automation is great.
That hand stuff with things like navigation system that assist us, those are great. But where the system takes over and then gives up, where it doesn't know enough, that's horrible. That's where the accident happens. So I was trying to figure it out. You want a system that is basically always telling you what it's thinking, what its assumptions are, what it's about to do, where it has come from, where it is, how close it is to the transition, and what your alternatives are.
That's what you want to know all the time. But you don't want it always bothering you and interrupting you and chattering. You're driving the car. You don't want the car always bombarding you with signals and so on and saying, I'm tensing up the brakes, I'm straightening up the seat. You're getting close to the car in front of you, but it's still safe. You don't want that. Yet, in some sense, you do. The question is, how can you do it in a way that feels natural? So that's what I'm trying to look at. And I think it's time for a science of natural interaction.
And I realize that we actually know enough to do it, or at least to begin. Some time ago, I talked about something I call natural mapping, which was how do you figure out what switch controls what light or what control? And the argument was simple. You want a spatial relationship, like the spatial... It's a mapping from space to space. If the controls were spatially arranged in the same manner as the things you were controlling,
then the relationship would be clear. Well, it turns out that you can find many similar such things. So for example, I'm flying my airplane, and my speed is dropping, and I'm trying to climb, and there's therefore not enough airspeed over the wings to give me lift. That's called stalling, and it's a dangerous situation. And pilots don't always detect it. They don't detect it. The standard way now of warning pilots about it is to vibrate control sticks.
It's very effective. It's extremely effective. If you're flying along and this thing starts vibing, the automatic response is, something is wrong and I'm going to deal with it. I don't like to know that this is a signal for phone. I know just how to deal with it. I either apply speed or I, better yet, go down. So there's one. General Motors has actually tried this in a car with important things. As you're driving along, that car comes too close to your right or to your left or in front or in back.
the seat vibrates from the left, right, front, or back. And people who have driven this car say they don't have to be instructed. It's just obvious. They are always aware of where the cars are around. Many cars now have backup monitors to tell you how far away you are from the items behind you. And the one car I have is Beeps. So as I back up, and if you catch somebody behind me, It goes beep.
They're like, oh, closer. It goes beep, beep, beep, beep, beep, beep, beep, beep, beep, beep, beep, beep, beep, beep, beep, beep. You don't have to read the manual. No one has to explain it to you. The relationship is very clear. So actually in all of these, well, here's a different example. Someone is guiding you to a parking spot. What do they tend to do? Again, you don't have to talk to them. You don't have to agree about a convention. They go like that.
And as you get closer, they do this. All of these are mapping of a dimension into another dimension. You're taking things that are controlled by amount. Distance, speed, intensity, loudness, more. if you're talking I do that that means louder
1:00:00–1:05:00
Listen from 1:00:09 (MP3 audio)
if you're talking I do that that means slow up and that means speed up that means quieter and there's a wide range of very natural things if you actually examine the ones that are easy to understand at every case it's because it's a natural mapping on some dimension is relevant to a different dimension that's relevant. And so I actually believe that we can come up with a whole bunch of basic design principles.
Listen from 1:00:41 (MP3 audio)
Here's a different example. Here is the map that Yahoo Maps gives me of the trip from my home in Palo Alto to A, to B, here. And you've all seen this, and it looks reasonable. It's a map and so on, but it's really hard to follow. so Manish Shagralla did his PhD thesis at Stanford a number of years ago and he worked with Barbara Tversky he's a psychologist who studies maps and they actually looked and see how people
Listen from 1:01:12 (MP3 audio)
drew maps and talked about maps and people aren't like that everything must be accurate, everything must be shown in the same scale, no they make things bigger where things are important and they compress where things don't matter and they only show the important landmark. So here, I go down Channing Avenue to Middlefield, turn left to Middlefield, to Willow, turn right to Willow, and they basically keep going. Don't worry about the fact you have to turn right and so on. It's automatic. You just follow until you hit 880
Listen from 1:01:43 (MP3 audio)
and then turn left and just go for a long distance until you hit, in this case, 980 and just keep going to 52nd Street to telegraph, etc. So actually, he actually made this into a commercial product. It was actually done by a company called MapQuest or something, which Microsoft bought. And it's now hidden away deep inside MSN Maps. You can actually find it. It's called Line Drive Maps. It's hard to find, but it still exists.
Listen from 1:02:15 (MP3 audio)
I find it a wonderful example of taking the system and make it into a way that we really tend to like and tend to use. Now, it's interesting because most people actually are not completely happy with that. They want that and they want this. So when you think about it, we really do need both. So, for example, altimeters used to be like clock handles, and people made errors all over the place. People can't read the clock, by the way. Clots are horrible.
Listen from 1:02:48 (MP3 audio)
So they switched into digital. But digital we have are horrible also. Because the pilots didn't really need to know the real exact allergy goes through time. They wanted to know if they're going up or down. And the analog hands are very nice and they're watching the spin. So what they've done is they've made the most of the different digits digital. The least of the different digits analog. So you can see the major parts about how high you are. But then you can see the hand which is analoged to the right which means you're decreasing
Listen from 1:03:19 (MP3 audio)
and how fast tells you about how fast so there's no reason why you need to have a single answer you'd have multiple answers rumble strips automatic signaling of where you are roughly on the road and here's rumble strips for the blind so this is in Tokyo and you can see that the patterning is a kind of a rumble strip and the blind use it to locate where they are, and guess what? As is almost always the case of these devices, people who aren't blind use it also.
Listen from 1:03:50 (MP3 audio)
If you're busy talking and so on, you can roughly figure out where you are. Now, why shouldn't the precision of a display match the accuracy? So when this ship was on dead reckoning, well, take off the hundredths of a minute. Take it off. This would be a clear sign that something was different. First of all, it gets rid of the accuracy within 60 feet. And second of all, it's a very clear perceptual marker that something is different.
Listen from 1:04:20 (MP3 audio)
In fact, the more I think about it, it's one minute. A hundredth of a minute is about 60 feet. One minute is roughly a mile. And one degree is 69 miles. So it's possible that actually you want to get rid of the minutes as well. Although So possibly what you should do is wait. And for the first couple of hours, you would leave the minutes on, and after 10 hours, get rid of the minutes. So why not have the accuracy of the display reflect, if you like, the precision of the
Listen from 1:04:56 (MP3 audio)
display reflects the accuracy of your reading? Same with the radar set. Instead of this nice, precise dot, why not say, there's some place in this area. laughter laughter I really do believe that we can have a science of natural interaction and therefore what we can do is I have a whole new philosophy of devising intelligent machines to recognize the intelligence is not in the machine
1:05:00–1:10:00
Listen from 1:05:26 (MP3 audio)
the intelligence is in the designer what the designer tried to do is imagine all the possible things that could possibly happen and figure out how you would measure a particular to the field so you would know where you were and then figure out some way of dealing with it. And that's all done beforehand from imagination and it's doomed to fail. And if we realize that we can make very powerful, intelligent, useful machines, but let them communicate with us in this peripheral way, in this natural way. Use sound a lot more. Use vibration a lot more.
Listen from 1:06:02 (MP3 audio)
make things a natural matter to our perceptual understanding. So that's the direction that my research has now moved and that's the direction where I wish to be going for the next several years. And that's what this book will be about if I ever finish it. So thank you very much. Thank you.
Listen from 1:06:33 (MP3 audio)
I want to hit a couple of questions. Yeah. It sounds like you're not just advocating zero automation or full automation, but for the middle, increased notice about uncertainty. The last few graphs you showed, you had some sort of notice of uncertainty. Is that sort of a refinement? So the question was whether, I'm not just saying really, I said earlier that I want full automation or no automation. and the basic question is that it's doing what you're saying, is it?
Listen from 1:07:06 (MP3 audio)
I think that's correct. That's not what I'm saying. When automation works, it's nice. The problem with the Royal Majesty was that they had been using the automation about five years with never any problem. And they had therefore gotten lazy. Now, lazy is a really the wrong term. they've come to trust and rely upon the automation. And so when it failed, and it failed in the subtle way that it did,
Listen from 1:07:37 (MP3 audio)
they didn't notice it. So what I'm suggesting is that, you know, it was pretty good when it worked. And so why not try to take advantage of those times when it worked, but let's try to change this interaction between the systems and us so that we're always much more aware of what's going on. And the trick is to have it, Today, people have said that a long time. There's a huge literature of people. There's all sorts of papers on how to make a system into an intelligent team player.
Listen from 1:08:08 (MP3 audio)
There's all sorts of systems on coordination, on cooperative systems. There's a lot of work trying to bring these together. I think they all miss the vote, because I think they all are assuming that if only we could get the communication better, the cooperation better, the common assumptions out there, it would work better. And I now believe that's fundamentally wrong. I've been arguing with a colleague of mine who's a team player advocate, David Woods at Ohio State, who's very, very good, and he finally
Listen from 1:08:38 (MP3 audio)
at his aspiration said, did I have the fundamental asymmetry hypothesis? And I said, yeah, that's right. Fundamentally, I believe it's asymmetric. We can have a certain class of knowledge that we don't know how to put into the machines unless you realize that. So that's what I'm saying. I think there is actually a ground where we can take advantage of the useful machine, but not get suddenly surprised when it gives up on it. I mean, there are all sorts of other stories I could tell you, like the plane, the automatic system, there's a fuel.
Listen from 1:09:12 (MP3 audio)
The right fuel tank is losing fuel. So the plane is getting light on the right side and heavier on the left, and it would normally be on the middle like that. But no, the automatic pilot is quietly compensating. So nobody notices. flying along and along and along and then suddenly the automatic climate says to itself i give up i can't do it anymore and boom now if the person applying the person would say what does the matter i've been having to try to compensate for some sort of imbalance and then the person would say and i'm getting closer and closer to where i can't do it anymore
Listen from 1:09:47 (MP3 audio)
so that's what i'm asking for but it has to be in a way that doesn't bother you A lot of people were suggested giving all this information, but it's always talking, always chattering. And I've watched that. If you actually drive in Japan and listen to their navigation systems, they're so polite. And therefore it takes a long time to say anything. And then it's always saying, please pay more attention to criticism. And we are coming to an important intersection with criticism.
1:10:00–1:14:55
Listen from 1:10:20 (MP3 audio)
And so what happened is I was in a car for like two hours, and I don't think it was ever quiet. So nobody listened. Nobody listened at all. And I'm sure that if something important actually happened, they wouldn't have heard. So you have to avoid it. Yes? You mentioned the refrigerator. You like to fix up the gear from the refrigerator, but you can't handle the refrigerator door. is the door something that we need that badly or is it just a holdover from another era of
Listen from 1:10:59 (MP3 audio)
possible appliance design so you know there are there are there times when actually we can accommodate the machine better instead of the machine accommodating us right so that's like that's what I was going to say, right. But yeah, a lot of our stuff in the house was designed for us. Not everything, surprisingly enough. So of course it's not surprising that the doorknob is our height
Listen from 1:11:31 (MP3 audio)
and that the refrigerator door, which has to keep quite shut, often has a magnet that's holding it shut and it takes a lot of force to open it. And that's deliberate. But it's not appropriate for devices that don't have much force so they have wheels on them so they can't pull, etc. So you're right, there might be other things. If you actually look at electric outlets, they're not designed for people, they're designed for electricians. And so there are lots of things that are, you're right, I can imagine that we will change the homes
Listen from 1:12:03 (MP3 audio)
to accommodate our electric. And that's not as surprising as it might sound, and we have changed the homes to accommodate the wiring, the plumbing, and the heating, and ventilation, and all the infrastructure. So what we've done with highways and parking structures and street signs and sidewalks and so on to accommodate vehicles. So I have no doubt that will happen. All right, one final question. I'm sorry, I'd like to give you a long answer. Let's see what's going on here. Yes? Going back to the actual situation,
Listen from 1:12:34 (MP3 audio)
which seems to me a highly social, maybe inherently social situation, where the interaction is mediated by rural roads so it doesn't have to be mediated by Newtonian physics, things like that. The mind reading, the essential mind reading is reading the mind of the idiot in front of you and him is reading the mind of the idiot behind him. We have devices like turn signals that are sort of semi-optional, very optional in California.
Listen from 1:13:06 (MP3 audio)
And thankfully brake lights that aren't optional. If they were optional, then we'd be losing to the house. Do you see any difference in the point of view where the automation is to support communication in contrast to support control of advice? I once wrote a book whose title was Term Signals of the Facial Expressions of Automobiles. Because I decided that the term signal was quite unique
Listen from 1:13:37 (MP3 audio)
in everything in the automobile and that it was signaling your intention to the other. So you had creatures. A creature is a car plus driver, which is a single unit entity. So this car plus driver is signaling its intention to the other car plus driver through the use of turn signals. And many of you will probably signal false intentions, like the car behind you is too close. You tap the brake pedal or put on your lights, hoping that it will scare the car behind you. And there are other things one does. And yeah, there is this communication.
Listen from 1:14:10 (MP3 audio)
We've actually evolved all sorts of ways of communicating to other vehicles, and you try to guess what the other vehicles are doing. Part of that is codified into the legal system, too, to the traffic lanes. So we don't, in theory, you don't have to pay attention to the car on your ride. As long as it stays within the traffic lane, you don't think about it. I think this is a very rich and wonderful topic and I can go on for a long time and I have but I'm totally running out of time
Listen from 1:14:41 (MP3 audio)
so yeah it's a good topic thank you very much Don applause applause
Generation and audio provenance
- Generated
- 2026-09-09T23:02:21.605583+00:00
- Speech-recognition engine
- mlx-whisper 0.4.3
- Model
- mlx-community/whisper-large-v3-turbo
- Model revision
a4aaeec0636e6fef84abdcbe3544cb2bf7e9f6fb- Model weights SHA-256
951ed3fc1203e6a62467abb2144a96ce7eafca8fa77e3704fdb8635ff3e7f8a6- Original audio file
- Donald_Norman_UCiSchool_01Mar2006.mp3 (34139272 bytes)
- Original audio SHA-256
4c4a2c4cc61b71a5d3b1727b47f210ca25d6e2aea9a583f6071bcec9a8c285f4- Completion receipt SHA-256
cd89b98adf7f53a33a0c80b7ca3ecd87611c9195863c23d54556d26cc3831e42- Normalized generated text SHA-256
7ce3456617fbb58f7ecfe31c0abf5c7a6fa1070156d7b5cfb7ebd774bba5fd51
Audio stayed local during generation. This provenance identifies the source and process; it does not certify transcription accuracy.
Back to the top of this transcript · Return to the original talk post