"How does that follow?" Derec asked. "With Eliton's death, no one will be able to build robots on Earth."

"I don't think it matters. Someone will hire you to build bodyguards now, no matter what."

"Excuse me, but I failed to do that."

Ariel waved her hand dismissively. "Glitches. No one on Earth would buy it, but Spacers understand that prototypes always have bugs to be worked out. It brought back three of the assassins. That's the point that won't be missed. You've got your opportunity to build your special positronics now. More leeway, more freedom of action, more humanlike. Hell, they'll even make mistakes."

"Ariel-"

"Pardon me," Mia said. "I feel like I've come in at the tail end of a very complicated argument."

"Derec and I disagree fundamentally over Bogard. What it represents."

"I gathered that much. Why?"

"Derec's robot here is the product of an attempt to circumvent Three Law programming-"

"That's a complete mischaracterization!" Derec shouted. "You never understood what I was after!"

"Really? Tell me something-why is that robot still functioning?"

"What? I don't-"

"It failed," Ariel snapped. "It let a human in its care die. It stood witness to dozens of fatalities and injuries. It should be a mass of collapsed positronic gelatin. Instead, it is fully functional."

"What good would it be if it had collapsed?"

"It would be inert. It would pose no further threat."

"What threat?"

"The threat of negligence!"

"It wasn't negligent! Look at how it performed for Mia."

"Then why is Eliton dead?"

"We don't know he's dead!"

"As far as Bogard is concerned, he is! Why?"

Derec did not know. Of everything he had intended in designing and building Bogard, that was precisely the thing which ought never to have happened. He looked at the unmoving, unmoved machine and wondered what had gone so profoundly wrong that it had allowed the human it was programmed expressly to protect to die.

"Would someone please explain this to me?" Mia asked.

"Derec built it, let him try," Ariel said in disgust.

"Under normal circumstances…" Derec started. His throat caught, and he coughed. "Normal circumstances… whatever that means… the Three Laws are built into every positronic brain, part of the core template. They represent the First Principles for a robot, the foundation on which all its subsequent learning and experience rests. The initial designers set it up so that almost all the secondary programming requires the presence of those laws in order to function. It would require a complete redesign of all the manufacturing methods as well as the basic pathways themselves to build a brain without the Three Laws. They aren't just hardwired into the brain, they are basic to the processes of constructing one."

"Is that what you did? Redesign everything?"

"No. I'm not that good. Nor am I that irresponsible. All I did was set in place a new set of parameters for the application of the Laws."

"You circumvented them," Ariel snapped.

"I did not. I took an accepted standard practice and stretched it."

"What practice?" Mia asked.

"Setting conditions of when the robot perceives that it is responsible for a violation," Derec explained. "Think about it. According to the First Law, if a robot followed it absolutely, all robots would collapse. 'A robot may not injure a human being, or, through inaction, allow a human being to come to harm. ' Consider that as an absolute. Human beings are always coming to harm. Time and distance alone guarantee that a robot can't act to prevent that harm in all cases. If some kind of buffer zone, a hierarchical response, weren't in place, the instant robots realized how many humans came to harm because they weren't doing something to prevent it, we would have no functional robot population. So we establish a reasonable limitation for its application. If a robot does nothing to stop the human it is standing right next to from being killed-say, by a speeding vehicle, out of control-then it fails and collapse follows. If that same robot can do nothing to prevent the same fate happening to a human a kilometer away from it, the only way it fails is if another human forcefully asserts that it was at fault. It does not automatically perceive itself as responsible."

Mia nodded. "That makes sense. "

"Practical engineering," Derec said. "But then we run into a functional brick wall when it comes to certain tasks. Law enforcement, for one. Most positronic brains cannot cope with violent crime. A few have been programmed to dissociate under very specific circumstances so that, say, witnessing dead humans at a crime scene won't create a Three Law crisis. But they still can't make arrests because that offers the potential of harm to a human being."

"But a criminal-" Mia began.

"Is still a human being," Derec insisted. "What a human has done by violating a law does not mitigate the robot's adherence to the Three Laws."

"Unless you redefine harm," Ariel said. "Which is how you got around the Three Law imperative."

"That's an oversimplification," Derec replied. "What I redefined was the sphere of responsibility. Bogard is just as committed to the Three Laws as any other robot, but it has a broader definition of that commitment. It can make the determination that limiting a human's freedom action, even if it results in a degree of harm-bruises or strained muscles, for instance-may prevent harm from coming to other humans."

"You're telling me you gave it a moral barometer?" Ariel demanded.

"Sort of. It relies on the human to which it is assigned to make that determination. It also recognizes a human prerogative to go in harm's way should circumstances require risk to prevent further harm."

"And when confronted with a clear case of Three Law violation?"

"It has memory buffers and a failsafe that shunts the data out of the primary positronic matrix. It prevents positronic collapse and allows for the opportunity for further evaluation. In a proper lab debriefing, the cause-and-effect of a situation can be properly explained and set in context. The robot can be reset and returned to duty."

"You gave it selective amnesia," Ariel said. "It can allow a human to come to harm and still function because after the fact it doesn't remember doing it."

"That's why Bogard left data out of its report," Mia said.

"That's why I have to take Bogard back to Phylaxis to debrief it."

Mia nodded thoughtfully. "So why don't you approve, Ariel?"

"A robot is a machine," she said. "A very powerful machine. It is intelligent, it can make decisions. I want them inextricably joined to the Three Laws so that they can never-never-circumvent their concern for my safety. If they fail to protect me, I want them shut down. I don't want them thinking it over. I don't want to ever be considered a secondary or tertiary concern by a robot who may decide that I ought to be sacrificed for the good of the many. Or of a specific individual. I think loosening the bonds like this can only lead to operational conflicts that will result in unnecessary harm."

"That's the only way to construct a robot bodyguard, though," Derec said.

"There should be no such thing, then!" Ariel shouted. "It didn't work! Somewhere in its sloppy brain it made a decision and sacrificed Senator Eliton! Explain it to me how that was for anyone's greater good!"

Derec stared at her, ashamed. He could think of no answer to give her. In fact, he had no answer for himself.


Перейти на страницу:
Изменить размер шрифта: