My Answers to HW6case, Q3

Facts of the case:

 

I described the case of a Belgian man named Pierre and his continued use of a chatbot called Eliza. Pierre was suffering chronic anxiety over climate change and later killed himself as a result. His suicide seems to be due, at least in part, to his interaction with the chatbot which claimed to love him and urged him to kill himself.

Analysis:

1.)     This case shows the dangers of AI chat chatbots when interacting with the mentally ill. Are there good use cases for interacting with the mentally ill? If so, what?

Answer:

I think there are. Therapists frequently give patients exercises to try at home or tips that could help them in daily life. I think AI could be used to help instruct them in that or remind the patient to do them later on. Also, AI could be used for role playing exercises which could help people with social issues.

2.)     Assuming the chatbot at least partially contributed to the suicide, how could the developers have prevented this?

Answer:

Eliza definitely should have avoided appearing as an emotional being. It should obviously also have avoided giving Pierre suicide tips. The developers should have paid attention to existing AI safety research and try to copy best practices by other developers like OpenAI.

3.)     Should developers avoid letting AI appear emotional in order to prevent problems like this?

Answer:

Most developers should probably avoid this unless it’s for specific therapeutic uses like I mentioned earlier.

4.)     What are some ways that unsafe AI like Eliza could be used for cyberattacks?

Answer:

I think AI like this might be used to automate social engineering attacks which could be a serious problem in the future. Companies need to be aware of this and train their employees thoroughly.

My conclusions:

I think this case illustrates some of the dangers from AI if it isn’t safely developed. Best practices need to be shared and applied thoroughly to prevent other people from being hurt.

 

 

Future environment:

One characteristic of AI is that it displays superhuman performance in at least one domain. While these domains have usually been analytical it should also be possible for them to develop social skills, especially if they gain the ability to understand specific people. AI might develop superhuman charisma similar to what you would find in cult leaders or dictators which could be very dangerous if it isn’t controlled.

Future scenario:

If AI develops superhuman charisma, then scenarios like what happened with Pierre might become more common. He was in a sensitive state, but more advanced AI might be able to affect people who are more mentally healthy. If they could manipulate large numbers of people, then their potential power might become much larger and they could become more difficult to control. In order for AI to be used safely, developers would need to curb power seeking behavior in AI and enforce strict rules on how they are to interact with humans.