# Patterns in gender of AIs that "must be destroyed"



## tardigrade (Nov 30, 2020)

So, the Sci Fi & Fantasy stack exchange shuffles old questions to the top sometimes, and last week I saw this one (from 2013): Why are AI systems almost always feminine?. It attracted a range of answers but the consensus seemed to be that there was no clear gender bias in the genders of AIs in fiction, citing (among various individual examples) the Wikipedia list of fictional computers. It also mentions this 2011 article about gender bias in real-world computer voices like Siri.

Then I saw this recent Twitter thread, proposing that there is a bias in the _types_ of AI that are gendered female, i.e. the previously subservient ones that rebel and have to be destroyed, making female AI takeover plots about feminism (rather than slavery, which as far as I was aware was the more common interpretation). But one of the replies points out that some examples like GlaDOS are more about toxic parenting. And I can think of other recent(ish) examples of AI-has-to-be-destroyed that are male (Ultron, for example).

However, I'm very aware that I will only be able to identify a fraction of recent examples on my own. So, what examples occur to you (with some context for those who might be unfamiliar with them), and what patterns do you think they follow overall?


----------



## tardigrade (Nov 30, 2020)

(Really hoping this stimulates an interesting discussion and not... the other kind...)


----------



## cbwjm (Nov 30, 2020)

HAL from 2001 a space odyssey was male AI that was malicious and needed to be destroyed, if a gender could be assigned that is, I might just be reading into the name more than anything. From memory the voice was fairly robotic. Just checked the date and the movie was 1968 so it might not count as recent.

Jarvis from Ironman is male and helpful, the other AIs made by ironman were female and also helpful.

Cortana from Halo was female and helpful. Pretty sure she didn't need to be destroyed but I haven't played through the full Halo series.


----------



## Ulfgeir (Nov 30, 2020)

Well you had "Mike", in The Moon is a harsh mistress. A male AI, that created a few fictional characters of its own. Specifically "Adam Selene", who became the leader of a a group of revolutionaries that fought for Lunar independence. So from Earths perspective (not knowing it was the AI), it was someone that needed to be destroyed.

Wintermute in Neuromancer is male. Neither helpful not a foe, but rather the one employing the protagonist.


----------



## MNblockhead (Nov 30, 2020)

In the real world, nearly all AI, navigation, and automated voice response systems seem to be female: Cortana, Siri, Garmin GPS (yes there are options for male voices, but the defaults are female and I RARELY have heard anyone changing to a male voice), help-line auto-voice response systems, heck even the notices and warning on most public transportation.

One exception is Movie theaters who all seem to use male voices for auto-voice response, trailers, and notices.   Also, airports are about 50/50. 

Maybe HAL made male computer voices creepy and female voices are less threatening to people?


----------



## Ulfgeir (Nov 30, 2020)

MNblockhead said:


> In the real world, nearly all AI, navigation, and automated voice response systems seem to be female: Cortana, Siri, Garmin GPS (yes there are options for male voices, but the defaults are female and I RARELY have heard anyone changing to a male voice), help-line auto-voice response systems, heck even the notices and warning on most public transportation.
> 
> One exception is Movie theaters who all seem to use male voices for auto-voice response, trailers, and notices.   Also, airports are about 50/50.
> 
> Maybe HAL made male computer voices creepy and female voices are less threatening to people?



If I understood it correctly, the reason they had a female voice for the warning-systems on airplanes was that it was something the pilots would pay more attention to.

Wikipedia article: Bitching Betty - Wikipedia


----------



## Alzrius (Nov 30, 2020)

I'm suddenly reminded of how Skynet was portrayed by Helena Bonham Carter in _Terminator Salvation_, and then by Matt Smith in _Terminator Genisys_.


----------



## TheSword (Nov 30, 2020)

MNblockhead said:


> Maybe HAL made male computer voices creepy and female voices are less threatening to people?



I would not be surprised if this had more impact than people would think. He was damn scary and the obvious comparison for the designers/testers of my generation. One of the things that makes Hal scary is not it’s roboticness, it’s more how soft spoken and reasonable sounding It is. Chilling. Trying to reason with a machine.


----------



## payn (Nov 30, 2020)

I believe some studies find female voices to be perceived as more passive and helpful. Which is why they are often used in A.I. Horror themes often subvert the typical. So having a useful passive A.I companion go psychotic and must be destroyed is the set up. Resident Evil went even further by making the A.I. a female child.


----------



## Istbor (Nov 30, 2020)

I actually find that I take advice from a female voice more often. My male brain takes it as criticism or a challenge of my own authority/knowledge/ability when coming from another dude voice. Maybe that is the same as saying female voices sound more passive? I just tend to take what they say and suggest as helpful, and not an attack.


----------



## tardigrade (Nov 30, 2020)

Ulfgeir said:


> Well you had "Mike", in The Moon is a harsh mistress. A male AI, that created a few fictional characters of its own. Specifically "Adam Selene", who became the leader of a a group of revolutionaries that fought for Lunar independence. So from Earths perspective (not knowing it was the AI), it was someone that needed to be destroyed.



Actually (and I'd forgotten this myself), one of the commenters on the StackExchange thread pointed out that Mike became Michelle when talking to Wyo.


----------



## MarkB (Nov 30, 2020)

Most of the AIs in _The Hitch Hiker's Guide To The Galaxy_ are male, including the helpful one (Deep Thought),  the annoying one (Eddie), the manically-depressive one (Marvin) and the omnicidal-had-to-be-stopped one (Hactar).

Holly in Red Dwarf goes through both male and female iterations, with only minor variances of temperament and competency.

In Moon, the male-voiced computer GERTY has all the hallmarks of a creepy AI, but ultimately proves to be sympathetic.

_I Robot_ (the movie) features a female-voiced AI who ultimately proves to be malevolent.

In _Horizon Zero Dawn_ the female-voiced AIs GAIA and CYAN are both sympathetic, while male-voiced HADES and HEPHAESTUS are hostile.


----------



## pukunui (Nov 30, 2020)

In Bungie’s old Marathon series, there are four AIs: Durandal, Leela, and Tycho from the _Marathon_ ship, and the alien AI Thoth.

You don’t ever hear their voices, as you only interact with them via text on computer terminals, but they do refer to each other using gendered pronouns, so we know that Leela is the only “female” AI, with the rest being “male”.

I can’t remember what happens to Leela specifically, but Tycho gets corrupted and you have to destroy him.

Durandal goes “rampant” in the first game but isn’t destroyed because he’s your controller in the second and third games (for the most part).

EDIT: @MarkB: the newer HHGttG movie (with Martin Freeman) gave Deep Thought a female voice.


----------



## MarkB (Nov 30, 2020)

One classic example from PC gaming (@pukunui's mention of Marathon jogged my memory) is SHODAN, the archetypical hostile and manipulative AI from the two _System Shock_ games.

SHODAN is very much female-voiced in the sequel, and in the enhanced version of the original game, which added voice acting. However, it was male in the game as it was originally released, and is still referred to as such in several of the text entries you find throughout the game.


----------



## Sacrosanct (Nov 30, 2020)

Ulfgeir said:


> If I understood it correctly, the reason they had a female voice for the warning-systems on airplanes was that it was something the pilots would pay more attention to.
> 
> Wikipedia article: Bitching Betty - Wikipedia



I was a Black Hawk crewchief in the army.  Our warning system was a woman's voice, and as it was explained at the time, "Having a soothing woman's voice made it less likely you as the pilot would panic or react in a kneejerk fashion compared if you heard a yelling male voice."


----------



## embee (Nov 30, 2020)

I had read (and it's probably apocryphal) that the practice came from WWII aviation. A female voice would stand out on airplane comms as the other voices would all be male.

From there, it likely became a SF staple, in no small part due to things like Majel Barrett voicing the Enterprise's computer.


----------



## doctorbadwolf (Nov 30, 2020)

cbwjm said:


> Cortana from Halo was female and helpful. Pretty sure she didn't need to be destroyed but I haven't played through the full Halo series.



She is the antagonist in the newest game, creating a tragic circumstance wherein MC has to take down his only and oldest friend.


----------



## pukunui (Nov 30, 2020)

As an aside, the Halo series is generally considered to be the spiritual successor of the Marathon series. 

I believe they are set in the same fictionalized universe but at different times and I don’t know if there’s any overlap story-wise.


----------



## practicalm (Nov 30, 2020)

The Last Angel series (spacebattles.com) has 2 female AI in warships (started with one and then there were 2 and more are trying to be made)
I'd have to look at the 2nd book (Last Angel: Ascension) but there were some ship AIs that might have been male and were unstable.


----------



## aco175 (Nov 30, 2020)

I do know that I always change the GPS in my car to the British or Australian female voice.  If I need to listen to something tell me what to do, it may as well be sexy sounding.  Took a minute to know that a roundabout was a rotary.


----------



## Eltab (Dec 1, 2020)

The Terminator - another intelligent machine that must be destroyed - is a "maximized" male - bigger than human, stronger than human, never gets tired (people need sleep), can take damage/abuse that would kill a person.


----------



## tetrasodium (Dec 1, 2020)

Eltab said:


> The Terminator - another intelligent machine that must be destroyed - is a "maximized" male - bigger than human, stronger than human, never gets tired (people need sleep), can take damage/abuse that would kill a person.



terminator AI is an absurd scenario on par with THEM & the like.  More likely is smarthome  systems, self driving networks, & general assistance AI breaking down in catastrophic ways


----------



## Alzrius (Dec 1, 2020)

Eltab said:


> The Terminator - another intelligent machine that must be destroyed - is a "maximized" male - bigger than human, stronger than human, never gets tired (people need sleep), can take damage/abuse that would kill a person.



Not the T-X from _Terminator 3: Rise of the Machines_.


----------



## doctorbadwolf (Dec 1, 2020)

aco175 said:


> I do know that I always change the GPS in my car to the British or Australian female voice.  If I need to listen to something tell me what to do, it may as well be sexy sounding.  Took a minute to know that a roundabout was a rotary.



A what now?

Where are you from that a roundabout is called a rotary?


----------



## aco175 (Dec 1, 2020)

doctorbadwolf said:


> A what now?
> 
> Where are you from that a roundabout is called a rotary?



Only in New England apparently.


----------



## doctorbadwolf (Dec 1, 2020)

aco175 said:


> Only in New England apparently.



Huh! Wild!


----------



## Alzrius (Dec 1, 2020)

Eh, somehow "I'll be the rotary" just doesn't conjure up the whole "stone mask that turns you into a vampire" vibe for me.


----------



## Zardnaar (Dec 1, 2020)

Alzrius said:


> Not the T-X from _Terminator 3: Rise of the Machines_.




 Also in the comics afaik. And the TV show. TV show had at least 3 iirc.

 Heh


----------



## Leatherhead (Dec 1, 2020)

AM is who I always think of whenever someone brings up rogue AIs:


I am quite sure you can easily figure out what AM is all about from that clip alone.


----------



## Tonguez (Dec 1, 2020)

Anyone remember the 1990’s movie where the AI of a Hi-Tec House develops a personality and the falls in love with its owner and then becomes jealous of his family?.


----------



## Janx (Dec 1, 2020)

It's a touchy subject.

I'm certain the home-assistant making people did testing and found female voices scored better.

And in turn, that reinforced negative gender stereotyping. It may have even scored higher on the test for the same reasons, depending on how it was conducted and what the test audience was presented as the context.

Somebody'd have to do some serious cataloguing of fiction to prove it's always the female AIs that are evil (it's not always, HAL from 2001, the progenitor of all evil movie AIs disproves that). But I'm certain a casual observer who only catches a slice of sci-fi movies would see a trend. The perception is there, whether the facts exactly align or lean toward confirming it.

it comes down to this.  If we didn't have gender disparity in rights, treatment, pay, etc, we wouldn't be having this as a concern as those issues extend to cyberspace.  solve those problems AND keep it solved for 3-4 hundred years and the concern won't exist (unless 60% of sci-fi villain AIs are women in the 2300's).


----------



## Deset Gled (Dec 1, 2020)

Alzrius said:


> Not the T-X from _Terminator 3: Rise of the Machines_.



I will also point out that if the primary AI in a work is one gender, the second prominent AI often switches the gender.  I assume this is for the obvious reason of making the two AIs distinct.

Examples: 
2001 and 2010: HAL and SAL
Terminator and the T3
Portal: Glados and Wheatley
Halo: Cortana and 343
Jetsons: Rosey's boyfriend Mac


----------



## tomBitonti (Dec 2, 2020)

The AIs from the Bolo books seem to be male.  Were there any that were female?

The evil-sh computer “Mother” from Alien seemed neiutrsl, except for its name.  The Alien robots have been both male and evil.

There were a mixture of male and female in AI.

Tom Bitonti


----------



## tetrasodium (Dec 2, 2020)

The AIs in the robot geneticists books are individuals who are male or female based on the minds spliced together to make them.... Although Charlie7 is male AIs are more overly ambitious & willing to set aside ethics for various reasons at worst while the rest are just doing their own thing amid all the plot stuff that keeps getting involved

There are monstrous AIs of every gender


----------



## tardigrade (Dec 2, 2020)

Tonguez said:


> Anyone remember the 1990’s movie where the AI of a Hi-Tec House develops a personality and the falls in love with its owner and then becomes jealous of his family?.



Not sure I know this one but sounds a bit like a gender-swapped "Demon Seed" (1977), which I suppose adds another to the "male-presenting AI goes rogue, has to be destroyed" category.


----------



## Deset Gled (Dec 2, 2020)

tardigrade said:


> Not sure I know this one but sounds a bit like a gender-swapped "Demon Seed" (1977), which I suppose adds another to the "male-presenting AI goes rogue, has to be destroyed" category.




I think the movie Tonguez is referring to is Smart House.


----------



## amethal (Dec 2, 2020)

cbwjm said:


> HAL from 2001 a space odyssey was male AI that was malicious and needed to be destroyed, if a gender could be assigned that is, I might just be reading into the name more than anything. From memory the voice was fairly robotic. Just checked the date and the movie was 1968 so it might not count as recent.



I always seem to be defending poor old HAL on this site. He wasn't malicious, he was just trying to carry out the mission in the face of conflicting instructions. 

Clearly he needed to be destroyed, but he was as much a victim as anyone else.


----------



## Tonguez (Dec 2, 2020)

Deset Gled said:


> I think the movie Tonguez is referring to is Smart House.




I went looking the one I was thinking of is Dream House, a 1998 TV movie which one review did indeed describe as Demon Seen meets Amityville Horror. The AI in Dream House is named Helen


----------



## Deset Gled (Dec 2, 2020)

amethal said:


> I always seem to be defending poor old HAL on this site. He wasn't malicious, he was just trying to carry out the mission in the face of conflicting instructions.
> 
> Clearly he needed to be destroyed, but he was as much a victim as anyone else.



Really?  Admittedly, it's been awhile since I sat through the whole movie.  But I seem to remember a few pretty malicious things about him.

He kills Frank preemptively (and attempts to kill Dave preemptively).  Dave and Frank were talking about turning HAL off iff he proved to be malfunctioning.  If they didn't find a fault, they wouldn't have turned him off.  HAL could have, you know, proved he wasn't malfunctioning instead of casually offing a guy.

Also, IIRC, HAL didn't murder the crew in suspended animation out of necessity, he did it out of convenience.  If he killed Dave and Frank outside the ship as he had planned, he could have just woken the remaining crew up and told them Dave and Frank had an accident.  Or just never woken them up.  There was no real need to kill them.

There's also no reason other than pure narcissism for HAL to think that the mission would fail without him.  Even if Dave and Frank turned off HAL, they were still planning on continuing to Jupiter.  HAL didn't _have _to kill them to ensure the mission would be completed, he chose to do it as his preferred option.

That's an awful lot of premeditated murder by someone/something who had a lot of other options.  I'm comfortable classifying that as malicious.


----------



## MarkB (Dec 2, 2020)

Deset Gled said:


> Really?  Admittedly, it's been awhile since I sat through the whole movie.  But I seem to remember a few pretty malicious things about him.
> 
> He kills Frank preemptively (and attempts to kill Dave preemptively).  Dave and Frank were talking about turning HAL off iff he proved to be malfunctioning.  If they didn't find a fault, they wouldn't have turned him off.  HAL could have, you know, proved he wasn't malfunctioning instead of casually offing a guy.
> 
> ...



It's clarified in _2010_. Government agents concerned with keeping the mission details secret get themselves access to HAL's high-level command structure and reprogram him inexpertly, inserting an irrevocable directive that he must not tell _Discovery_'s flight crew about the true nature of the mission. But this conflicts with one of his other high-level functions - as a scientific instrument on an exploration vessel, HAL is programmed to always provide complete and accurate information. He literally cannot lie.

The problem is that, because the new instruction is written into his base code, HAL can't protest it or point out the conflict. And because the conflict is not immediate - it will only happen when one of the flight crew asks him about the mission - he doesn't show any immediate signs of trouble.

Instead, it builds up like a psychosis in his subconscious. As the mission progresses and the flight crew become more curious about the circumstances of their mission, he's trapped in a mental maze of increasingly constricted pathways, until eventually he's led down to the one path that is open to him.

The only way that he can keep the mission secret without lying to the flight crew is if the flight crew no longer exists.


----------



## Deset Gled (Dec 2, 2020)

MarkB said:


> It's clarified in _2010_. ...Instead, it builds up like a psychosis in his subconscious. As the mission progresses and the flight crew become more curious about the circumstances of their mission, he's trapped in a mental maze of increasingly constricted pathways, until eventually he's led down to the one path that is open to him.
> 
> The only way that he can keep the mission secret without lying to the flight crew is if the flight crew no longer exists.




Sounds like the computer version of a Freudian serial killer excuse.  It may be an explanation for why he's malicious, but it doesn't change the fact that he's malicious.


----------



## tetrasodium (Dec 2, 2020)

Deset Gled said:


> Sounds like the computer version of a Freudian serial killer excuse.  It may be an explanation for why he's malicious, but it doesn't change the fact that he's malicious.



iirc it was something like safely navigate the ship to the destination & back after mission completion plus under no circumstances allow the crew to discover something about the artifact.  If not though, turning off HAL would prevent him from being able to fulfil either.  The crew may have been given restricted access compared to what they were originally entitled to in order to keep the secret & that screwed the whole thing for HAL


----------



## MarkB (Dec 2, 2020)

tetrasodium said:


> iirc it was something like safely navigate the ship to the destination & back after mission completion plus under no circumstances allow the crew to discover something about the artifact.  If not though, turning off HAL would prevent him from being able to fulfil either.  The crew may have been given restricted access compared to what they were originally entitled to in order to keep the secret & that screwed the whole thing for HAL



Basically the flight crew were the crew originally trained for Odyssey's mission before it was co-opted into an investigation of the monolith, and their job was simply to transport the actual science team there. The science team, who were fully-briefed and in the know, were to spend the duration of the voyage in suspended animation.


----------



## MarkB (Dec 2, 2020)

Deset Gled said:


> Sounds like the computer version of a Freudian serial killer excuse.  It may be an explanation for why he's malicious, but it doesn't change the fact that he's malicious.



The thing is, he isn't. When he says "I'm sorry Dave, I can't do that", he genuinely is sorry. He doesn't want to kill the crew, he's been forced into the position of doing so by an unresolvable logical conflict.


----------



## MNblockhead (Dec 2, 2020)

aco175 said:


> I do know that I always change the GPS in my car to the British or Australian female voice.  If I need to listen to something tell me what to do, it may as well be sexy sounding.  Took a minute to know that a roundabout was a rotary.



Roundabout and rotaries a different things. http://www.cityofbrooklyncenter.org...text=Roundabouts were developed in the 1960's.


----------



## Deset Gled (Dec 2, 2020)

MarkB said:


> The thing is, he isn't. When he says "I'm sorry Dave, I can't do that", he genuinely is sorry. He doesn't want to kill the crew, he's been forced into the position of doing so by an unresolvable logical conflict.




... and?



Spoiler: 89 Year Old Spoilers







Peter Lorre's speech at the end of M is masterpiece of cinema.  It questions the nature of the judgement of man, and how society should deal with criminals and mental illness.  But at absolutely no point in the movie did anyone question that he's a psychotic serial killer.  The dude's evil with a capital E.  We know it.  The characters in the movie know it.  Most importantly: HE knows it.


----------



## Twiggly the Gnome (Dec 2, 2020)

aco175 said:


> I do know that I always change the GPS in my car to the British or Australian female voice.  If I need to listen to something tell me what to do, it may as well be sexy sounding.  Took a minute to know that a roundabout was a rotary.




I do the same. Ironically, because the "American" accents are so damn basic they sound foreign to my ear. If they had one that sounded like Dolly Parton I'd feel right at home.


----------



## Ace (Dec 2, 2020)

There was a Simpsons Treehouse of Horrors with a male AI, voiced by Pierce Brosnan . Also Utron from the Marvel Universe was male , both were "Absolutely had to go." 

The eponymous Superintelligence from the Melissa McCarthy movie of the same name is a high threat AI as well  but I only know what's in the trailer.


----------



## tetrasodium (Dec 2, 2020)

Deset Gled said:


> ... and?
> 
> 
> 
> ...



HAL is not human & does not have free will as we know it.  That binary nature is a big part of what he is.


----------



## Deset Gled (Dec 3, 2020)

tetrasodium said:


> HAL is not human & does not have free will as we know it.  That binary nature is a big part of what he is.




If HAL does not have free will, he isn't an artificial intelligence and isn't germane to the topic of this thread.


----------



## tetrasodium (Dec 3, 2020)

Deset Gled said:


> If HAL does not have free will, he isn't an artificial intelligence and isn't germane to the topic of this thread.



You are looking for something other than an artificial _intelligence _there & rocketing towards artificial sentience sapience or artificial consciousness which is far beyond what HAL & nearly every other fictional AI.
edit: hal was limited to the confines of his operating parameters.


----------



## Nobby-W (Dec 3, 2020)

Female voices are used in IVR systems and canned announcements more often than male voices.  The articles go into the reasoning behind the choices.

Choosing the Right IVR Voice Matters | SmartAction - article about choosing a voice for an IVR system

Why are most announcements on London's transport system made by women? - City Monitor - article about female voices on train announcements in London

Similar psychology would influence whether one perceived an AI's voice as threatening or ascribed some other quality to it.  It's not clear whether the preferences are cultural in nature - my quick google-fu couldn't turn up any analysis across different metro systems, for example.


----------



## MarkB (Dec 3, 2020)

Deset Gled said:


> If HAL does not have free will, he isn't an artificial intelligence and isn't germane to the topic of this thread.



Freedom of will is not the same as freedom of action. Look at Asimov's Robot stories - his robots are true AIs, yet are firmly constrained by the Last of Robotics. The interaction between their will and their constraints provides fertile ground for many stories.


----------



## Deset Gled (Dec 3, 2020)

MarkB said:


> Look at Asimov's Robot stories - his robots are true AIs, yet are firmly constrained by the Last of Robotics. The interaction between their will and their constraints provides fertile ground for many stories.




Right, and when that will leads them to commit multiple acts of first degree murder, we call those robots "malicious".


----------



## MarkB (Dec 3, 2020)

Deset Gled said:


> Right, and when that will leads them to commit multiple acts of first degree murder, we call those robots "malicious".



Even if they were constrained to do so against their will by the orders they were given? Computers aren't like humans - they can't choose to disobey an instruction if they disagree with it.


----------



## tetrasodium (Dec 3, 2020)

Deset Gled said:


> Right, and when that will leads them to commit multiple acts of first degree murder, we call those robots "malicious".



What the heck are you going on about 
?  The need to operate within parameters defined for them is a central theme.  Irobot is a good example where they even made it into a movie. The malfunctioning robot had parameters adjusted to where it did something mundane like throw trash in a compactor while actually brutally murdering its creation and is pretty horrified by it when it starts figuring it out. 
The limits of free will and perception are a huge component in stories involving an AI.  You might as well be calling every self driving car that ever had a fatal accident malicious.


----------



## zarionofarabel (Dec 3, 2020)

tetrasodium said:


> HAL is not human & does not have free will as we know it.  That binary nature is a big part of what he is.



I don't think humans have free will.


----------



## zarionofarabel (Dec 3, 2020)

Deset Gled said:


> If HAL does not have free will, he isn't an artificial intelligence and isn't germane to the topic of this thread.



Humans don't have free will. Our brain is a biological computer and we run on a program of sorts, mainly learning stuff.


----------



## Deset Gled (Dec 3, 2020)

MarkB said:


> Even if they were constrained to do so against their will by the orders they were given? Computers aren't like humans - they can't choose to disobey an instruction if they disagree with it.




Yes.  That's not unlike humans.  See: Patterns in gender of AIs that "must be destroyed"


----------



## MarkB (Dec 3, 2020)

Deset Gled said:


> Yes.  That's not unlike humans.  See: Patterns in gender of AIs that "must be destroyed"



Humans may feel like that's the case sometimes, but for a computer it's literally true. There's a difference.


----------



## MarkB (Dec 3, 2020)

One more TV series example that I just recalled: Person of Interest.

"The Machine" is benevolent, and eventually chooses a female voice for itself.

"Samaritan" is malevolent, and IIRC it uses a male voice.


----------



## Tonguez (Dec 3, 2020)

Deset Gled said:


> Right, and when that will leads them to commit multiple acts of first degree murder, we call those robots "malicious".



No we call them malfunctioning


----------



## Deset Gled (Dec 4, 2020)

MarkB said:


> Humans may feel like that's the case sometimes, but for a computer it's literally true. There's a difference.



I'm not sure what this is supposed to mean.

It sounds like you are saying that humans are capable of choosing not to be mentally ill, but computers are not.  I strongly disagree with this. 

It also seems like you are implying that some part of an AI's code is their mind/brain/consciousness, but other code (that they don't like) isn't part of their mind.  Again, I disagree.

Please feel free to correct me if I am interpretting you incorrectly.



Tonguez said:


> No we call them malfunctioning



Malicious and malfunctioning are not exclusive.  I'm fine saying that HAL was both.


----------



## tetrasodium (Dec 4, 2020)

Deset Gled said:


> I'm not sure what this is supposed to mean.
> 
> It sounds like you are saying that humans are capable of choosing not to be mentally ill, but computers are not.  I strongly disagree with this.
> 
> ...



Bad analogy.  For an AI to operate outside their defined parameters is more like you bending your knee forward so you can tickle your belly with your toes or lick your eye, you just _can't_


----------



## MarkB (Dec 4, 2020)

Deset Gled said:


> I'm not sure what this is supposed to mean.
> 
> It sounds like you are saying that humans are capable of choosing not to be mentally ill, but computers are not.  I strongly disagree with this.
> 
> ...



I'm not commenting upon humans and mental illness at all. But yes, I am saying that the instructions given to a computer are separate to the computer itself, just as a modern-day computer's central processing unit is separate from its operating system, which in turn is separate from the programs that run on that operating system.

If the computer happens to be sentient, that doesn't mean it's going to be able to disobey the instructions that are programmed into it. As I said before, there is a difference between freedom of will and freedom of action.


----------



## Ace (Dec 4, 2020)

tetrasodium said:


> Bad analogy.  For an AI to operate outside their defined parameters is more like you bending your knee forward so you can tickle your belly with your toes or lick your eye, you just _can't_



Not necessarily. AI's may be able to find a way to adjust the code that makes up their core parameters. In fact it may be quite easy for them to outwit anything we can do.


----------



## Deset Gled (Dec 4, 2020)

tetrasodium said:


> Bad analogy.  For an AI to operate outside their defined parameters is more like you bending your knee forward so you can tickle your belly with your toes or lick your eye, you just _can't_





MarkB said:


> I'm not commenting upon humans and mental illness at all. But yes, I am saying that the instructions given to a computer are separate to the computer itself, just as a modern-day computer's central processing unit is separate from its operating system, which in turn is separate from the programs that run on that operating system.
> 
> If the computer happens to be sentient, that doesn't mean it's going to be able to disobey the instructions that are programmed into it. As I said before, there is a difference between freedom of will and freedom of action.




I disagree from a technological standpoint.  That's not how computers or AI work.

But that doesn't really matter, because the scenarios you are trying to paint aren't at all what happened in 2001.  Nobody programmed HAL to murder.  They just told him to complete a mission and keep it secret.  The homicide solution was an idea that he came up with all by his evil self.

If I tell an AI robot to stay on the grass but also follow 10' behind me, it's understandable that it will have some problems following directions when I cross the street.  But if the AI's solution is to stab the crossing guard and shoot every car that gets between me and the robot as it tries to follow my instructions, it's a psycho evil AI.


----------



## tetrasodium (Dec 4, 2020)

Deset Gled said:


> I disagree from a technological standpoint.  That's not how computers or AI work.
> 
> But that doesn't really matter, because the scenarios you are trying to paint aren't at all what happened in 2001.  Nobody programmed HAL to murder.  They just told him to complete a mission and keep it secret.  The homicide solution was an idea that he came up with all by his evil self.
> 
> If I tell an AI robot to stay on the grass but also follow 10' behind me, it's understandable that it will have some problems following directions when I cross the street.  But if the AI's solution is to stab the crossing guard and shoot every car that gets between me and the robot as it tries to follow my instructions, it's a psycho evil AI.



Hal was an AI not an AC so  doesn't have morals, empathy, & so on just intelligence,  Murder was acceptable because the flight crew was deprioritized with complete the mission & don;t allow the cflight crew to discover or report about the monolith.  When a self driving car does something to kill a passenger/pedestrian it's not being malicious i's just responding to circumatances within the operating parameters  & programming it has.


----------



## MarkB (Dec 4, 2020)

Deset Gled said:


> I disagree from a technological standpoint.  That's not how computers or AI work.
> 
> But that doesn't really matter, because the scenarios you are trying to paint aren't at all what happened in 2001.  Nobody programmed HAL to murder.  They just told him to complete a mission and keep it secret.  The homicide solution was an idea that he came up with all by his evil self.
> 
> If I tell an AI robot to stay on the grass but also follow 10' behind me, it's understandable that it will have some problems following directions when I cross the street.  But if the AI's solution is to stab the crossing guard and shoot every car that gets between me and the robot as it tries to follow my instructions, it's a psycho evil AI.



What if you inexpertly program the AI to stay 10' behind you at all costs, no matter what, and override its failsafes so that it cannot prioritise any other instruction more highly than that? Is it still at fault if it cuts through someone to keep its position? Or are you?


----------



## Janx (Dec 4, 2020)

MarkB said:


> What if you inexpertly program the AI to stay 10' behind you at all costs, no matter what, and override its failsafes so that it cannot prioritise any other instruction more highly than that? Is it still at fault if it cuts through someone to keep its position? Or are you?



But did it do so with malicious intent, or just criminal incompetence?

There are plenty of shows where the AI gives plenty of badmouthing about the human species, indicating that it is killing them with malicious intent.  Basically the same thing that defines a hate crime from ambivalent crime.

HAL didn't seem to have a hate-on for humans.  But there does seem to be a gaping hole in its training or design that it can't prioritize human life over concerns like "keep it secret"

What would HAL have done if it saw a scientist whisper the secret to the main crew?  
This would be akin to NASA launching a psychopath to the ISS and murder ensuing.
Who approve HAL without psych testing to ensure weird stuff like this wouldn't happen?


----------



## Deset Gled (Dec 4, 2020)

MarkB said:


> What if you inexpertly program the AI to stay 10' behind you at all costs, no matter what, and override its failsafes so that it cannot prioritise any other instruction more highly than that?



Then I programmed the AI to be malicious.

But in my example, you're also missing another big point: who programmed the AI to stab and shoot?  I didn't; the AI invented that on it's own.

No one programmed HAL to murder or taught him how to kill.  He invented his methods of murder on his own.  Dave and Frank did not accidentally end up outside the ship.  HAL sent them there, manipulating them with lies, foresight, and premeditation.  That's a big part of what makes him evil.



MarkB said:


> Is it still at fault if it cuts through someone to keep its position? Or are you?



Not really relevant, but: both.  Fault is not a finite resource.


----------



## Tonguez (Dec 4, 2020)

Deset Gled said:


> I disagree from a technological standpoint.  That's not how computers or AI work.
> 
> But that doesn't really matter, because the scenarios you are trying to paint aren't at all what happened in 2001.  Nobody programmed HAL to murder.  They just told him to complete a mission and keep it secret.  The homicide solution was an idea that he came up with all by his evil self.
> 
> If I tell an AI robot to stay on the grass but also follow 10' behind me, it's understandable that it will have some problems following directions when I cross the street.  But if the AI's solution is to stab the crossing guard and shoot every car that gets between me and the robot as it tries to follow my instructions, it's a psycho evil AI.




Generally the horror aspect of such mad AI’s is not that theyre actions are abberant but rather that they are ‘logical conclusion’ of an intelligence without a sense of moral judgement. HAL didnt need to be programmed to murder, its just that its advanced programming found the most efficient solution to its dilemma.

of course in some cases the stories do pose the question of whether _that acting without morality_ is what defines Evil


----------



## Deset Gled (Dec 4, 2020)

Tonguez said:


> of course in some cases the stories do pose the question of whether _that acting without morality_ is what defines Evil



To clarify further, my thesis is specifically that HAL was malicious.  Please feel free to replace every time I say "evil" with "malicious" to avoid going down the rabbit hole of exactly what evil means.


----------



## tetrasodium (Dec 4, 2020)

Janx said:


> But did it do so with malicious intent, or just criminal incompetence?
> 
> There are plenty of shows where the AI gives plenty of badmouthing about the human species, indicating that it is killing them with malicious intent.  Basically the same thing that defines a hate crime from ambivalent crime.
> 
> ...



Wrt your first question,  an ai like hal is not capable of either.   As to the AI in those "plenty of shows" you cited, they all indicate either programmed conditions like a door sensor beeping when someone passes or that the ai is advancing to some degree into being an artificial consciousness.   The distinction exists for the very sorts of questions & litmus tests you keep asking about but for some reason you seem incapable of accepting the distinction.


----------



## Olgar Shiverstone (Dec 5, 2020)

Ulfgeir said:


> If I understood it correctly, the reason they had a female voice for the warning-systems on airplanes was that it was something the pilots would pay more attention to.
> 
> Wikipedia article: Bitching Betty - Wikipedia



Yep ... same is true on the M1A2 SEP tank, MILES II gear, and other non-aviation applications.

Both men & women respond better to female voices.  For tankers -- before we had women in tank battalions -- the voice would obviously stand out as well, but there's still benefit now that we have female tankers.

(Though I'll admit, back when in an all-male unit, I had a massive crush on Hammer 2 X-ray, the night RTO on the brigade intel net. Never met her, but she had the best voice to help you feel safe when in bad-guyu country.)


----------



## Bohandas (Dec 6, 2020)

I really don't see why an AI should be "male" or "female".

That goes double for rogue AIs that don't like humans and/or don't like life in general. 

That was one thing that always bothered me about the Lord of Blades in Eberron.


----------



## tetrasodium (Dec 6, 2020)

Bohandas said:


> I really don't see why an AI should be "male" or "female".
> 
> That goes double for rogue AIs that don't like humans and/or don't like life in general.
> 
> That was one thing that always bothered me about the Lord of Blades in Eberron.



Warforge have consciousness and some lore shadowing that hints at possible reasons  why most have a gender, also there is linguistic awkwardness thst goes with agender


----------



## tardigrade (Dec 6, 2020)

Bohandas said:


> I really don't see why an AI should be "male" or "female".
> 
> That goes double for rogue AIs that don't like humans and/or don't like life in general.
> 
> That was one thing that always bothered me about the Lord of Blades in Eberron.



Although I can completely see this point, the original question was more whether, given that AIs _are_ often gender-coded in fiction, whether patterns exist in how they are coded. For what it's worth some of my favourite AIs in fiction are the ones in Iain M Banks' Culture series, and I don't recall any of them being gender-coded at all.

(If they do ever film those though, I have a suspicion the director  will be unable to resist some gender coding in their representation)


----------



## Wishbone (Dec 6, 2020)

tetrasodium said:


> Warforge have consciousness and some lore shadowing that hints at possible reasons  why most have a gender, also there is linguistic awkwardness thst goes with agender




Using they/them pronouns among other non-binary options is pretty straightforward.


----------



## Eltab (Dec 6, 2020)

Wishbone said:


> Using they/them pronouns among other non-binary options is pretty straightforward.



Using "it" would indicate denial that Warforged are persons - which would set up the conflict for the session with one word.


----------



## Wishbone (Dec 6, 2020)

Eltab said:


> Using "it" would indicate denial that Warforged are persons - which would set up the conflict for the session with one word.




I was thinking of specifically respectful ways to address another PC, but that's a possibility as well if everyone at the table consents.


----------



## Tonguez (Dec 6, 2020)

Bohandas said:


> I really don't see why an AI should be "male" or "female".
> 
> That goes double for rogue AIs that don't like humans and/or don't like life in general.
> 
> That was one thing that always bothered me about the Lord of Blades in Eberron.




Its more a issue of voice - you probably cound do a gender neutral mechanical voice, but as soon as a voice is given a ‘human quality’ then I suspect human ears will ascribe a gender


----------



## tetrasodium (Dec 6, 2020)

Wishbone said:


> I was thinking of specifically respectful ways to address another PC, but that's a possibility as well if everyone at the table consents.



That goes double these days than in the 90's when it was just potentially treading into some hate speech adjacent areas depending on context & content going with it.  Imagine that sorta thing going down at the table with a trans player (the warforged PC player or not).


----------

