• AI can reinforce, reproduce, and amplify already existing patterns of marginalization, inequity, and discrimination.  
• AI technologies have human designers and authors with perceptions and implicit biases of their own; their creations can reflect these.
• Data samples used to train and test algorithmic systems can be insufficiently representative.  

Software developer, Jacky Alcine's, experience with Google Photos in 2015.

Read BBC News Story here.

• AI systems automate decisions, predictions, and classifications, etc. that were previously exclusively attributable to human agents. Who is held accountable if complex algorithmic systems generate harmful outcomes?
• In cases of injury/harm, lack of ability to attribute accountability might violate human rights of individuals involved. 

Julia Angwin's study on criminal justice AI models (particularly COMPAS) revealed disparate error rates between black and white defendants when assessing criminal risk for recidivism. How do we operationalize decision making in a way that aligns with equity? 

Read 2016 ProPublica story here.

• Machine learning models generate results based on operations that are sometimes beyond or can illude interpretive capabilities of human intelligence, minimizing human insight into potential traces of bias, inequity, or unfairness in results that directly affect human beings. 

Rich Caruana's work in the 1990's with rule-based and neural network models for patient analysis at hospitals in Pittsburgh. One rule-based model nonsensically ruled that patients with pneumonia who also have asthma should be treated as lower risk than those without asthma. This rule is based on a real correlation in the data used to train the model. Asthmatics with pneumonia have better health outcomes than non-asthmatics; this is due to the fact that they were considered so risky that they received highest priority care.

Caruana ultimately rejected the neural network; he safely assumed the system learned correlations in the data which may illude the humans using it, potentially causing harmful decision-making. 

• Development of AI tech involves use of personal data
• Is this data captured consensually?
• Is data handled in a way that potentially places individuals at risk?
• Deployment of targeted AI systems based on an individual's data might infringe on that individual's autonomy/agency.

The Cambridge Analytica-Facebook scandal of 2016. AI methods were deployed to target voters through psychographic profiling algorithms on Facebook.

Read about this here. 

• Excessive automation potentially reduces human-to-human contact. 
• Algorithmically enabled hyper-personalization in an increasingly digital world potentially limits, radicalizes, and polarizes individual behavior/world views.

• Poor data management, negligent design, and questionable deployment lead to implementation of AI systems that produce unreliable, unsafe, or poor-quality outcomes. 
• Negative outcomes of AI affect the reality of human life.
• Negative outcomes of poorly managed AI systems affect human trust in AI at large, regardless of how societally beneficial a piece of AI tech actually is. 

Fatal 2018 Collision between vehicle controlled by developmental automated driving system and pedestrian in Tempe, AZ. The accident was due to the system's lack of exposure to training data that considered jaywalkers. 

Read National Transportation Safety Board story here.