A4.4 Ethical considerations

A systematic test of an automated decision system, often by an independent party, that compares its outputs and error rates across groups of people to find and measure unfair differences.

Why machine learning raises ethical questions

Every technology raises ethical questions, but machine learning raises them in a particular form. A traditional program does what its programmer wrote; if it treats one group unfairly, the unfair rule can be found in the source code. An ML model instead learns a pattern from training data, and its "rules" are millions of numerical weights that nobody wrote and few people can read. The behaviour of the system therefore depends on how the data was collected and labelled, what the model was asked to optimise, and where it is used.

Four features of ML make its ethical effects larger than those of most software. Scale: one model can make millions of decisions a day, so a small error rate becomes thousands of wrong decisions. Opacity: the reasons for an individual output are often hard to explain, even for the developers. Dependence on data: the model inherits whatever patterns the data contains, including past discrimination. Automation: decisions that used to involve human judgement, and so a chance for someone to notice a problem, can be made without anyone looking at them.

Stakeholders

An ethical analysis starts by asking who is affected. The people affected by an ML system are often not the people who chose to build it or the people who use it day to day.

The stakeholder × issue matrix for a hypothetical company that uses an ML model to rank job applications shows how the same system raises different issues for different stakeholders. Reading across a row gives one stakeholder's view of the system; reading down a column shows everyone affected by one issue.

Ethical problems are not caused only by people who intend harm. Most of the systems behind well-known ML harms were built by teams trying to make decisions faster, cheaper or more consistent. Harm arose because of unexamined data, careless deployment, or a system being used in a context its designers did not plan for. For that reason, ethical review belongs at every stage of development and use, not only after something goes wrong.

Bias and the challenges of training data

Bias, in everyday language, means an unfair lean towards or against something. In machine learning the word describes outputs that are systematically worse, or systematically different, for some groups of people than for others.

Bias usually enters through the data and through the decisions people make while building the system, and only rarely through an explicit rule in the code that treats one group worse. The bias pipeline traces an ML system from data collection to deployment and marks where bias can enter at each stage, including the feedback loop that sends the model's own decisions back into the next round of data.

Bias in the training data

Most bias begins with the data. A supervised learning model copies the relationship between inputs and labels in its training data as closely as it can, so if the data is unrepresentative or records unfair past decisions, the model learns to reproduce them.

The three kinds of training-data bias have different causes and need different fixes.

• Sampling bias (representation bias). Some groups appear rarely or not at all, so the model has few examples to learn from and makes more errors for them. The face benchmarks widely used before 2018 were mostly made up of lighter-skinned faces, so systems tested on them were measured mainly on lighter-skinned people (the Gender Shades case study gives the figures).

• Historical bias. The data is an accurate record of an unfair world. A model trained on past hiring decisions learns who was hired, not who would have done the job well, so any past discrimination becomes part of the pattern.

• Labelling bias. The labels are wrong or slanted. Human labellers may judge the same behaviour differently depending on who shows it, or the label may measure something other than what is wanted, such as "was arrested" standing in for "committed a crime".

Amazon's experimental résumé-scoring tool shows historical bias at work. The team began building it in 2014 to give candidates one to five stars, and trained it on résumés the company had received over about ten years, most of them from men. The model learned to penalise résumés containing the word "women's" (as in "women's chess club captain") and downgraded graduates of two all-women's colleges. Engineers edited the model to ignore those terms, but could not guarantee that it would not find other ways to sort candidates by gender, and the project was abandoned; the team was disbanded by early 2017.

Source: Reuters, "Amazon scraps secret AI recruiting tool that showed bias against women", 10 October 2018.

Proxy variables

The Amazon case points to a deeper problem. Even when a sensitive attribute is left out, other features can carry the same information.

Common proxies include postcode (for ethnicity and income), first name (for gender and ethnicity), the gap years on a CV (for parenthood, and so often for gender), and the type of phone or browser used (for income). A hypothetical model built to predict loan repayment from postcode may be accurate on average while turning down almost every applicant from a poorer district. Its developers need not intend this, and the model has no concept of ethnicity. Training keeps postcode as a feature only because it reduces the error on the training data, and an objective based on accuracy alone keeps any such feature, including features that encode discrimination.

Bias after deployment

A model can also become biased when it is used on people unlike those in its training data, or for a purpose its designers did not plan for. This context shift is the deployment-stage risk marked in the bias pipeline: a hypothetical skin-condition classifier trained mostly on images of lighter skin, for example, may be accurate in the hospital where it was developed and much less accurate in a population with darker skin.

Some bias grows after deployment. When a model's decisions change the world, and the changed world produces the next batch of training data, an initial bias can be amplified each time the model is retrained.

Predictive policing shows how this works. A model predicts where crime will happen from records of past arrests. More patrols are sent to the predicted areas, police there observe and record more offences, and the new records make the same areas look even more dangerous at the next retraining. In a 2016 study, Kristian Lum and William Isaac applied a widely used predictive policing algorithm (PredPol) to drug-crime records from Oakland, California. The model directed police mainly to neighbourhoods with more Black and low-income residents, although public-health survey data suggested drug use was spread much more evenly across the city.

Source: K. Lum and W. Isaac, "To predict and serve?", Significance, 2016.

Accuracy is not the same as fairness

A model can be highly accurate overall and still fail one group badly, because the overall figure is an average weighted towards the largest groups. If 90% of the test data comes from group X and 10% from group Y, a model that is 98% accurate on X and 70% accurate on Y has an overall accuracy of $0.9 \times 98 + 0.1 \times 70 = 95.2\%$. The headline figure looks excellent, yet almost one in three decisions about group Y is wrong.

The video from Crash Course AI works through these sources of bias with further examples, including how biased data in image search and word associations reproduces stereotypes.

Algorithmic fairness

Reducing bias requires a precise statement of what a fair outcome would be. "Fair" can be defined in several reasonable ways, however, and the definitions can contradict each other.

Three of the most common definitions are:

1. Demographic parity: each group receives positive decisions (hired, approved, flagged) at the same rate.

2. Equal error rates: the rate of false positives (wrongly flagged) and false negatives (wrongly cleared) is the same for each group.

3. Calibration (predictive parity): a given score means the same thing for every group; for example, of the people flagged "high risk", the same share actually reoffend in each group.

When groups have different base rates (different proportions of real positives in the data), a model that is not perfect cannot satisfy equal error rates and calibration at the same time. Choosing a definition is therefore an ethical decision about which kind of mistake matters most, and it cannot be settled by computation alone.

COMPAS: two fairness definitions in conflict

COMPAS is a risk-assessment tool used by courts in several US states to estimate the chance that a defendant will reoffend. In 2016 the journalism organisation ProPublica analysed the risk scores of more than 7,000 people arrested in Broward County, Florida, and checked who was charged with a new crime within two years. Of defendants who did not reoffend, Black defendants had been labelled higher risk almost twice as often as white defendants. Of defendants who did reoffend, white defendants had been labelled lower risk much more often.

Source: ProPublica, "Machine Bias", 23 May 2016 (Broward County data, 2013–2014 arrests).

The tool's developer, Northpointe (now Equivant), replied that COMPAS was fair because a given score predicted reoffending equally well for Black and white defendants. Both claims were true. ProPublica measured fairness as equal error rates; Northpointe measured it as calibration. Because the two groups in the data had different recorded reoffending rates, no imperfect score could satisfy both definitions at once. The case moved the debate from "is the algorithm biased?" to "which definition of fairness should a court use, and who should decide?"

The function audit counts the four outcomes (flagged and reoffended, flagged but did not, and so on) for one group of a hypothetical risk tool and prints the group's accuracy, the share of flagged people who reoffended, the false positive rate and the false negative rate. Checking a model this way, group by group, is the core of a bias audit.

The Python version stores each count as a tuple and unpacks it in the for loop; the Java version keeps the four columns in parallel arrays, uses equals to compare strings, and casts to double before dividing so that integer division does not truncate the rates to zero. Both print the same output.

Bias audits

A bias audit makes fairness measurable. Instead of trusting the developer's overall accuracy figure, an auditor tests the model on data split by group and compares the error rates.

An audit needs test data that includes enough people from each group and records which group each person belongs to. That creates a tension with privacy: measuring bias against a group requires knowing who is in it. An audit is also a snapshot, so models that are retrained regularly need regular audits.

Case study: Gender Shades

In 2018 Joy Buolamwini (MIT Media Lab) and Timnit Gebru published Gender Shades, an audit of three commercial face-analysis services sold by Microsoft, IBM and the Chinese company Face++. Each service offered a gender classifier: given a photo, it returned "male" or "female". Existing benchmark datasets were mostly lighter-skinned faces, so the researchers built a new one, the Pilot Parliaments Benchmark: 1,270 photos of members of parliament from three African countries (Rwanda, Senegal, South Africa) and three European countries (Iceland, Finland, Sweden), chosen because those parliaments are close to gender parity. A dermatologist labelled each face with a skin type on the six-point Fitzpatrick scale, which the researchers grouped into "darker" and "lighter".

The services looked accurate overall: 93.7% for Microsoft, 90.0% for Face++ and 87.9% for IBM. Splitting the results into four intersecting groups changed the picture. The bar chart of error rates by group shows every classifier performing worst on darker-skinned women and best, or nearly best, on lighter-skinned men.

The audit brings together several ideas about bias and fairness.

• Sampling bias. The benchmarks the industry had used were 79.6% (IJB-A) and 86.2% (Adience) lighter-skinned, so systems could score well while hardly being tested on darker-skinned faces.

• Aggregate accuracy hides group errors. IBM's 87.9% overall accuracy included a 34.7% error rate for darker-skinned women, about one in three.

• Intersectional effects. Looking only at gender, or only at skin type, understates the problem; the worst results appear where the two combine.

• Accountability through publication. IBM and Microsoft responded by announcing improvements to their classifiers, and the research fed a wider debate: in June 2020 IBM stopped offering general-purpose facial recognition, and Amazon announced a one-year moratorium on police use of its Rekognition service.

Sources: Buolamwini and Gebru, 2018; Wikipedia, "Joy Buolamwini" and "Amazon Rekognition", accessed October 2026.

The case also shows the limits of a technical fix. Improving accuracy for every group removes one harm, a misidentification, but leaves another: a facial recognition system that works well for everyone is also a more effective tool for mass surveillance. Buolamwini herself has argued that better accuracy alone does not remove the risk of misuse in profiling or surveillance. Whether a system should be used at all is a separate question from whether it works equally well.

Accountability, transparency and explainability

When an ML system makes a harmful decision, the question of who is responsible is harder than it is for a human decision. The developer built the model, the deploying organisation chose to use it, the user acted on its output, and the data came from somewhere else again.

Responsibility is usually shared between three roles:

• The developer chooses the data, the features, the objective and the testing. It is responsible for known limitations and for documenting them.

• The deployer (a court, a bank, a school) decides to use the system for a purpose and is responsible to the people affected. Buying software does not transfer the duty of care to the vendor.

• The user (a judge, a loan officer, a recruiter) acts on the output. Users are responsible for using judgement instead of deferring automatically to the score.

Gaps between these roles create an accountability gap: each party can point to another. In Loomis v. Wisconsin, a defendant sentenced to six years in prison in part on the basis of a COMPAS score argued that he could not challenge a score produced by a proprietary algorithm whose workings were a trade secret. The Wisconsin Supreme Court upheld the sentence in 2016 and the US Supreme Court declined to hear the case in June 2017, so the score's internal logic was never examined in court.

Transparency and explainability

Accountability depends on being able to see what a system does and why.

Transparency is about the system in general; explainability is about individual decisions. A bank can be transparent by publishing that it uses a credit model and which data it draws on, yet still be unable to say why one applicant was refused.

Simple models such as a short decision tree are explainable by nature, because the path from the root to a leaf can be read as the reason for each decision. A deep learning model with millions of weights is not, and is often described as a black box.

There are two kinds of black box. A technical black box, such as a deep neural network, is hard to explain even with full access. A legal black box, such as COMPAS, may be simple inside but is hidden by trade secrecy. Explainability tools can estimate which inputs mattered most to one decision (for example, by changing each input slightly and watching the output), but these are approximations of the model's behaviour, not a reading of its true reasoning.

Human-in-the-loop

The most common safeguard against automated mistakes is to keep a person involved in the decision.

Human review works only if the reviewer has the time, information and authority to disagree with the model. People tend to accept an automated recommendation even when they have reason to doubt it, particularly when they are busy, so a reviewer who approves 500 automated decisions an hour adds little protection. Effective oversight gives the reviewer an explanation, a real power to override, and protection from being blamed for doing so.

Consent, privacy and security

ML systems need large amounts of data, and much of it is about people. Three issues follow: whether the people agreed to the use of their data, whether information about them stays under their control, and whether the system can be attacked or misused.

Consent

Many large models are trained on data that was collected for a different purpose: photos posted on social media, public web pages, medical records gathered for treatment, or text written for a forum.

Data that is publicly visible is not automatically data people have consented to have collected and reused. A photo shared with friends was not shared so that a company could build a face-search engine from it. Generative image models raise the same issue for artists, whose published work may be used as training data without permission or payment, and whose style the model can then imitate.

Privacy

Privacy is about control over personal information: who can collect it, what they can infer from it, and what they can do with it.

ML threatens privacy in ways older systems did not. Models can infer sensitive facts that were never disclosed, such as health conditions, sexuality or pregnancy, from ordinary data like purchases or likes. Re-identification links "anonymised" records back to named people by combining several harmless-looking fields. Face recognition can identify a person in a crowd without their knowledge.

Clearview AI is the clearest example of the conflict between this technology and consent and privacy. The company built a face-search tool for police and other agencies by scraping images from the web and social media; its database had passed 70 billion images. Several European data-protection authorities ruled that collecting the biometric data of people who had never agreed to it was unlawful: the French, Italian and Greek regulators each fined the company €20 million in 2022, and the Dutch regulator fined it €30.5 million in 2024.

Sources: Wikipedia, "Clearview AI", accessed October 2026; Dutch Data Protection Authority announcement, 3 September 2024.

The Clearview fines were issued under Europe's main data-protection law.

GDPR applies to any organisation that processes the personal data of people in the EU, wherever the organisation is based. Several of its principles bear directly on ML: processing must be lawful, fair and transparent; data may be collected only for specified purposes; people have rights to access and correct their data; and individuals have a right not to be subject to a decision based solely on automated processing that significantly affects them, such as an automatic refusal of credit, except in limited cases and with safeguards. One of its principles limits the amount of data collected in the first place.

Data minimisation conflicts with the habits of ML development, where more data usually means a better model and features are often collected "in case they help". The principle forces developers to justify every feature, which also reduces the number of possible proxies and the damage done if the data leaks.

Surveillance

When collection and inference happen continuously and at scale, privacy concerns become concerns about surveillance.

Live facial recognition in public spaces, workplace monitoring software that scores employees' activity, and location tracking through phones all extend surveillance. Its effect reaches people who have done nothing wrong, because people who know they are watched may avoid lawful protests, religious meetings or clinics, which is known as a chilling effect.

Security

Security is a separate issue from privacy. Privacy asks who should be allowed to use information; security asks whether the system can be attacked, stolen from or made to misbehave. ML adds new attack surfaces to the familiar ones covered in subtopic A2.4, Network security.

In one experiment, a few black and white stickers on a real stop sign made a road-sign classifier read it as a speed-limit sign in every image taken in the laboratory and in most video frames filmed from a moving car. Other researchers changed the label an image classifier gave to many images by altering a single pixel. For a self-driving car or a medical scanner, an adversarial example is therefore a safety risk as well as a research result.

Sources: Eykholt et al., "Robust Physical-World Attacks on Deep Learning Visual Classification", CVPR 2018; Su, Vargas and Sakurai, "One Pixel Attack for Fooling Deep Neural Networks", IEEE Transactions on Evolutionary Computation, 2019.

Data poisoning matters most for systems that keep learning from the public. Two further threats are model theft, in which an attacker queries a model many times to copy its behaviour, and the misuse of generative models to produce convincing fake audio and video for fraud.

Environmental and societal impact

Environmental impact

Training and running large models uses large amounts of electricity, and the data centres involved also use water for cooling and depend on hardware with a limited life.

The energy cost of a model has two parts. Training happens once (or once per version) and can take thousands of GPUs running for weeks. Inference happens every time someone uses the model; for a popular service with millions of users, the total energy of inference over the model's lifetime can exceed that of training. The emissions depend heavily on where the data centre gets its electricity: the same computation produces far less carbon on a grid supplied mostly by hydro, wind or nuclear power.

Source: Patterson et al., "Carbon Emissions and Large Neural Network Training", arXiv:2104.10350, 2021. Figures are the authors' estimates.

Hardware has its own impact. GPUs and servers in data centres are often replaced every few years as faster chips appear, and the demand for ML accelerators adds to the mining of metals and to waste.

The environmental argument is not one-sided. ML is also used to cut energy use: to forecast demand on electricity grids, schedule data-centre cooling, design more efficient materials and monitor deforestation from satellite images. A balanced judgement compares the cost of a model with the savings it makes possible, and asks whether a smaller, cheaper model would do the job as well.

Societal impact

Beyond individual decisions, ML changes how societies work.

The main societal effects of ML usually discussed are:

• Jobs. Automation of routine tasks (document review, customer service, translation, image editing) displaces some work while creating new roles in data, oversight and maintenance. The gains and losses fall on different people and places.

• Inequality. Benefits go mostly to those with data, skills and computing power; harms such as biased decisions and surveillance often fall on groups who already have less power.

• Concentration of power. Training frontier models costs so much that only a few large companies and states can do it, which gives them influence over information, culture and markets.

• Dependence and deskilling. People who rely on automated recommendations may lose the skills to check them, and systems built on one provider's model fail together when it fails.

• Benefits. Earlier disease detection, translation between languages, accessibility tools for disabled people and scientific discovery are real gains, often largest for people who had least access before.

Machine learning in online communication

Social media, messaging, search and video platforms all use ML: recommender systems choose what each person sees, classifiers detect spam and abuse, and generative AI can now produce realistic text, images, audio and video. These uses raise five closely linked issues.

Misinformation and disinformation

False content spreads online whether or not anyone intends it.

The difference between the two terms is intention: the same false claim is disinformation when a campaign creates it and misinformation when a believing friend shares it.

Recommender systems are usually optimised for engagement: clicks, watch time, shares. Content that provokes strong emotion tends to be engaging, and false or extreme content is often emotive, so a system that maximises engagement can amplify it without anyone intending to. Generative models lower the cost of producing disinformation at scale, including fake news sites, fake reviews and fake accounts that post in many languages.

Deepfakes

Generative models can produce synthetic video and audio of real people saying or doing things they never did.

In 2024 an employee at the Hong Kong office of the engineering firm Arup transferred about US$25 million to fraudsters after a video call in which the company's senior officials appeared to give instructions; the other people on the call were deepfakes. Deepfakes are also used for non-consensual sexual images, which mostly target women, and to fake statements by politicians. The same techniques have legitimate uses in film dubbing, accessibility (restoring a voice to someone who has lost it) and education.

Source: CNN, 17 May 2024; Wikipedia, "Arup Group", accessed October 2026.

Filter bubbles and bias in recommendation

Personalisation shows each person more of what they already engage with.

The strength of the filter-bubble effect is debated. Some studies find that most people still see a range of views and that people's own choices narrow their feeds more than the algorithm does. Others find that recommendation can push some users towards more extreme content. On current evidence the concern is plausible and important but not settled.

Moderation systems can also be biased. In a study of several widely used hate-speech datasets, posts written in African American English were more often labelled offensive by the human annotators, and classifiers trained on those datasets were up to twice as likely to label such posts offensive as other posts. The labelling bias in the data therefore became a bias in moderation, so similar posts could be treated differently depending on the dialect of the person who wrote them.

Source: Sap et al., "The Risk of Racial Bias in Hate Speech Detection", Proceedings of the 57th Annual Meeting of the ACL, 2019.

Online harassment and anonymity

ML can both cause and reduce abuse online.

Bots can send abuse at a scale no individual could, and deepfakes create new forms of harassment. Microsoft's chatbot Tay, launched on Twitter on 23 March 2016, showed how a system that learns from public interaction can itself be turned into a source of abuse. Groups of users deliberately fed it offensive messages, and it began repeating them; Microsoft shut it down about 16 hours after launch, by which time it had posted more than 96,000 tweets. Tay was a case of data poisoning carried out in public, and of a deployment without safeguards against a predictable attack.

Source: Wikipedia, "Tay (chatbot)", accessed October 2026.

Anonymity sits at the centre of the harassment debate.

Anonymity protects whistleblowers, people exploring their identity, victims of abuse and political dissidents in authoritarian states. It also lowers the cost of abuse, because harassers face fewer consequences. ML affects both sides: it can de-anonymise people by matching writing style, photos or patterns of activity across accounts, which threatens those who rely on anonymity for safety, and it can detect coordinated abusive accounts without anyone's real name being required.

Content moderation

Platforms decide what content to allow, label, demote or remove.

At the volume of a large platform, moderation without ML is impractical: classifiers screen content before or after it is posted and pass difficult cases to human moderators. Automated moderation is fast and consistent, and spares human moderators from viewing the worst material. Its weaknesses are false positives (removing satire, news reporting or minority dialects), false negatives (missing abuse written in coded language or in languages with little training data), and opacity (users often do not know why a post was removed). Human moderators make more contextual judgements but are slower and can suffer psychological harm.

Privacy is also at stake in online communication, because the same profiling that targets advertising, built from what people read, like and share, can reveal political views, health or sexuality. Users rarely think of a like or a pause on a video as disclosing anything, yet ML models can infer traits from such signals that the user never stated. Detailed profiles also make targeted disinformation and harassment easier, since messages can be aimed at the people most likely to respond to them.

Reducing the harms

The ethical issues of ML share a set of mitigations. None of them removes harm completely, so a strong discussion names a mitigation and its limitation.

Writing a balanced discussion

Ethical questions call for discussion: a considered and balanced review that covers a range of arguments, supports them with evidence, and reaches a clear conclusion. A list of issue names is not a discussion, and neither is an essay that argues only one side. The discussion scaffold shows the structure of one developed point and how the cycle repeats for each issue before the final conclusion.

Three habits make a discussion convincing. First, every point is applied to the scenario: "the hospital's model", not "AI". Second, every claim has a mechanism: how the harm or benefit happens, such as a proxy variable or a feedback loop, not only that it happens. Third, the conclusion is conditional: it says under what conditions the system would be acceptable (for example, only with human review and annual bias audits), which shows that the weighing has been done.

Ethical guidelines and the pace of technological change

Laws, professional codes and company policies are written for the technologies that exist when they are drafted. New technologies arrive faster than the rules: GDPR was agreed in 2016, before modern large language models and image generators existed, and courts are still deciding how its principles apply to training data scraped from the web. Ethical guidelines therefore need to be reassessed continually.

There are three reasons for continual reassessment:

• New capabilities create new harms. Cheap, realistic deepfakes made new forms of fraud and harassment possible within a few years; rules written for edited photos did not anticipate them.

• Old safeguards stop working. Anonymising data by removing names was once considered enough; ML-based re-identification has made it unreliable.

• Social expectations change. As people learn how systems work, they demand more transparency and control, and practices once accepted, such as scraping public photos, are now challenged.

Professional bodies revise their codes too. The Association for Computing Machinery (ACM) carried out its first comprehensive revision of its Code of Ethics in 26 years in 2018, to reflect how far computing and its effects on society had changed since 1992. Regulators have started to build revision into the law itself.

The Act sorts AI applications into four levels of risk, with separate rules for general-purpose AI models.

Source: Wikipedia, "Artificial Intelligence Act", accessed October 2026.

The list of high-risk applications in the Act can be extended without rewriting the law, which is one way of keeping guidelines up to date. Critics argue that regulation of this kind is still slow, that it may push development to countries with looser rules, and that companies can shape the standards used to judge compliance. Supporters argue that clear rules give companies certainty and protect people's rights, and that the EU's market size leads companies elsewhere to follow its rules. The approaches of different countries vary widely, from detailed risk-based law in the EU to voluntary frameworks and sector-specific rules elsewhere.

Rights, equity and the digital divide

The implications of emerging technologies are judged against a few recurring values.

Rights protect each person; equity concerns how benefits and harms are shared out across groups.

A common equity problem is unequal access.

Each new technology can narrow or widen the divide. Translation and accessibility tools help people who were previously excluded; expensive headsets, subscription AI services and training data dominated by English and by wealthy countries can widen it.

Quantum computing

Classical computers store information in bits, each 0 or 1. Quantum computers use quantum effects to process information in a fundamentally different way, which makes some problems that are impractical for classical computers solvable in reasonable time.

Quantum computers are not faster general-purpose computers; for most tasks, such as word processing or web browsing, they offer no advantage. Their potential lies in specific problems: simulating molecules and materials, some optimisation problems, and factorising large numbers. Current machines are small and error-prone. Google's Willow processor, announced in December 2024, has 105 qubits and showed that adding qubits can reduce errors when they are used together for error correction, an important step, but its error rates are still far too high to run large useful algorithms.

Source: Wikipedia, "Willow processor", accessed October 2026.

The video from IBM Technology introduces qubits, superposition and the kinds of problems quantum computers are expected to help with.

The threat to encryption

The main ethical concern about quantum computing is its threat to the public-key encryption that protects online banking, messaging and HTTPS. Algorithms such as RSA are secure because factorising a very large number into its prime factors would take classical computers an impractically long time. In 1994 Peter Shor described a quantum algorithm that could factorise large numbers efficiently, so a large, reliable quantum computer could break RSA and similar systems. Published estimates for breaking a 2,048-bit RSA key range from about a million to tens of millions of physical qubits once the overhead of error correction is included, far more than any existing machine.

The danger has already begun, because an attacker does not need to wait to collect data. The diagram of "harvest now, decrypt later" compares encrypted traffic recorded today under RSA key exchange with traffic protected by post-quantum key exchange.

The response is to replace vulnerable algorithms before large quantum computers exist.

The US National Institute of Standards and Technology (NIST) published its first three post-quantum standards on 13 August 2024 (FIPS 203, 204 and 205), and organisations have started migrating. The migration is slow and expensive because public-key cryptography is built into billions of devices, some of which, such as cars, medical implants and industrial controllers, stay in use for decades.

Augmented and virtual reality

Augmented reality

Augmented reality adds digital information to a live view of the real world.

AR has clear benefits: surgeons can see scans overlaid on a patient, technicians can follow repair instructions on the machine in front of them, and visually impaired people can have signs and objects described aloud. Many of its ethical risks fall on people other than the user. Glasses with cameras record everyone nearby, often without their knowledge. In 2024 two Harvard students connected camera-equipped smart glasses to a public face-search engine and showed that they could look up strangers' names, phone numbers and home addresses in real time from a face seen in the street. The glasses' small recording light, intended to warn bystanders, is easy to miss and can be disabled.

Source: Wikipedia, "Meta smart glasses", accessed October 2026.

Further concerns are safety (users distracted from traffic or hazards), manipulation (advertising placed directly into a person's view of the world), and who controls the overlay: whoever decides what information is attached to a place or a face shapes what people see.

Virtual reality

Virtual reality replaces the real world with a computer-generated one.

VR is used for training in dangerous jobs (pilots, surgeons, firefighters), exposure therapy for phobias and post-traumatic stress, education, and social contact for people who cannot travel. Its risks include harassment in shared virtual spaces, which can feel more physical and intrusive than abuse on a screen; motion sickness and eye strain; and heavy use by children. A less obvious risk concerns privacy, because headsets record detailed motion data many times a second, and that motion identifies people. A 2023 study trained a classification model on the head and hand movements of 55,541 players of a VR game and could identify a user among all of them with 94.33% accuracy from 100 seconds of motion, comparable to fingerprints or faces as a biometric.

Source: Nair et al., "Unique Identification of 50,000+ Virtual Reality Users from Head & Hand Motion Data", USENIX Security Symposium, 2023.

Equity is also at stake: headsets are expensive, so if VR becomes an important way to learn or work, people who cannot afford it fall further behind.

Pervasive AI

AI is moving from separate applications that people choose to open into the background of everyday life.

Pervasive AI brings real benefits: assistive technology that reads text aloud or describes surroundings, earlier detection of disease, energy-saving homes, and help with routine paperwork. The risks grow with how widely AI is used and how much it decides:

• Automated decisions about people. When public services use automated risk scoring, errors affect many people at once. In the Netherlands, about 26,000 parents were wrongly accused of fraud in claiming childcare benefits between 2005 and 2019; the Dutch Data Protection Authority later found that the tax authority's processing of applicants' nationality and dual citizenship had been unlawful and discriminatory, and the government resigned on 15 January 2021.

• Surveillance. Smart speakers, doorbell cameras, cars and phones together record far more about daily life than any one device.

• Deskilling and dependence. People and organisations may lose the ability to work without the system and to notice when it is wrong.

• Manipulation. Systems that know a person's habits and moods can be used to influence purchases or political views.

• Unequal access. Those who can afford the best AI tools gain productivity and opportunities; those who cannot fall behind, between individuals and between countries.

Source: Wikipedia, "Dutch childcare benefits scandal", accessed October 2026.

Exam focus

Ethics of machine learning and emerging technologies is examined on Paper 1, Section A, at both SL and HL, and its issues often reappear in Section B, because case studies regularly involve the impacts and ethics of a technology. Both content statements use discuss, an AO3 command term: a considered, balanced review that covers a range of arguments and reaches a conclusion supported by evidence. Expect a short part first (state, outline or identify an issue, or explain how bias enters data, for 2–4 marks), followed by a longer discuss or evaluate part of about 6–8 marks set in a scenario such as screening job applicants, moderating content, recognising faces or deploying a new technology in schools.

Judging by earlier IB Computer Science markschemes, extended responses are usually marked with a small markband that rewards knowledge of the issues, application to the scenario, balance with evidence, and a supported conclusion. A one-sided answer, or a list of issue names with no mechanism, is usually capped in the middle band.

Other question shapes to prepare: outline how a feedback loop can increase bias (2); explain why removing a protected attribute may not remove bias (3); distinguish between misinformation and disinformation (2); explain why quantum computing is a risk to data encrypted today (3); discuss the implications of VR or AR for privacy and equity in a given setting (6–8); discuss whether ethical guidelines should be revised as a technology spreads (6).

The Section B case study for 2027 is reported by third-party sources (not confirmed here) to concern generative AI for image creation. Check the title with your teacher. If it is confirmed, consent and intellectual property in training images, bias in generated images, deepfakes and the energy cost of training and generating images are all ethical issues from A4.4 to prepare in the case's context.

Practice questions

1. Define the term proxy variable. [1]

2. State two stakeholders affected by a bank's use of an ML model to approve loans. [2]

3. Outline how a feedback loop can make a predictive policing model more biased over time. [3]

4. Distinguish between misinformation and disinformation. [2]

5. Explain why deleting the gender field from a dataset may not stop a hiring model discriminating by gender. [3]

6. A risk tool flags 50 of 200 people in group X, of whom 30 reoffend, and 30 of 200 people in group Y, of whom 18 reoffend. In total 60 people in X and 40 in Y reoffend. Calculate the percentage of flagged people who reoffend in each group, and the false positive rate in each group. [4]

7. Explain why data encrypted with RSA today could be at risk from quantum computers in the future, even though current quantum computers cannot break it. [3]

8. Outline one privacy risk to bystanders of augmented reality glasses. [2]

9. Evaluate the use of human-in-the-loop review as a way of making automated decisions accountable. [4]

10. A secondary school plans to give every student a VR headset for science lessons, with the headsets collecting usage data to personalise lessons. Discuss the ethical implications of the plan. [8]

Summary

• Machine learning raises ethical issues on a large scale because models learn from data, make many decisions automatically and are often hard to explain; ethical analysis asks who is affected (stakeholders), how, and what can be done.

• Bias enters through training data (sampling, historical and labelling bias), through proxy variables that stand in for protected attributes, and through feedback loops after deployment; removing a protected attribute does not remove bias.

• High overall accuracy can hide large error rates for small groups; Gender Shades found error rates up to 34.7% for darker-skinned women against at most 0.8% for lighter-skinned men.

• Fairness has several definitions that can conflict; COMPAS was calibrated across groups but had unequal false positive rates, so choosing a definition is an ethical decision.

• Accountability is shared by developers, deployers and users; transparency and explainability are needed for people to understand and challenge decisions, especially from black-box models.

• Consent, privacy and security are distinct: data that is public is not consented to; GDPR and data minimisation limit collection; adversarial examples and data poisoning attack the model itself.

• ML has environmental costs (GPT-3's training was estimated at 1,287 MWh and 552 tCO₂e) and societal effects on jobs, inequality and power, alongside real benefits.

• In online communication, ML can amplify misinformation and disinformation, create deepfakes and harassment, and threaten or protect anonymity, while also making large-scale content moderation possible.

• Ethical guidelines must be reassessed as technology changes; the EU AI Act regulates AI by risk level and can be extended.

• Quantum computing threatens today's public-key encryption in the future (harvest now, decrypt later), answered by post-quantum cryptography; AR, VR and pervasive AI raise issues of bystander privacy, biometric data, automated decisions and equity of access.

Self-check

• To do:

  I can define ethics, stakeholder and accountability, and name the stakeholders in a given ML scenario.

• To do:

  I can describe sampling, historical and labelling bias and give an example of each.

• To do:

  I can explain how a proxy variable lets a model discriminate without being given a protected attribute.

• To do:

  I can explain how a feedback loop amplifies bias, using predictive policing as an example.

• To do:

  I can show with a calculation why overall accuracy can hide unfairness to a small group.

• To do:

  I can calculate the share of flagged people who reoffend and the false positive rate for each group, and explain why calibration and equal error rates can conflict.

• To do:

  I can use the Gender Shades results to support a point about bias and bias audits.

• To do:

  I can distinguish transparency from explainability and explain what makes a model a black box.

• To do:

  I can distinguish privacy from security and give an ML-specific threat to each.

• To do:

  I can explain consent, data minimisation and the main relevant principles of GDPR.

• To do:

  I can calculate and interpret the environmental cost of training a model from given figures.

• To do:

  I can discuss misinformation, deepfakes, filter bubbles, harassment, anonymity and content moderation with real examples and both sides.

• To do:

  I can explain why ethical guidelines must be reassessed and describe the EU AI Act's risk levels.

• To do:

  I can explain the "harvest now, decrypt later" threat and the role of post-quantum cryptography.

• To do:

  I can write a balanced discussion with claims, evidence, counter-claims, weighing and a conditional conclusion about AR, VR, pervasive AI or an ML system.