Within Health Proxy
Why costs can misread medical need
Healthcare costs can look like medical need, but they also reflect unequal access, coverage, and treatment patterns.
On this page
- What healthcare spending actually measures
- How unequal access distorts risk scores
- Why accuracy on costs can still fail patients
Page outline Jump by section
Introduction
In healthcare AI, a common mistake is to treat healthcare spending as if it were a direct measure of how sick a patient is. At first glance, the assumption seems reasonable: people with greater medical needs often generate higher costs. However, healthcare spending reflects far more than illness. It is shaped by insurance coverage, access to doctors, local prices, treatment patterns, administrative systems, and historical inequalities. As a result, an AI model that predicts future costs may become highly accurate at forecasting spending while remaining poor at identifying who actually needs care. This distinction matters because risk-scoring systems often determine which patients receive additional support, preventive interventions, or care-management resources. When cost is used as a proxy for need, the model can systematically overlook patients whose illnesses are serious but whose healthcare spending has historically been lower. [arXiv]arxiv.orgUnderstanding racial bias in health using the Medical Expenditure Panel Survey dataNovember 4, 2019…
What healthcare spending actually measures
Healthcare spending is not a pure measure of disease burden. Instead, it combines multiple factors that influence how much money is spent on a patient.
A patient’s total cost can rise because they are severely ill, but it can also rise because they live in a high-price region, receive more intensive treatment, have better insurance coverage, or encounter providers who order more services. Researchers studying spending variation have found that prices, utilisation patterns, and local healthcare systems all contribute substantially to differences in expenditure. Two patients with similar health conditions can therefore generate very different costs. [PMC]pmc.ncbi.nlm.nih.govOpen source on nih.gov.
This creates a measurement problem for AI systems. If a model is trained to predict future spending, it learns relationships between these spending drivers and future costs. The model does not automatically distinguish between money spent because of genuine medical need and money spent because of pricing, insurance arrangements, coding practices, or treatment intensity. [Health Justice Monitor]healthjusticemonitor.orgHealth Justice MonitorHealth Justice MonitorIt’s the Prices and More - Health Justice Monitor…
An important implication follows: healthcare spending is partly a measure of access to healthcare. People who receive more services generally generate more claims and more expenditure. Those who face barriers to care may generate lower spending even when they have substantial unmet medical needs. [PubMed]pubmed.ncbi.nlm.nih.govUsual source of care: an important source of variation in health care spending - PubMed…
How unequal access distorts risk scores
The most widely discussed example comes from healthcare risk algorithms used to identify patients for care-management programmes. Researchers found that a commonly used commercial system predicted future healthcare costs rather than future illness burden. Because spending differed across racial groups for reasons beyond health status, the resulting risk scores inherited those disparities. [arXiv]arxiv.orgUnderstanding racial bias in health using the Medical Expenditure Panel Survey dataNovember 4, 2019…
The mechanism was straightforward. Historical healthcare data showed that Black patients often incurred lower healthcare spending than White patients with comparable levels of illness. Lower spending did not necessarily indicate better health. It frequently reflected differences in access, treatment patterns, and healthcare utilisation. When the algorithm learned from those spending records, it interpreted lower expenditure as lower future risk. [ellagreenslade.com+2arXiv]ellagreenslade.comAlgorithm Bias in Healthcare | Ella GreensladeAlgorithm Bias in Healthcare | Ella Greenslade
As a result, many Black patients had to be substantially sicker before receiving the same risk score assigned to White patients. The system was not explicitly programmed to discriminate. Instead, the bias emerged because the target variable—future cost—already contained the effects of unequal healthcare access. The model faithfully reproduced those patterns. [arXiv]arxiv.orgUnderstanding racial bias in health using the Medical Expenditure Panel Survey dataNovember 4, 2019…
This illustrates a broader AI lesson. Removing sensitive variables such as race does not necessarily eliminate bias if the outcome being predicted is itself distorted by historical inequalities. A flawed target can transmit inequities even when protected characteristics are absent from the model inputs. [Reddit]reddit.comRacial bias in a medical algorithm favors white patients over sicker black patients…
Why accuracy on costs can still fail patients
One of the most important insights from this case is that a model can be accurate and harmful at the same time.
Suppose a healthcare organisation asks an AI system to predict which patients will generate the highest costs next year. The model may achieve excellent statistical performance. It may correctly forecast expenditures, identify future high-cost patients, and pass standard validation tests. From a machine-learning perspective, it appears successful.
The problem is that healthcare organisations often care about something different. Their real objective may be to identify patients with severe illness, unmet needs, or elevated risk of deterioration. Cost prediction and need prediction overlap, but they are not the same task. When the chosen target differs from the real-world goal, optimisation can produce misleading results. The model becomes better at predicting spending than at finding the patients who most need help. [arXiv]arxiv.orgUnderstanding racial bias in health using the Medical Expenditure Panel Survey dataNovember 4, 2019…
This is a classic example of a proxy-target failure. The algorithm optimises exactly what developers asked it to optimise. The failure arises not from poor modelling but from selecting a target variable that captures the wrong phenomenon. In AI system design, this is often described as a problem-formulation error: the mathematical objective does not fully represent the human objective. [arXiv]arxiv.orgUnderstanding racial bias in health using the Medical Expenditure Panel Survey dataNovember 4, 2019…
The distinction is especially important in high-stakes settings such as healthcare. A model that accurately predicts spending may allocate resources towards patients who are expensive to treat while overlooking patients whose needs are substantial but historically under-served. In that situation, improving predictive accuracy on costs does not improve fairness or clinical effectiveness. It merely improves the prediction of a flawed proxy. [ellagreenslade.com]ellagreenslade.comAlgorithm Bias in Healthcare | Ella GreensladeAlgorithm Bias in Healthcare | Ella Greenslade
What this teaches about AI targets
The healthcare-cost example demonstrates that the quality of an AI system depends not only on its algorithms but also on the variables chosen to represent success.
Before training a model, developers must ask whether the target truly measures the outcome they care about. If the target reflects access, prices, administrative processes, or social inequalities in addition to the intended outcome, the model may reproduce those distortions at scale.
For healthcare risk scoring, illness burden, chronic disease indicators, clinical outcomes, and unmet health needs are often closer to the underlying objective than raw spending. The broader lesson extends beyond healthcare: whenever an AI system relies on an easily measured proxy instead of the real goal, strong predictive performance can conceal systematic mistakes. The healthcare-cost case remains one of the clearest demonstrations that choosing the wrong target can matter as much as choosing the wrong algorithm. [arXiv+2Health Justice Monitor]arxiv.orgUnderstanding racial bias in health using the Medical Expenditure Panel Survey dataNovember 4, 2019…
Amazon book picks
Further Reading
Books and field guides related to Why costs can misread medical need. Use these as the next step if you want deeper reading beyond the article.
Deep Medicine
Examines how AI should support genuine patient needs rather than crude proxies.
Weapons of Math Destruction
Explains dangers of using poor proxy measures in decision systems.
Invisible Women
Shows how flawed data and measurement choices create unequal outcomes.
The Patient Will See You Now
Provides healthcare context for data-driven decision making.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: arxiv.org
Link: https://arxiv.org/abs/1911.01509Source snippet
Understanding racial bias in health using the Medical Expenditure Panel Survey dataNovember 4, 2019...
Published: November 4, 2019
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC8414822/ -
Source: ellagreenslade.com
Title: Algorithm Bias in Healthcare | Ella Greenslade
Link: https://www.ellagreenslade.com/2023/04/09/algorithm-bias-in-healthcare/ -
Source: reddit.com
Link: https://www.reddit.com/r/neoliberal/comments/dmpgbqSource snippet
Racial bias in a medical algorithm favors white patients over sicker black patients...
-
Source: healthjusticemonitor.org
Link: https://www.healthjusticemonitor.org/its-the-prices-and-more/Source snippet
Health Justice MonitorHealth Justice MonitorIt’s the Prices and More - Health Justice Monitor...
-
Source: pubmed.ncbi.nlm.nih.gov
Link: https://pubmed.ncbi.nlm.nih.gov/19276017/Source snippet
Usual source of care: an important source of variation in health care spending - PubMed...
Additional References
-
Source: youtube.com
Title: AI for the Fair-Minded: Bias in AI in Under 9 Minutes
Link: https://www.youtube.com/watch?v=dfYwjBpxg3cSource snippet
DEI Special Seminar: Algorithmic bias and data platforms (Ziad Obermeyer)...
-
Source: youtube.com
Title: Dissecting Racial Bias in an Algorithm that Guides Health [Decisions]({{ ‘decisions/’ | relative_url }}) for Millions
Link: https://www.youtube.com/watch?v=y6eo0FZIqjk -
Source: youtube.com
Title: Dissecting Algorithmic Bias | Ziad Obermeyer | AI FOR GOOD DISCOVERY
Link: https://www.youtube.com/watch?v=U5MlyFsMi-ESource snippet
London Speaker Bureau...
-
Source: youtube.com
Title: DEI Special Seminar: Algorithmic bias and data platforms (Ziad Obermeyer)
Link: https://www.youtube.com/watch?v=6xo2tzuOyY4 -
Source: youtube.com
Title: Keynote Presentation: Dissecting Algorithmic Bias
Link: https://www.youtube.com/watch?v=JfKYO1W4uuASource snippet
London Speaker Bureau...
Topic Tree