States that spend more on health tend to have higher life expectancy.
Intro to Computational Journalism · Session 8
Thinking with data
From numbers to evidence
Keep asking
What can I responsibly sayfrom this data?
The calculation is rarely the end of the reporting question.
₹25,000₹28,000₹30,000₹31,000₹32,000₹35,000₹38,000₹42,000₹48,000₹4,00,000
What is a typical household income here?
Same values · Different summaries
Median · centre observation₹33,500
Near the centre of most households.
Mean · pulled by the extreme₹70,900
Every value contributes—including ₹4 lakh.
The mean answers one question. The median answers another.
Same mean · Different shape
Most values cluster near the mean
No values sit near the mean
Whenever someone gives you an average, ask to see the distribution.
OUTLIER ≠ BAD DATA
Unusual value
Typo
Different unit
Data-entry problem
Genuine extreme case
Reporting lead
Investigate it before you delete it.
Section 02
Variables move together.
That is a pattern. It is not yet an explanation.
As X changes, what tends to happen to Y?
Positive
Negative
Little / no correlation
Correlation coefficient · A compact description
−1+1
Strong negativeLittle linear relationship · 0Strong positive
Direction
Strength
We are interpreting r—not calculating it by hand.
Income per capita × life expectancy
Higher-income states tend to have higher life expectancy.
Why?
The scatterplot shows a relationship. It does not tell us what produced it.
Section 03
CORRELATION ≠ CAUSATION
Possibility 1 · X causes Y
Health spending
→
Better health outcomes
Possible. But the observed correlation alone does not prove this path.
Reverse causality · Y may affect X
Police numbers
←
Crime
Higher crime may lead governments to hire more police.
Confounder · A third variable affects both
Wealth
↙ ↘
Health spending
Life expectancy
Other reasons a relationship can appear
Measurement
How were the variables recorded?
Selection
Who entered the dataset?
Coincidence
Could this pattern be chance?
Which claim does the data support?
Higher health spending makes people live longer.
Causal language
causes
leads to
drives
results in
Descriptive language
is associated with
is correlated with
tends to accompany
was higher among
coincided with
“Correlation ≠ causation” does not mean “causation can never be established.”
Randomized experimentsNatural experimentsLongitudinal evidenceQuasi-experimental designsConverging sources
Your language should not bestronger than your evidence.
A poll surveys 1,500 voters
47%
support Candidate A
Did 47% of all voters tell us this?
Population
Everyone we want to understand
${''.repeat(40)}
Sample
The people we actually observe
${''.repeat(12)}
Which poll would you trust?
Poll A
100,000voluntary Instagram responses
Poll B
2,000carefully sampled voters
REPRESENTATIVENESS > SHEER SIZE
Who had a chance to appear in the data?
Selection bias
An online poll reaches people who chose to visit.
Non-response bias
People who answer may differ from people who do not.
Survivorship bias
Successful startups are visible; failed startups disappear.
Measurement bias
A confusing question changes the answer recorded.
REPORTED CRIME ≠ ALL CRIME
Actual crimeReporting behaviourPolice registrationLegal definitionsAwarenessAdministrative capacity
Administrative data measures systems as well as reality.
Candidate A · 47% · ±3 percentage points
44%47%50%
Candidate A · 47%
Candidate B · 45%
Can we confidently say A is leading?
Overall survival
Hospital A
85%Hospital B
90%Which hospital looks better?
Wait. What happened?
Low-risk patients98%96%
High-risk patients75%70%
Columns: Hospital A · Hospital B
Hospital A
Hospital B
Hospital A treats many more high-risk patients.
The aggregate hid who was in each group.
BREAK IT DOWN.
GenderAgeIncomeCasteGeographyUrban / ruralYear
State-level relationship
States with higher literacy have lower fertility.
Does not automatically mean
Literate women have fewer children.
What does one row represent?
18,000 people died.
Is that high?
Compared with last year?
Ten years ago?
Another state?
The population?
A national average?
A government target?
You can calculate everything correctly and still reach the wrong conclusion.
Counts vs rates
UP has more road deaths than Goa.
Per 100,000 people? Vehicles? Kilometres of road?
Percent vs percentage points
5% → 6%+1 percentage point · +20% relative increase
Big number
₹80,000 crore → ₹1 lakh crore
Is that actually
a large increase?
Inflation
Population
GDP
Time
Actual vs allocated
2019 · 100
2020 · 70
2021 · 85
2022 · 92
2023 · 98
2024 · 105
2020 → 2024 = +50%
2019 → 2024 = +5%
Why did you choose your starting year?
Do not begin with df.dropna()
Who is missing?
Missing observations may be poorer, rural, conflict-affected
or served by weaker administrative systems.
30 states × 20 indicators × 10 years
Eventually, something will look interesting.
FIND
CHECK
CONTEXTUALISE
REPORT
VERIFY
An interesting correlation is the beginning of reporting
Everything we just discussed can also go wrong visually.
Cherry-picked time period→Cropped timeline
Missing denominator→Totals instead of rates
Ignoring distribution→Only showing the average
Exaggerating change→Truncated y-axis
Hiding subgroups→Aggregate-only chart
Correlation ≠ causation→Causal reading of a scatterplot
Missing data→Unexplained blanks on a map
01Compare categories
Which states have the highest unemployment?
Bar chart
02Change over time
How has inflation changed?
Line chart
03Relationship
Are income and life expectancy related?
Scatterplot
04Distribution
Where do most incomes fall?
Histogram
CHARTS ARE ARGUMENTS.
What to show
What to omit
Where the axis starts
Totals or rates
How to aggregate
How to order
Which period
Which comparison
What headline
Y-axis · 0–100
Y-axis · 89–92
A chart can contain no false numbers and still mislead.
Return to the question
What can I responsibly say from this data?
What exactly is being measured, and who or what is missing?
What is the right denominator, baseline and comparison?
What does this evidence allow me to say—and what does it not allow me to say?
NEXT · Seeing the story · Data visualization