10 → 5,000
Checking ten company filings manually may be manageable; checking 5,000 requires automation.
Intro to Computational Journalism
How coding can help journalists
We are going to use code as a reporting tool.
MARKDOWNWelcome.md
You don't need to memorise everything, and expect your code to fail regularly. The important skill is learning how to understand what happened and what to try next.
01
What is computational journalism?
MARKDOWNIn [1]
Computational journalism means using structured methods, data and code to support reporting.
The code is useful only because it helps us ask or answer a journalistic question.
When we want to work at a different scale
Every month
A script can download the same monthly table every time it is updated.
Apply the same cleaning rule to every row rather than relying on manual edits.
A colleague can rerun the notebook and see how the result was produced.
Expose outliers that become reporting leads.
Audit public claims, procurement data and company disclosures.
Code does not replace reporting.
It helps us decide where to report, what claim to test and what evidence to request. Computational work does not remove journalistic judgement.
02
The anatomy of a dataset
One row
One observation
Depending on the dataset, that observation might be one company, one person, one district, one transaction or one financial year.
| company | year | profit | |
|---|---|---|---|
| 0 | Aruna Power | 2024 | ₹84m |
| 1 | Nila Energy | 2024 | ₹51m |
| 2 | Vaigai Solar | 2024 | ₹73m |
One column
One variable
A variable is one characteristic recorded about each observation. In tables, variables appear as columns.
| company | year | profit | |
|---|---|---|---|
| 0 | Aruna Power | 2024 | ₹84m |
| 1 | Nila Energy | 2024 | ₹51m |
| 2 | Vaigai Solar | 2024 | ₹73m |
Variables can be different types
84Numeric
Numbers that we may calculate with.
"solar"Categorical
Groups observations into labels.
2026-07-28Date
Records time.
TN-0042Identifier
Distinguishes one record from another.
How to read a dataset
A data dictionary explains what each variable means, how it was measured, what format it uses and what its possible values represent.
Without one, an obvious column name can still mislead.
What does profit mean?
operating profitprofit before taxnet profitquarterly profitannual profit
At minimum
- What does each row represent?
- What does each column mean?
- What unit is being used?
- Which time period does the value cover?
MORE ROBUSTdictionary.md
- Are the figures estimates, provisional values or final figures?
- How are missing values recorded?
- Have definitions changed over time?
No data dictionary? That becomes a reporting problem.
- Look for methodology notes.
- Inspect the source website.
- Contact the data provider.
- Compare the data with published reports.
- Create your own working dictionary as you investigate.
Data
The file extension is a clue: it tells us what a file can preserve, how it is structured and how we might open it.
Data
CSV
companies.csvXLSX
survey.xlsxSHP
districts.shpPDF
report.pdfJSON
api.jsonHTML
table.htmlZIP
archive.zip?
README.mdPlain-text table
Portable and easy to analyse. Does not preserve highlight colours, formulas or multiple sheets when exported.
Excel workbook
May include several sheets, formulas, merged cells and formatting.
Document, not data
Tables may look structured but be difficult to extract reliably if they are not machine readable.
Webpage language
Useful when collecting tables or repeated page elements.
Structured data commonly returned by APIs.
A compressed container that may hold one or many data files.
Commonly used to store geographic boundaries. It is both data and geometry.
03
Trying out Google Colab
A workspace and a record
The same notebook can later become a transparent record of the analysis.
New tabsession_1.ipynb
colab.research.google.com
- It runs in the browser.
- You don’t need to install Python.
- Notebooks combine code, notes and output in one document.
Say hello to the newsroom
CODEIn [11]
print("Hello, newsroom")
Hello, newsroom
CODEIn [12]
print("Hello, newsroom")
A function: a named instruction Python already knows.
The text inside quotation marks is a string.
Parentheses hold the information being passed to the function.
The output appears directly below the cell.
publication = "The Hindu"
print(publication)
print(publication)
04
Your first Pandas exercise
A library for tabular data
A library is reusable code written by other people. We import it rather than rebuilding its functions ourselves.
Python library
PANDAS
read_csv()
head()
shape
columns# tools we can reuse
Read one line,
one part at a time
CODEIn [15]
importloadpandasthe libraryaspdshorter nickname
Open a dataset
url = ...Store the dataset address in a variable.
pd.read_csv(url)Read the CSV and turn it into a dataframe.
dfA conventional short name for the dataframe.
Ask what is inside
CODEIn [17]
df.head()
df.shape
df.columns
df.shape
df.columns
head() shows five rows. shape returns rows and columns. columns lists names exactly as Python sees them.
Journalistic inspection
- What does one row represent?
- How many records are present?
- Which columns might answer a reporting question?
- Do the column names make sense?
- Do any values immediately look suspicious?
- What information would we need from a data dictionary or source note?
Optional final command if the class is moving comfortably.
05
Errors and debugging
CODEIn [18]
df.heed()
ERRORTraceback
-------------------------------------------------------------------
AttributeError: 'DataFrame' object has no attribute 'heed'
Did you mean: head() ?
AttributeError: 'DataFrame' object has no attribute 'heed'
Did you mean: head() ?