Big data

Key people

• Carme ArtigasSpanish politician and businessperson
• Juliana FreireBrazilian computer scientist
• Éric MoulinesFrench researcher in statistical learning
• Bruno Sánchez-Andrade NuñoSpanish astrophysicist

Timeline

• 1990Commercial vendors historically offered parallel database management systems for big data beginning in the 1990s.
• 2012Big data "size" is a constantly moving target; as of 2012 ranging from a few dozen terabytes to many zettabytes of data.
• 2013Based on an IDC report prediction, the global data volume was predicted to grow exponentially from 4.4 zettabytes to 44 zettabytes between 2013 and 2020.
• 2014India Big data analysis was tried out for the BJP to win the 2014 Indian General Election.
• 2017As of 2017, there are a few dozen petabyte class Teradata relational databases installed, the largest of which exceeds 50 PB.

Read

• Weapons of Math DestructionCathy O'Neil · 2016Book
• Big Data AnalyticsKiran Chaudhary · 2021Book
• The Art of InvisibilityKevin D. Mitnick · 2017Book
• Big DataBalamurugan Balusamy · 2021Book

Watch

• Big Data In 5 Minutes | What Is Big Data?| Big Data Analytics | Big Data Tutorial | SimplilearnSimplilearnVideo
• Big Data - Tim SmithTED-EdVideo
• Big Data ExplainedIBM TechnologyVideo

Listen

• Big dataHadoop Application ArchitecturesPodcast
• Women in Big Data Podcast: Career, Big Data & Analytics InsightsDesiree TimmermansPodcast
• Making Data SimpleIBM Big Data & Analytics HubPodcast
• The Data Podcast - Datacenters, Cloud, Big Data, AI, ML, Enterprise Hardware and much moreServer FactoryPodcast

Voices to follow

• Bruno Sánchez-Andrade Nuño@Brunosan · XSpanish astrophysicist
• Seth Stephens-Davidowitz@SethS_D · XAmerican data scientist and economist
• Craig M. Dalton@CraigMDalton · XGeographer
• Mark Litwintschik@marklit82 · XGeospatial consultant

By the numbers

• 73.6Internet users (%) — global, 2025 (World Bank)
• 111.5Mobile subs / 100 — global, 2025 (World Bank)

Debates

• Should personal data be freely used for public benefit?One view: Yes, anonymized data can drive medical breakthroughs and improve urban planning for societal good. · Another: No, individuals have a right to privacy, and their data should not be used without explicit consent, even if anonymized.Open question
• Can AI systems built on big data be truly unbiased?One view: Yes, with careful data curation, diverse datasets, and rigorous algorithmic auditing, biases can be significantly reduced or eliminated. · Another: No, human biases are inherently embedded in the data used to train AI, making it nearly impossible to create a perfectly neutral system.Open question

Glossary

• Big DataExtremely large datasets that are too complex for traditional data processing software to handle.
• Data MiningThe process of discovering patterns, trends, and insights from large datasets using various analytical techniques.
• Machine LearningA type of artificial intelligence that enables systems to learn from data and improve performance without explicit programming.
• Artificial Intelligence (AI)The simulation of human intelligence processes by machines, including learning, reasoning, and problem-solving.
• Cloud ComputingThe delivery of on-demand computing services—including servers, storage, databases, networking, software, analytics, and intelligence—over the Internet.
• Data VisualizationThe graphical representation of information and data to make complex patterns and insights more accessible and understandable.

Careers

Roles this can lead toward

Student research

Published policy papers by One Young India delegates — every delegate leaves published under their own name.

• AI-Augmented Multimodal Risk Prediction of Breast Cancer Using Polygenic Scores, Lifestyle Data, and Temporal EMR ModelingAnjali Desai
• The Plasticity Paradox: Technology’s Impact on The Adolescent BrainNeeti Singh
• Rethinking Education in the Digital Age: Bridging the Gap Between Technology and AccessGovind Mehra
• Untangling The Threads of the TikTok Ban's Impact On Global Trade via Trade, Technology, and DiplomacyAanya Menon

Threads 9

Where this connects to other fields — and why it's worth knowing.

• Ritual and Practice Religion

  A scientist once fed pigeons food at totally random times, and they started repeating whatever they happened to be doing when it arrived, like spinning or bowing, as if it caused the food. Computers do the exact same thing: a program can 'learn' a lucky fluke and treat noise as a rule. Both are superstition, a dance built around pure chance.

• Supply, Demand & Price Discovery Economics

  A price tag looks simple, but it's secretly doing a giant math problem. Millions of strangers who never meet, each knowing one tiny thing about supply or demand, all push and pull until the price lands. No single computer or government planner could crunch all that; the market does it without anyone in charge.

• Psychometrics Psychology

  Old personality tests asked you a hundred nosy questions. Now hiring and lending algorithms skip the quiz and just guess your personality from your clicks and likes. Psychology's controversial mind-measuring is back, running silently in the background without your consent or a single form to fill.

• Courts, Judges & Legal Reasoning Law

  When a judge decides a case by looking up the most similar past cases and copying their outcome, they're doing by hand exactly what a simple AI does: 'find the nearest match, predict the same answer.' Which means a legal 'hard case' is just an input that sits far away from anything the judge has seen before, with no close match to copy.

• Genre & Form Literature

  Feed thousands of novels into a computer that tracks whether the mood rises or falls, and something wild pops out: nearly every story fits one of just about six emotional shapes, like 'rags to riches' or 'tragedy.' The endless variety of stories turns out to be as countable as a handful of geometric forms. Data mining found the skeleton hiding inside all our tales.

• Map Scale and Gerrymandering Geography

  Split the same data into different groups and the answer can flip completely: a medicine that looks helpful overall can look harmful once you separate men and women. Mapmakers hit the identical trap when they redraw district lines and change the winner. Same numbers, opposite conclusion, just from how you slice them.

• The Holocaust and Genocide History

  To murder millions, the Nazis first needed to find and sort them, and for that they used early data machines. IBM's punch-card counters let them census people, tag them by category, and track them down. Killing at that scale needed a database before it needed weapons.

• Non-Western Art Arts

  AI image generators learn from huge picture archives that are mostly Western art. So when they create 'art,' they repeat that old bias, quietly baking a Europe-centered idea of what counts as beautiful into the very tools everyone now uses to make new images.

• Economic Systems Economics

  Hayek said a government can never plan an economy well, because prices hold knowledge scattered across millions of people that no planner could ever gather. So here's the modern twist: can big data and AI finally gather it all — or is the problem just too big, even now?