ImageNet: influences, peers and legacy

Every connection with its receipt

ImageNet began in 2007 as Fei-Fei Li's bet that data, not algorithms, was the bottleneck in computer vision — a 14-million-image database scaffolded onto Princeton's WordNet and labeled by Amazon Mechanical Turk workers. This mix traces its lineage from lexical databases and cognitive psychology's theories of categories through the dataset culture of the 2000s, and out to the deep learning boom its 2012 challenge detonated.

The Kynda mix for ImageNet

• Key Influence · WordNet by George A. Miller (1985). ImageNet's entire architecture is borrowed: its 20,000+ categories are WordNet synsets, and the project's original goal was to populate every noun synset in Miller's Princeton lexical database with hundreds of verified images. Fei-Fei Li was teaching at Princeton when she started, in the same institution that built WordNet. Without Miller's hierarchy of hypernyms and hyponyms there is no organizing spine for the images — the name itself is an homage.,
• Influencia Obscura · Peekaboom: A Game for Locating Objects in Images by Luis von Ahn (2006). Von Ahn's Carnegie Mellon work on the ESP Game and Peekaboom proved that strangers on the internet would produce accurate image labels and even object locations if the task were structured properly — the direct intellectual precursor to ImageNet's turn to Amazon Mechanical Turk after in-house labeling proved impossibly slow. The crowdsourced annotation layer, not the scraping, is what made ImageNet's scale achievable.,
• Local Roots · Caltech-256 by Pietro Perona (2007). Perona was Fei-Fei Li's doctoral advisor at Caltech, and this Griffin–Holub–Perona follow-up to the earlier 101-category set is the immediate ancestor ImageNet was designed to dwarf: 256 categories, roughly 30,000 images, clutter added to answer critiques of its predecessor's uniform poses. The Caltech vision lab's object-category datasets are the scene from which ImageNet's ambitions were launched.,
• Beyond the Medium · Basic Objects in Natural Categories by Eleanor Rosch (1976). Rosch's work on prototype theory and the privileged "basic level" of categorization — why people say "chair" rather than "furniture" or "Windsor chair" — underpins the whole question of what granularity an image dataset should label. ImageNet's inheritance of WordNet's nested hypernyms made this psychological problem a computational one: fine-grained dog breeds sit beside broad artifacts, and the resulting confusions are Rosch's thesis made measurable.,
• Peer · The PASCAL Visual Object Classes Challenge by Mark Everingham (2010). Running annually from 2005, PASCAL VOC was the benchmark competition ImageNet's organizers explicitly modeled their challenge on — twenty classes, careful annotation, a withheld test server and a yearly workshop. The ILSVRC began in 2010 as a PASCAL-affiliated "taster" competition before eclipsing it, and the Oxford-led challenge supplied the evaluation protocols and mean-average-precision culture that ImageNet inherited wholesale.,
• Key Collaborator · WordNet: An Electronic Lexical Database by Christiane Fellbaum (1998). Fellbaum, the Princeton linguist who directed WordNet after Miller, consulted on ImageNet's use of the synset hierarchy and is credited in the project's early acknowledgements. Her edited volume is the authoritative account of how the lexical database's senses, hypernym chains and glosses are structured — the document you need to understand why ImageNet's class list looks the way it does, breeds and artifacts and all.,
• Legacy · ImageNet Classification with Deep Convolutional Neural Networks by Alex Krizhevsky (2012). AlexNet, written with Ilya Sutskever and Geoffrey Hinton, cut the challenge's top-five error rate from roughly 26 percent to 15 percent and is universally cited as the start of the deep learning era. The architecture was not radically new; what was new was a million labeled images and two GPUs. The paper's title names its dependency outright — the dataset is the co-author of the result.,

What influenced ImageNet

• WordNet by George A. Miller (1985). WordNet (George A. Miller) — titan for ImageNet “As a result of this meeting, Li went on to build ImageNet starting from the roughly 22,000 nouns of WordNet and using many of its features.” (Wikipedia)
• Irving Biederman. Biederman’s estimate inspired the scale of the ImageNet project. (also via Recognition-by-components: A theory of human image understanding) “In 2007, while at Princeton, Li began developing ImageNet with the goal of building a large-scale visual dataset inspired by an estimate from cognitive psychologist Irving Biederman that humans recognize approximately 30,000 object categ…” (Wikipedia)
• LabelMe by Bryan Russell (2008). LabelMe (Bryan Russell) — ghost for ImageNet
• Recognition-by-components: A theory of human image understanding by Irving Biederman (1987). Recognition-by-Components: A Theory of Human Image Understanding (Irving Biederman) — culture for ImageNet
• 80 Million Tiny Images by Antonio Torralba (2008). 80 Million Tiny Images (Antonio Torralba) — titan for ImageNet
• Vision: A Computational Investigation into the Human Representation and Processing of Visual Information by David Marr (1982). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (David Marr) — culture for ImageNet
• MNIST database by Yann LeCun (1998). MNIST database (Yann LeCun) — titan for ImageNet
• Basic Objects in Natural Categories by Eleanor Rosch (1976). Basic Objects in Natural Categories (Eleanor Rosch) — culture for ImageNet
• Peekaboom: A Game for Locating Objects in Images by Luis von Ahn (2006). Peekaboom: A Game for Locating Objects in Images (Luis von Ahn) — ghost for ImageNet

Peers and kindred spirits

• CIFAR-10 by Alex Krizhevsky (2009). CIFAR-10 (Alex Krizhevsky) — peer for ImageNet “Building on Convolutional Neural Networks and Sutskever's Deep Neural Network approach of deepening the neural layers far beyond the convention of the time—as well as adding Dropout for training resilience—AlexNet won the ImageNet challe…” (en.wikipedia.org)
• CS231n: Convolutional Neural Networks for Visual Recognition by Andrej Karpathy (2015). CS231n: Convolutional Neural Networks for Visual Recognition (Andrej Karpathy) — geography for ImageNet “Andrej Karpathy estimated in 2014 that with concentrated effort, he could reach 5.1% error rate, and ~10 people from his lab reached ~12–13% with less effort.” (en.wikipedia.org)
• WordNet: An Electronic Lexical Database by Christiane Fellbaum (1998). WordNet: An Electronic Lexical Database (Christiane Fellbaum) — collaborator for ImageNet “In 2007, Li met with Princeton professor Christiane Fellbaum, one of the creators of WordNet, to discuss the project.” (en.wikipedia.org)
• ImageNet Large Scale Visual Recognition Challenge by Olga Russakovsky (2015). ImageNet Large Scale Visual Recognition Challenge (Olga Russakovsky) — collaborator for ImageNet “However, as one of the challenge's organizers, Olga Russakovsky, pointed out in 2015, the ILSVRC is over only 1000 categories; humans can recognize a larger number of categories, and also (unlike the programs) can judge the context of an…” (en.wikipedia.org)
• Caltech-256 by Pietro Perona (2007). Caltech-256 (Pietro Perona) — geography for ImageNet
• Microsoft COCO: Common Objects in Context by Tsung-Yi Lin (2014). Microsoft COCO: Common Objects in Context (Tsung-Yi Lin) — peer for ImageNet
• The PASCAL Visual Object Classes Challenge by Mark Everingham (2010). The PASCAL Visual Object Classes Challenge (Mark Everingham) — peer for ImageNet
• Visual Genome by Ranjay Krishna (2016). Visual Genome (Ranjay Krishna) — geography for ImageNet
• Deep Learning for Generic Object Detection: A Survey by Jia Deng (2020). Deep Learning for Generic Object Detection: A Survey (Jia Deng) — collaborator for ImageNet

Who ImageNet influenced

• ImageNet Classification with Deep Convolutional Neural Networks by Alex Krizhevsky (2012). ImageNet Classification with Deep Convolutional Neural Networks (Alex Krizhevsky) — legacy for ImageNet “Building on Convolutional Neural Networks and Sutskever's Deep Neural Network approach of deepening the neural layers far beyond the convention of the time—as well as adding Dropout for training resilience—AlexNet won the ImageNet challe…” (en.wikipedia.org)
• Deep Residual Learning for Image Recognition by Kaiming He (2015). Deep Residual Learning for Image Recognition (Kaiming He) — legacy for ImageNet
• Learning Transferable Visual Models From Natural Language Supervision by Alec Radford (2021). Learning Transferable Visual Models From Natural Language Supervision (Alec Radford) — legacy for ImageNet