MapReduce, Hadoop and Spark (big-data processing)

Programming model for parallel batch jobs over clusters of cheap machines; Hadoop open-sourced it and launched the 'big data' industry.

Data, cloud & securitySoftware
  1. 2004

    First version

    Jeffrey Dean and Sanjay Ghemawat (Google); Hadoop by Doug Cutting and Mike Cafarella; Spark by Matei Zaharia (UC Berkeley)

    Image: Google Inc. (logo by Ruth Kedar, 1999) · Unclear; no licence stated (Google trad… · google.com home page logo file, Wayback Machine…
  2. 2006

    Apache Hadoop

    Open-source MapReduce and HDFS, scaled at Yahoo, spreads big-data clusters.

    Image: Apache Software Foundation · Apache License 2.0 · Wikimedia Commons
  3. 2010

    Spark research release

    In-memory resilient distributed datasets make iterative analytics far faster than MapReduce.

    Image: Apache Software Foundation · Apache License 2.0 · Wikimedia Commons
  4. 2011

    Apache Kafka stream processing

    LinkedIn open-sources a distributed log that moves big-data work from nightly batches to continuous real-time streams.

    Image: ™/®Apache Software Foundation · Apache License 2.0 · Wikimedia Commons

See it on the timeline