【spark】Andy Konwinski-spark-summit-china-spark-keynote.pdf

The Spark Project And What’s Next Today Andy Konwinski @andykonwinski Community Project History Spark started as research project in 2009 Open sourced in 2010 Entered Apache Incubator June 2013 Graduated to Top Level Project in 8 months Development Community Over 100 contributors, from 30+ companies »Databricks, Yahoo!, Intel, Adobe, Cloudera, Bizo, … Development Community One of the most active communities in big data Comparison: Storm (48), Giraph (52), Drill (18), Tez Past 14 months: more active devs than Hadoop MapReduce ! Development Community Healthy across the whole ecosystem Release Growth Spark 0.9: 83 Spark 0.8: contributors 67 contributors Spark 0.7: 31 Spark 0.6: contributors 17 contributors Oct ‘12 Feb ‘13 Sept ‘13 Feb ‘14 Some Community Contributions YARN support (Yahoo!) Columnar compression in Shark (Yahoo!) Fair scheduling (Intel) Metrics reporting (Intel, Quantifind) New RDD operators (Bizo, ClearStory) Scala 2.10 support (Imaginea) Events Growing Exponentially 1000 900 900 800 s 700 e e 600 d n 500 e t t 400 440 A 300 200 220 100 150 0 AMP Camp 1 AMP Camp 2 Spark Summit Spark Summit (Aug 2012) (Aug 2013) (Nov 2013) (Jun 2014) Meetup Groups Taking Off Spark meetups n