High Performance Spark

Best Practices for Scaling and Optimizing Apache Spark

Holden Karau, Rachel Warren (Autoren)

Buch | Softcover

358 Seiten

2017
O'Reilly Media (Verlag)
978-1-4919-4320-5 (ISBN)

Artikel merken

Apache Spark is amazing when everything clicks. But if you haven’t seen the performance improvements you expected, or still don’t feel confident enough to use Spark in production, this practical book is for you. Authors Holden Karau and Rachel Warren demonstrate performance optimizations to help your Spark queries run faster and handle larger data sizes, while using fewer resources.

Ideal for software engineers, data engineers, developers, and system administrators working with large-scale data applications, this book describes techniques that can reduce data infrastructure costs and developer hours. Not only will you gain a more comprehensive understanding of Spark, you’ll also learn how to make it sing.

With this book, you’ll explore:

How Spark SQL’s new interfaces improve performance over SQL’s RDD data structure
The choice between data joins in Core Spark and Spark SQL
Techniques for getting the most out of standard RDD transformations
How to work around performance issues in Spark’s key/value pair paradigm
Writing high-performance Spark code without Scala or the JVM
How to test for functionality and performance when applying suggested improvements
Using Spark MLlib and Spark ML machine learning libraries
Spark’s Streaming components and external community packages

Holden Karau is a Canadian, Apache Spark committer, and an active open source contributor. When not in San Francisco working as a software development engineer at IBM's Spark Technology Center, Holden talks internationally on Spark and holds office hours at coffee shops at home and abroad. She makes frequent contributions to Spark, specializing in PySpark and Machine Learning. Prior to IBM she worked on a variety of distributed, search, and classification problems at Alpine, Databricks, Google, Foursquare, and Amazon. She graduated from the University of Waterloo with a Bachelor of Mathematics in Computer Science. Outside of software she enjoys playing with fire, welding, scooters, poutine, and dancing.

Chapter 1 Introduction to High Performance Spark
Chapter 2 How Spark Works
Chapter 3 DataFrames, Datasets & Spark SQL
Chapter 4 Joins (SQL & Core)
Chapter 5 Effective Transformations
Chapter 6 Working with Key/Value Data
Chapter 7 Going Beyond Scala
Chapter 8 Testing & Validation
Chapter 9 Spark MLlib and ML
Chapter 10 Spark Components and Packages
Appendix Spark Tuning and Cluster Sizing

Erscheinungsdatum	08.06.2017
Verlagsort	Sebastopol
Sprache	englisch
Maße	178 x 233 mm
Gewicht	618 g
Einbandart	kartoniert
Themenwelt	Informatik ► Datenbanken ► Data Warehouse / Data Mining
Schlagworte	Apache • Big Data • High Performance • Spark • SQL
ISBN-10	1-4919-4320-3 / 1491943203
ISBN-13	978-1-4919-4320-5 / 9781491943205
Zustand	Neuware