Hands-On Big Data Analytics with PySpark: Analyze large datasets and discover techniques for testing, immunizing, and parallelizing Spark jobs - Softcover

Digital, Colibri; Lai, Rudy; Potaczek, Bartłomiej

Softcover

ISBN 10: 183864413X ISBN 13: 9781838644130

Verlag: Packt Publishing, 2019

Inhaltsangabe

Use PySpark to easily crush messy data at-scale and discover proven techniques to create testable, immutable, and easily parallelizable Spark jobs

Key Features:

- Work with large amounts of agile data using distributed datasets and in-memory caching

- Source data from all popular data hosting platforms, such as HDFS, Hive, JSON, and S3

- Employ the easy-to-use PySpark API to deploy big data Analytics for production

Book Description:

Apache Spark is an open source parallel-processing framework that has been around for quite some time now. One of the many uses of Apache Spark is for data analytics applications across clustered computers. In this book, you will not only learn how to use Spark and the Python API to create high-performance analytics with big data, but also discover techniques for testing, immunizing, and parallelizing Spark jobs.

You will learn how to source data from all popular data hosting platforms, including HDFS, Hive, JSON, and S3, and deal with large datasets with PySpark to gain practical big data experience. This book will help you work on prototypes on local machines and subsequently go on to handle messy data in production and at scale. This book covers installing and setting up PySpark, RDD operations, big data cleaning and wrangling, and aggregating and summarizing data into useful reports. You will also learn how to implement some practical and proven techniques to improve certain aspects of programming and administration in Apache Spark.

By the end of the book, you will be able to build big data analytical solutions using the various PySpark offerings and also optimize them effectively.

What You Will Learn:

- Get practical big data experience while working on messy datasets

- Analyze patterns with Spark SQL to improve your business intelligence

- Use PySpark s interactive shell to speed up development time

- Create highly concurrent Spark programs by leveraging immutability

- Discover ways to avoid the most expensive operation in the Spark API: the shuffle operation

- Re-design your jobs to use reduceByKey instead of groupBy

- Create robust processing pipelines by testing Apache Spark jobs

Who this book is for:

This book is for developers, data scientists, business analysts, or anyone who needs to reliably analyze large amounts of large-scale, real-world data. Whether you're tasked with creating your company's business intelligence function or creating great data platforms for your machine learning models, or are looking to use code to magnify the impact of your business, this book is for you.

Über diesen Titel

Über die Autorin bzw. den Autor

Bart¿omiej Potaczek is a software engineer working for Schibsted Tech Polska and programming mostly in JavaScript. He is a big fan of everything related to the react world, functional programming, and data visualization. He founded and created InitLearn, a portal that allows users to learn to program in a pair-programming fashion. He was also involved in InitLearn frontend, which is built on the React-Redux technologies. Besides programming, he enjoys football and crossfit. Currently, he is working on rewriting the frontend for tv.nu-Sweden's most complete TV guide, with over 200 channels. He has also recently worked on technologies including React, React Router, and Redux.

Details zur Ausgabe

• Verlag: Packt Publishing
• Erscheinungsdatum: 2019
• Sprache: Englisch
• ISBN 10: 183864413X
• ISBN 13: 9781838644130
• Einband: Taschenbuch
• Anzahl der Seiten: 182
• Kontakt zum Hersteller: Nicht verfügbar
• Verantwortliche Person: gpsr@libri.de; gpsr@libri.de; ; Friedensallee 273; Hamburg; 22763; Deutschland