Principles of Data Wrangling. Practical Techniques Kolonowskie

A key task that any aspiring data-driven organization needs to learn is data wrangling, the process of converting raw data into something truly useful. This practical guide provides business analysts with an overview of various data wrangling techniques and tools, and puts the practice of data wrangling into context by asking, "What are you trying to do and why?"Wrangling data consumes roughly 50-80% of an analyst...s time before any kind of analysis is possible. Written by key executives at Trifacta, this book walks you through the wrangling process by exploring several factors-time, granularity, scope, and structure-that you need to consider as you begin to work with data. You...ll learn a shared language and a comprehensive understanding of data wrangling, with an emphasis on recent agile analytic processes used by many of today...s data-driven organizations.Appreciate the importance-and the satisfaction-of wrangling data the right way.Understand what kind of data is availableChoose which data to use and at what level of detailMeaningfully combine multiple sources of dataDecide how to distill the results to a size and shape that can drive downstream analysis Spis treści: Foreword 1. Introduction Magic Thresholds, PYMK, and User Growth at Facebook 2. A Data Workflow Framework How Data Flows During and Across Projects Connecting Analytic Actions to Data Movement: A Holistic Workflow Framework for Data Projects Raw Data Stage Actions: Ingest Data and Create Metadata Ingesting Known and Unknown Data Creating Metadata Structure Granularity Accuracy Temporality Scope Refined Data Stage Actions: Create Canonical Data and Conduct Ad Hoc Analyses Designing Refined Data Addressing structural issues Addressing granularity issues Addressing accuracy issues Addressing scope issues Refined Stage Analytical Actions Production Data Stage Actions: Create Production Data and Build Automated Systems Creating Optimized Data Designing Regular Reports and Automated Products/Services Data Wrangling within the Workflow Framework 3. The Dynamics of Data Wrangling Data Wrangling Dynamics Additional Aspects: Subsetting and Sampling Core Transformation and Profiling Actions Data Wrangling in the Workflow Framework Ingesting Data Describing Data Assessing Data Utility Designing and Building Refined Data Ad Hoc Reporting Exploratory Modeling and Forecasting Building an Optimized Dataset Regular Reporting and Building Data-Driven Products and Services 4. Profiling Overview of Profiling Individual Value Profiling: Syntactic Profiling Individual Value Profiling: Semantic Profiling Set-Based Profiling Profiling Individual Values in the Candidate Master File Syntactic Profiling in the Candidate Master File Set-Based Profiling in the Candidate Master File 5. Transformation: Structuring Overview of Structuring Intrarecord Structuring: Extracting Values Positional Extraction Pattern Extraction Complex Structure Extraction Intrarecord Structuring: Combining Multiple Record Fields Interrecord Structuring: Filtering Records and Fields Interrecord Structuring: Aggregations and Pivots Simple Aggregations Column-to-Row Pivots Row-to-Column Pivots 6. Transformation: Enriching Unions Joins Inserting Metadata Derivation of Values Generic Proprietary 7. Using Transformation to Clean Data Addressing Missing/NULL Values Addressing Invalid Values 8. Roles and Responsibilities Skills and Responsibilities Data Engineer Data Architect Data Scientist Analyst Roles Across the Data Workflow Framework Organizational Best Practices 9. Data Wrangling Tools Data Size and Infrastructure Data Structures Excel SQL Trifacta Wrangler Transformation Paradigms Excel SQL Trifacta Wrangler Choosing a Data Wrangling Tool