Test Data Engineering

97 Things Every Data Engineer Should Know

Take advantage of the sky-high demand for data engineers today. With this in-depth book, current and aspiring engineers will learn powerful, real-world best practices for managing data big and small. Contributors from Google, Microsoft, IBM, Facebook, Databricks, and GitHub share their experiences and lessons learned for overcoming a variety of specific and often nagging challenges. Edited by Tobias Macey from MIT Open Learning, this book presents 97 concise and useful tips for cleaning, prepping, wrangling, storing, processing, and ingesting data. Data engineers, data architects, data team managers, data scientists, machine learning engineers, and software engineers will greatly benefit from the wisdom and experience of their peers. Projects include: Building pipelines Stream processing Data privacy and security Data governance and lineage Data storage and architecture Ecosystem of modern tools Data team makeup and culture Career advice.

Feature Engineering and Selection

The process of developing predictive models includes many stages. Most resources focus on the modeling algorithms but neglect other critical aspects of the modeling process. This book describes techniques for finding the best representations of predictors for modeling and for nding the best subset of predictors for improving model performance. A variety of example data sets are used to illustrate the techniques along with R programs for reproducing the results.

Bulletin of the United States Bureau of Labor Statistics