Most of your time is going to involve processing/cleaning/munging data. How do you know your data is clean? Sometimes you know what you need beforehand, but other times you don't. We'll cover the basics of looking at your data and getting started with the Pandas Python library, and then focus on how to "tidy" and reshape data. We'll finish with applying customized processing functions on our data.
Total time (including breaks): 3 Hours
First hour: Pandas DataFrame Basics
Before we start cleaning data, let's begin by covering the basics of the Pandas library. We'll cover importing libraries in Python, and how to load your own datasets into Pandas. From there, you'll typically want to look around your data, so we'll cover various ways we can filter and look at our data, calculate simple aggregate statistics and visualize them. This section will end with how to save our data into files we can share with others.
Second hour: Tidy data
Knowing what is a "clean" and "tidy" dataset will help you look for common data problems and give you an idea what your final dataset should look like. Once your data is tidy, it can be easily transformed to other shapes you need for analysis. Understanding what kinds of data manipulation steps are needed will help you with the "how" to do it, i.e., it is language agnostic, and won't matter what language you use.
Third hour: Applying Functions
Sometimes we need a more complex method to tidy our data.
Other times, we need to perform more complex tasks on our data.
Here we'll cover how to write functions in Python and how to apply them to our data.
This way, if a method does not exist to perform the task we want,
or if we want to combine multiple tasks together,
we can write our own custom functions to process our data.
Conclusion: Getting ready for modeling
We clean data so we can do something with it. A common task is to fit some statistical model on our data. One last processing task will be to convert our categorical variables into "dummy variables" for a model.