Group By Part II: Data Transformation

Aggregate Vs. transform

Question: Aggregate Vs. transform

Answer: Once we have created a grouping structure, we can use it for aggregating the data (i.e., passing from monthly to yearly data points, see the previous section) as well as for expanding the source data. This procedure is called transformation and does not affect the number of rows in the DataFrame.

In the code examples of this section, we get the cumulative sum of a product’s sales for each period. Having defined the grouping structure, we use the .transform() method to apply NumPy’s cumsum function to the values of sales that regard the same product. The outcome of the transformation is assigned to a new column like cumulative_sales.

# import the pandas library with the socially accepted alias 'pd'
>>> import pandas as pd

# the data frame
>>> df = pd.DataFrame.from_dict(
    {
        "product": ['a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c'],
        "month": [0, 0, 0, 1, 1, 1, 2, 2, 2],
        "sales": [10,  5,  0,  7,  6,  5,  4,  6,  9]
    }
    )

# data view
>>> df
  product  month  sales
0       a      0     10
1       b      0      5
2       c      0      0

This section includes detailed examples showing how transformation differs from aggregation by maintaining the original number of rows while creating new calculated columns based on group-wise operations.

This section demonstrates various transformation operations that can be applied within groups, such as calculating running totals, standardizing values within groups, or computing group-relative statistics.