YeoJohnsonTransformer

The Yeo-Johnson transformation is defined as:

where Y is the response variable and λ is the transformation parameter.

API Reference

class feature_engine.transformation.YeoJohnsonTransformer(variables=None)

The YeoJohnsonTransformer() applies the Yeo-Johnson transformation to the numerical variables.

The Yeo-Johnson transformation implemented by this transformer is that of SciPy.stats: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.yeojohnson.html

The YeoJohnsonTransformer() works only with numerical variables.

A list of variables can be passed as an argument. Alternatively, the transformer will automatically select and transform all numerical variables.

Parameters

variableslist, default=None

The list of numerical variables that will be transformed. If None, the transformer will automatically find and select all numerical variables.

Attributes

lambda_dict_ :

Dictionary containing the best lambda for the Yeo-Johnson per variable.

References

1

Weisberg S. “Yeo-Johnson Power Transformations”. https://www.stat.umn.edu/arc/yjpower.pdf

Methods

fit:	Learn the optimal lambda for the Yeo-Johnson transformation.
transform:	Apply the Yeo-Johnson transformation.
fit_transform:	Fit to data, then transform it.

fit(X, y=None)

Learn the optimal lambda for the Yeo-Johnson transformation.

Parameters

Xpandas dataframe of shape = [n_samples, n_features]

The training input samples. Can be the entire dataframe, not just the variables to transform.

ypandas Series, default=None

It is not needed in this transformer. You can pass y or None.

Returns

self

Raises

TypeError

• If the input is not a Pandas DataFrame

• If any of the user provided variables are not numerical

ValueError

• If there are no numerical variables in the df or the df is empty

• If the variable(s) contain null values

transform(X)

Apply the Yeo-Johnson transformation.

Parameters

XPandas DataFrame of shape = [n_samples, n_features]

The data to be transformed.

Returns

Xpandas dataframe

The dataframe with the transformed variables.

rtype

DataFrame ..

Raises

TypeError

If the input is not a Pandas DataFrame

ValueError

• If the variable(s) contain null values.

• If the dataframe not of the same size as that used in fit().

Example

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split

from feature_engine import transformation as vt

# Load dataset
data = data = pd.read_csv('houseprice.csv')

# Separate into train and test sets
X_train, X_test, y_train, y_test =  train_test_split(
            data.drop(['Id', 'SalePrice'], axis=1),
            data['SalePrice'], test_size=0.3, random_state=0)

# set up the variable transformer
tf = vt.YeoJohnsonTransformer(variables = ['LotArea', 'GrLivArea'])

# fit the transformer
tf.fit(X_train)

# transform the data
train_t= tf.transform(X_train)
test_t= tf.transform(X_test)

# un-transformed variable
X_train['LotArea'].hist(bins=50)

# transformed variable
train_t['LotArea'].hist(bins=50)