YeoJohnsonTransformer

The Yeo-Johnson transformation is defined as:

../_images/yeojohnsonformula.png

where Y is the response variable and λ is the transformation parameter.

API Reference

class feature_engine.variable_transformers.YeoJohnsonTransformer(variables=None)[source]

The YeoJohnsonTransformer() applies the Yeo-Johnson transformation to the numerical variables.

The Yeo-Johnson transformation implemented by this transformer is that of SciPy.stats: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.yeojohnson.html

The YeoJohnsonTransformer() works only with numerical variables.

A list of variables can be passed as an argument. Alternatively, the transformer will automatically select and transform all numerical variables.

Parameters

variables (list, default=None) – The list of numerical variables that will be transformed. If None, the transformer will automatically find and select all numerical variables.

lamda_dict\_

The dictionary containing the {variable: best lambda for the Yeo-Johnson transformation} pairs.

Type

dictionary

fit(X, y=None)[source]

Learns the optimal lambda for the Yeo-Johnson transformation.

Parameters
  • X (pandas dataframe of shape = [n_samples, n_features]) – The training input samples. Can be the entire dataframe, not just the variables to transform.

  • y (None) – y is not needed in this transformer. You can pass y or None.

transform(X)[source]

Applies the Yeo-Johnson transformation.

Parameters

X (pandas dataframe of shape = [n_samples, n_features]) – The data to be transformed.

Returns

X_transformed – The dataframe with the transformed variables.

Return type

pandas dataframe of shape = [n_samples, n_features]

Example Use

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split

from feature_engine import variable_transformers as vt

# Load dataset
data = data = pd.read_csv('houseprice.csv')

# Separate into train and test sets
X_train, X_test, y_train, y_test =  train_test_split(
            data.drop(['Id', 'SalePrice'], axis=1),
            data['SalePrice'], test_size=0.3, random_state=0)

# set up the variable transformer
tf = vt.YeoJohnsonTransformer(variables = ['LotArea', 'GrLivArea'])

# fit the transformer
tf.fit(X_train)

# transform the data
train_t= tf.transform(X_train)
test_t= tf.transform(X_test)

# un-transformed variable
X_train['LotArea'].hist(bins=50)
../_images/lotarearaw.png
# transformed variable
train_t['LotArea'].hist(bins=50)
../_images/lotareayeojohnson.png