Feature Importance and Model Interpretation

Published

Jun 2026

  • ID: DS-L13
  • Type: Premium
  • Audience: Intermediate to Advanced
  • Theme: From stable models to interpretable models

In the previous lesson, we focused on stability.

We used:

Now we move to the next question:

What is the model actually using to make predictions?

This is the transition from:

stable models → interpretable models

A model may produce reasonable predictions, but that is not enough.

In applied data science, we also need to understand how the model behaves, what features it relies on, and what kinds of claims are safe to make from its outputs.

How to Run This Lesson

Run the supporting script from the project root:

python scripts/python/13a_interpret_model_features.py

This creates feature interpretation outputs in the reports/ directory:

reports/diabetes-linear-model-coefficients.csv
reports/diabetes-tree-feature-importance.csv
reports/figures/diabetes-linear-model-coefficients.png
reports/figures/diabetes-tree-feature-importance.png

Then render the Quarto site:

quarto render

You can also run the code blocks inside this chapter interactively.

The script-based workflow is preferred because it creates reusable project artifacts that can be inspected, compared, committed, and reused in later chapters.

Load the Dataset

We continue using the same diabetes dataset saved earlier in the project data/ directory.

import pandas as pd

df = pd.read_csv("data/diabetes.csv")

X = df.drop(columns=["disease_progression"])
y = df["disease_progression"]

df.head()

The feature table contains the predictors used by the model.

X.head()

The target column contains the outcome we are trying to predict.

y.head()

Using the same saved dataset keeps the modeling workflow stable across lessons.

Why Interpretation Matters

Model evaluation tells us how well a model predicts.

Model interpretation helps us understand how the model predicts.

These are different questions.

A model can have acceptable performance but still be difficult to explain. A model can also be easy to explain but not accurate enough to use.

Applied data science requires both views:

  • performance: is the model useful?
  • stability: is the result reliable?
  • interpretation: what is the model relying on?
  • caution: what claims are safe to make?

Interpretation is especially important when model outputs may influence decisions, reports, recommendations, or real-world actions.

Fit a Linear Model

We start with a linear regression model because its coefficients are directly inspectable.

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

linear_model = LinearRegression()
linear_model.fit(X_train, y_train)

Now we extract the coefficients.

coefficients_df = pd.DataFrame({
    "feature": X.columns,
    "coefficient": linear_model.coef_
})

coefficients_df["absolute_coefficient"] = coefficients_df["coefficient"].abs()

coefficients_df = coefficients_df.sort_values(
    "absolute_coefficient",
    ascending=False
).reset_index(drop=True)

coefficients_df.round(3)

The coefficient column shows the direction and size of each feature’s contribution in the fitted linear model.

The absolute_coefficient column is useful for ranking features by magnitude.

Understanding Coefficients

A coefficient reflects how much the model prediction changes when a feature changes, holding other features constant.

In a linear regression model:

  • a positive coefficient increases the predicted outcome
  • a negative coefficient decreases the predicted outcome
  • a larger absolute coefficient means stronger model influence

However, coefficients must be interpreted carefully.

A coefficient does not automatically imply:

  • causation
  • clinical importance
  • biological mechanism
  • independent real-world truth

Instead, a coefficient describes how the fitted model uses that feature within this dataset, under this modeling setup, and given the presence of the other features.

This distinction is critical.

Coefficients represent model behavior, not proof of real-world mechanisms.

Save Linear Model Coefficients

In an applied workflow, interpretation outputs should be saved just like evaluation outputs.

from pathlib import Path

Path("reports").mkdir(exist_ok=True)

coefficients_df.to_csv(
    "reports/diabetes-linear-model-coefficients.csv",
    index=False
)

coefficients_df.head()

This creates a reusable artifact:

reports/diabetes-linear-model-coefficients.csv

Visualize Linear Model Coefficients

A coefficient plot makes it easier to see which features the linear model uses most strongly.

import matplotlib.pyplot as plt

plot_df = coefficients_df.sort_values("coefficient")

plt.figure(figsize=(8, 5))
plt.barh(plot_df["feature"], plot_df["coefficient"])
plt.axvline(0, linewidth=1)
plt.xlabel("Coefficient value")
plt.ylabel("Feature")
plt.title("Linear Model Coefficients")
plt.tight_layout()
plt.show()

The plot shows both direction and magnitude.

Features on the positive side increase the prediction in the fitted model. Features on the negative side decrease the prediction in the fitted model.

Fit a Tree-Based Model

Linear models use coefficients.

Tree-based models use feature importance.

A decision tree does not estimate a single coefficient for each feature. Instead, it measures how much each feature contributes to splitting the data and reducing prediction error inside the tree.

from sklearn.tree import DecisionTreeRegressor

tree_model = DecisionTreeRegressor(random_state=42)
tree_model.fit(X_train, y_train)

importance_df = pd.DataFrame({
    "feature": X.columns,
    "importance": tree_model.feature_importances_
})

importance_df = importance_df.sort_values(
    "importance",
    ascending=False
).reset_index(drop=True)

importance_df.round(3)

Feature importance is useful, but it is not the same as a coefficient.

It gives relative contribution, not direction.

Save Tree Feature Importance

We also save the tree-based interpretation output.

importance_df.to_csv(
    "reports/diabetes-tree-feature-importance.csv",
    index=False
)

importance_df.head()

This creates another reusable artifact:

reports/diabetes-tree-feature-importance.csv

Visualize Tree Feature Importance

plot_importance_df = importance_df.sort_values("importance")

plt.figure(figsize=(8, 5))
plt.barh(plot_importance_df["feature"], plot_importance_df["importance"])
plt.xlabel("Feature importance")
plt.ylabel("Feature")
plt.title("Decision Tree Feature Importance")
plt.tight_layout()
plt.show()

This plot shows which features the decision tree relies on most.

Unlike the coefficient plot, it does not show whether a feature increases or decreases the prediction.

It only shows relative contribution inside the fitted tree.

Compare the Two Interpretation Views

The linear model and the decision tree provide different views of feature influence.

Linear model coefficients

The linear model provides:

  • direction of influence
  • magnitude of influence
  • a relatively simple interpretation
  • dependence on linear assumptions

A positive coefficient means higher feature values are associated with higher predictions in the fitted model.

A negative coefficient means higher feature values are associated with lower predictions in the fitted model.

Decision tree feature importance

The tree model provides:

  • relative contribution
  • no direction of effect
  • possible nonlinear splitting behavior
  • sensitivity to tree structure

A feature with high importance is one the tree used heavily for prediction.

But the importance value does not say whether the feature increases or decreases the predicted outcome.

Interpretation Is Model-Specific

Different models may highlight different features.

That does not necessarily mean one model is telling the truth and the other is wrong.

It means the models represent the data differently.

A linear model distributes influence through coefficients.

A tree model uses threshold-based splits.

Because the models learn in different ways, they can produce different interpretations.

This is why interpretation should always be tied to the model being interpreted.

We should say:

This model relies strongly on these features.

Not:

These features are universally important.

That difference matters.

What Can Go Wrong During Interpretation

Confusing Prediction With Causation

A feature can be useful for prediction without causing the outcome.

For example, a variable may be correlated with the target because it is a proxy for another factor.

A predictive model can use that proxy even when the proxy itself is not causal.

Ignoring Feature Scale

For linear models, coefficient size depends on feature scale.

If features are measured on very different scales, raw coefficients can be misleading.

In this dataset, the features are already standardized, which makes coefficient comparison more meaningful.

In other datasets, scaling should be considered before comparing coefficient magnitudes.

Overinterpreting One Model

One fitted model gives one interpretation under one modeling setup.

Interpretation becomes stronger when similar patterns appear across:

  • repeated splits
  • different model specifications
  • domain knowledge
  • external validation

Treating Feature Importance as Explanation

Feature importance can show what a model used.

It does not fully explain why the real-world process behaves the way it does.

That distinction becomes especially important in medical, biological, financial, and policy settings.

CDI Interpretation

These results show how models use features differently to represent the same data.

The linear model describes feature influence through directional coefficients.

The decision tree describes feature influence through relative importance.

Both are useful.

Neither should be treated as automatic proof of mechanism.

The safest interpretation is:

Feature influence describes how the model behaves, not how the real world works.

A feature appearing important means the model depends on it for prediction.

It does not prove that the feature causes the outcome.

From Interpretation to Claims

At this point, we can make model-level statements such as:

In this fitted linear model, certain features have larger coefficients than others.

Or:

In this decision tree, prediction depends heavily on a smaller subset of features.

But we should avoid stronger real-world claims such as:

This feature causes disease progression.

Or:

This feature is clinically important.

Those claims require additional evidence beyond model output.

This is the bridge to the next lesson.

Model interpretation is useful, but real-world claims require stricter reasoning.

Summary

In this lesson, we moved from stable models to interpretable models.

We:

  • loaded the saved diabetes dataset
  • fitted a linear regression model
  • extracted and saved model coefficients
  • visualized coefficient direction and magnitude
  • fitted a decision tree model
  • extracted and saved tree feature importance
  • compared coefficient-based and tree-based interpretation
  • separated model behavior from real-world causation

The key idea is:

Interpretation tells us how a model uses features. It does not automatically tell us why the real world behaves that way.

What Comes Next

Now that we can inspect model outputs, we need to learn how to translate them responsibly.

The next lesson focuses on the boundary between:

  • model results
  • analytical conclusions
  • real-world claims

This is where many analyses become misleading.

A model can support a claim, but it does not automatically prove one.

→ From model outputs to real-world claims