Making clusters part of a supervised pipeline#
This notebook explores how K-means clustering can be used as a feature engineering step to improve the performance of a regression model.
Here we use the California Housing dataset, which includes information about the geographic location (latitude and longitude).
Our goal is to predict the median house value (MedHouseVal) using a ridge regression model, to investigate whether adding features derived from applying K-means to geographic coordinates can improve the pipeline’s predictive performance.
from sklearn.datasets import fetch_california_housing
data, target = fetch_california_housing(return_X_y=True, as_frame=True)
target *= 100 # rescale the target in k$
We can first design a predictive pipeline that completly ignores the coordinates:
import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import Ridge
from sklearn.compose import make_column_transformer
data_train, data_test, target_train, target_test = train_test_split(
data, target, test_size=0.2, random_state=0
)
geo_columns = ["Latitude", "Longitude"]
model_drop_geo = make_pipeline(
make_column_transformer(("drop", geo_columns), remainder="passthrough"),
StandardScaler(),
Ridge(alpha=1e-12),
)
test_error_drop_geo = -cross_val_score(
model_drop_geo, data_train, target_train, scoring="neg_mean_absolute_error"
)
print(
"The test MAE without geographical features is: "
f"{test_error_drop_geo.mean():.2f} ± {test_error_drop_geo.std():.2f} k$"
)
The test MAE without geographical features is: 57.35 ± 0.53 k$
We observe a Mean Absolute Error of approximately 57k$ when dropping the geographical features.
As seen in the previous notebook, we suspect that the price information may be linked to the distance to the nearest urban center, and proximity to the coast:
import plotly.express as px
fig = px.scatter_map(
data,
lat="Latitude",
lon="Longitude",
color=target,
zoom=5,
height=600,
labels={"color": "price (k$)"},
)
fig.update_layout(
mapbox_style="open-street-map",
mapbox_center={
"lat": data["Latitude"].mean(),
"lon": data["Longitude"].mean(),
},
margin={"r": 0, "t": 0, "l": 0, "b": 0},
)
fig.show(renderer="notebook")
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[3], line 12
8 zoom=5,
9 height=600,
10 labels={"color": "price (k$)"},
11 )
---> 12 fig.update_layout(
13 mapbox_style="open-street-map",
14 mapbox_center={
15 "lat": data["Latitude"].mean(),
File /opt/hostedtoolcache/Python/3.11.16/x64/lib/python3.11/site-packages/plotly/graph_objs/_figure.py:217, in Figure.update_layout(self, dict1, overwrite, **kwargs)
213 BaseFigure
214 The Figure object that the update_layout method was called on
215
216 """
--> 217 return super().update_layout(dict1, overwrite, **kwargs)
File /opt/hostedtoolcache/Python/3.11.16/x64/lib/python3.11/site-packages/plotly/basedatatypes.py:1415, in BaseFigure.update_layout(self, dict1, overwrite, **kwargs)
1391 def update_layout(self, dict1=None, overwrite=False, **kwargs):
1392 """
1393 Update the properties of the figure's layout with a dict and/or with
1394 keyword arguments.
(...) 1413 The Figure object that the update_layout method was called on
1414 """
-> 1415 self.layout.update(dict1, overwrite=overwrite, **kwargs)
1416 return self
File /opt/hostedtoolcache/Python/3.11.16/x64/lib/python3.11/site-packages/plotly/basedatatypes.py:5105, in BasePlotlyType.update(self, dict1, overwrite, **kwargs)
5103 with self.figure.batch_update():
5104 BaseFigure._perform_update(self, dict1, overwrite=overwrite)
-> 5105 BaseFigure._perform_update(self, kwargs, overwrite=overwrite)
5106 else:
5107 BaseFigure._perform_update(self, dict1, overwrite=overwrite)
File /opt/hostedtoolcache/Python/3.11.16/x64/lib/python3.11/site-packages/plotly/basedatatypes.py:3859, in BaseFigure._perform_update(plotly_obj, update_obj, overwrite)
3857 err = _check_path_in_prop_tree(plotly_obj, key, error_cast=ValueError)
3858 if err is not None:
-> 3859 raise err
3861 # Convert update_obj to dict
3862 # --------------------------
3863 if isinstance(update_obj, BasePlotlyType):
ValueError: Invalid property specified for object of type plotly.graph_objs.Layout: 'mapbox'
Did you mean "map"?
Valid properties:
activeselection
:class:`plotly.graph_objects.layout.Activeselection`
instance or dict with compatible properties
activeshape
:class:`plotly.graph_objects.layout.Activeshape`
instance or dict with compatible properties
annotations
A tuple of
:class:`plotly.graph_objects.layout.Annotation`
instances or dicts with compatible properties
annotationdefaults
When used in a template (as
layout.template.layout.annotationdefaults), sets the
default property values to use for elements of
layout.annotations
autosize
Determines whether or not a layout width or height that
has been left undefined by the user is initialized on
each relayout. Note that, regardless of this attribute,
an undefined layout width or height is always
initialized on the first call to plot.
autotypenumbers
Using "strict" a numeric string in trace data is not
converted to a number. Using *convert types* a numeric
string in trace data may be treated as a number during
automatic axis `type` detection. This is the default
value; however it could be overridden for individual
axes.
barcornerradius
Sets the rounding of bar corners. May be an integer
number of pixels, or a percentage of bar width (as a
string ending in %).
bargap
Sets the gap (in plot fraction) between bars of
adjacent location coordinates.
bargroupgap
Sets the gap (in plot fraction) between bars of the
same location coordinate.
barmode
Determines how bars at the same location coordinate are
displayed on the graph. With "stack", the bars are
stacked on top of one another With "relative", the bars
are stacked on top of one another, with negative values
below the axis, positive values above With "group", the
bars are plotted next to one another centered around
the shared location. With "overlay", the bars are
plotted over one another, you might need to reduce
"opacity" to see multiple bars.
barnorm
Sets the normalization for bar traces on the graph.
With "fraction", the value of each bar is divided by
the sum of all values at that location coordinate.
"percent" is the same but multiplied by 100 to show
percentages.
boxgap
Sets the gap (in plot fraction) between boxes of
adjacent location coordinates. Has no effect on traces
that have "width" set.
boxgroupgap
Sets the gap (in plot fraction) between boxes of the
same location coordinate. Has no effect on traces that
have "width" set.
boxmode
Determines how boxes at the same location coordinate
are displayed on the graph. If "group", the boxes are
plotted next to one another centered around the shared
location. If "overlay", the boxes are plotted over one
another, you might need to set "opacity" to see them
multiple boxes. Has no effect on traces that have
"width" set.
calendar
Sets the default calendar system to use for
interpreting and displaying dates throughout the plot.
clickanywhere
If true, `plotly_click` events will fire for any click
position within the plot area, not just over traces.
When clicking where there is no trace data, the event
will have an empty `points` array but will include
`xvals` and `yvals` with click coordinates in data
space, and `xPixel` and `yPixel` with click coordinates
in pixels, relative to the top-left corner of the graph
div.
clickmode
Determines the mode of single click interactions.
"event" is the default value and emits the
`plotly_click` event. In addition this mode emits the
`plotly_selected` event in drag modes "lasso" and
"select", but with no event data attached (kept for
compatibility reasons). The "select" flag enables
selecting single data points via click. This mode also
supports persistent selections, meaning that pressing
Shift while clicking, adds to / subtracts from an
existing selection. "select" with `hovermode`: "x" can
be confusing, consider explicitly setting `hovermode`:
"closest" when using this feature. Selection events are
sent accordingly as long as "event" flag is set as
well. When the "event" flag is missing, `plotly_click`
and `plotly_selected` events are not fired.
coloraxis
:class:`plotly.graph_objects.layout.Coloraxis` instance
or dict with compatible properties
colorscale
:class:`plotly.graph_objects.layout.Colorscale`
instance or dict with compatible properties
colorway
Sets the default trace colors.
computed
Placeholder for exporting automargin-impacting values
namely `margin.t`, `margin.b`, `margin.l` and
`margin.r` in "full-json" mode.
datarevision
If provided, a changed value tells `Plotly.react` that
one or more data arrays has changed. This way you can
modify arrays in-place rather than making a complete
new copy for an incremental change. If NOT provided,
`Plotly.react` assumes that data arrays are being
treated as immutable, thus any data array with a
different identity from its predecessor contains new
data.
dragmode
Determines the mode of drag interactions. "select" and
"lasso" apply only to scatter traces with markers or
text. "orbit" and "turntable" apply only to 3D scenes.
editrevision
Controls persistence of user-driven changes in
`editable: true` configuration, other than trace names
and axis titles. Defaults to `layout.uirevision`.
extendfunnelareacolors
If `true`, the funnelarea slice colors (whether given
by `funnelareacolorway` or inherited from `colorway`)
will be extended to three times its original length by
first repeating every color 20% lighter then each color
20% darker. This is intended to reduce the likelihood
of reusing the same color when you have many slices,
but you can set `false` to disable. Colors provided in
the trace, using `marker.colors`, are never extended.
extendiciclecolors
If `true`, the icicle slice colors (whether given by
`iciclecolorway` or inherited from `colorway`) will be
extended to three times its original length by first
repeating every color 20% lighter then each color 20%
darker. This is intended to reduce the likelihood of
reusing the same color when you have many slices, but
you can set `false` to disable. Colors provided in the
trace, using `marker.colors`, are never extended.
extendpiecolors
If `true`, the pie slice colors (whether given by
`piecolorway` or inherited from `colorway`) will be
extended to three times its original length by first
repeating every color 20% lighter then each color 20%
darker. This is intended to reduce the likelihood of
reusing the same color when you have many slices, but
you can set `false` to disable. Colors provided in the
trace, using `marker.colors`, are never extended.
extendsunburstcolors
If `true`, the sunburst slice colors (whether given by
`sunburstcolorway` or inherited from `colorway`) will
be extended to three times its original length by first
repeating every color 20% lighter then each color 20%
darker. This is intended to reduce the likelihood of
reusing the same color when you have many slices, but
you can set `false` to disable. Colors provided in the
trace, using `marker.colors`, are never extended.
extendtreemapcolors
If `true`, the treemap slice colors (whether given by
`treemapcolorway` or inherited from `colorway`) will be
extended to three times its original length by first
repeating every color 20% lighter then each color 20%
darker. This is intended to reduce the likelihood of
reusing the same color when you have many slices, but
you can set `false` to disable. Colors provided in the
trace, using `marker.colors`, are never extended.
font
Sets the global font. Note that fonts used in traces
and other layout components inherit from the global
font.
funnelareacolorway
Sets the default funnelarea slice colors. Defaults to
the main `colorway` used for trace colors. If you
specify a new list here it can still be extended with
lighter and darker colors, see
`extendfunnelareacolors`.
funnelgap
Sets the gap (in plot fraction) between bars of
adjacent location coordinates.
funnelgroupgap
Sets the gap (in plot fraction) between bars of the
same location coordinate.
funnelmode
Determines how bars at the same location coordinate are
displayed on the graph. With "stack", the bars are
stacked on top of one another With "group", the bars
are plotted next to one another centered around the
shared location. With "overlay", the bars are plotted
over one another, you might need to reduce "opacity" to
see multiple bars.
geo
:class:`plotly.graph_objects.layout.Geo` instance or
dict with compatible properties
grid
:class:`plotly.graph_objects.layout.Grid` instance or
dict with compatible properties
height
Sets the plot's height (in px).
hiddenlabels
hiddenlabels is the funnelarea & pie chart analog of
visible:'legendonly' but it can contain many labels,
and can simultaneously hide slices from several
pies/funnelarea charts
hoveranywhere
If true, `plotly_hover` events will fire for any cursor
position within the plot area, not just over traces.
When the cursor is not over a trace, the event will
have an empty `points` array but will include `xvals`
and `yvals` with cursor coordinates in data space, and
`xPixel` and `yPixel` with cursor coordinates in
pixels, relative to the top-left corner of the graph
div. A `plotly_unhover` event fires when the cursor
leaves the plot area.
hoverdistance
Sets the default distance (in pixels) to look for data
to add hover labels (-1 means no cutoff, 0 means no
looking for data). This is only a real distance for
hovering on point-like objects, like scatter points.
For area-like objects (bars, scatter fills, etc)
hovering is on inside the area and off outside, but
these objects will not supersede hover on point-like
objects in case of conflict.
hoverlabel
:class:`plotly.graph_objects.layout.Hoverlabel`
instance or dict with compatible properties
hovermode
Determines the mode of hover interactions. If
"closest", a single hoverlabel will appear for the
"closest" point within the `hoverdistance`. If "x" (or
"y"), multiple hoverlabels will appear for multiple
points at the "closest" x- (or y-) coordinate within
the `hoverdistance`, with the caveat that no more than
one hoverlabel will appear per trace. If *x unified*
(or *y unified*), a single hoverlabel will appear
multiple points at the closest x- (or y-) coordinate
within the `hoverdistance` with the caveat that no more
than one hoverlabel will appear per trace. In this
mode, spikelines are enabled by default perpendicular
to the specified axis. If false, hover interactions are
disabled.
hoversort
Determines the order of items shown in unified hover
labels. If "trace", items are sorted by trace index. If
*value descending*, items are sorted by value from
largest to smallest. If *value ascending*, items are
sorted by value from smallest to largest. Only applies
when `hovermode` is *x unified* or *y unified*.
hoversubplots
Determines expansion of hover effects to other subplots
If "single" just the axis pair of the primary point is
included without overlaying subplots. If "overlaying"
all subplots using the main axis and occupying the same
space are included. If "axis", also include stacked
subplots using the same axis when `hovermode` is set to
"x", *x unified*, "y" or *y unified*.
iciclecolorway
Sets the default icicle slice colors. Defaults to the
main `colorway` used for trace colors. If you specify a
new list here it can still be extended with lighter and
darker colors, see `extendiciclecolors`.
images
A tuple of :class:`plotly.graph_objects.layout.Image`
instances or dicts with compatible properties
imagedefaults
When used in a template (as
layout.template.layout.imagedefaults), sets the default
property values to use for elements of layout.images
legend
:class:`plotly.graph_objects.layout.Legend` instance or
dict with compatible properties
map
:class:`plotly.graph_objects.layout.Map` instance or
dict with compatible properties
margin
:class:`plotly.graph_objects.layout.Margin` instance or
dict with compatible properties
meta
Assigns extra meta information that can be used in
various `text` attributes. Attributes such as the
graph, axis and colorbar `title.text`, annotation
`text` `trace.name` in legend items, `rangeselector`,
`updatemenus` and `sliders` `label` text all support
`meta`. One can access `meta` fields using template
strings: `%{meta[i]}` where `i` is the index of the
`meta` item in question. `meta` can also be an object
for example `{key: value}` which can be accessed
%{meta[key]}.
minreducedheight
Minimum height of the plot with margin.automargin
applied (in px)
minreducedwidth
Minimum width of the plot with margin.automargin
applied (in px)
modebar
:class:`plotly.graph_objects.layout.Modebar` instance
or dict with compatible properties
newselection
:class:`plotly.graph_objects.layout.Newselection`
instance or dict with compatible properties
newshape
:class:`plotly.graph_objects.layout.Newshape` instance
or dict with compatible properties
paper_bgcolor
Sets the background color of the paper where the graph
is drawn.
piecolorway
Sets the default pie slice colors. Defaults to the main
`colorway` used for trace colors. If you specify a new
list here it can still be extended with lighter and
darker colors, see `extendpiecolors`.
plot_bgcolor
Sets the background color of the plotting area in-
between x and y axes.
polar
:class:`plotly.graph_objects.layout.Polar` instance or
dict with compatible properties
scattergap
Sets the gap (in plot fraction) between scatter points
of adjacent location coordinates. Defaults to `bargap`.
scattermode
Determines how scatter points at the same location
coordinate are displayed on the graph. With "group",
the scatter points are plotted next to one another
centered around the shared location. With "overlay",
the scatter points are plotted over one another, you
might need to reduce "opacity" to see multiple scatter
points.
scene
:class:`plotly.graph_objects.layout.Scene` instance or
dict with compatible properties
selectdirection
When `dragmode` is set to "select", this limits the
selection of the drag to horizontal, vertical or
diagonal. "h" only allows horizontal selection, "v"
only vertical, "d" only diagonal and "any" sets no
limit.
selectionrevision
Controls persistence of user-driven changes in selected
points from all traces.
selections
A tuple of
:class:`plotly.graph_objects.layout.Selection`
instances or dicts with compatible properties
selectiondefaults
When used in a template (as
layout.template.layout.selectiondefaults), sets the
default property values to use for elements of
layout.selections
separators
Sets the decimal and thousand separators. For example,
*. * puts a '.' before decimals and a space between
thousands. In English locales, dflt is ".," but other
locales may alter this default.
shapes
A tuple of :class:`plotly.graph_objects.layout.Shape`
instances or dicts with compatible properties
shapedefaults
When used in a template (as
layout.template.layout.shapedefaults), sets the default
property values to use for elements of layout.shapes
showlegend
Determines whether or not a legend is drawn. Default is
`true` if there is a trace to show and any of these: a)
Two or more traces would by default be shown in the
legend. b) One pie trace is shown in the legend. c) One
trace is explicitly given with `showlegend: true`.
sliders
A tuple of :class:`plotly.graph_objects.layout.Slider`
instances or dicts with compatible properties
sliderdefaults
When used in a template (as
layout.template.layout.sliderdefaults), sets the
default property values to use for elements of
layout.sliders
smith
:class:`plotly.graph_objects.layout.Smith` instance or
dict with compatible properties
spikedistance
Sets the default distance (in pixels) to look for data
to draw spikelines to (-1 means no cutoff, 0 means no
looking for data). As with hoverdistance, distance does
not apply to area-like objects. In addition, some
objects can be hovered on but will not generate
spikelines, such as scatter fills.
sunburstcolorway
Sets the default sunburst slice colors. Defaults to the
main `colorway` used for trace colors. If you specify a
new list here it can still be extended with lighter and
darker colors, see `extendsunburstcolors`.
template
Default attributes to be applied to the plot. This
should be a dict with format: `{'layout':
layoutTemplate, 'data': {trace_type: [traceTemplate,
...], ...}}` where `layoutTemplate` is a dict matching
the structure of `figure.layout` and `traceTemplate` is
a dict matching the structure of the trace with type
`trace_type` (e.g. 'scatter'). Alternatively, this may
be specified as an instance of
plotly.graph_objs.layout.Template. Trace templates are
applied cyclically to traces of each type. Container
arrays (eg `annotations`) have special handling: An
object ending in `defaults` (eg `annotationdefaults`)
is applied to each array item. But if an item has a
`templateitemname` key we look in the template array
for an item with matching `name` and apply that
instead. If no matching `name` is found we mark the
item invisible. Any named template item not referenced
is appended to the end of the array, so this can be
used to add a watermark annotation or a logo image, for
example. To omit one of these items on the plot, make
an item with matching `templateitemname` and `visible:
false`.
ternary
:class:`plotly.graph_objects.layout.Ternary` instance
or dict with compatible properties
title
:class:`plotly.graph_objects.layout.Title` instance or
dict with compatible properties
transition
Sets transition options used during Plotly.react
updates.
treemapcolorway
Sets the default treemap slice colors. Defaults to the
main `colorway` used for trace colors. If you specify a
new list here it can still be extended with lighter and
darker colors, see `extendtreemapcolors`.
uirevision
Used to allow user interactions with the plot to
persist after `Plotly.react` calls that are unaware of
these interactions. If `uirevision` is omitted, or if
it is given and it changed from the previous
`Plotly.react` call, the exact new figure is used. If
`uirevision` is truthy and did NOT change, any
attribute that has been affected by user interactions
and did not receive a different value in the new figure
will keep the interaction value. `layout.uirevision`
attribute serves as the default for `uirevision`
attributes in various sub-containers. For finer control
you can set these sub-attributes directly. For example,
if your app separately controls the data on the x and y
axes you might set `xaxis.uirevision=*time*` and
`yaxis.uirevision=*cost*`. Then if only the y data is
changed, you can update `yaxis.uirevision=*quantity*`
and the y axis range will reset but the x axis range
will retain any user-driven zoom.
uniformtext
:class:`plotly.graph_objects.layout.Uniformtext`
instance or dict with compatible properties
updatemenus
A tuple of
:class:`plotly.graph_objects.layout.Updatemenu`
instances or dicts with compatible properties
updatemenudefaults
When used in a template (as
layout.template.layout.updatemenudefaults), sets the
default property values to use for elements of
layout.updatemenus
violingap
Sets the gap (in plot fraction) between violins of
adjacent location coordinates. Has no effect on traces
that have "width" set.
violingroupgap
Sets the gap (in plot fraction) between violins of the
same location coordinate. Has no effect on traces that
have "width" set.
violinmode
Determines how violins at the same location coordinate
are displayed on the graph. If "group", the violins are
plotted next to one another centered around the shared
location. If "overlay", the violins are plotted over
one another, you might need to set "opacity" to see
them multiple violins. Has no effect on traces that
have "width" set.
waterfallgap
Sets the gap (in plot fraction) between bars of
adjacent location coordinates.
waterfallgroupgap
Sets the gap (in plot fraction) between bars of the
same location coordinate.
waterfallmode
Determines how bars at the same location coordinate are
displayed on the graph. With "group", the bars are
plotted next to one another centered around the shared
location. With "overlay", the bars are plotted over one
another, you might need to reduce "opacity" to see
multiple bars.
width
Sets the plot's width (in px).
xaxis
:class:`plotly.graph_objects.layout.XAxis` instance or
dict with compatible properties
yaxis
:class:`plotly.graph_objects.layout.YAxis` instance or
dict with compatible properties
Did you mean "map"?
Bad property path:
mapbox_style
^^^^^^
We first feed the coordinates directly to the linear model without transformation.
model_naive_geo = make_pipeline(StandardScaler(), Ridge(alpha=1e-12))
test_error_naive_geo = -cross_val_score(
model_naive_geo,
data_train,
target_train,
scoring="neg_mean_absolute_error",
)
print(
"The test MAE with raw geographical features is: "
f"{test_error_naive_geo.mean():.2f} ± {test_error_naive_geo.std():.2f} k$"
)
Including the geospatial data naively improves the performance a bit, however, we suspect that we can do better by introducing features that represent proximity to points of interests (urban centers, the coast, parks, etc.).
We could look for a dataset containing all the coordinates of the city centers, the coast line and other points of interest in California, then manually engineer such features. However this would require a non-trivial amount of code. Instead we can rely on K-means to achieve something similar implicitly: we use it to find a large number of centroids from the housing data directly and consider each centroid a potential point of interest.
The KMeans class implements a transform method that, given a set of data
points as an argument, computes the distance to the nearest centroid for each
of them. As a result, KMeans can be used as a preprocessing step in a
feature engineering pipeline as follows:
from sklearn.cluster import KMeans
from sklearn.linear_model import Ridge
from sklearn.pipeline import make_pipeline
model_cluster_geo = make_pipeline(
make_column_transformer(
(KMeans(n_clusters=100), geo_columns),
remainder="passthrough",
force_int_remainder_cols=False, # Silence a warning in scikit-learn v1.6
),
StandardScaler(),
Ridge(alpha=1e-12),
)
test_error_cluster_geo = -cross_val_score(
model_cluster_geo,
data_train,
target_train,
scoring="neg_mean_absolute_error",
)
print(
"The test MAE with clustered geographical features is:"
f" {test_error_cluster_geo.mean():.2f} ±"
f" {test_error_cluster_geo.std():.2f} k$"
)
The resulting mean test error is much lower. Furthermore, KMeans is now part
of a supervised pipeline, which means we can use a grid-search to tune
n_clusters as we have learned in previous modules of this course.
from sklearn.model_selection import GridSearchCV
param_name = "columntransformer__kmeans__n_clusters"
param_grid = {param_name: [10, 30, 100, 300, 1_000, 3_000]}
grid_search = GridSearchCV(
model_cluster_geo, param_grid=param_grid, scoring="neg_mean_absolute_error"
)
grid_search.fit(data_train, target_train)
results_columns = [
"mean_test_score",
"std_test_score",
"mean_fit_time",
"std_fit_time",
"mean_score_time",
"std_score_time",
"param_" + param_name,
]
grid_search_results = pd.DataFrame(grid_search.cv_results_)[results_columns]
grid_search_results["mean_test_error"] = -grid_search_results[
"mean_test_score"
]
grid_search_results = (
grid_search_results.drop(columns=["mean_test_score"])
.rename(columns={"param_" + param_name: "n_clusters"})
.round(3)
)
grid_search_results.sort_values("mean_test_error", ascending=False)
Larger number of clusters increases the predictive performance at the cost of larger fitting and prediction times.
labels = {
"mean_fit_time": "CV fit time (s)",
"mean_test_error": "CV score (MAE)",
}
fig = px.scatter(
grid_search_results,
x="mean_fit_time",
y="mean_test_error",
error_x="std_fit_time",
error_y="std_test_score",
hover_data=grid_search_results.columns,
labels=labels,
)
fig.update_layout(
title={
"text": "Trade-off between fit time and mean test score",
"y": 0.95,
"x": 0.5,
"xanchor": "center",
"yanchor": "top",
}
)
fig.show(renderer="notebook")
We can finally evaluate the best model found by our analysis and see how well it can generalize.
best_n_clusters = grid_search.best_params_[param_name]
print(
f"The test MAE with {best_n_clusters} clusters is: "
f"{-grid_search.score(data_test, target_test):.3f} k$"
)
This notebook demonstrates one way to leverage clustering for non-linear feature engineering, but there are many ways to compose unsupervised models in a supervised-learning pipeline.
We can finally observe that even if K-means was not the best clustering algorithm from a qualitatively point of view (as presented in the previous notebook), it is still helpful at crafting predictive features.