treeple.UnsupervisedRandomForest

class treeple.UnsupervisedRandomForest(n_estimators=100, *, criterion='twomeans', max_depth=None, min_samples_split='sqrt', min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features='sqrt', max_leaf_nodes=None, min_impurity_decrease=0.0, bootstrap=False, oob_score=False, n_jobs=None, random_state=None, verbose=0, warm_start=False, max_samples=None, clustering_func=None, clustering_func_args=None)

Unsupervised random forest.

An unsupervised random forest is inherently a clustering algorithm that also simultaneously computes an adaptive affinity matrix that is based on the 0-1 tree distance (i.e. do samples fall within the same leaf).

Parameters:

n_estimatorsint, optional

Number of trees to fit, by default 100.

criterion{“twomeans”, “fastbic”}, default=”twomeans”

The function to measure the quality of a split. Supported criteria are “twomeans” for maximizing the variance and “fastbic” for the maximizing the Bayesian Information Criterion (BIC), see Mathematical formulation.

max_depthint, default=None

The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples.

min_samples_splitint or float, default=2

The minimum number of samples required to split an internal node:

• If int, then consider min_samples_split as the minimum number.

• If float, then min_samples_split is a fraction and ceil(min_samples_split * n_samples) are the minimum number of samples for each split.

In unsupervised trees, it is recommended by [1] to use the sqrt of two times the number of samples in the dataset.

min_samples_leafint or float, default=1

The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least min_samples_leaf training samples in each of the left and right branches. This may have the effect of smoothing the model, especially in regression.

• If int, then consider min_samples_leaf as the minimum number.

• If float, then min_samples_leaf is a fraction and ceil(min_samples_leaf * n_samples) are the minimum number of samples for each node.

min_weight_fraction_leaffloat, default=0.0

The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided.

max_features{“sqrt”, “log2”, None}, int or float, default=”sqrt”

The number of features to consider when looking for the best split:

• If int, then consider max_features features at each split.

• If float, then max_features is a fraction and max(1, int(max_features * n_features_in_)) features are considered at each split.

• If “auto”, then max_features=sqrt(n_features).

• If “sqrt”, then max_features=sqrt(n_features).

• If “log2”, then max_features=log2(n_features).

• If None, then max_features=n_features.

Note: the search for a split does not stop until at least one valid partition of the node samples is found, even if it requires to effectively inspect more than max_features features.

max_leaf_nodesint, default=None

Grow trees with max_leaf_nodes in best-first fashion. Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes.

min_impurity_decreasefloat, default=0.0

A node will be split if this split induces a decrease of the impurity greater than or equal to this value.

The weighted impurity decrease equation is the following:

N_t / N * (impurity - N_t_R / N_t * right_impurity
                    - N_t_L / N_t * left_impurity)

where N is the total number of samples, N_t is the number of samples at the current node, N_t_L is the number of samples in the left child, and N_t_R is the number of samples in the right child.

N, N_t, N_t_R and N_t_L all refer to the weighted sum, if sample_weight is passed.

bootstrapbool, optional

Whether to bootstrap, by default False.

oob_scorebool or callable(), default=False

Whether to use out-of-bag samples to estimate the generalization score. By default, calinski_harabasz_score() is used. Provide a callable with signature metric(X, predicted_labels) to use a custom metric. Only available if bootstrap=True. Other supported functions from scikit-learn are sklearn.metrics.silhouette_score(), sklearn.metrics.calinski_harabasz_score(), and sklearn.metrics.davies_bouldin_score().

n_jobsint, optional

Number of CPUs to use in joblib parallelization for constructing trees, by default None.

random_stateint, optional

Random seed, by default None.

verboseint, optional

Verbosity, by default 0.

warm_startbool, optional

Whether to continue constructing trees from previous instant, by default False.

max_samplesint or float, default=None

If bootstrap is True, the number of samples to draw from X to train each base estimator.

• If None (default), then draw X.shape[0] samples.

• If int, then draw max_samples samples.

• If float, then draw max_samples * X.shape[0] samples. Thus, max_samples should be in the interval (0.0, 1.0].

clustering_funccallable()

Scikit-learn compatible clustering function to take the affinity matrix and return cluster labels. By default, sklearn.cluster.AgglomerativeClustering.

clustering_func_argsdict

Clustering function class keyword arguments. Passed to clustering_func.

Attributes:

estimator_UnsupervisedDecisionTree

The child estimator template used to create the collection of fitted sub-estimators.

estimators_list of UnsupervisedDecisionTree

The collection of fitted sub-estimators.

n_features_in_int

Number of features seen during fit.

feature_names_in_ndarray of shape (n_features_in_,)

Names of features seen during fit. Defined only when X has feature names that are all strings.

feature_importances_ndarray of shape (n_features,)

The impurity-based feature importances.

labels_ndarray of shape (n_samples,)

Labels of each point.

oob_score_float

Score of the training dataset obtained using an out-of-bag estimate. This attribute exists only when oob_score is True.

oob_decision_function_ndarray of shape (n_samples, n_samples)

Affinity matrix computed with only out-of-bag estimate on the training set. If n_estimators is small it might be possible that a data point was never left out during the bootstrap. In this case, oob_decision_function_ might contain NaN. This attribute exists only when oob_score is True.

Methods

apply(X)	Apply trees in the forest to X, return leaf indices.
compute_similarity_matrix(X)	Compute the similarity matrix of samples in X.
decision_path(X)	Return the decision path in the forest.
fit(X[, y, sample_weight])	Fit estimator.
fit_predict(X[, y])	Perform clustering on X and returns cluster labels.
fit_transform(X[, y])	Fit to data, then transform it.
get_leaf_node_samples(X)	For each datapoint x in X, get the training samples in the leaf node.
get_metadata_routing()	Get metadata routing of this object.
get_params([deep])	Get parameters for this estimator.
predict(X)	Predict clusters for X.
predict_quantiles(X[, quantiles, method])	Predict class or regression value for X at given quantiles.
set_fit_request(*[, sample_weight])	Request metadata passed to the fit method.
set_output(*[, transform])	Set output container.
set_params(**params)	Set the parameters of this estimator.
transform(X)	Transform X to a cluster-distance space.

References

[1]

Meghana Madhyastha, Percy Li, James Browne, Veronika Strnadova-Neeley, Carey E. Priebe, Randal Burns, and Joshua T. Vogelstein. Geodesic forests. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ‘20, 513––523. New York, NY, USA, 2020. Association for Computing Machinery. URL: https://doi.org/10.1145/3394486.3403094, doi:10.1145/3394486.3403094.

apply(X)

Apply trees in the forest to X, return leaf indices.

Parameters:

X{array_like, sparse matrix} of shape (n_samples, n_features)

The input samples. Internally, its dtype will be converted to dtype=np.float32. If a sparse matrix is provided, it will be converted into a sparse csr_matrix.

Returns:

X_leavesndarray of shape (n_samples, n_estimators)

For each datapoint x in X and for each tree in the forest, return the index of the leaf x ends up in.

compute_similarity_matrix(X)

Compute the similarity matrix of samples in X.

Parameters:

Xarray_like of shape (n_samples, n_features)

The input data.

Returns:

sim_matrixarray_like of shape (n_samples, n_samples)

The similarity matrix among the samples.

decision_path(X)

Return the decision path in the forest.

New in version 0.18.

Parameters:

X{array_like, sparse matrix} of shape (n_samples, n_features)

The input samples. Internally, its dtype will be converted to dtype=np.float32. If a sparse matrix is provided, it will be converted into a sparse csr_matrix.

Returns:

indicatorsparse matrix of shape (n_samples, n_nodes)

Return a node indicator matrix where non zero elements indicates that the samples goes through the nodes. The matrix is of CSR format.

n_nodes_ptrndarray of shape (n_estimators + 1,)

The columns from indicator[n_nodes_ptr[i]:n_nodes_ptr[i+1]] gives the indicator value for the i-th estimator.

fit(X, y=None, sample_weight=None)

Fit estimator.

Parameters:

X{array_like, sparse matrix} of shape (n_samples, n_features)

The input samples. Use dtype=np.float32 for maximum efficiency. Sparse matrices are also supported, use sparse csc_matrix for maximum efficiency.

yIgnored

Not used, present for API consistency by convention.

sample_weightarray_like of shape (n_samples,), default=None

Sample weights. If None, then samples are equally weighted. Splits that would create child nodes with net zero or negative weight are ignored while searching for a split in each node. In the case of classification, splits are also ignored if they would result in any single class carrying a negative weight in either child node.

Returns:

selfobject

Returns the instance itself.

fit_predict(X, y=None, **kwargs)

Perform clustering on X and returns cluster labels.

Parameters:

Xarray_like of shape (n_samples, n_features)

Input data.

yIgnored

Not used, present for API consistency by convention.

**kwargsdict

Arguments to be passed to fit.

New in version 1.4.

Returns:

labelsndarray of shape (n_samples,), dtype=np.int64

Cluster labels.

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

Xarray_like of shape (n_samples, n_features)

Input samples.

yarray_like of shape (n_samples,) or (n_samples, n_outputs), default=None

Target values (None for unsupervised transformations).

**fit_paramsdict

Additional fit parameters.

Returns:

X_newndarray array of shape (n_samples, n_features_new)

Transformed array.

get_leaf_node_samples(X)

For each datapoint x in X, get the training samples in the leaf node.

Parameters:

Xarray_like of shape (n_samples, n_features)

Dataset to apply the forest to.

Returns:

leaf_node_samplesa list of array_like

Each sample is represented by the indices of the training samples that reached the leaf node. The n_leaf_node_samples may vary between samples, since the number of samples that fall in a leaf node is variable. Each array-like has shape (n_leaf_node_samples, n_outputs).

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routingMetadataRequest

A MetadataRequest encapsulating routing information.

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

paramsdict

Parameter names mapped to their values.

predict(X)

Predict clusters for X.

The predicted class of an input sample is a vote by the trees in the forest, weighted by their probability estimates. That is, the predicted class is the one with highest mean probability estimate across the trees.

Parameters:

X{array_like, sparse matrix} of shape (n_samples, n_features)

The input samples. Internally, its dtype will be converted to dtype=np.float32. If a sparse matrix is provided, it will be converted into a sparse csr_matrix.

Returns:

yndarray of shape (n_samples,) or (n_samples, n_outputs)

The predicted classes.

predict_quantiles(X, quantiles=0.5, method='nearest')

Predict class or regression value for X at given quantiles.

Parameters:

X{array_like, sparse matrix} of shape (n_samples, n_features)

Input data.

quantilesfloat, optional

The quantiles at which to evaluate, by default 0.5 (median).

methodstr, optional

The method to interpolate, by default ‘linear’. Can be any keyword argument accepted by numpy.quantile().

Returns:

yndarray of shape (n_samples, n_quantiles, [n_outputs])

The predicted values. The n_outputs dimension is present only for multi-output regressors.

set_fit_request(*, sample_weight='$UNCHANGED$')

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in fit.

Returns:

selfobject

The updated object.

set_output(*, transform=None)

Set output container.

See Introducing the set_output API for an example on how to use the API.

Parameters:

transform{“default”, “pandas”, “polars”}, default=None

Configure output of transform and fit_transform.

• "default": Default output format of a transformer

• "pandas": DataFrame output

• "polars": Polars output

• None: Transform configuration is unchanged

New in version 1.4: "polars" option was added.

Returns:

selfestimator instance

Estimator instance.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**paramsdict

Estimator parameters.

Returns:

selfestimator instance

Estimator instance.

transform(X)

Transform X to a cluster-distance space.

In the new space, each dimension is the distance to the cluster centers. Note that even if X is sparse, the array returned by transform will typically be dense.

Parameters:

X{array_like, sparse matrix} of shape (n_samples, n_features)

New data to transform.

Returns:

X_newndarray of shape (n_samples, n_samples)

X transformed in the new space.

property estimators_samples_

The subset of drawn samples for each base estimator.

Returns a dynamically generated list of indices identifying the samples used for fitting each member of the ensemble, i.e., the in-bag samples.

Note: the list is re-created at each call to the property in order to reduce the object memory footprint by not storing the sampling data. Thus fetching the property may be slower than expected.

property feature_importances_

The impurity-based feature importances.

The higher, the more important the feature. The importance of a feature is computed as the (normalized) total reduction of the criterion brought by that feature. It is also known as the Gini importance.

Warning: impurity-based feature importances can be misleading for high cardinality features (many unique values). See sklearn.inspection.permutation_importance() as an alternative.

Returns:

feature_importances_ndarray of shape (n_features,)

The values of this array sum to 1, unless all trees are single node trees consisting of only the root node, in which case it will be an array of zeros.