K-means

Sat, 10 Oct 2026 00:00:00 +0000

K-means is an unsupervised learning algorithm that assigns labels to the data points (clustering) with an iterative optimization that repeats until convergence.

Optimization
#

initialize clusters centers (also called centroids) randomly
update cluster labels by assigning points to the nearest cluster center
update cluster centers by setting the center to the mean of the corresponding cluster

Loss function
#

Euclidean distance is the metric used to find the nearest centroids.

K-means from scratch

Sat, 10 Oct 2026 00:00:00 +0000

Algorithm and implementation
#

K-means algorithm assigns labels to data points by iteratively updating centroids and clusters’ labels and using euclidean distance to calculate their proximity until convergence (or until a maximum number of iterations is reached).

Class structure
#

main attributes:
- the number of clusters K
- n_iterations
- clusters, which is the list of sample indices for each cluster initialized in the __init__
- centroid which is the center (or mean vector) for each cluster
predict method It is worth to note that there’s no fit method and we can do the predict straight away.

import numpy as np

class KMeans:
	def __init__(self, K=5, n_iterations=100):
		self.K = K
		self.n_iterations = n_iterations
		
		#list of sample indices for each cluster
		self.clusters = [[] for _ in range(self.K)]
		
		#mean vector for each cluster
		self.centroids = []
		
	def predict(self, X):
		pass

Inference
#

The predict method consists of the following steps:

Clustering on davethehuman.com

K-means

Optimization #

Loss function #

K-means from scratch

Algorithm and implementation #

Class structure #

Inference #

Optimization
#

Loss function
#

Algorithm and implementation
#

Class structure
#

Inference
#