Build a Keyword Clustering Tool with Python

Grouping keywords by topic sounds like a straightforward SEO task, but anyone who has stared down a spreadsheet with 12,000 rows and a blank “group by intent” column knows the truth. Manual clustering simply does not scale. Rule-based grouping, meanwhile, fails to capture the semantic overlap between phrases that share no common words yet clearly belong together.

I recently refactored a keyword clustering script I originally built several years ago. This framework relies on TF-IDF vectorization to generate feature vectors, which are then grouped using HDBSCAN, a density-based clustering algorithm. You can access the script here.

The core challenge with keyword clustering

Keyword clustering is the engine behind effective topic generation. Instead of briefing content writers with hundreds of disjointed keywords, clustering bundles semantically related queries into coherent topics. This makes it far easier to produce content that covers a broader range of search intent.

The payoff is a set of topics that can strengthen semantic relationships, topical authority, internal linking, and visibility across related queries. But two major hurdles stand in the way: preprocessing the data and clustering the topics themselves.

Preprocessing the data

SEO keyword exports are notoriously noisy, especially when pulled from internal databases. Removing stopwords and non-ASCII characters manually is impossible at scale, which is why automating data cleaning is essential. Python offers a straightforward path, giving you the flexibility to build a pipeline that cleans, normalizes, and transforms large keyword datasets with minimal manual effort.

Clustering the topics

When you start analyzing keyword data, you rarely know how many topical groups exist until you explore the dataset. This makes algorithms like k-means a poor choice, since they require you to specify the number of clusters upfront.

A combination of TF-IDF and HDBSCAN proves far more effective. TF-IDF converts each keyword into a numerical vector by assigning heavier weight to distinctive terms within the dataset, while down-weighting terms that appear frequently across many keywords. These vectors then feed into HDBSCAN, a density-based algorithm that identifies natural groupings without needing a predefined cluster count.

One of HDBSCAN’s standout features is its ability to identify noise. Instead of forcing every keyword into a cluster, it assigns outliers a -1 label, effectively excluding queries that don’t belong to any topic. This is especially valuable for SEO datasets, where exports often contain highly specific long-tail queries that don’t naturally fit into broader thematic groups. Rather than degrading cluster quality by forcing these terms into arbitrary groups, HDBSCAN isolates them, producing cleaner and more coherent topical clusters.

Sourcing the keyword list from BigQuery

Before any clustering happens, you need a keyword list. If your Search Console property is already exporting to BigQuery, that’s a far better source than the UI export, since it isn’t capped at 1,000 rows and isn’t sampled. A simple pull against the standard GSC BigQuery export schema looks like this:

Export the result as a CSV, strip it down to a single query column, save it as a .txt file with one keyword per line, and you have your clustering input. If you don’t have BigQuery export set up, you can still work with Search Console data, but you’ll have a more limited dataset. The notebook only cares that you feed it a list of keywords.

Prompting the AI

A few years ago, I relied on a less polished version of this script. AI made several things significantly easier when rebooting and fine-tuning the script for enhanced output.

First, I asked for tunable parameters instead of hardcoded ones. Cluster sensitivity and minimum cluster size behave very differently depending on whether you’re clustering 50 keywords or 50,000. Making these adjustable variables at the top of the script meant I could retune without touching the logic.

Second, I specified the environment. The original version was a plain Python script meant to run from a terminal. Since the actual workflow is “pull keywords, run notebook, hand a client a file,” I requested a Google Colab-specific version. This changed the code structure significantly, adding try/except blocks around Colab-only imports and removing argparse. It also enabled better visualization by leveraging Plotly to its fullest.

Running the code

The core of the script remains simple. It takes a flat list of keywords and automatically groups them into topic clusters. All you have to do is upload a .txt file with one keyword per line, and it cleans the text by stripping special characters, stopwords, and non-English entries.

The key to the entire framework is using tunable parameters. Play around with the sensitivity and minclustersize settings, adjusting them based on the size of your keyword list. Once you’re satisfied with the number of clusters, TF-IDF scores how important each word is within the set, and HDBSCAN finishes the job.

Each cluster receives an auto-generated label based on its most distinctive terms. The results are exported to an Excel file with both a grouped cluster view and a full keyword-by-keyword breakdown.

Where AI adds value

The decisions that truly matter, from choosing TF-IDF over word embeddings to selecting HDBSCAN because the number of topics is unknown, still require an understanding of how keyword clustering works. What AI removed was the repetitive work of rebuilding boilerplate code, wiring libraries together, and refining the notebook into something reusable.

The final result is a lightweight clustering tool that processes thousands of keywords in minutes, producing a sensible first draft of your topical taxonomy. It won’t replace editorial judgment, but it will eliminate hours of manual grouping and give your content team a far more structured starting point. If you already export Search Console data into BigQuery, this becomes a natural step in your workflow: extract your queries, run the notebook, review the clusters, and start planning content based on topics rather than isolated keywords.