N-grams for writers and SEOs: finding the phrases you repeat

Every writer has phrases they lean on without noticing. "At the end of the day." "It's important to note." "A wide range of." Read your own drafts closely and you'll find a handful of verbal tics repeating every few hundred words — invisible to you because they're your tics, obvious to every reader. The fastest way to catch them is not careful re-reading. It's counting. An n-gram analyzer counts every multi-word phrase in your text and ranks them by frequency, and the phrases at the top are usually a mix of things you meant to repeat and things you didn't.

What an n-gram is

An n-gram is a contiguous run of n words from a text. The n just sets the length:

• Bigrams are two-word phrases: "machine learning", "of the", "climate change".
• Trigrams are three words: "artificial neural network", "at the end".
• 4-grams and 5-grams capture longer set phrases: "at the end of the day".

The analyzer slides a window of length n across your text one word at a time, recording every phrase it sees. A 1,000-word article contains roughly 999 bigrams (most appearing once), and the interesting signal is in the few that appear many times. Those are your patterns — for better or worse.

How the analyzer counts and filters

Counting raw n-grams produces a lot of noise, so our n-gram analyzer applies a few filters that turn the raw list into something useful:

• Minimum frequency. You set a floor — say, "only show phrases appearing at least three times" — so the one-off phrases drop away and the genuine repeats surface.
• Stopword phrases. Phrases made entirely of function words ("of the", "in the", "and the") are statistically the most common n-grams in any English text and almost never what you care about. The analyzer can drop any phrase composed purely of stopwords, leaving the content-bearing ones.
• Lowercase normalization. By default "The Cat" and "the cat" are treated as the same phrase, so capitalisation at the start of a sentence doesn't split one phrase into two separate counts. That's almost always what you want for analysis.

The stopword filter deserves emphasis. Without it, the top of every n-gram list is the same boring scaffolding — "of the", "to the", "in a" — that tells you nothing about a specific text. Drop those and the list suddenly reflects your subject matter and your habits.

Why lowercasing is a real decision

Lowercasing is the right default, but it isn't free. Once you fold case, "Apple" the company and "apple" the fruit become the same token, and proper nouns merge with ordinary words. For most writing-quality work that's fine — you care about phrase repetition, not entity disambiguation. But if you're analysing a text where capitalisation carries meaning (brand names, code, titles), keep case sensitivity in mind when reading the results. The tool gives you the lever; the right setting depends on the question you're asking.

Three things n-grams are good for

1. Catching your own repetition

This is the self-editing use. Run a finished draft, set the minimum frequency to three or four, drop stopword phrases, and read the top of the list. Set phrases that appear five times in a 1,200-word post are the ones a reader will notice. Some are fine — your core topic should repeat. Others are crutches you can vary. The list makes the difference visible in a way re-reading rarely does.

2. Keyword and SEO research

N-grams are the backbone of keyword analysis. Running a competitor's page or a corpus of top-ranking articles through the analyzer surfaces the two- and three-word phrases that define the topic — the natural language people actually use, not just single keywords. Trigrams especially tend to map to real search queries ("best running shoes", "how to invest"). Pair the n-gram view with a keyword density analyzer for single-word frequency and you have both halves of the picture.

3. Style fingerprinting

Repeated phrasing is a signature. The same n-gram analysis that catches your tics can characterise a voice — an author's favourite constructions, a brand's recurring taglines, the boilerplate that creeps into corporate writing. Comparing the n-gram profiles of two texts is a quick, quantitative way to see how their phrasing differs.

Putting it to use

Start with trigrams — they're the sweet spot, long enough to be meaningful and short enough to recur. Paste your text into the n-gram analyzer, set a minimum frequency of two or three, switch on the drop-stopword-phrases option, and read down the ranked list. For a self-edit, every content phrase near the top is a candidate to vary; for research, every phrase is a clue about how people frame the topic. Then drop to bigrams for finer-grained patterns or climb to 4- and 5-grams to catch the long set phrases — "at the end of the day" only shows up once you look at five-word windows. The counting does the noticing your own eyes can't.