6 vital lenses for effective keyword research

More keyword data isn’t your problem. 

No, what’s holding you back has nothing to do with your advanced ability to slice and dice keyword data dumps. 

Data dumps on their own won’t help you; they’ll probably only hurt you.

Here’s why, how this mindset routinely sabotages keyword research and how to fix it.

Why keyword research is hard (and misunderstood)

Anyone anywhere can fire up a keyword research tool and export a giant list of “relevant” keywords. That’s not the problem. 

The problem is that this haphazard, spray-and-pray approach doesn’t work. Not when your competitors (both direct but more importantly the indirect ones) actually know what they’re doing in competitive spaces.

In other words, your problem isn’t sifting through a ton of random keyword ideas, using arbitrary (and misleading) filters like keyword difficulty (KD) to sort which ones are a “good” target vs. a “bad” one.

My friend Ben Goodey helped illustrate the problem with keyword research in a recent LinkedIn post:

“Most keyword research “spreadsheet dumps” people make are NOT useful.

Easy keyword research:

– Full dump of entire set of keywords in “your industry”.
– Difficulty and volume appended.
– Clustered into related topics.

Useful keyword research:

– Keywords you can *actually* win based on your current authority
– Keywords that show YOUR buyer is looking for YOUR product
– Keywords where competing content is weak

Takes time and experience. It’s slow.

But it’s where the magic *actually* happens.”

See, SEO is part art, part science.

Yes, it can be technically complex at times. But for the most part, it’s more similar to weight loss:

What’s hard is the consistent execution, and not the actual theory or strategy (eat less; workout more).

So yes, pivot tables can be helpful, especially when you’re working with tons of data points.

But they’re only helpful to a point.

Because the best keyword research strategies actually blend multiple different types of subjective analysis at the same time, including:

• The likelihood or ability to actually rank, for… 
• Business-generating topics, within a… 
• Realistic time frame given your goals/objectives, considering…
• The competitive landscape you’re up against versus… 
• Your site’s current inherent strengths and… 
• The resources you’re able to allocate during these pre-defined time periods.

In a world of sophisticated SEO, over-engineered pivot tables, fancy formulas and similar basically become more fake busy work.

Let’s step through these six views or “lenses” so you can properly analyze keywords from different angles that are a better predictor for success.

1. The likelihood of actually ranking

This first one sounds trite but often isn’t in practice.

Yes, you should be targeting the biggest and best commercial terms in your space. But that brings us back to the age-old SEO Catch-22:

• The “best” keywords are often the most competitive.
• This means they’re also going to take the longest to rank for (so, you won’t see any results anytime soon).

If you’re ready and willing and able to stomach years of no results, then go for it!

Otherwise, if bosses or peers or clients are going to be breathing down your neck six months from now (read: most, if not all, of them), you’re gonna need to re-adjust your sights.

For instance, feast your eyes on this awesome “live streaming platform” keyword:

Yikes. “Super hard” keyword difficulty (KD) isn’t great. 

(It’s not all bad per se; keep reading below to see why KD often lies).

However, unless your domain rating already reads ~80-90+, you’re looking at 12+ months at the earliest for any top page (let alone top five) rankings.

So. You adjust your sights – out and down!

What’s a similarly relevant keyword that you can rank for in under 12 months to help “balance the books” while you’re waiting to rank for this more competitive target?

You look for similar underlying intent and relevance but one that’s much more of a short-to-medium-term goal than a long-term one.

Like this: 

No, you probably still won’t rank in the top 5 for this out of the gate. But the related long-tail variations are even more promising.

So, with the right approach, you’re able to:

• Pick up the long-tail variations in the short term (0-6 months).
• Get this “VOD streaming” pillar within ~6-12 months.
• Leverage this dominance to go back for “live streaming platform” 12+ months from now.

In other words, your keyword research + topic selection + content strategy should be more holistic. 

One cluster sets up another, which sets up another, which sets up another.

No pivot tables required – only long-term thinking and strategizing.

Dig deeper: The trifecta of keyword research strategy:  Volume, difficulty, intent

2. Business-generating topics

Ranking for all the top-of-the-funnel keywords in your space is a great start. 

It might be enough for B2C companies, depending on the rest of their customer acquisition channels and options.

But, those terms alone often don’t move the needle for long, drawn-out, complex sales cycles.

Translation: Using AI to pump out all the crappy content in the world isn’t actually going to drive you new business. 

If anything, the substandard quality will likely erode the trust and credibility you’ll need to drive five-plus-figure ARPUs.

That’s why B2B brands especially need to balance both ends of the buyer’s journey:

• You need enough bottom-of-the-funnel content to convert attention into interest (where demos, paid trials or sales enablement are more of the norm).
• You must not neglect the fact that the vast majority of your ICPs don’t know who you are – so you’ll need to address pain points and education.

You should also realize that sophisticated buyers are already comparing you to alternatives before ever reaching out in the first place. 

So piggybacking on competitors and alternatives of your product – even if they’re not direct competitors but simply better known (or funded) – can help you thread the needle.

This way, you can double down on expanding the “vs” or “alternatives”-queries you’re targeting and actually pushing into the top five. Like so:

If you’re just doing keyword research with a giant data dump, chances are you’re missing these critical “bridge”-type keywords to help transform searchers into leads. 

‘Cause you’re setting artificial minimums around things like “volume,” even though “volume” across most keyword research tools are:

• Completely inaccurate.
• Wildly different from tool to tool.

So keep in mind that keyword research data is all relative. Not absolute.

You don’t need to care about “how much volume” there is for a “vs” or “alternative” keyword. ‘Cause in real life, you already know you’re being compared against alternatives. 

Learn to trust your gut, intuition and customer feedback more than any other artificial number in a spreadsheet.

Dig deeper: B2B keyword research: A comprehensive guide

3. Ranking in a realistic time frame (given your goals and objectives)

Here’s the thing:

Everyone wants to rank for the “best” keywords, except not everyone deserves to rank for the best keywords.

Check out this example below and look at the brand names dominating the top 10 for “what is CRM”:

Now, ask yourself:

If you’re not already a household name in the CRM space, what is the likelihood of you unseating one of these competitors?

Seriously, what’s the likelihood that you’ll rank in the top 10 for this keyword in one year? 

And then, more importantly, what is it actually gonna take you – today – to get there tomorrow?

No. You probably don’t deserve to rank in the top 10 for this query. 

Not today, and probably not over the next 12 months. 

But two or three or five years from now? Sure! Why not?!

The Beachhead Principle can help, but only if you put in the hard work over the next few years to achieve it.

---

4. The competitive landscape’s relative strength

Keyword difficulty lies.

Not some of the time, but all of the time, across pretty much every single keyword research tool.

This means simply adding a filter for KD when doing keyword research isn’t good enough.

At best, it sets you up for bad expectations. At worst, it manifests in “pretty good” future rankings that never will actually break the bank (in a good way).

See, “keyword difficulty” usually relates specifically to the number (not even quality!) of backlinks to each individual piece of content currently ranking in the top ten or so. 

However, it does not only ignore the quality of said backlinks but also the overall domain strength of the site. 

Check out this “localization issues” example below to see what I mean:

Nice! We SEOs like “Easy,” right?

Except, this:

Sure, the two green boxes (page-level strength) are kinda/sorta easy.

However, the red box? Domain strength overall? With one exception, the rest are sitting at 70+ domain ratings. 

So no. This keyword isn’t “easy.”

This is why filtering a spreadsheet for “low KD” is misleading and inaccurate.

5. Your site’s current strengths

Topical authority can work practically in two ways:

• It can be a rising tide that lifts all boats (err, other similar keywords in the same space you’re deemed an expert), or…
• It can be a glass ceiling, giving you the dreaded “pretty good” rankings that never actually come to fruition (without a ton of extra work in other areas.

The simplest way to assess topical authority (or lack thereof) is to see if you have good rankings (top five or 10, at least) for lots of related content around these same queries.

Wanna see what that looks like? Kinda like this:

Tons of top five rankings for related keywords around “live streaming.” Great!

But the inverse is true, too.

No strong rankings for other related topics or concepts? This means you have a lot more work to do!

Take a look at this pillar page below and specifically the sub-sections being called out:

Now, let’s assess each area to see if this site already has good rankings for those sections.

Example 1: A single 12th position, so not really.

Example 2: Two 60+ positions – not at all!

Example 3: Lots of 40-80+ positions. So clearly, no.

The point?

That first pillar page will always struggle to rank until you can also get these other supporting pages to help, you know, actually support it!

In other words, there is no existing topical authority in this space. Creating and reinforcing this should be priority number one.

And yet? No Excel sheet or formula is going to tell you this. At least, not without analyzing the problem of not ranking from different perspectives or angles like this.

6. Resources you’re able to allocate in a specific time frame

Let’s recap and combine the first five ideas. Then, add a sixth consideration: how many resources (i.e., people, time, money) you can realistically allocate to each keyword you might want to target over the next year. 

Once again, back to “live streaming platform.”

Take a peek at both the domain and page-level strengths below:

Now, an important caveat:

• You’ll notice the huge number of referring domains (in the right-hand orange box).
• That’s why this KD was listed as “Super Hard” earlier.
• But, those of you with eagle eyes will now see that the ones with thousands or tens of thousands of referring domains are actually homepages!
• This means that “Super Hard” ranking earlier is also overstated. 
• Ahrefs is simply averaging out these tens of thousands of domains across the other results, too.
• If search intent is truly a comparison post, it will mean that you don’t need nearly as many links to actually rank for this query.

Confused yet?

Yes, this keyword is still pretty difficult. Don’t get it twisted. 

But it’s not nearly as difficult or unattainable as that KD would have originally alluded to.

Once again, no Excel formulas would have helped you uncover this issue. Only your eyes, ears and experience can.

This is one of the many reasons why brands routinely fail at link building. When you see “1,000+” backlinks needed, you build tons and tons of cheap, low-quality links. 

Yet, you’d probably be better off with better content, topical authority and fewer (dozens to hundreds) of really high-quality, editorial-based links instead.

Yes, you still need relevance and scale either way. 

The type of campaigns and, therefore, the effective budget required for this one URL (compared to all the other URLs you might be juggling) should also be completely different.

A smarter, more intentional keyword research approach

Let’s face it:

SEOs are kinda geeky. We love data and want to live in a world of logic. 

We believe everything we do is quantifiable and data-driven. But don’t take this way too far.

You don’t need fancy formulas and pivot tables when doing keyword research. The more data sets you’re juggling often lead you astray.

SEO is more art than pure science in this regard.

What’s hard about SEO is reading the tea leaves and then consistently executing over the long haul (read: 24+ months, minimum).

None of these can be answered by ChatGPT, Ahrefs, etc., with a simple prompt or export button.

This is why people and companies fail at SEO. 

Not because “SEO is difficult/technical/doesn’t work in my space/enter other excuses here.”