SEO Has a Younger Sibling: It’s On-Site Search, and It Deserves Attention

January 9, 2017

On-site search, also known as internal search, is a critical yet undervalued and underrepresented tactic in the search industry. In 2014,eConsultancy released a report that showed that 15% of companies dedicate resources to optimizing the on-site search experience, 42% fold on-site search into other online measurement responsibilities, and 42% of companies ignore on-site search. This report was released more than two years ago and after some extensive Googling, some Duck Duck Go-ing, and some slight Bing-ing, I couldn’t find any reputable site releasing similar, more recent, reports.

Wait a minute… Nearly 84% of companies don’t actively optimize or measure their on-site search?

Consider the following benefits of active on-site search optimization:

  • On average, 30% of visitors will perform an on-site search
  • When comparing revenue gained from people who performed an on-site search vs. people who did not perform an on-site search, the people who performed an on-site search generated more revenue than those who did not.
  • Performing an on-site search is a strong behavioral predictor of intent to convert:
    • People who perform a site-search are twice as likely to convert.
    • People who perform an on-site search are more likely to return to the site with an intent to purchase.
    • In the research, companies had an average overall ecommerce conversion rate of 2.77%. However, the conversion rate nearly doubled to 4.63% from people who used on-site search and found what they were looking for.

Data sources: eConsultancy, ConversionXL

It’s honestly surprising that, in an industry that focuses so intently on data-driven marketing and conversion optimization, we don’t put more effort into on-site search, harnessing users who are engaged and interested in our content and serves. My point is that for a tactic that can become an incredible force multiplier, there’s so little information on it.

Okay, okay, you’re kinda starting to rant. What are my first steps to understanding on-site search?

The first thing you need to understand before starting to optimize your on-site search is what data is being collected by analytics from your on-site search platform. For this exercise, it’s irrelevant whether you’re using Solr, the Google Search Appliance, or a baked-in search platform that came with your content management system.

Ultimately, the goal is to understand “Search Quality” by quantifying it:

When discussing quantifying on-site search, I tend to use the metaphor of resolution. I started playing video games as a kid and that’s continued to today. So, seeing little blocky characters evolve into something incredibly complex is a part of my experience. Clever designers were able to create experiences using only 8-bit resolution. As hardware grew more capable, 16-, 32-, and 64-bit consoles became more mainstream. Each generation of hardware supported greater resolution and the amount of visual information become more complex, more immersive, and more descriptive.

This is how I view measuring and quantifying on-site search. Start by understanding the basic metrics. Start with 8-bit resolution. This is where you’ll start seeing patterns emerge and behaviors evolve. As your experience grows, add more complexity to your metrics, look at the interplays between different types of content, look at how your on-site search results change as your content changes, look at the time it takes for people to find what they’re looking for. It won’t take too long to where you’re looking at your on-site search data at 64-bit resolution.


Image source

As your measurement resolution improves, you can understand the story deeper and with more clarity, and can use that understanding to impact greater change.

Search behavior

Search behavior is the measure of the quantitative actions a user takes on your website. This includes keywords, clicks, and calculated measurements like CTR. Measuring search behavior allows the search professional to see if any changes to the way on-site search works results in a net benefit to on-site search.

How to quanitify search behavior

The simplest metric to use to examine user behavior is CTR.

First of all, there’s an implicit assumption that clicks signal that the user has found something useful and the click is a signal of that found value. Not all clicks are indicators of search success, and there are users that click multiple links on a SERP as a way to refine their search. For now, in our 8-bit world, we need to understand that assumption, yet it’s important to measure overall CTR as a signal of search quality.

When you first start looking at SERP CTR, it’s likely yours will be low. When I first started measuring on-site search, we used the Google Search Appliance and never surpassed a 24% SERP CTR. The first question I was asked was “What’s a good on-site search CTR?” I was able to answer this by examining our different content types (blogs, videos, articles, etc). What I found through customer surveys was that a higher CTR generally correlated with a higher satisfaction with the search experience. The surprising part? Fewer people than expected had a “neutral” experience. This showed me that when it comes to on-site search, people are happy (and remember) a good search experience and people are frustrated (and remember) a poor search experience. The room for a neutral feeling has a very small window. If we had a greater than 65% CTR on our search results, we would be able to show a positive search experience and a greater chance for conversion, assuming that a good search experience meant that the user was able to find what they were looking for.

Screenshot from 2016-11-15 12:07:02.png

Higher-resolution metrics on CTR can include different elements of the SERP:

  • Filter Section CTR
  • Promoted Section CTR
  • Category Result CTR
  • Sponsored Result CTR
  • Natural Search CTR

Screenshot from 2016-11-15 12:11:07.png

SERP Source

Understanding how all of these SERP elements influence clicks can identify opportunities for CTR optimization with either better results or a better UI. This is probably the most superficial metric to measure behavior — I haven’t even mentioned keyword refinements, revenue per search, revenue per keyword, or conversions gained through search. All are very important to measure the overall success of on-site search, but probably deserve their own article.

Search findability

Findability is the measurement of how a user finds the content they’re looking for. Findability is directly tied to the content’s rank for particular keywords. When the on-site SERP shows the results, knowing which piece of content the user clicks on, combined with the average rank of that content, can reveal if good content is being clicked, thus making it “findable.”

How to quantify findability

I take a fairly simple approach to measuring findability. I take a look at it from a content-level perspective. For each piece of content, I measure the SERP impressions, the amount of times a piece of content shows up in the top ten results. I look at the average rank, then measure the CTR of the piece of content. Using this information I can identify pieces of content that show up a lot in the SERP and have a low CTR. Looking at the average rank tells me if, on average, the content is seen. If it has an average rank of less than 10 (such as “content 2” in the table below) there’s a reason why it shows up frequently but rarely gets clicked on, despite being in a good ranking position. Conversely, when I look at “content 5,” I see that it has high impressions, low clicks, high rank, and a low CTR. I then take a look at this piece of content to see if I can make it rank a bit higher in our on-site search. If I see the CTR improve, then I’ve increased its findability as well as its value.

Screenshot from 2016-11-15 13:51:52.png

A higher-resolution version of findability would include the number of keywords that trigger an impression and a visibility scoring model that would describe how “visible” a piece of content is. Factoring in clicks and CTR into the visibility model would give you an overall findability score that could quickly identify valuable content that is not found and non-valuable content that is frequently found.

Screenshot from 2016-11-15 14:00:09.png

Result set quality

Determining result set quality is answering the question, “How good are the results for given keywords?” For example, if a common search term on your site is “shoes” and your top result is the page where the user can filter down to the right kind of shoe, and that page is in the pathway to a good conversion rate, then that’s a good result for the keyword. However, if the results for “shoes” are more general — “athletic shoes,” “red shoes,” “ballet shoes” — and the results for this keyword have a poor conversion rate, then the result set quality can be improved.

I use a fairly simple method to make sure that my results are performing according to the best interests of the company. I look at the number of times a SERP appeared and then measure the number of times that SERP resulted in a conversion or a consumption (a user reading a page that doesn’t have a call-to-action on it). Then, looking at the successful events that occur after a SERP occurs, I have a metric that I can track, trend, and show the value of on-site search.

Screenshot from 2016-11-15 14:22:53.png

Which metrics are core to measuring on-site search?

At the most basic level, these are the metrics we’ll need to start measuring on-site search. Many of the nuances we’re measuring in SEO applies to onsite-search as well. While most of these metrics we likely already know, I want to describe them in the context of on-site search.

  • Query

    • A query is a package of information that is sent to the search platform. While “query” and “keyword” can generally be interchanged, keywords and keyword phrases have meaning because they’re a part of language. A query is a packet of information because it is the “thing” that is sent to the search engine to be parsed, recognized, and compared against a…

Get more information about this news here: Read More