Providing the Not Provided
The Rise of “Not Provided” Keywords
The days of knowing which specific keywords brought traffic or lead to conversions from Google are virtually gone. Now the focus will be on overall traffic and conversions (as well as other aspects not addressed herein such as Social). Tracking the rankings of pages based on keywords combined with search volume data will allow you to gain insight into what keywords are bringing people to your site. When used in combination with other data you can further define successful keywords as well as keyword opportunities. These include internal site search data and keyword segmenting (or building out deeper pages for targeting important long-tail keywords).
The biggest loser in the new keyword data blackout which results in the virtual skull and crossbones named “Not Provided” is the long-tail keyword. Tracking “trophy” or general terms is easier but finding out what long-tail terms are bringing traffic and in what quantities will be far more difficult. This is why legacy analytics data and keyword segmenting can be very beneficial.
The following is a list of tips, tricks, software and information that can assist you in determining what keywords people used to find you, what keywords Google finds you relevant to, what pages those keywords are found on and keyword opportunities that you may not be currently taking advantage of.
The first thing to do is to arm yourself with the necessary tools. I recommend the following to start:
Google Adwords Keyword Planner
Google Analytics (or another analytics program)
Google Webmaster Tools
A powerful Rank Tracking Software
Moz Open Site Explorer
There are many other tools that can be used and I look forward to hearing which ones you’ve used, how you’ve done so and what the results are!
First Things First!
Master Keyword List
At this point it is important to build out your Master Keyword List (though you may have another name for it), it is the super-document which you use to manage your keywords. Segmenting it into groups, or “buckets” as Moz calls them, can allow you to more effectively manage them and use them eventually to determine which keywords can be bundled together to support each other on a page or which should have their own pages. It’s also a good idea to associate a page on your site to each of those keywords. When you build your list you’ll also want to include as many long-tails as possible, even slight variations of the same term but just ordered differently. The more the merrier.
Again, there are many tools you can use to build your list but some of the main ones are listed below.
Using Google for Keyword Research:
Adwords Keyword Planner: Ad Group Ideas
Using the Adwords Keyword Planner offers you a mix of their previous Contextual Targeting Tool and the original Keyword Tool.
Use the Keyword Planner Ad Group Ideas tool by placing in a keyword and pulling out the different keywords from each of the results which you feel are relevant to your site/page/target market, etc. The ones listed under “Ad group (by relevance)” are the terms that are considered highly relevant to the term you searched for.
The keywords in the next column “Keyword Ideas” are the ones most relevant to the corresponding term under “Ad Group Ideas”. You can then take one of those terms (from either column) and place it back into the search box to find more terms that are considered highly relevant. This process can go on and on as you build somewhat of a tree of relevant terms. Remember that these are based on the context of the term so the more context you can provide, the more targted the results.
You can also place your domain name or brand name into this tool to see what terms Google finds are most relevant to you. The same goes for your competitors’ domains and brand names.
Keyword Planner: Keyword Ideas
As with the Ad Group Ideas tool you can use the Keyword Ideas tool to refine and build out your list. Run the tool for your desired keyword and pull the results and then run the tool for each of the results, and so on.
Filtering out the good relevant terms from the irrelevant terms can take quite a bit of time as this can generate thousands of potential keywords.
Keyword Planner: Traffic Estimates
Once you have your list then you can run it through the Traffic Estimates tool to see which terms appear to be valid targets based on search quantities. Don’t necessarily throw out terms (especially long-tails) if they don’t show a measureable search volume. These are the long-tails that can add contextual relevance to your pages and generate an army of low-quantity long-tail traffic.
Using Google Search:
Google’s Search Engine is a great tool itself for building out your keyword list as there are many different functions available which allow you to dive further into keywords based on their actual relevance in Google’s eyes.
Google Search: Related Searches
At the bottom of SERPs Google offers suggested terms of related searches which provide an excellent source of keywords which are actively searched for on Google. As with the Keyword Planner you can build out your list by again searching for one of these recommended terms to find more related to it.
Google Search: Google Site Operator Search
When you want to find a keyword, term or phrase that exists on your site you can use Google’s Site Operator to find it. Take the term you want and put it into Google like this (site:mysite.com keyword). So if I’m on Danny Sullivan’s team and I want to optimize for “Recovering from Panda” then I may want to see what pages on my site are currently using that term so I search Google’s index of SearchEngineWatch.com for “recovering from panda”.
I find that there are 132 results which have my target term exactly as I want it since I included it in quotation marks. This will help with contextual targeting. Now perhaps I want to see just those pages which have my target term in the Title tag. I then modify my search to include other Google operators such as “intitle”. This produces surprising results:
None of the pages which include my target term in the content have that term in Title and only one out of those 132 pages has all of the words from my term included, though not in order. This can provide me with potential opportunities for optimizing my Titles for this term which should show a benefit to those seeking it, especially with a more contextually-targeted Google. SEW ranks at the top for this term so this is useful for when you’re not already ranking or want to target other pages on your site.
Google Insights: Google Trends
Using the Google Insights Trends tool allows you more “insight” into how keywords are performing in search over time and location. It includes “related terms” as mentioned above but more importantly it shows you how terms are rising or falling in interest and usage. This is ideal for people with a seasonal or trend related focus. If you sell flowers then your traffic is affected by seasons as you’ll see a spike in related searches around Valentine’s Day or Mother’s Day while if you sell DVDs your timing will be affected by new releases or movie star news.
This is also a great tool for seeing the trending of terminology which dominates your industry.
Using Google’s Webmaster Tools
The Search Queries page shows a range of information on the queries which resulted in one of your website’s pages being provided as a result for a specific keyword search.
Google describes it thus:
The Search Queries page provides information about Google Web Search queries that have returned pages from your site. In addition, you can also see information about the pages on your site that were seen most often in search results (top pages). To specify the period for which you want to see data, use the calendar dropdowns above the graph. (By default, we'll show the last 30 days of data, and compare the daily average for the selected period with the daily average for the previous period.)
Google goes on to list the data available via this section:
Search queries data includes the following:
Queries: The total number of search queries that returned pages from your site over the given period.
Query list: Specific user queries for which your site appeared in search results. Webmaster Tools shows data for the top 2,000 queries that returned your site at least once or twice in search results in the selected period. This list reflects any filters you’ve set (for example, a search query for [flowers] on google.ca is counted separately from a query for [flowers] on google.com).
Impressions: The number of times pages from your site appeared in search results, and the percentage increase/decrease in the daily average impressions compared to the previous period. The number of days per period defaults to 30, but you can change it at any time. (These numbers can be rounded, and may not be exact.)
Clicks: The number of times a user clicked your site's listing in search results for a particular query, and the percentage increase/decrease in the average daily clicks compared to the previous period. (These numbers can be rounded, and may not be exact.)
CTR (clickthrough rate): The percentage of impressions that resulted in a click to your site, and the number of percentage points increase or decrease in the daily average CTR compared to the previous period. For example, if your CTR went from 40% to 30%, this column would show a change of -10.
Average position: The average top position of your site on the search results page for that query, and the change compared to the previous period. Green indicates that your site's average top position is improving.
To calculate average position, we take into account the top ranking URL from your site your site for a particular query. For example, if Jane’s query returns your site as the #1 and #2 result, and David’s query returns your site in positions #2 and #7, your average top position would be 1.5.
There is a button on the report that allows you to view the percentage of change of each keyword. Make sure this is clicked so that you can see the results of your optimization efforts.
There is a date range with drop-downs so that you can custom define your desired range. Content is normally not provided for the previous two days.
You can view up to 90 days of historical data. However, you can only see change data for time periods of 30 days or less.
Filters are very important when it comes to determining keywords and successful pages based on the medium which generates the query result. So if you want to determine how many came from the web or from mobile or from video, or how many came from which country, then you can use the filters to view the details.
One benefit to the filters is if you generate more than 2,000 queries then you can use the filters to provide the top queries for each filter type. This can result in data on more than 2,000 overall keywords. Also the results are updated daily so you can change the date range by day to get more keyword data. Combining this with the filters can provide you with a lot of keyword data.
Once you’ve generated a report using the date range and the filters then you can download the report as a CSV file to Excel by clicking on the “Download this table” button at the top of the report. Once in Excel there is a lot you can do with it.
For example; if you take the time to download reports daily then you can accumulate the data so that you can both increase the amount of keywords your able to track as well as the accumulative data on each keyword.
Google provides some further insight into how you can leverage the data found in the Search Queries report:
How to use Search Queries data
This data can provide valuable information about your site. We recommend the following steps:
Review the Query list for expected keywords. If keywords you expect to see don't appear, your site may not have enough useful content relevant to those keywords. If unexpected words (like "Viagra" or "casino") appear, it's likely that your site has been hacked.
Compare Impressions and CTR to identify how you can improve your content.(Tip: Sort by Change to see queries with significant new activity.) There are several steps you can take to make your content appear more compelling so that users click your site in search results pages. Your page title appears in the results, so make sure it's relevant and accurate. Google can display the text in your pages' meta descriptions in search results, so review your meta descriptions.
If you have an AdWordsaccount, review the Query list for keyword ideas. (Looking for more ideas? Check out the Publisher's Guide to Toolbar.)
Another informative section of Webmaster Tools is the Content Keywords section.
Google describes it thus:
The Content Keywords page lists the most significant keywords and their variants Google found when crawling your site. When reviewed along with the Search Queriesreport and your site's listing in actual search results for your targeted keywords, it provides insight into how Google is interpreting the content of your site.
The significance of each keyword reflects how often it's found on your site's pages. Click each keyword to see a sampling of pages on which it appears. Both keywords and their variants are listed in order of frequency of appearance.
The interesting point here is how they describe them as the “most significant keywords” found. This shows the significance to Google of the usage (location, prominence and quantity, etc) of the terms on your pages, throughout the entire site.
It also shows variants of terms which is another indicator of significance. It is advisable to include many variants as well as synonyms of keywords as you can in a relevant and legible manner.
Using the Content Keywords section to drill-down into your keywords can provide some interesting and useful data on how keywords are spread throughout your site. You can also change the date range to view different keywords which can be helpful if you have a site that is large and constantly adding or modifying content.
You can also use this section to help with keyword cannibalization by seeing what words are found on which pages.
Use it also to see words that are missing, in other words – if your number 3 word is something you’re not trying to rank for then you may want to review its usage on your site and minimize it while inversely increasing the usage of other target terms.
Links to Your Site
The Links to Your Site section shows you how your data is linked to externally. It can be useful in seeing what keywords are actually being used in anchor text which points to your site and which pages are being pointed to and from. This allows you to see if your site has backlinks from relevant sources or not. This is an invaluable tool for link audits and for cleaning out unwanted or unnatural links.
You can also use this tool to obtain keyword opportunities by reviewing the list you may see that people have begun linking to a specific page on your website and with a specific set of keywords in the anchor text and from within content that is specific to a certain subject. You may have only provided one piece of information that may have started getting attention and links and this knowledge now allows you the opportunity to build out that content and optimize for more keywords surrounding that topic.
It also provides great insight into sites that are willing to link to your quality content!
Some other great tools are Moz’ Open Site Explorer & Link Detective.
Organic Keyword Referrals:
GA still provides a wealth of information on keyword referrals from other sources so using that data we can get an idea of which terms are sending traffic (and how much of it) from Google.
Since the standard Organic Keyword Referrals report doesn’t quite conveniently allow you to see the Source/Medium which drove traffic for each keyword, I like to create a custom report which does.
Click on “Customization” in the top orange bar and then modify the different Metric Groups and Dimension Drilldowns to like the following:
First I set it up with the first Metric Group of Organic Searches and add in some other metrics that I’ll initially find helpful (bounce rate, unique visitors and pageviews). Then I add the dimension drilldowns of keyword, source/medium and mobile. Mobile is an extra that can be very useful especially with Hummingbird since Mobile tends to have different query aspects (e.g. more long-tails).
After running this report I can see the keywords that brought traffic to my site and then I can click on a specific keyword and instantly see the source and medium which brought it as well as how it performed from each with bounce rate, unique visitors and pageviews. You can add and swap out Metric Groups and Dimension Drilldowns as you see fits your desired goal.
Since you won’t be seeing any organic keywords from Google you can still see any brought by Adwords. You will however see organic traffic from other engines like Bing and Yahoo. You can pull down reports in CSV (Comma-Separated Values) which you can view and manage with Excel.
If you modify this report placing Source/Medium above Keyword in the Dimension Drilldowns then you can use that to see all keywords by search engine so that you can pull that data into Excel specifically.
The benefit to that is that when you pull it in you can then use it to estimate the traffic received from Google (See the “Guestimating Google Keyword Referrals” section below).
Not Provided: URLs/Title Keywords
Another great method of determining the keywords used to find you is to look at the URLs, Titles and other data on your pages which received traffic from search. In order to do this we can create another GA custom report which pulls the data based on entrances so set up your custom report like this:
Running this report at first gives you the Titles of pages that received organic search entrances. With all of the Titles presented there in the report you can use the search feature to find specific keywords you want to track which are located in the Title tag. Next you can drilldown to see the landing page URL for this Title. Using this allows you to see also where multiple pages may share the same Title. Next you can click to see the Source and Medium which brought traffic to that page and finally you can click to view the actual keywords. Knowing that there will only be one keyword in Google (i.e. Not Provided) we can use the steps up to the Source/Medium (e.g. Google/Organic) to see which Titles and URLs had the keywords that we’re looking for which actually generated organic traffic from Google. Combining this with the Google Site Operator Search of your own site, along with the keyword referral data from the other Search Engines, data from Screaming Frog, Ranks.nl and your other tools, can show you a lot about the keywords that you’re site is optimized for which may be part of those “Not Provided” terms.
Internal Site Search
Many sites have an internal search engine which allows a user to find what they’re looking for within the site. Some use a custom engine while others use a search engine like Google. Internal search engines can provide a wealth of keyword data as the person who lands on your site and uses that search places a specific query which does not show as Not Provided! There is a strong likelihood that it is similar to the search query placed in Google which brought them to you.
Internal search engines are more commonly found on larger sites but you can install a search engine even on smaller sites (recommended for sites with 100+ pages). Your website may only have 50 pages and it may not make sense to have an internal search with that few pages but it does inspire you to begin increasing the pages of your site and that provides the excellent opportunity for Keyword Segmenting which is talked about below.
Google Analytics allows you to plug Site Search data into it so that you can run reports. I highly recommend this.
Screaming Frog is a great program used for crawling your website and providing you a macro view of your site in a convenient spreadsheet. You can see your pages and their respective URLs, Titles, Meta tags, Headings and even such things as Canonicals. When you use this along with the aforementioned tools you can see how your website uses keywords in those elements which are most significant to optimization. Screaming Frog doesn’t delve into the on-page textual content but for the other data it’s a powerful tool. Make sure you export the data into Excel as a CSV.
Also www.Ranks.nl provides a long-standing and powerful keyword density reporting tool (as well as other data) which can show you a lot of information on how your site and its pages are using keywords.
Once that specific Google organic referral keyword data is gone we’ll need better ways of determining what keywords are effective in driving traffic and conversions. Rank tracking is once again a dominant factor in providing this.
Once you have your Master Keyword List then you’ll want to run your ranking report. Run as many keywords as possible for your baseline report as you can always run targeted groups later and compare them against your baseline report.
Once you have your stats you can start to see how you rank across terms, groups of terms, topics, specific long-tails and more general “trophy” terms. When you see that one of your pages receives traffic, which you’ve verified via Analytics, then you can use the rankings to determine which keywords are likely responsible for the traffic. Again this is a relative measurement which is no replacement for actual keyword data - but it lends a hand.
Guestimating Google Keyword Referrals
How do you provide the Not Provided? Having the available keyword referral data for other engines, knowing what your pages are optimized for and ranking for and knowing what keywords your targeting can all culminate in added awareness. In the end it’s going to be down to testing and that means gaining what direction you can from the data available to you and implementing it. When a company is implementing multiple marketing initiatives which can generate organic traffic it becomes difficult to divine which initiative brought which traffic. Limited Control Testing can provide quantifiable results that can be extrapolated across the entire campaign and site so that if you can raise the rankings and traffic for your target terms for a limited part of the site then you can take that rise and compare and contrast it with the growth across the site once you implement globally, despite other coinciding initiatives. This also allows one to judge the success of those other initiatives more precisely.
Building a Question List
Sit down and take your keyword list in hand and then start writing down next to your keywords questions that people would apply to them. If it’s a product then ask questions about its usage, its durability, its benefits, the warranty on it, the history of it, etc.
Use other existing Q&A sites to generate queries that already drive traffic.
· What are the best ski boots on the market?
· How do I choose the right ski boot?
· Do ski boot sizes affect performance?
A lot of these you can find online.
There are a number of ways you can use Keyword Segmenting to improve your site’s ranking (especially on long-tails) while increasing your conversion rate and time-on-site which can also decrease your bounce rate and other negative factors.
Keyword Segmenting is the process of splitting out keywords that are related so that you can get people to the more-specific data they’re seeking while also tracking which keywords they’re interested in. Knowing their motivation allows you to create content to supply or answer their desires or demands and this process tells you what content they were most interested in after they arrived on your site while also providing you with more relevant and contextually-targeted content pages for Google’s index.
You have a webpage dedicated to “Ice Cream Machines” and on it you have a paragraph of content about them as well as links to your different machines available for sale. You have a nice list of long-tails that you want to create content around so you take your list: “best ice cream machines”, “ice cream machine reviews”, “electric ice cream machines”, “homemade ice cream maker”, ”ice cream recipes”, etc and you create a small page for each with around 200 words at least of relevant unique content.
You can link to these pages from different areas of the site whether from within content, within a section at the bottom of the page for Related Content or some other area you feel they fit. They should also be included in any sitemaps whether html or xml.
The use of dictionaries/glossaries, specific tips and tricks sections, or the above mentioned articles of a couple of hundred words can all provide excellent opportunities to target long-tails. These pages also increase the quantity of relevant pages in the index, more internal backlinks and more contextually targeted content for searchers.
To test the theory you can create a controlled test case where you can choose a page which ranks not so well for a specific term and optimize for it. Then you can track the ranking and the organic entrances via analytics and see if that rise in ranking corresponded with a rise in organic entrance traffic. If you’re able to generate a sizeable rise in traffic then this can lead you to segment related terms by creating new content pages and link to them from that related page.
If you have a page which is optimized for “Ski Boots” and ranks well for it but also ranks well for “Ski Boot Sizing” (which is mentioned on the page) then you can increase the optimization of that term a bit, though not to the detriment of the main term, and see if it ranks higher. Then compare that against your organic entrance traffic to that page. If it increases with the increased rankings then you have a solid indicator that this term is valuable (as it produces traffic). Then you can decide to segment the term to a new page which is primarily optimized for it. Once that page is indexed you can run your ranking report on it and see if it outranks, replaces or ranks along with your “Ski Boots” page. The goal would be to have it rank higher but not replace the other for this term so that you can have better placement for that more specific page while the other acts as a second listing providing you with 20% coverage of the organic results on that SERP. Google no longer limits rankings to two per domain so you can potentially have even more of your pages rank for a specific term.
You can continue this process of optimizing for long-tails and then segmenting to target other similar terms such as “Ski Boot Sizes”. The content would need to be sufficiently unique for each of those two similar terms. The first can speak about how one goes about sizing the right boot for their foot, along with information on how there should be sufficient room for ski socks. The other page can actually list the sizes that are available in a convenient size chart with a bit of unique content on sizes, perhaps about different sizes for men’s, women’s and children’s boots. This provides you with two different pages for similar terms with sufficiently unique content. Using link volume data to determine which words to target only increases your likelihood of a good return on your efforts. Such variations can provide a wealth of traffic. For example “Ski Boot Sizing” and “Ski Boot Sizes” receive 1,600 and 720 average monthly searches respectively according to the Adwords Keyword Planner.
The reporting benefit is that you’re able to then see the path that people follow throughout your site as they drill deeper to find the information they’re seeking. By tracking this you can see the keywords that they’re interested in by their choice to click on a link (due to the content surrounding and the anchor text of the link). The same goes for the site’s internal search as they’re now directly providing you with their keywords.
The ranking report you ran shows you where a page is (or pages are) in terms of position for a keyword. Your analytics data will provide you with the entrances to that page from organic search (including Google) and keywords which resulted in entrances (excluding Google). Combining this data with some keyword volume data from the Adwords Keyword Planner (and like tools from other sources) and an idea of Google’s market share can give you a decent idea of the quantities you are likely to receive from each keyword.
My ranking report shows that my page on “Ski Boot Flex Ratings” is ranked 1, 3, 7 and 9 for my four main keywords for that topic (e.g. “sk boot flex index, ski boot flex ratings, ski boot flex index chart, ski boot flex chart) in Google. This will be far more effective if you use a much higher quantity of keywords but we’ll limit it to four for this example. My analytics program shows that I received 350 organic entrance visits this week to that page. Bing and Yahoo generated 50 visits each while Google generated 250.
I’ve taken all of this data and placed it in a spreadsheet with each keyword taking up a row and my data on each of those in the columns of that row. There is one for my Google ranking, Bing ranking, and Yahoo ranking and then total traffic for each keyword from Bing and Yahoo and the relevant quantities from WMT Search Queries. You then have another column for Search Volume and one for “Not Provided” which you expect to fill.
(The keywords that you used in your ranking report may differ from the ones that actually generated traffic in Bing and Yahoo so you’ll need to either match closely related/ranked ones or simply include them all)
Running those keywords through search volume tools can give you an idea of the amount of traffic you’d expect those queries would get and again you can use WMT to see how many impressions and clicks you got (although WMT isn’t very accurate).
Now you have a nice spreadsheet with current data and you have a glaring blank spot under the “Not Provided” column. The next thing to do is to assign a percentage to each ranking position of Google’s first page.
Here are some links to available data and opinions on this:
There are others so you can determine for yourself what you feel is the rate which you want to apply.
Take what search volume you have and apply your percentages. Next take those figures and compare them to Bing and Yahoo and see how they match up. You can use real world differences to tweak your percentages. Next apply the same percentages to Google based on the available search volume data, ranking position, your newly defined ranking/clicks percentage and total entrances to that page. Now when you tally up the expected amount of traffic for each keyword that resulted in an organic entrance to that page from Google you can get an idea of which keywords are performing.
Keeping this as a baseline report allows you to optimize that page further and rank higher for keywords where you’re not number one. If you rise up from position 9 to position 3 for a keyword that has 700 monthly visits then you can take an average of that 700 monthly visits over 30 days (23 visits per day or 163 per week) and compare that against the traffic you received to that page. If you didn’t have any other rises in ranking for similar keywords then you can figure that a lot of that traffic resulted from this keyword or one very similar. If you equally rose in Bing and Yahoo then you can see the amounts of increases you received from those specific keywords and use that against your Google “guestimation”.
Branded vs. Non-Branded
Distinguishing between branded and non-branded traffic with “Not Provided” is not impossible.
One of the best ways to determine branded vs. non-branded traffic is to check the referrals to a page from other search engines. Then take the keywords that sent referrals from them and place them into your Google rank checker and find out if you’re ranking well for those terms. The likelihood that page which ranks well for a term is generating traffic from organic non-branded searches is far higher than one that doesn’t rank well for terms unless the brand is included in the query.
Also it’s good to look at the landing page and determine from historical Google keyword data how much traffic that page received prior to the loss of keyword data (or if it’s a new page then find one like it which has historical data).
Get to know your site (or client’s site) and see where most branded traffic has come in before and still arrives on other engines. Do some testing and compare the results to what you or your client is doing as far as branding. If the term is “ski boots” and your client is the ski boot maker Salomon then you can assume that people will be doing a lot of branded searches. One thing is that branded searches don’t normally fluctuate much unless you’re proactively pushing a branded campaign or something comes out, like news, which affects your branded traffic. However if you’re ranked #14 for ski boots like Salomon is and you get that up to first page and you see a jump in traffic to your homepage then you can assume that most of that gain came from the increased organic ranking with non-branded traffic.
When doing branded promotions it’s a good thing to use targeted landing pages so that you can more easily track the results as well as making sure that you run a lot of reports and grab a snapshot of where you stand before the launch so that you can determine the results of the campaign even on homepage organic traffic.
One of the big things about SEO now, with Hummingbird and the rise of Authorship, is building out a breadth of topically related content, building interaction for user-generated content and connecting with others in your industry and sharing this topically related content with them.
The keyword data will help you and your clients or bosses understand how a campaign is faring but thinking also about building up customer retention schemes, branding, viral marketing and other programs to keep your audience/customer/fan base happy, coming back and bringing other people.
Best of luck!
Meteorsite is a Los Angeles SEO Company since 2002, specializing in ethical Search Engine Optimization, Local SEO, Web Development and more. Please contact us for a free consultation.
Partner / SEO Director