How To Do a Content Audit – Step-by-Step

Our focus in this example will be to audit the content with an eye to improve and/or Remove low performing pages, followed by keyword research and a keyword matrix for the primary pages, including the home page, categories, blog home and key product pages, as well as certain other topical landing pages.

As it turns out, this hypothetical website has lots of manufacturer-supplied product descriptions. We’re going to need to prioritize which ones get rewritten first because the client does not have the cash-flow to do them all at once. When budget and time is a concern, we typically shoot for the 80/20 rule: Write great content for the top 20% of pages right away, and do the other 80% over the course of 6-12 months as time/budget permit.

Because this site doesn’t have an existing penalty, we will recommend that all pages stay indexed. If they had a penalty already, we would recommend they noindex,follow the bottom 80% of pages, gradually releasing them back into the index as they are rewritten. This may not be the way you choose to handle the same situation, which is fine, but the point is you can easily sort the pages by any number of metrics to determine a relative “priority”. The bigger the site and tighter the budget, the more important it is to prioritize what gets worked on first.

Causes of Content-Related Penalties

For the purpose of a content audit we are only concerned with content-related penalties (as opposed to links and other off-page issues), which typically fall under three major categories: Quality, Duplication, and Relevancy. These can be further broken down into other issues, which include – but are not limited to:

• Typical low quality content
  • Poor grammar, written primarily for search engines (includes keyword stuffing), unhelpful, inaccurate…
• Completely irrelevant content
  • OK in small amounts, but often entire blogs are full of it.
  • A typical example would be a “linkbait” piece circa 2010.
• Thin / Short content
  • Glossed over the topic, too few words, all image-based content…
• Curated content with no added value
  • Comprised almost entirely of bits and pieces of content that exists elsewhere.
• Misleading Optimization
  • Titles or keywords targeting queries for which content doesn’t answer or deserve to rank
  • Generally not providing the information the visitor was expecting to find
• Duplicate Content
  • Internally duplicated on other pages (e.g. categories, product variants, archives, technical issues…)
  • Externally duplicated (e.g. manufacturer product descriptions, product descriptions duplicated in feeds used for other channels like Amazon, shopping comparison sites and eBay, plagiarized content…)
• Stub Pages (e.g. “No content is here yet, but if you sign in and leave some user-generated-content then we’ll have content here for the next guy”. By the way, want our newsletter? Click an AD!)
• Indexable internal search results
• Too many indexable blog tag or blog category pages
• And so forth and so-on…

If you are unsure about the scale of the site’s content problems, feel free to do step 2 before deciding on a scenario…

Step 2: Scan the site

We use Screaming Frog for this step, but you can adapt this process to whatever crawler you want. This is how we configure the spider’s “Basic” and “Advanced” tabs…

 And the advanced tab…

Notice that “crawl all subdomains” is checked. This is optional, depending on what you’re auditing. We are respecting “meta robots noindex”, “rel = canonical” and robots.txt. Also notice that we are not crawling images, CSS, JS, flash, external links…. This type of stuff is what we look at in a Technical SEO Audit, but would needlessly complicate a “Content” Audit. What we’re looking for here are all of the indexable HTML pages that might lead a visitor to the site from the SERPs, though it may certainly lead to the discovery of technical issues.

Export the complete list of URLs and related data from Screaming Frog into a CSV file.

Step 3: Import the URLs and start the tool

We have our own internal “Content Auditing Tool”, which takes URLs and data from Screaming Frog and Google Analytics, de-dupes them, and pulls in data from Google Webmaster Tools, Moz, Social Count and Copyscape for each URL. The tool is a bit buggy at times, however, so I’ve been experimenting with URL Profiler, which can essentially accomplish the same goal with fewer steps and less upkeep. We need the “Agency” version, which is about $400 per year, plus tax. That’s not too bad, considering we’d already spent several thousand on our internal tool by the time Gareth Brown released URL Profiler publicly. :-/

Below is a screenshot of what you’ll see after downloading the tool. I’ve highlighted the boxes we currently check, though it depends on the tools/APIs to which you already subscribe and will differ by user. We’ve only just started playing with uClassify for the purpose of semi-automating our topic bucketing of pages, but I don’t have a process to share yet (feel free to comment with advice)…

Right-click on the URL List box and choose “Import From File”, then choose the ScreamingFrog export or any other list of URLs. There are also options to import from the clipboard or XML sitemap. Full documentation for URL Profiler can be found here. Below are two output screenshots to give you an idea of what you’re going to end up with…

The output changes depending on which boxes you check and what API access you have.

Step 4: Import the tool output into the dashboard

As described in the 50,000 foot overview above, we have a spreadsheet template with multiple tabs, one of which is the “Content Audit” tab. The tool output gets brought into the Content Audit tab of the dashboard. Our internal tool automatically ads columns for Action, Strategy, Page Type and Source (of the URL). You can also add these to the tab after importing the URL Profiler output. Page Type and URL Source are optional, but Action and Strategy are key elements of the process.

Our hypothetical client requires a Keyword Matrix. However, if your “scenario” does not involve keyword research (i.e. if it is a big site with content penalty risks) you can skip steps 5-7 and move straight to “Step 8 – Time to Analyze and Make Some Decisions”.

Step 5: Import GWT data

Match existing URLs from the content audit to keywords for which they already rank in Google Webmaster Tools

There may be a way to do this with URL Profiler. If so, I haven’t found it yet. Here is what we do to grab the landing page and associated keyword/query data from Google Webmaster Tools, which we then import into two tabs (GWT Top Queries and GWT Top Pages). These tabs are helpful when filling out the Keyword Matrix because they tell you which pages Google is already associating with each ranking keyword. This step can actually be skipped altogether for huge sites with major content problems because the “Focus” is going to be on pruning the site of low quality content, rather than doing any keyword research or content gap analysis.

Instructions for Importing Top Pages from GWT

• Log into GWT from a Chrome browser
• Go to Search Traffic —> Search Queries
• Switch the view to “Top pages” (default is “Top queries”)
• Change the date range to start as far back as possible (i.e. 3 months)
• Expand the amount of rows to show to the maximum of 500 rows
  • This will put the s=500 parameter in the URL. Change s=500 to s=10000 or however many rows of data are available
  • See bottom of GWT page (e.g. 1-500 of ####).
• In the Chrome menu go to View —> Developer —> Javascript Console
• Copy and Paste the following script into the console window and press Enter.
• This action should expand all of the drop-downs to show the keywords under each “page” URL and then open up a dialog window that will ask you to save a CSV file: (more info here and here).
• The script is also available in a javascript bookmarklet on Lunametrics.com…

1. (function(){eval(function(p,a,c,k,e,r){e=function(c){return(c<a?'':e(parseInt(c/a)))+((c=c%a)>35?String.fromCharCode(c+29):c.toString(36))};if(!''.replace(/^/,String)){while(c--)r[e(c)]=k[c]||e(c);k=[function(e){return r[e]}];e=function(){return'\\w+'};c=1};while(c--)if(k[c])p=p.replace(new RegExp('\\b'+e(c)+'\\b','g'),k[c]);return p}('C=M;k=0;v=e.q(\'1g-1a-18 o-y\');z=16(m(){H(v[k]);k++;f(k>=v.c){15(z);A()}},C);m H(a){a.h(\'D\',\'#\');a.h(\'11\',\'\');a.F()}m A(){d=e.10(\'Z\').4[1].4;2=X B();u=B.W.R.Q(d);7=e.q(\'o-G-O\');p(i=0;i<7.c;i++){d=u.J(7[i]);2.K([d,7[i].4[0].4[0].j])}7=e.q(\'o-G-14\');p(i=0;i<7.c;i++){d=u.J(7[i]);2.K([d,7[i].4[0].4[0].j])}2.N(m(a,b){P a[0]-b[0]});p(i=2.c-1;i>0;i--){r=2[i][0]-2[i-1][0];f(r===1){2[i-1][1]=2[i][1];2[i][0]++}}5="S\\T\\U\\V\\n";9=e.q("o-y-Y");6=0;I:p(i=0;i<9.c;i++){f(2[6][0]===i){E=2[6][1];12{6++;f(6>=2.c){13 I}r=2[6][0]-2[6-1][0]}L(r===1);2[6][0]-=(6)}5+=E+"\\t";l=9[i].4[0].4.c;f(l>0)5+=9[i].4[0].4[0].j+"\\t";17 5+=9[i].4[0].w+"\\t";5+=9[i].4[1].4[0].w+"\\t";5+=9[i].4[3].4[0].w+"\\n";5=5.19(/"|\'/g,\'\')}x="1b:j/1c;1d=1e-8,"+1f(5);s=e.1h("a");s.h("D",x);s.h("1i","1j.1k");s.F()}',62,83,'||indices||children|thisCSV|count|pageTds||queries|||length|temp|document|if||setAttribute||text|||function||url|for|getElementsByClassName|test|link||tableEntries|pages|innerHTML|encodedUri|detail|currInterval|downloadReport|Array|timeout1|href|thisPage|click|expand|expandPageListing|buildCSV|indexOf|push|while|25|sort|open|return|call|slice|page|tkeyword|timpressions|tclicks|prototype|new|row|grid|getElementById|target|do|break|closed|clearInterval|setInterval|else|block|replace|inline|data|csv|charset|utf|encodeURI|goog|createElement|download|GWT_data|tsv'.split('|'),0,{}))})();
    

   Ignore any dialog windows that pop up.
   You can check “Prevent this page from creating additional dialogs” to disable them.

   

   • Import the resulting download.csv file from GWT into the “GWT Top Pages” tab in the Content Auditing Dashboard.

   Instructions for Importing Top Queries from GWT
   

   1. Within GWT switch back to Top Queries.
   2. Adjust the date to go back as far as you can.
   3. Expand the amount of rows to show to the maximum of 500 rows
      1. This will put the s=500 parameter in the URL. Change s=500 to s=10000 or however many rows of data are available
      1. See bottom of GWT page (e.g. 1-500 of ####).
   4. Select “Download this table” as a CSV file
   5. Import the resulting TopSearchQueries.csv file from GWT into the “GWT Top Queries” tab in the Content Auditing Dashboard.

   

   Step 6: Perform keyword research

   This is another optional step, depending on the focus/objective of the audit. It is also highly customizable to your own KWR process. Use whatever methods you like for gathering the list of keywords (e.g. brainstorming, SEMRush, Google Trends, Uber Suggest, GWT, GA…). Ensure all “junk” and irrelevant keywords are removed from the list, and run the rest through a single tool that collects search volume and competition metrics. We use the Google Adwords Keyword Planner, which is outlined below.

   1. Go to www.google.com/sktool/ while logged into our Google email account associated AdWords.
   2. Select “Get search volume for a list of keywords or group them into ad groups”, paste in your list of keywords and click “Get search volume”.
      1. Note: At this point you should have already expanded the list as much as you need/want to so you’re just gathering data and organizing them now.
      2. Note: The copy/paste method is limited to 1,000 keywords. You can get up to 3,000 by uploading your simple .txt file.
   3. Go to the “Keyword Ideas” tab on the next screen and Add All keywords to the plan.
   4. Go to the “Ad Group Ideas” tab and choose to Add All of the ad groups to the plan.
   5. Download the plan, as seen in the screenshot below.
   6. Import the data into the AdWords Data tab of the Content Auditing Dashboard

   

   Use the settings below when downloading the plan:

   

   Step 7: Tying the keyword data together

   Again, you don’t need to do this step if you’re working on a large site and the focus is on pruning out low quality content. The GWT Queries and KWR steps provide data needed to develop a “Keyword Matrix” (KWM), which isn’t necessary unless part of your focus is on-page optimization and copywriting of key pages. Sometimes you just need to get a client out of a penalty, or remove the danger of one. The KWM comes in handy for the important pages marked as “Improve” within the Content Audit tab just so the person writing the copy understands which keywords are important for that page. It’s SEO 101 and you can do it anyway you like using whatever tools you like.

   Google Adwords has given you the keyword, search volume and competition. Google Webmaster Tools has given you the ranking page, average position, impressions, clicks and CTR for each keyword. Pull these together into a tab called “Keyword Research” using Vlookups. You should end up with something like this:

   

   The purpose of these last few steps was to help with the KWM, an example of which is shown below:

   

   Step 8: Time to analyze and make some decisions!

   All of the data is right in front of you, and your path has been laid out using the Content Audit Scenarios tool. From here on the actual step-by-step process becomes much more open to interpretation and your own experience / intuition. Therefore, do not consider this a linear set of instructions meant to be carried out one after another. You may do some of them and not others. You may do them a little differently. That is all fine as long as you are working toward the goal of determining what to do, if anything, for each piece of content on the website.

   • Sort by Copyscape Risk Score
     • Which of these pages should be rewritten?
       • Rewrite key/important pages, such as categories, home page, top products
       • Rewrite pages with good Link and Social metrics
       • Rewrite pages with good traffic
       • After selecting “Improve” in the Action column, elaborate in the Strategy column:
         • “Improve these pages by writing unique, useful content to improve the Copyscape risk score.”
     • Which of these pages should be removed / pruned?
       • Remove guest posts that were published elsewhere
       • Remove anything the client plagiarized
       • Remove content that isn’t worth rewriting, such as:
         • No external links, no social shares, and very few or no entrances / visits
       • After selecting “Remove” from the Action column, elaborate in the Strategy column:
         • “Prune from site to remove duplicate content. This URL has no links or shares and very little traffic. We recommend allowing the URL to return 404 or 410 response code. Remove all internal links, including from the sitemap.
     • Which of these pages should be consolidated into others?
       • Presumably none, since the content is already externally duplicated
     • Which of these pages should be marked “Leave As-Is”
       • Important pages which have had their content stolen
         • In the Strategy column provide a link to the CopyScape report and instructions for filing a DMCA / Copyright complaint with Google.
   • Sort by Entrances or Visits (filtering out any that were already finished)
     • Which of these pages should be marked as “Improve”?
       • Pages with high visits / entrances but low conversion, time-on-site, pageviews per session…
       • Key pages that require improvement determined after a manual review of the page
     • Which of these pages should be marked as “Consolidate”?
       • When you have overlapping topics that don’t provide much unique value of their own, but could make a great resource when combined.
         • Mark the page in the set with the best metrics as “Improve” and in the Strategy column outline which pages are going to be consolidated into it. This is the canonical page.
         • Mark the pages that are to be consolidated into the canonical page as “Consolidate” and provide further instructions in the Strategy column, such as:
           • Use portions of this content to round out /canonicalpage/ and then 301 redirect this page into /canonicalpage/ Update all internal links.
       • Campaign-based or seasonal pages that could be consolidated into a single “Evergreen” landing page (e.g. Best Sellers of 2012 and Best Sellers of 2013 —> Best Sellers).
     • Which of these pages should be marked as “Remove”?
       • Pages with poor link, traffic and social metrics related to low-quality content that isn’t worth updating
         • Typically these will be allowed to 404/410.
       • Irrelevant content
         • The strategy will depend on link equity and traffic as to whether it gets redirected or simply removed.
       • Out-of-Date content that isn’t worth updating or consolidating
         • The strategy will depend on link equity and traffic as to whether it gets redirected or simply removed.
     • Which of these pages should be marked as “Leave As-Is”?
       • Pages with good traffic, conversions, time on site, etc… that also have good content.
         • These may or may not have any decent external links

   Another Way of Thinking About It…

   

   For big sites It is best to use a hatchet-approach as much as possible, and finish up with a scalpel in the end. Otherwise you’ll spend way too much time on the project, which eats into the ROI.

   This is not a process that can be documented step-by-step. For the purpose of illustration, however, here are a few different examples of hatchet approaches and when to consider using them.

   • Parameter-based URLs that shouldn’t be indexed
     • Defer to the Technical Audit, if applicable. Otherwise, use your best judgement:
       • e.g. /?sort=color, &size=small
       • Assuming the Tech Audit didn’t suggest otherwise these pages could all be handled in one fell swoop. Below is an “example” action and an “example” strategy for such a page:
       • Action = Consolodate
       • Strategy = Rel canonical to the base page without the parameter
   • Internal search results
     • Defer to the Technical Audit if applicable. Otherwise, use your best judgement:
       • e.g. /search/keyword-phrase/
       • Assuming the Tech Audit didn’t suggest otherwise:
       • Action = Remove
       • Strategy = Apply a noindex meta tag. Once they are removed from the index, disallow /search/ in the robots.txt file.
   • Blog tag pages
     • Defer to the Technical Audit if applicable. Otherwise…:
       • e.g. /blog/tag/green-widgets/ , blog/tag/blue-widgets/ …
       • Assuming the Tech Audit didn’t suggest otherwise:
       • Action = Remove
       • Strategy = Apply a noindex meta tag. Once they are removed from the index, disallow /search/ in the robots.txt file.
   • eCommerce Product Pages with Manufacturer Descriptions
     • In cases where the “Page Type” is known (i.e. it’s in the URL or was provided in a CMS export) and Risk Score indicates duplication…
       • e.g. /product/product-name/
       • Assuming the Tech Audit didn’t suggest otherwise:
       • Action = Improve
       • Strategy = Rewrite to improve product description and avoid duplicate content
   • eCommerce Category Pages with No Static Content
     • In cases where the “Page Type” is known…
       • e.g. /category/category-name/ or category/cat1/cat2/
       • Assuming NONE of the category pages have content…
       • Action = Improve
       • Strategy = Write 2-3 sentences of unique, useful content that explains choices, next steps or benefits to the visitor looking to choose a product from the category.
   • Out-of-Date Blog Posts, Articles and Other Landing Pages
     • In cases where the Title tag includes a date or…
     • In cases where the URL indicates the publishing date….
       • Action = Improve
       • Strategy = Update the post to make it more current if applicable. Otherwise, change Action to “Remove” and customize the Strategy based on links and traffic (i.e. 301 or 404)

   Step 9: Content gap analysis and other value-adds

   Although most of these could be put as optional items during the keyword research process, I prefer to save them until last because I never knows how much time I’ll have after taking care of more pressing issues.
   

   Content gaps
   If you’ve gone through the trouble of identifying keywords and the pages already ranking for them, it isn’t much of a step further to figure out which keywords could lead to ideas about how to fill content gaps.

   

   At Inflow we like to use the “Awareness Ladder” developed by Ben Hunt, as featured in his book Convert!. You can learn more about it here.

   Content levels
   If time permits, or the situation dictates, we may also add a column to the Keyword Matrix or Content Audit which identifies which level of content the page would need to compete in its keyword space. We typically choose from Basic, Standard and Premium. This goes a long way in helping the client allocate copywriting resources to work where they’re needed the most (i.e. best writers do the Premium content).

   Landing page or keyword topic buckets
   If time permits, or the situation dictates, we may provide topic bucketing for landing pages and/or keywords. More than once this has resulted in recommendations for adding to or changing existing taxonomy with great results. The most frequent example is in the “How To” or “Resources” space for any given niche.

   Keyword relevancy scores
   This is a good place to enlist the help of a client, especially in complicated niches with a lot of jargon. Sometimes the client can be working on this while the strategist is doing the content audit.

   Step 10: Writing up the content audit strategy document

   The Content Strategy, or whatever you decide to call it, should be delivered at the same time as the audit, and summarizes the findings, recommendations and next steps from the audit. It should start with an Executive Summary and then drill deeper into each section outlined therein.

   Here is a real example of an executive summary from one of Inflow’s Content Audit Strategies:

   As a result of our comprehensive content audit, we are recommending the following, which will be covered in more detail below:

   • Removal of about 624 pages from Google index by deletion or consolidation:
     • 203 Pages were marked for Removal with a 404 error (no redirect needed)
     • 110 Pages were marked for Removal with a 301 redirect to another page
     • 311 Pages were marked for Consolidation of content into other pages
       • Followed by a redirect to the page into which they were consolidated
   • Rewriting or improving of 668 pages
     • 605 Product Pages are to be rewritten due to use of manufacturer product descriptions (duplicate content), these being prioritized from first to last within the Content Audit.
     • 63 “Other” pages to be rewritten due to low-quality or duplicate content.
   • Keeping 26 pages as-is with no rewriting or improvements needed unless the page exists in the Keyword Matrix, in which case it requires on-page optimization best practices be reviewed/applied.
   • On-Page optimization focus for 25 pages with keywords outlined in the Keyword Matrix tab.

   These changes reflect an immediate need to “improve or remove” content in order to avoid an obvious content-based penalty from Google (e.g. Panda) due to thin, low-quality and duplicate content, especially concerning Representative and Dealers pages with some added risk from Style pages.

   
   The Content Strategy should end with recommended next steps, including action items for the consultant and the client. Here is a real example from one of our documents:

   We recommend the following actions in order of their urgency and/or potential ROI for the site:

   1. Remove or consolidate all pages in the “Prune” tab of the Content Audit Dashboard
      1. Detailed instructions for each page can be found in the “Strategy” column of the Prune tab
   2. Begin a copywriting project to improve/rewrite content on Style pages to ensure unique, robust content and proper keyword targeting.
      1. Inflow can provide support for your own copywriters, or we can use our in-house copywriters, depending on budget and other considerations. As part of this process, these items can also be addressed:
         1. Improve/rewrite all pages in the Keyword Matrix to match assigned keywords.
            1. Include on-page optimization (e.g. Title, description, alt attributes, keyword use, etc.)
               1. See the “Strategy” column for more complete instructions for each page.
         2. Improve/rewrite all remaining pages from the “Content Audit” tab listed as “Improve”.

   Resources, links, and post-scripts…

   Example Content Auditing Dashboard
   
   Make a copy of this Google Docs spreadsheet, which is a basic version of how we format ours at Inflow.

   Content Audit Strategies for Common Scenarios
   
   This page/tool will help you determine where to start and what to focus on for the majority of situations you’ll encounter while doing comprehensive content audits.

   How to Conduct a Content Audit on Your Site by Neil Patel of QuickSprout
   Oh wait, I can’t in send everyone to a page that makes them navigate a gauntlet of pop-ups to see the content, and another one to leave. So nevermind…

   How to Perform a Content Audit by Kristina Kledzik of Distilled
   This one focuses mostly on categorizing pages by buying cycle stage.

   Expanding the Horizons of eCommerce Content Strategy by Dan Kern of Inflow
   Dan wrote an epic post recently about content strategies for eCommerce businesses, which includes several good examples of content on different types of pages targeted toward various stages in the buying cycle.

   Distilled’s Epic Content Guide
   See the section on Content Inventory and Audit.

   The Content Inventory is Your Friend by Kristina Halvorson on BrainTraffic
   Praise for the life-changing powers of a good content audit inventory.

   How to Perform a Content Marketing Audit by Temple Stark on Vertical Measures
   Temple did a good job of spelling out the “how to” in terms of a high-level overview of his process to inventory content, assess its performance and make decisions on what to do next.

   Why Traditional Content Audits Aren’t Enough by Ahava Leibtag on Content Marketing Institute’s blog
   While not a step-by-step “How To” like this post, Ahava’s call for marketing analysts to approach these proejcts from both a quantitative (content inventory) and qualitative (content quality audit) resonated with me the first time I read it, and is partly responsible for how I’ve approached the process outlined above.