We have been working with a client who asked us to help him rebuild his e-commerce website that has not seen any major updates in over 15 years. The site has continued to expand with new products over the years, but his content management system was in need of a massive upgrade.
Whenever we take on any new web projects, one of the first things we analyze is the SEO of the site. We ask ourselves, how do we keep the current rankings and how do we improve the rankings after the launch. This site defiantly had some unique challenges, today I am going to share how we are approaching the URL redirects for our client’s aged website.
We had a few ideas, for this store with thousands of products. Our ideas started with the concept of ripping it off like a bandaid and just 301 everything. Then we thought maybe it would be better to launch the new site and put a canonical tag to the new location with a slow role out of 301 redirects. Then we thought well we don’t want to the traffic to dip to the top pages, so maybe we could redirect all but the top 500 pages, then change a few URL’s every week.
It has been a topic of discussion at our office for a few weeks and even at our Monthly Advanced SEO meeting. I just renewed my SEOmoz membership and I thought I would start using it by asking them a question on this internal debate.
Here is the answer from Everett at SEOmoz.
I bet after 15 years you’re going to see a lot of long, term positive results by fixing up all of the legacy issues you probably deal with.
The bad news is that traffic from search is likely to dip a little, maybe even a lot, for a period of time after you do this. Even if you do everything perfectly right (and we all make mistakes on big moves like this), it will take a little time for search engines to update their indexes and compute all the necessary data to apply actual rankings of old pages to your new ones. On a medium-sized ecommerce site, it should take 2-6 weeks. Any more than that and you should start troubleshooting.
OK, let’s get into the actual process. First of all, I’d wait until after Christmas if you can. As mentioned above, you’re going to see a dip no matter how you handle it, so if at all possible I’d wait until the shopping season slows down.
Second, don’t try to be too clever about it or roll things out in phases. It will only confuse search engines. If all of your URLs are changing and you 301 them correctly, that should be enough. By changing some, but not others, or 301ing some and relying on rel=canonical for others, you would be sending mixed messages and making it more difficult for yourself, and search engines.
More to the point: I would 301 redirect ALL of the old URLs to the new ones. There are things you can do to make this go more smoothly, but that’s the short answer. Assuming there is some logic to the way the old URLs are made that can be tied into the new ones, you or your developers can probably take care of this with regular expressions instead of writing page-to-page redirects for every single product and category. In one case years ago, there was no logic to the old URLs that we could “latch onto” for this purpose, so I ended up doing page to page redirects for every single category, and then just used Webmaster Tools to find out which of our pages had the most links. I redirected all product detail pages (PDPs) with external links using a page-to-page 301 redirect, and then just grouped all of the other ones together and sent them all to the top-level category page.
Once you figure out how you’re going to do the redirects, make sure you have an up-to-date XML sitemap of those old (current) URLs submitted to Google. You’ll need it in the next steps.
1. Gather all of your data on current URLs. Using Analytics, log files, Webmaster Tools, or whatever other means you have at your disposal, find out where the links are coming from for each page on your site. Export this data now because some of it might not be available later. You want to know where, both internally and from other sites, each of the old pages has links from.
2. Implement the redirects.
3. Resubmit the OLD sitemap so Google and Bing know to recrawl the old URLs. This way they get to see that 301 header status code faster than if they’d just have to restumble upon those old URLs again through a link or random crawl.
4. Once all of the old URLs have been recrawled (usually within 24 hours of submiting that XML sitemap) replace the old XML sitemap with a new one that has the NEW URLs. Keep the old one around on your hard drive for future reference.
5. Begin a massive campaign to clean up any links you have control of. Remember the data you collected in step #1? Now it will come in handy. First, update all internal links you have pointing to the old URLs. Even if the 301 works, best practice is to go straight to the new URL instead of relying on the redirect. For one thing, it has been my experience that 301s do not pass 100% of the page rank 100% of the time. This is another reason your rankings will drop a little no matter how hard you try. Also, if you ever have to do this again you don’t want to be hopping through multiple redirects so it’s best to just update the ones you have control over.
6. Once you’ve changed what you have direct control of (i.e. the links from your own site/s) move on to updating the most important links from external sites. This is a case-by-case thing. If CNN is linking to your old page, don’t send them an email asking for it to be updated unless you have friends working there. Just count your blessings. But if you sent Suzy Blogger a sample to review and you have a developed a comfortable relationship with her over time, or if you have purchased a directory/sponsor/whatever link somewhere – those are the ones you should be asking to have updated so they point to the new URL.
7. You may have missed a few. I always do. The best way to find them is by looking at where the sources of 404s are coming from. If someone is linking to an old URL that isn’t redirecting properly (maybe it had some weird query string or escaped characters like %20, etc…) you will see this in the 404 report either in your log files or in Webmaster Tools. Have them fix the link directly if you can, and get your developers to make sure these URLs are redirecting too.
To sum up how I approach a major site taxonomy change like this: Gather data, implement 301 redirects, resubmit old xml sitemap, submit new xml sitemap, update internal links, update external links, monitor 404 errors and rankings.
