WMF Unified mobile routing 2025

Unifying our mobile and desktop domains – [[WM:TECHBLOG]]

by

How we achieved 20% faster mobile response times, improved SEO, and reduced infrastructure load.

How we achieved 20% faster mobile response times, improved SEO, and reduced infrastructure load.

Until now, when you visited a wiki (like en.wikipedia.org), the server responded in one of two ways: redirecting to a desktop page, or the equivalent mobile URL (e.g. en.m.wikipedia.orgThis mobile URL, in turn, served the mobile version of MediaWiki’s page. Our servers have been working this way since 2011, when we deployed MobileFrontend.

Before: Wikimedia CDN responds to requests from mobile clients with a redirect from en.wikipedia.org to en.m.wikipedia.org, and en.m.wikipedia.org then responds with mobile HTML. After: Wikimedia CDN reacts directly with mobile HTML.
Diagram of technological change.

Over the past two months we have integrated mobile and desktop domains for all wikis (Timeline). This means we no longer redirect mobile users to a different domain during page load.

We completed the change on Wednesday, October 8, after deploying it to the English Wikipedia. The mobile domains were disabled within 24 hours, which confirms that the majority of mobile traffic came to Wikipedia through the standard domain and experienced redirects thus far.[1][2]

Why?

Why did we have a separate mobile domain? And, why did we believe that changing it might benefit us?

The year is 2008 and all types of websites, big and small, have a mobile subdomain. The coveted m-dot domain has appeared on the BBC, IMDb, Facebook and in newspapers around the world. For Wikipedia, a separate mobile domain made launching a mobile experiment less risky and avoided technical limitations. This became the default via redirect in 2011.

Fast forward seventeen years and a lot has changed. It is no longer common for websites to have M-dot domains. Its use of Wikipedia is surprising to our current audience, and may reduce the perceived strength of domain branding. The technical limitations we had in 2008 have long been resolved, with the Wikimedia CDN having efficient and well-tested support for variable responses under a single URL. And on top of that, we had reason to believe that Google stopped supporting individual mobile domains, which prompted the project to be abandoned at the same time it started.

You can find detailed history and engineering analysis in the Mobile Domain Sunsetting RFC, along with weekly updates on Mediawiki.org.

site speed

Google used to link mobile search results directly to our mobile domain, but it stopped last year. This exposed a larger portion of our audience to mobile redirects and reduced mobile response times by 10-20%.[2]

Google supported mobile domains in 2008 by letting you advertise a separate mobile URL. While Google only indexed the desktop site for content, they stored this mobile URL and linked to it when searching from a mobile device.[3] This allowed Google referrals to redirect.

Google introduced a new crawler in 2016 and gradually re-indexed the Internet with it.[4-7] This new “mobile-first” crawler works like a mobile device rather than a desktop device, and removes the ability to advertise a separate mobile or desktop link. Now that’s a link for everyone! Wikipedia.org was one of the last sites that Google switched to as part of a clear change window in May 2024.[2] This means that 60% of incoming pageviews referred by Google now have to wait for the same redirect that the other 40% of referrals have experienced since 2011.[8]

mobile domain sunsetting fawiki responsestart 2025 09
Persian Wikipedia saw a quarter second reduction in the “responsestart” metric from 1.0s to 0.75s.

Integrating our domains eliminated the redirect and resulted in 20% improvement in mobile response time,[2] This recovery is both a recovery And A net-improvement because it applies to everyone! This recaptures the regression that Google-referred traffic began experiencing last year, but also improves response times for all other traffic by the same amount.

The graphs below show how the change was felt around the world. “Worldwide P50” is tailored to your experience in Germany or Italy, with fast connectivity close to our data centres. “Worldwide p80” is similar to what you might experience when browsing Persian Wikipedia in Iran.

Wordwide P80 moved back 11% from 0.63 to 0.70, then dropped 18% from 0.73 to 0.60. Wordwide p75 went back 13% to 0.61s, then dropped 19% to 0.52s. Wordwide p50 went back 22% to 0.33 seconds, then dropped 21% to 0.27 seconds. Full table in the linked comment on Fabricator.
Check the perf report to find out the underlying data and other fields.

SEO

The first site to be affected was not Wikipedia but Commons. Wikimedia Commons is a free media repository used by Wikipedia and its sister projects. Tim Starling found in June that only half of the 140 million pages on Commons were known to Google.[9] And 20 million of these known pages were also removed due to mobile redirects. It was increasing by one million unlisted pages every month.[10] The reason for delisting turned out to be mobile redirect. You see, the new Google crawler will have to follow mobile redirects, just like your browser does.

After following the redirect, the crawler reads our page metadata which points back to the standard domain as the preferred domain. This creates a loop that can prevent a page from being updated or listed in Google Search. Delisting is not a matter of ranking, but whether a page is even in the search index.

Tim and I disabled mobile redirects for “Googlebot on Commons” via an emergency intervention on June 23. After this the referrals started coming back and continued to increase for eleven weeks, until reaching a 100% increase in Google referralsFrom a baseline of 3 million weekly pageviews to 6 million, Google’s data on clickthroughs shows a similar increase from 1M to 1,8M “clicks”,[9]

Wikimedia Commons pageviews whose type is equal to user (i.e. not a known bot or spider), and referrer is equal to Google. After July 2025, it increases from 3 million to 6 million per week.
Google-referred pageviews in 2025.
A steady 1.0 million clicks per week in June and early July, then increased to 1.8 million clicks per week in mid-July and remained there.
Weekly Clicks (according to Google Search Console).

We reversed last year’s decline And Set a new all-time high. There are three reasons we believe Commons will reach new heights:

  1. Redirects consume half of the crawl budget, thus limiting how many pages can be crawled.[10][11]
  2. Google turned Commons into its new crawler a few years before Wikipedia.[12] The index was probably already shrinking two years ago.
  3. Pages on Commons have a sparse link graph. Wikipedia has a rich network of links between articles, whereas pages on Commons represent a photo with an image description that rarely links to other files. This unique page structure makes it difficult to find Commons pages through recursive crawling without a sitemap.

Integrating our domains removed a limitation we didn’t know existed!

The MediaWiki software has a built-in sitemap generator, but we disabled it on Wikimedia sites a decade ago.[13] We decided to enable it for Commons and submitted it to Google on August 6th.[14][15] Google indexed 70 million new pages for Commons, up 140% from June.[9]

We also found that less than 0.1% of videos on Commons were recognized by Google as video watch pages (Google search for “video” tab). I brought this up at a partnership meeting with Google Search, and it may have been a bug on their part. Commons began appearing in Google Video a week later.[16][17]

Link Sharing UX

When sharing links from mobile devices, such links previously had the mobile domain hardcoded. Links shared from a mobile device provide you with the same access to the mobile site as you would on a desktop. The “Desktop” link in the footer of the mobile site points to the standard domain and disables the standard-to-mobile redirect for you, assuming you reached the mobile site via a redirect. The “Desktop” link didn’t remember your preferences on the mobile domain, and there was no equivalent mobile-to-standard redirect for when you got there. This means that a shared mobile link always renders the mobile site, even after opting out on desktop.

Now everyone shares the same domain which naturally shows the appropriate version.

There is a long chain of static referrals from news articles, research papers, blogs, talk pages and mailing lists that reference the mobile domain. We plan to support it indefinitely. To limit operational complexity, we now serve these through a simple whole-domain redirect. This has the advantage of fixing a UX problem retroactively because Old mobile links now redirect to standard domain,[18]

It addresses a long-standing bug with a solution in the form of a shared user script,[19] browser extension,[20] and individual scripts.[24]

infrastructure load

After publishing an edit, MediaWiki instructs the Wikimedia CDN to clear (“purge”) the cache of affected articles. It has always been a concern from the SRE teams at WMF that our CDN purification rates are not sustainable. For each purge from MediaWiki core, the MobileFrontend extension will add a copy for the mobile domain.

wikimedia cdn purge rate in 2025
Daily purification workload.

After unifying our domains we stopped these duplicate purges, and cut the MediaWiki purge rate by 50%. Wikimedia CDN processed approx. in last weeks 4 billion less cleanses a dayMediaWiki used to send purges at a baseline rate of 40K/sec with increases up to 300K/sec, and both have been halved, In keeping with other services, Wikimedia CDN now receives 20% to 40% fewer purges per second overall, depending on editing activity,[18]

  1. T403510: Main rollout, Wikimedia Fabricator.
  2. T405429: Detailed traffic statistics and performance reports, Wikimedia Fabricator.
  3. Running desktop and mobile versions of your site (2009), developers.google.com.
  4. Mobile-First Indexing (2016), developers.google.com.
  5. Google makes mobile-first indexing default for new domains (2019), TechCrunch.
  6. Mobile-first indexing is here (2023), developers.google.com.
  7. Mobile Indexing vLast Updated (June 2024), developers.google.com.
  8. Mobile Domain Sunsetting RFC § Footnote: Wikimedia pageviews (February 2025), MediaWiki.org.
  9. T400022: Commons SEO Review, Wikimedia Fabricator.
  10. T54647: Image pages are not indexed by Google, Wikimedia Fabricator.
  11. Crawl Budget Management for Large Sites, developers.google.com.
  12. I have no idea when Google switched Commons to their new crawler. I indicated May 2024 as the switch date for Wikipedia based on new redirects affecting page load times (i.e. a non-zero fetch delay). For Commons, this fetch delay was already non-zero since at least 2018. This suggests that Google’s older crawler linked mobile users to the Commons canonical domain, unlike Wikipedia which linked to the mobile domain until last year. Raw full data: P73601.
  13. History of sitemaps on Wikimedia, by Tim Starling, wikitech.wikimedia.org.
  14. T396684: Develop sitemap API for MediaWiki
  15. T400023: Deploy Sitemap API to Commons
  16. T396168: Video pages are not indexed by Google, Wikimedia Fabricator.
  17. Google video search results for commons.wikimedia.org.
  18. T405931: Cleanup and redirects, Wikimedia Fabricator.
  19. Wikipedia:User scripts/list at en.wikipedia.org. These include NeverUseMobileVersion, AutomobileRedirect, and UnmobilePlus.
  20. Redirect (10,000 users), Chrome Web Store.
  21. How can I force my desktop browser to never use mobile Wikipedia (2018), StackOverflow.
  22. Mobile Wikipedia (726 users), Skip Firefox add-on.
  23. Search for “mobile Wikipedia”, Firefox add-ons.
  24. Mobile domain sunset 2025 announcement § Individual script workarounds (September 2025), MediaWiki.org.

About this post

Featured image by PierreSelim, CC BY 3.0, via Wikimedia Commons.



<a href

Leave a Comment