Last Updated January 10, 2018
It’s the middle of the night and you’re staring at your PC probing for bugs. Since you’re using Google Chrome, you hit F12 and pull up Chrome DevTools to help with your mobile website debugging. Up pops a sweet suite of developer tools, which includes a mobile testing emulator that lets you imitate the properties of any phone or tablet under the sun. It’s Google so it must be completely accurate, right?
Now here’s the news you didn’t want to hear: Chrome’s mobile emulator often gets it wrong, delivering a number of unwelcome surprises. And these surprising mobile testing results are a no-fun basket of “Easter eggs.” Instead, they could seriously mislead your team about your website’s performance on mobile, just when you’re trying to engage and convert customers.
Lurking Dangers in Chrome DevTools for Mobile
Chrome’s developer tools offer an emulator, not an actual phone or tablet, to simulate your website on any given mobile device. This emulation is based solely on changing the screen resolution to match the dimensions of the device. Consequently, Chrome’s display of your website on mobile may be completely inaccurate. That’s right, Google may be getting it totally wrong!
Chrome’s departure from actual phones and tablets may result in the dreaded “false positive”: you think your newly designed website is bug-free on the latest iPhone 6s plus and ready to go live because Chrome’s device mode emulator shows you it is. But serious bugs infecting your code or poor optimization for mobile may be delivering a poor UX, painfully slow load speeds, and decreased conversion rates. Yes, you are soaring with confidence over your website’s sure-to-be successful launch, even while Chrome’s inaccuracies are setting you, your team, and your brand up for a fall.
Why Chrome DevTools’ Mobile Emulator Delivers Inaccurate Results
DevTools is an example of an emulator, which is an application that imitates the operating properties of mobile device hardware and operating systems and displays the results on your desktop. Sounds perfect, right?
But since emulators do not replicate the software or the hardware of the actual physical mobile devices, they have serious limitations, as Google acknowledges. Simply put, these limitations result in serious testing errors.
Google even admits their emulator fails to accurately duplicate the behavior of the GPU and CPU in the mobile device hardware. But here are two additional big deficiencies in the Chrome mobile emulator that Google doesn’t mention:
- Chrome mobile emulator uses a different rendering engine than either Safari on iOS or the native Android browser. The rendering engine is what takes some HTML and turns it into a web page that you can see and interact with on the screen.
- Chrome mobile emulator uses a different JavaScript engine from Safari in particular. Chrome uses V8 while Safari runs JavaScriptCore/Nitro.
Chrome’s mobile emulator deviates significantly from components of actual phones and tablets. In fact, Chrome’s web dev testing tools not only employ different rendering engines and different JavaScript engines, but also different hardware — arguably everything that’s important in how a web page actually renders.
Finally, additional inaccuracies are introduced by Chrome DevTools, since they often use the latest OS for each device, but that differs from consumers’ use in the real world. For example, Android 4.x is still the most popular and most used OS, operating in more than 50% of Android devices, still trumping the later 5.x and 6.x combined.
Chrome Mobile Device Mode vs. Mobilizer
Round 1: Taco Bell on the iPhone 5c
Let’s look at actual examples of mistaken mobile test results from Chrome’s F12 and why their mobile emulator failed to perform with 100% accuracy.
Chrome Device Mode | Mobilizer Mobile Testing Lab |
On the left, Chrome DevTools shows a perfectly rendered mobile site. On the right, the Mobilizer test result from a real iPhone 5 located in the Mobilizer mobile device lab reveals that the background imagery on the Taco Bell website fails to appear, leaving only black.
The Chrome emulator is imitating the properties of the wrong OS and thus fails to expose the glitch. Had Taco Bell’s marketing, development and QA teams relied solely on Chrome mobile emulator, they would not have seen the error and would not have known that their customers were unable to view their delicious burrito background.
This error is remarkably common. Older operating systems account for more than 50% of traffic on Android devices and 5% of traffic on Apple devices. So, if you are using Chrome DevTools, you are likely failing to spot this issue with your website’s mobile performance and thereby delivering a poor UX to a significant share of your mobile traffic.
Chrome F12 Mobile Device Mode vs Mobilizer
Round 2: Paypal’s Snags on the iPad Mini
Chrome Device Mode | Mobilizer Mobile Testing Lab |
Chrome DevTools (on the left) makes it seem like PayPal’s site displays great on the iPad mini. However, as Mobilizer shows on an actual iPad mini, PayPal’s text is badly misaligned, with parts chopped off and obscuring the top navigation links. The background image is displayed much more zoomed out than Chrome’s mobile emulator leads you to believe, and the misalignment of the call to action button blocks out the women, so the customer can’t see how she is happily completing her PayPal payment. Chrome’s mobile emulator shows a perfectly laid-out page, misleading PayPal’s developers, designers, QA and marketing teams.
Chrome F12 Mobile Device Mode vs Mobilizer
Round 3: Ducati on Samsung’s Galaxy S6
Chrome Device Mode | Mobilizer Mobile Testing Lab |
Similar to the issues Chrome F12 has with iOS devices, on Samsung’s Galaxy S6, an Android device, critical errors can also occur. F12 misses glitches on the Ducati site, as shown on a real device (on the right) in Mobilizer, including the top image that is badly cropped, decapitating key messaging along with the motorcyclist’s head – ouch!
Chrome F12 Mobile Device Mode vs Mobilizer
Round 4: Orbit Gum on iPhone 5c
Chrome Device Mode | Mobilizer Mobile Testing Lab |
Chrome F12 tools can claim snafus exist when none occur on the actual mobile devices. For example, the Orbit Gum’s website appears flawlessly on the iPhone 5c, but Chrome’s mobile emulator thinks there are a plethora of bugs that cause the page to barely render.
Chrome Mobile Device Mode vs Mobilizer
Round 5: Cloudera on the iPad mini
Chrome Device Mode | Mobilizer Mobile Testing Lab |
On the left, Chrome devtools depicts Cloudera’s website as perfectly rendering on the iPad. But in fact, on an actual iPad mini the background image does not appear — as tests in Mobilizer’s device lab demonstrate. Those in charge of Cloudera’s website may not know their mobile customers aren’t receiving their full message of “togetherness,” which should be (but isn’t) emphasized by the imagery of the rowing team working in perfect unity.
Although the issue visually appears to be the same as the issues in Taco Bell’s mobile display, the hidden problem is actually quite different. Because Chrome devtools employs a different rendering engine, it doesn’t display the missing background image. Specifically, Chrome’s mobile emulator on the desktop is parsing and displaying results different than the actual browser on the iPad tablet.
A Comparison: Mobilizer vs. Chrome DevTools for Mobile
Let’s consider an alternative: Mobilizer’s online mobile testing platform, which delivers fast screenshots of your website on actual market-leading phones and tablets. Both Chrome device mode and Mobilizer offer distinct advantages for marketing teams. This chart highlights how the two tools offer complementary functionality to ensure easy, bug-free mobile development and a glitch-free, superlative user experience.
Advantages and Utility
Mobilizer | Chrome DevTools (F12) |
● Renders web pages on actual market-leading devices, ensuring 100% accuracy ● Tests across device OS versions ● Side-by-side display of devices enables fast identification of errors ● Simple, intuitive interface makes it easy for the whole marketing team, not just the developers ● Mobile-focused analytics can reveal performance issues and suggest where to focus your team’s time ● Enables workflow collaboration with easy-to-share mobile screenshots, reports, and Jira tickets ● Automates delivery of renders of web pages and proactive notification of issues ● Chrome Extension app for fast mobile testing of web pages | ● Emulates any device resolution, but only the viewport size is guaranteed to be accurate ● Allows for interactivity ● Examines code segments in detail and their display ● Offers a wide variety of tools that are useful to developers in building web applications ● Built into Chrome ● Free |
When an Emulator Just Won’t Do…
Chrome’s web developer tools are an excellent addition to the digital marketer’s toolset. But just as every tool has its special purpose for which it is perfectly adapted, so too the Chrome emulator best accomplishes only certain tasks. It can accelerate mobile web development by displaying your web page on any device next to its code. But development and QA for the mobile web are dangerously incomplete if you deploy only an emulator.
Before your website is set to go live and be seen by thousands, if not millions, of prospects and customers, you want 100% accurate testing and easy, fast inspection. You need to test your crafted new page design on the real thing, whether an online mobile lab or a phone sitting in the palm of your hand, not a simulated imitation.
- Can you trust Chrome’s Emulator for your responsive design testing? - January 26, 2018
- Mobile Emulators – The Very Real, Unhappy Truth - January 25, 2018
- 8 Reasons Why You Need Responsive Design Testing - January 17, 2018