The Publishing Project

I came across this question in Quora and prepared a detailed answer but the more I think about it the more I think it deserves a more elaborate answer because it starts from a false premise, that you must test in all versions of all the browsers available to them.

What is the fastest way to test a website on all browsers and browser version?

Quora

Now with context out of the way onto the answer

---

There are a few things to do before even considering what the fastest way to test is.

The first one is to acknowledge that it’s impossible to test across all browsers, versions and in all devices, your target audiences are likely to be using

The next aspect of this question is to identify your who’s your target audience? Are they browsing on desktop or mobile? what devices are they using to test(emerging markets tend to go for cheap and underpowered devices and this will affect performance during testing and in the real world)? Do you have access to the devices your target audience is likely to be using?

After you’ve answered these questions you’ll realize that, while it’s nice to do so, you can’t to test in all browsers, only the most likely ones for your market and audience.

Tools like StatCounter can help you research your target browsers to test with based on country, OS, browsers and other parameters.

Next is to consider the differences between mobile and desktop systems. An earlier answer indicated that all modern browsers worked the same so you could get away with testing in Chrome and Firefox. This is simply not true. The differences in performance between desktop and mobile processor performance can have a huge impact on how your application responds.

There are also Proxy Browsers like Opera Mini, Puffin and UC Browser that do server side rendering with only partial support for CSS and JS (Opera Mini servers will run your JS payload for 5 seconds and then stop). How will your site look on those browsers? Does your app work at all?

You might say that it doesn’t matter but it does… you never know where your users are coming from and if you’re targeting emerging markets proxy browsers are widely used to save money

The other part of this is that you’ll want a way to automate the tests so you don’t have to manually run tests on all your platforms.

Tools like Browserstack or Saucelabs can help you with the actual testing but to organize large scale test for all versions of all browsers is not only expensive but also unnecessary if you do your user analysis up front and refine it as your site gets traffic

It’s also important to point out that unless you build your own device lab with older hardware that supports the older browser you’ll never get all versions of all browsers (As far as I know most testing platforms will only go as far back as IE6 but not further than that… and even IE 6, 7, 8 and 9 are unsupported by Microsoft).

I know we’ve jumped all over the place but I want to distill some final thoughts

Put the user first

Seamless is a food delivery service that’s incredibly popular on the East Coast. In 2013, they merged with GrubHub, and recently, the two companies combined their backends and released a refreshed user interface.

The reaction to these updates was really strong. It was a total “who moved my cheese” scenario, as customers had gotten used to the application’s flow, regardless of its inefficiencies. Customers had already spent the mental energy to get from Point A (logging in, searching, placing order), to Point B (purchasing food), and it worked for them despite how counterintuitive it may have seemed.

We need to remember that familiarity is a more powerful concept than intuition. While we should always strive to make easy-to-use software, we need to remember that this doesn’t serve the people who already know how to use the software as it exists today.

From: No Flex Zone: Empathy Driven Development

It easier for developers and designers to assume we know what’s best for our users but that’s not always the case. Even when the changes make sense you should still test them, maybe A-B testing or some sort of work to make sure that the changes you’re making are necessary for the users and not just changing for change’s sake, no matter how important you think the changes are.

Accessibility is important

I’ll hammer this point again. Accessibility should be an essential component of any design and development process. And we should be working on accessibility accommodations for visually impaired or hearing impaired users. More accessibility issues exist than those we normally pay attention to when working with accessibility.

We may need to accessibility features ourselves for short periods of time. Imagine that you have an accident or surgery that will change the way(s) how you access your content, even if it’s temporary.

Links, resources, and further ideas

• Character encodings: Essential concepts
• ASCII
• A Closer Look At Personas: What They Are And How They Work (Part 1)
• A Closer Look At Personas: A Guide To Developing The Right Ones (Part 2)
• Informing Your Empathy for More Human Designs and Communities
• Why Empathy is the Secret Sauce for Good Software Development
• EMPATHY IS THE NEW KILLER APP
• No Flex Zone: Empathy Driven Development
• Why You Must Use Empathy Effectively in UX Design
• USER EMPATHY IN WEB DESIGN
• Empathy: an essential skill in Software Development
• Why empathy is a crucial skill for web designers
• Building websites and empathy my experience as a frontend design apprentice
• 8 Ways To Become A More Empathic Designer
• Speed up your development process by coding with empathy
• Developing empathy
• Practicing Empathy in Product Design
• Using VoiceOver to Evaluate Web Accessibility
• Using NVDA to Evaluate Web Accessibility

The final empathy area I want to discuss is process and tooling. Who are we doing this for and what tools we’re using to develop

Who?

There are three groups of people we should empathize with: Fellow team members, External developers who use our code and the end users of our product.

Our fellow team members are typically the smallest group (some times it’s just us), but they are most affected by the code we write. Because we share the same context it’s easier for us to write code that they can work with easily. If we miss the mark here they will most definitely grill us with feedback and suggestions for improvement. If it’s just us we can certainly make the changes to improve the project.

External Developers build on top of the code and APIs we create. The typical example is a Javascript framework we create and they consume. We shouldn’t assume that they know as much about our framework as we do. Instead of being having direct access to the Framework’s developers for questions and feedback, this group often has our code and its supporting documentation to solve issues with our code.

The final group affected by our code is our end users. These are the people using applications that developers (internal or external) create. This is usually the largest group of people our code will affect, and they probably not developers and doesn’t share many contexts with us. While we may never interact directly with end users, we need to consider how our code can help the integration of features, including – but not limited to – full accessibility and perceived performance.

So how do we walk in the shoes of each of these groups? I believe that the first part of the answer is communication both internally within the team and with external developers. How we document the code we write (both inline comments in the code and documentation outside the code itself) and the interfaces we provide to our libraries and APIs should be easy to understand and well documented without having to be a Javascript expert.

Tool chains and tools

Another way to make things easier for developers and, eventually, end users is to provide good tooling for the tools you create.

One thing I’ve always loved about tools like NPM, Yeoman, Polymer CLI, Imagemin and other command line tools is that they are very clear in what they do and what they want the user to do to configure the project using the tool. They also give you ways to edit or overwrite the configuration files created in the CLI so you can change the way the tool behaves when it comes to building the application.

Make it as easy as possible for users who are using your tools. Provide documentation and remember that the user of your project may be you in six months and may not remember how you implemented it.

Accessibility is like dirty laundry we all know we should worry about it but we never do until we have no choice (and I’m the first one to admit that I suck at being proactive about addressing accessibility issues in my projects).

Let’s take this hypothetical example:

Your team close to releasing an application that uses the latest and greatest technologies. It’s a progressive web application that uses web components for the front end and Express application to talk to MongoDB and IndexedDB to store data in the user’s browser.

The app works wonderfully and your team gets a lot of kudos from users and management.

Then you get a call from a co-worker that tells you the application is not quite working as expected. After you panic and freak out for a few seconds you ask your coworker what is not working as expected.

What he tells you floors you… the keyboard tab order doesn’t work as expected.

You invite your coworker to your desk and ask him to explain what he means by his comment about tab order. He sits at your computer and starts navigating (or trying to navigate) the app and you realize that it jumps all over the place, not in the order you’d expect it.

The two of you hunker down and figure out that the tabindex attribute wasn’t used properly. After looking over the code you discovered the following issue

• Developers used it in some elements and not others

Further more when reviewing how to use tabindex you discovered that:

• An element with tabindex=”0″, an invalid value or no tabindex attribute should get focus after elements with positive tabindex values in the sequential keyboard navigation order
• Setting tabindex attribute on a <div> element forces us to set tabindex on the div children content, otherwise the content inside the div is ignored when navigating with their keyboard

Your work together in fixing the tabindex issues and pushed the changes. They work and your coworker is happy that the changes work.

The accessibility evaluation firm that you hired to do an audit of your site calls you a few days later and they ask to meet with you and your development team to discuss some of their findings.

The meeting was surprising. They ran pages of your application through a screen reader and your team was amazed at how it sounded. When you look at the report you realize that there are many images with no alt attributes and form fields without labels.

They further suggested developers use a screen reader to work over the site. This would help them understand some of the issues facing vision impaired users.

The actual world

So how do we address accessibility?

In an ideal world, we would have users with disabilities test your site and give you feedback but the range of accessibility issues is too large to expect every aspect of it to be tested.

We don’t realize that the web platform, the technology stack that lives under our frameworks and APIs, does a lot to make the web accessible to people using assistive technologies. When we build our own components or extend the platform it becomes our responsibility to make sure the content remains accessible.

This may cause extra work for you as a developer. For example, you’ll have to add accessibility attributes and things that we take for granted when working on the web. Things like keyboard navigation, focus, and the functionality of custom versions of web elements like buttons, checkboxes and other elements.

The W3C Web Accessibility Initiative provides a set of authoring practices that give you an idea of the amount of work that you need to do to make accessible elements. Some times this is by choice but sometimes you don’t really have an option and need to build a custom version of an existing element or create something completely new for your application.

If you’re using custon elements, the howto-tabs sample element shows how to incorporate labels for customm elements.

If you’re working with React you can check Facebook Accessibility will show you the basics and an example of an accessible accordion will show you a specific example of how to build accessibility on your React elements and components.

An interesting option is to include tools like Axe Core command line or build system plugins as part of your build process. This will help you identify the issues but it’s up to you and your team to fix them. It takes away the excuse that you didn’t know how to do it.

Lighthouse, a tool from Google provides a series of reports that cover performance and accessibility. YOu can run this as command line utility, a Chrome browser extension or a build system plugin

Going back to our central premise of walking in our users’ shoes let’s try using a screen reader for our content. Rob Dodson has created basic tutorials for Voice Over (screen reader built into macOS) and NVDA (a free screen reader for Windows).

Once we start working with accessibility it’s tempting to concentrate in a small area of accessibility and not pay attention to areas that we might not consider accessibility related.

Seren Davies’ presentation from Render 2017 covers some of these nonvisual and temporary conditions. This is important because all our users have suffered or will suffer some form of accessibility impairment and we don’t always take these issues into consideration.

Another thing to do when looking at what our users do on our site is to see where they come from and what device they are browsing with. I used to not care as much about where users were coming from as much as I cared about what browsers they were using.

But then I started looking at the logs and some strange browsers appeared in the logs. Not many and not often enough to warrant making changes but enough to take a look at these funky browsers: Opera Mini and UC browser.

Opera Mini and UC browsers are proxy browsers. Rather than do all the processing in the device they send the URL to a central server to do the rendering and, limited, Javascript and CSS. The server sends a single file containing the server rendered version of your page (without most of your CSS and Javascript) back to the client at a fraction of the size of the individual resources put together. As you can imagine this helps when you access the web with low end devices that will not handle your 2MB of Javascript gracefully.

The reason why this is important is learning about your user. It helps you figure out the combination of geographical location, browser and (mobile) devices for your users. If you’re working on emerging markets or if you have users from countries you were not expecting it may help you to research those browsers.