When testing software, it’s important to realize that the program itself will need to run on more than just one system, and not just a single type of system either. The program will need to interact with countless variations of hardware and software, so compatibility is always going to be an issue. As a software tester, it’s important to cover as many hardware/software combinations as possible in order to catch many of the compatibility issues inherent in such a fast paced, market driven environment while at the same time staying practical about how much time and money is spent testing any one particular platform.
Today’s electronic market is flooded with multiple brands, with multiple chip sets, running multiple versions of multiple operating systems. New hardware is being produced at a breakneck pace and with it comes planned and unplanned incompatibilities with the current generation of software. On the other side of the coin, software is being created at an equally staggering rate with an ease never seen before. Patches are often being deployed within minutes of changes being developed and routine maintenance updates are a common occurrence, sometimes happening without the user even noticing or being informed.
Device manufacturers are businesses that need to make money first and foremost. In order to better accomplish this they may prioritize cheaper parts made by several different vendors as opposed to buying everything from a single brand name. This means the smart phone in your pocket or the desktop computer in your office could have every single part made by a different company. As much as we’d love to have every brand work interchangeably with each other, it’s just not always the case and hardware compatibility issues crop up that can negatively affect your software. Device manufacturers will also periodically reevaluate their build processes and exchange pieces in the middle of a device’s life cycle if supply of parts becomes scarce or prices unreasonable. It’s not very unusual for two devices bought a year apart to have a few of the components changed out, even though they still have the same model name. This is of course assuming the device is even marketed for over a year because today’s technology advances are moving so quickly that manufacturers are releasing multiple models every year just to keep from falling behind and losing their share of the market.
Software development is easier than ever. With the integration of advanced development tools, pre-written code libraries, and easy-to-use programming languages, school children are able to create simple pieces of software as homework assignments. Don’t misunderstand. Good software meant for the consumer still takes time and effort to create, but the ease of getting software ready for the market means that there are far more competitors than before. There used to be nothing but Microsoft vs Apple, but now there are dozens of different operating systems out there and countless utility programs. Device manufacturers usually take special efforts to ensure whatever software they put on their cobbled together devices are at least compatible together, but they make no such promises when other developers are trying to release their own apps and custom software on as many different platforms as possible.
COVERING ALL THE BASES
To be blunt, there are far too many hardware/software configurations to be able test to everything, so software testing becomes a delicate balancing act of verifying that the product functions as intended in as many relevant environments as possible while staying practical at the same time. It makes little sense to test on a 10 year old platform that isn’t even supported by the manufacturers anymore, nor is it advisable to devote large amounts of money testing on devices that make up a small market share.
And so, with all that in mind, it’s important to prioritize your testing to start with the current generation of devices running the current versions of their respective operating systems. After that you grab the same devices and go back a couple version of operating systems. You do this because not all users will update when a new version is released. That right there should cover a huge percentage of the market. From there it’s going to take a little market research to find out which models are the most popular, which chip sets are shared between multiple brands, what the operating system adoption rates are by version number, etc. From there you can make educated decisions about which devices and operating systems the product needs to be tested on.
Software testing, like any area of software development, can get expensive if you don’t understand the how’s and why’s of certain processes. With so many hardware/software configurations in the world it quickly becomes important to prioritize your time and money on the popular configurations that potentially affect the most customers. Failing to do so can quickly lead to high costs and most importantly, under-tested and potentially buggy software getting into the hands of the consumer.
By Alan Haugen