Device holder: who it is, why it is needed and how well it holds devices

I am Stas, QA Lead and Device holder at Surf, and no, I don't just hold devices.

We need a whole fleet of devices to cover as many configurations as possible, especially if there is more than one project:

• all current or supported versions of iOS on the project;

• all current or supported versions of Android on the project;

• devices with Google services support;

• devices with Huawei services support;

• a couple of tablets (yes, even in native iOS applications can run on iPad if we have not created a design for this);

• devices with a small diagonal;

• devices with a large diagonal and others.

How to choose the optimal combination for the fleet when exhaustive testing is impossible, and buying all the devices in the world is unrealistic?

You can use simulators or emulators. But these are not real devices: difficulties with geolocation, camera, biometrics. In addition, due to the peculiarities of simulation, not all bugs are reproduced on them.

Since the company needs a fleet of devices, someone has to look after it. For example, we currently have 100 smartphones and 11 tablets.

Here we come to the device holder, who manipulates these devices. But why, you ask, is this not done by the sysadmin? Sysadmins are not always as immersed in specific cases as testers. They have a lot of other work, but QA is well acquainted with business tasks and department tasks.

So let's talk about how to cover all the department's requests with devices and make sure that every QA is confident in having the necessary configuration.

Choosing devices

The first thing we need is global statistics. We use the service gs.statcounter.com.

Here we see the popularity of vendors and select devices that will help see the application through the eyes of more users. Usually, Apple smartphones lead, Samsung is in second place, and Xiaomi closes the top three.

Then we analyze the device market in the region where we release the application.

For example, we consider devices popular in Russia. We understand that the Tecno brand is in the top for us. But it is not in the global statistics. So we will give preference to this brand, rather than Motorola, which, although in 9th place in the global list, is not popular in Russia at all.

We collect statistics on projects. For this, we use connected analytics services, such as AppMetrica. There we track which devices are most often used by users. Based on this information, we buy the necessary ones.

Suppose we already have an impressive fleet of devices — 100 smartphones and 11 tablets (as of fall 2024): iPhone, Samsung, Xiaomi, Oppo, Vivo. Even Tecno is there.

And here’s the thought: “Maybe we’re missing something?” Yes, we are definitely missing something (proven by experience). And here we look at the axes that our project supports.

For example, we bought new iPhones and see that our application supports iOS 14. Here we remember that the SE 1st gen model still works on iOS 14. And there the diagonal is 4 inches — how does the design look on it? And how does the application work, since this is an old device? It turns out, we also need the SE 1st gen.

So, we have the entire set of iPhones and Androids, one smartphone from each vendor. Is that all? Did we buy everything we need? Here we remember that there are also different chipsets.

Choosing a device by chips is our option. Within one vendor, we can take several devices that will work on different chips. For example, if we have Huawei, then it can have both Qualcomm and Kirin. We take one device on different chips — here we look at the statistics again.

What about Samsung? They have Qualcomm and Exynos, so we need 2 devices on these chips again.

In fact, we found specific bugs not only on different chipsets but also on different generations of the same chip. Therefore, it is important not only the manufacturer but also the generations of processors. On one device of the 600 series Snapdragon, we caught crashes that were not reproduced on the 700 series.

It is important to remember this, even when it seems that everything is provided for!

In general, we take devices from different vendors, with different chipsets and generations.

Working with devices in the fleet

You can't just buy a bunch of smartphones and tablets and do nothing with them. They need to be updated, configured, and kept in order.

Device Search
We have a separate chat in case a device is "not at the workplace": developers can take them home or to their office.

To quickly find devices, we optimized the message format. The list of smartphones and tablets is stored in Confluence, so we use the names from this list in the texts. This makes it easier to find in the chat who has booked the desired device.

Updates

Firmware is tied to regions - European, Russian, Japanese, and others. And it often happens that the version of Android we need in one of the regions was released earlier.

You can "reflash" the device, but if you value the warranty or doubt your abilities, it's better not to do it.

How devices live with us

1. We bought charging stations with smart blocks. They cut off the power supply when the device's charge reaches 100%. This extends the life of the batteries (and prevents them from burning out at work).

We have 4 stations with 8 slots each.

2. We use stickers. "Self-adhesive" by hand turned out to be inconvenient - they are difficult to peel off, and they quickly wear out.

Our format is a regular sheet with the printed name of the device and its OS. We stick it on simple adhesive tape. These stickers are easy to read due to the uniform format. And it's easier to change them.

3. We made wallpapers with the name of the device and its OS on each device.

Why, we have already put stickers on? Yes, stickers help not to stand by the table with a hundred devices and spend half an hour looking for the right one.

Wallpapers give more information. For example, in bug reports. You can attach a video from different smartphones, and because of the wallpapers, it will immediately become clear from which device this video is.

To make life even easier, it is worth saving the wallpaper template. This is necessary to quickly change them during updates.

On the wallpapers, we indicate the name of the device and its OS. In the latest ones, we added the diagonal and screen resolution, which also, in the long run, saves time.

4. We actively use models of different colors or cases for them.

For testing, it does not matter what color the device is. But when there is a need to quickly find a device, the different color of the case or cover speeds up the search many times. Especially when there are more than 100 devices in the park.

5. We have created a separate list of devices without sensitive information.

It helps us quickly share with the customer the devices on which his service will be tested.

6. We maintain order and have introduced usage rules.

No, we are not evil overseers who prohibit everyone from using out of spite. We have regulations that specify, for example, when to put devices on charge (for us, if the phone has less than 75% charge).

This way we avoid situations where a tester takes a completely discharged phone and has to wait for it to charge.

How we, device holders, help the company

We help a lot. We and our predecessors have formed a flow of interaction with the park, which is used by QA, designers, developers, and sometimes analysts.

It's great that the guys don't have to think about which version of iOS is installed on that little red iPhone, whether all the necessary applications are on that device, and whether developer mode is enabled on it. They can approach the charging stations, choose the appropriate smartphone or tablet, and start working with it right away.

If there are difficulties with the device or something unusual is required for the project, the team knows who to approach. And we will do everything as quickly as possible: we will jailbreak the iPhone for testing the banking application, and give root rights on Android.

We also save the entire testing department's time.

To charge a phone from zero to the point where it turns on, it takes 7 minutes. There are 37 people in the department, multiply 8 minutes by 37, and then by 5 (working days).

It turns out that about 24.5 hours a week are spent just on charging smartphones. And this time is not something we want to waste.

The role of a device holder does not require 8 working hours every day, there are no "burning releases" here. Therefore, it is possible to properly distribute tasks and manage the workload.

Becoming a device holder is worth it for those who will enjoy it and will not consider it a burden, even if it is rewarded.

We hope that we have helped you figure out which path to choose if you decide to develop your device fleet.

In the meantime, we will continue to work on turning our fleet into a real farm (by the way, we found a lot of useful information about this here).

Cases and best practices in the field of QA, news, vacancies, and internships at Surf — in the Surf QA Lab telegram channel.

Join us!