Pokemon is near and dear, not just to the hearts of 90s kids, but pretty much everyone who ever played video games. Pokemon Go triggered the nostalgia and took smartphone gaming to the next level.
In fact, Pokemon Go is already the biggest US mobile game of all time, with 21 million active users.
Pretty much everything that could be said about the game has already been said — from how it’s just the beginning of augmented reality gaming on mobile, to how its putting lives in danger and need to be stopped. Some even went ahead and debated the ethics of Pokemon Go, and whether or not we want games to have access to user data or own the virtual space around your home.
But what made the most news about Pokemon Go is the issues people faced with it. I myself couldn’t login using my Google credentials because it ‘Failed to connect to the server.’ When I could eventually login, I couldn’t catch a Bulbasaur sitting two feet away from me because it ‘lost GPS connection’. There are a number of posts already available listing down all Pokemon Go problems and suggesting possible solutions. Google Play and iTunes are flooding with apps that try to ‘fix’ Pokemon Go problem and keep your hunt ongoing.
Almost every day of its first week, the Pokemon Go servers have crashed and people have reported numerous other problems with user experience. Thankfully nothing has affected the virality of the game as of now. Niantic, the company behind the game is already learning a lot from every hurdle and has fixed multiple issues already. However, considering that it just launched in a few countries for now, and is set to go worldwide soon, the game makers are in for even bigger and unforeseen challenges.
We are all about software quality at SmartBear. So we decided to list down our suggestions to the game creators and someone who’s trying to learn from Pokemon Go’s obstacles and develop the next viral game!
The CEO and founder of Niantic, John Hanke, is an ex-Googler who worked on Google Earth and Google Maps initial design and development. So let’s say he knows a thing or two about geolocation mapping and it’s potential.
Many players faced problems where the game couldn’t detect GPS location, or in some cases detected wrong locations.
There are multiple problems here:
Crowd-sourced data is phenomenal when it comes to building an augmented reality game, however the data is still user generated and is naturally susceptible to human errors. This post about Pokestops in the strangest places will give you a mere hint of the problem. Unnecessary traffic in some of these locations can raise security and public health and safety concerns. Who should have control over this? Of course the game makers!
This is just one example of user-generated data and its implications, but we all agree that there should be some kind of validation and testing done on user-generated data.
The front-end graphical user interface (GUI) of Pokémon depends not only on the API responses, but also on mobile device hardware/sensor data. For instance, in this case, the location of the player is determined by location-based sensors GPS, which in turn determines the GUI behavior of the game. An integrated GUI and API testing tool can help resolve the first issue. An ability to pinpoint issues at the GUI or the API layer allows GUI testers to cut down on intermediary steps. For instance, on realizing the issue is at the API layer, the GUI tester can reach out to the API developer without going to the GUI developer first. This saves testing time, which is critical in the case of mobile.
At the same time, to create a solid test automation strategy that scales, the focus should be on ensuring optimal behavior at GUI and API layer. This can be done by focusing on a test automation pyramid strategy, which is based on creating more unit tests, followed by API tests, and then fewer UI tests.
The game makers have big plans for this game post the worldwide launch, such as advertising and boosting retail traffic. The game is already creating news around how it is boosting foot traffic for local businesses and can even help sell houses. As this game becomes more important piece in the advertising industry, front end errors or usability issues wouldn’t be taken lightly.
Irrespective of the problem, 9 out of 10 times they blame the app for their bad user experience. When I first got Pokemon Go, it drastically drained the battery on my Android. Even after trying the battery save mode for a few days, I eventually had to delete the game to make my phone usable again. Per Think with Google research, 72% users delete the app permanently after initial bad experience. While only 23% of the consumers give an app a second chance.
While negative out of the box experience slowing down the adoption may seem like a faraway problem for Pokemon Go right now, as the virality of the game calms down, it could be a deciding factor for Pokemon Go’s future.
I strongly think that the game makers are just getting started and there are some amazing things in store for the game makers and us as augmented reality games gain popularity. There are multiple possibilities for implementation and growth across all verticals using augmented reality, and Pokemon Go is doing a great job of giving us a taste of it. While we don’t know what the future of this game is, we hope that Pokemon Go makers are learning from each and every experience and those wishing to follow their success consider our tips for software quality.
Happy catching ‘em all!
In fact, Pokemon Go is already the biggest US mobile game of all time, with 21 million active users.
Pretty much everything that could be said about the game has already been said — from how it’s just the beginning of augmented reality gaming on mobile, to how its putting lives in danger and need to be stopped. Some even went ahead and debated the ethics of Pokemon Go, and whether or not we want games to have access to user data or own the virtual space around your home.
But what made the most news about Pokemon Go is the issues people faced with it. I myself couldn’t login using my Google credentials because it ‘Failed to connect to the server.’ When I could eventually login, I couldn’t catch a Bulbasaur sitting two feet away from me because it ‘lost GPS connection’. There are a number of posts already available listing down all Pokemon Go problems and suggesting possible solutions. Google Play and iTunes are flooding with apps that try to ‘fix’ Pokemon Go problem and keep your hunt ongoing.
But this doesn’t stop us from asking a simple question: why so many problems?
Were the game creators not ready for such overwhelming response? Or did they not test all possible use cases — including geolocations, devices, operating systems, hardware, diverse transaction etc. — before launching?Almost every day of its first week, the Pokemon Go servers have crashed and people have reported numerous other problems with user experience. Thankfully nothing has affected the virality of the game as of now. Niantic, the company behind the game is already learning a lot from every hurdle and has fixed multiple issues already. However, considering that it just launched in a few countries for now, and is set to go worldwide soon, the game makers are in for even bigger and unforeseen challenges.
We are all about software quality at SmartBear. So we decided to list down our suggestions to the game creators and someone who’s trying to learn from Pokemon Go’s obstacles and develop the next viral game!
Lesson #1: You are as strong as your weakest link(s) — testing and monitoring the APIs
Pokemon Go is all about APIs. Right from logging into the app to identifying locations, seeing augmented reality, collecting pokemon, battling in PokeGym or just walking; everything is fueled by APIs that transfer data and requests. Some of those APIs are internally developed by Niantic and some of them are popular public APIs such as Google geolocation APIs.The CEO and founder of Niantic, John Hanke, is an ex-Googler who worked on Google Earth and Google Maps initial design and development. So let’s say he knows a thing or two about geolocation mapping and it’s potential.
Many players faced problems where the game couldn’t detect GPS location, or in some cases detected wrong locations.
There are multiple problems here:
- the availability of the API endpoint
- the speed at which it responds to the request
- the functional correctness of the payload
Lesson #2: Validate user-generated data
Pokemon Go isn’t Niantic’s first rodeo in the augmented reality arena. In early 2011, they had launched a beta version of Ingress, another multiplayer augmented reality game, which let users pick landmarks and historical locations as ‘portal locations’. Ingress gave Niantic a database of over 15 million portal locations which were used in Pokemon Go for gyms and Pokestops. Thousands of Ingress users made sure that you can play Pokemon Go at almost any remote place. In Pokemon Go, users can submit Pokestops.Crowd-sourced data is phenomenal when it comes to building an augmented reality game, however the data is still user generated and is naturally susceptible to human errors. This post about Pokestops in the strangest places will give you a mere hint of the problem. Unnecessary traffic in some of these locations can raise security and public health and safety concerns. Who should have control over this? Of course the game makers!
This is just one example of user-generated data and its implications, but we all agree that there should be some kind of validation and testing done on user-generated data.
Lesson #3: Test and monitor the front end
Pokémon shows the increasing dependency of mobile apps on back-end APIs. In this case, the APIs and GUI are interdependent on each other to collect information. Hence, testing apps both at the API and GUI layer becomes important.The front-end graphical user interface (GUI) of Pokémon depends not only on the API responses, but also on mobile device hardware/sensor data. For instance, in this case, the location of the player is determined by location-based sensors GPS, which in turn determines the GUI behavior of the game. An integrated GUI and API testing tool can help resolve the first issue. An ability to pinpoint issues at the GUI or the API layer allows GUI testers to cut down on intermediary steps. For instance, on realizing the issue is at the API layer, the GUI tester can reach out to the API developer without going to the GUI developer first. This saves testing time, which is critical in the case of mobile.
At the same time, to create a solid test automation strategy that scales, the focus should be on ensuring optimal behavior at GUI and API layer. This can be done by focusing on a test automation pyramid strategy, which is based on creating more unit tests, followed by API tests, and then fewer UI tests.
The game makers have big plans for this game post the worldwide launch, such as advertising and boosting retail traffic. The game is already creating news around how it is boosting foot traffic for local businesses and can even help sell houses. As this game becomes more important piece in the advertising industry, front end errors or usability issues wouldn’t be taken lightly.
Lesson #4: Prepare your app for all possible user scenarios
Just like every other app, gaming or not, no two users are alike. They access the app from different channels, different locations, and different devices but yet expect seamless performance. And they get easily frustrated when the app fails to do what they want it to do.Irrespective of the problem, 9 out of 10 times they blame the app for their bad user experience. When I first got Pokemon Go, it drastically drained the battery on my Android. Even after trying the battery save mode for a few days, I eventually had to delete the game to make my phone usable again. Per Think with Google research, 72% users delete the app permanently after initial bad experience. While only 23% of the consumers give an app a second chance.
While negative out of the box experience slowing down the adoption may seem like a faraway problem for Pokemon Go right now, as the virality of the game calms down, it could be a deciding factor for Pokemon Go’s future.
I strongly think that the game makers are just getting started and there are some amazing things in store for the game makers and us as augmented reality games gain popularity. There are multiple possibilities for implementation and growth across all verticals using augmented reality, and Pokemon Go is doing a great job of giving us a taste of it. While we don’t know what the future of this game is, we hope that Pokemon Go makers are learning from each and every experience and those wishing to follow their success consider our tips for software quality.
Happy catching ‘em all!
No comments:
Post a Comment