Post

For their peers in the space, retention is going up. But for themselves, it's going down. That's bad, so let's fix it. (Also yes the Y axis marker is missing. He cannot share hard data, but the gist is clear)

So let's find out: Legacy UX, do you rebuild or repair, and what is the lost knowledge of the original onboarding of users? Why was it done that way?

Starting off: Lots of primary research of players. Which creates a lot of data, and it was hard work to organize it all.

Data clearly showed issues on day 2, 3, and 4. Some kind of issue must be present there. What could it be?

Turns out users were grinding scenes without rewards. Progression was too slow. Users were unsure how to unlock new scenes.

For secondary research, playtests were held with the team. This is valuable, but solutions are often thought of too quickly. There is a risk of confirmation bias, and to not solve the actual underlying problems.

It still created valuable data though. You spot things you may not have otherwise spotted, also because you know what to look for from the initial user data.

Example: Even a screen with two buttons can be confusing for a new player. What does a blue button mean? What is a green button? Is the left button the most important button? Even the basics are hard!

Goal: Increase retention by 1%

They made the start screen just one button. Moved connection button to top right, much smaller. This improved retention by 2.5%

They added a hint button to help players. This reduced retention by 2.2%. Players would just hit the button all the time, and the game became a chore instead of playing an actual game and having to think.

They added an overlay on how to play the game. This increased retention by 1.8%

So did it work? Yes! But not 1% total. They hit 0.81%

Key takeaways: Keep it simple. User stories can save your sanity. Always include your team. Build your research process. So that when you need to apply research, you can do so right away.

(I missed the last two talks as I had a long and good discussion with someone about hyped technology systems and confirmation bias. I'll link some things here that I think are important to relate: 1. mikelovesrobots.substack.com/p/wheres-the... 2. metr.org/blog/2025-07...

(3. @edzitron.com's work: www.wheresyoured.at/ai-is-a-mone... 4. The video I made recently in relation to blockchain, metaverse, and LLM AI in gamedev: youtu.be/CeZ_rcDQg6Q?... The earlier talk today that mentioned confirmation bias made me realize this is all related to those hypes.)

Next, another talk about World of Warships. How do you measure player satisfaction? How do you make sense of it?

They have 4 facets of satisfaction: Early, immediate, in-depth, and long-term. Each has different data that is important depending on the questions they need to answer.

By having this data, they can readily see if released content is too difficulty or too drastic. Gathering the data always means you have a baseline to compare to. So questions can be answered right away. (Reminding me of a previous talk, of having the user research systems always ready to go)

Same with data to identify a satisfaction drop after a feature launch, and a 'trinket' problem. In this case by adding a question to the system right after a match. So players would get a single question, and that gave them enough data to infer impact from when they launched content.

Through the combination of all these elements, the entire 'ship' of user research can be built. If one element were missing, it would not float. Having all elements allows them to have the views they need to make accurate decisions.

Q: Do you ever have a hard time relating data to a particular change? A: (very long pause) ...yeah.

Next talk: Demystifying Sensitivity Reading to make games better (not 'just' inclusive)