Sky Squadron Mission Report #4 – Player Progression

Great to see you alive, Pilots! Your efforts have help the boffins make excellent advances in adding rewards and progression to Sky Squadron.

As a reminder, here is what changed in the last build:

  • A brand new mission system with new objectives
  • XP rewards and levelling up
  • Complete for infamy on the global leaderboard
  • Variable mission difficulty
  • A deadly new enemy attack – homing missiles
  • Floaty numbers!

The short version:

Players have been completing missions and levelling up. While we didn’t see as many people reach max level as we would have liked there was still enough to show us that players will level up, unlock new items, and use them. While people were keen to engage with levelling up, very few people bothered investigating the “Infamy” global leaderboard.

We saw plenty of people playing the new missions and some very positive feedback. Most interestingly we saw that by simply changing the kind of objective for a mission type, we can drastically alter the difficulty.

The long version:

Player Progression

We added a number of systems to give players medium and long term goals to aim for. The hope here was we would see people engaging with the new systems, and increase the number of people that hang around and play.

The targets were:
50% of players will reach level 3
20% of players will reach level 5 (max level)

After running the build for a couple of weeks, we collected these results:

Level 1 (Blue), Level 2 (Green), Level 3 (Yellow), Level 4 (Red)

Unfortunately we didn’t hit our targets, but we still had some solid engagement with levelling up. We only saw about 25% of people reaching level 3, and no people reaching level 5. While we haven’t met our goals it’s still a promising to see there are some people who are playing through most of the content.

We also added rewards for levelling up, and were pleased to see people using them. We had a new plane skin unlocked at level 2, 4 and 5. Looking at the skins people equipped, we can see the engaged with the level 2 skin unlock.

Plane Skin Customisation
Blue: Free Skin 1
Green: Free Skin 2
Yellow: Free Skin 3
Red: Level 2 Unlock Skin

While checking some other metrics, we actually had some massive improvements in other areas. We saw a 50% bump in the number of people that come back after one day, moving to ~20% Day 1 retention, and actually had people coming back after a week with a Day 7 retention of ~9%. We also saw people doubling their average session length to ~2 minutes.

The final progression system we added was “Infamy”. Infamy is effectively your cumulative score over the lifetime of the game, and is ranked on a global leaderboard.

Infamy Leaderboard

While we saw plenty of people using the leaderboard (it happens automatically), very few people viewed it. In fact, we only had one person open the Infamy leaderboard. There could be several reasons for this, but we can at least say that a global ranking doesn’t seem to be a primary progression motivator.

Take Away:

  • While we didn’t hit our targets, we still saw some good engagement with levelling up
    The current progression system does not drive people through the existing content. That may be because there isn’t enough of an incentive.
  • We may need to be clearer about why people want to level up
  • Don’t expect to get this perfect first go, but good to see there is some engagement. It will need more work.
  • While we didn’t get any level 4 or level 5 unlock skins used, we did see people who unlocked a skin at level 2 use it, but that was only ~25% of people who reached it
  • Only one person opened the leaderboard, so the infamy really needs some work. It’s not tied to any progression at the moment, but we may find that people just don’t care about global rankings.

Mission Engagement

A major feature we added in this update was mission objectives. While the missions have a long way to go, we wanted to test a few simple mission types and see if people like them. For our changes to be successful at this we wanted to see a positive feedback in our overall mission rating, with a value of at least 3 out of 5. With a final result was 4.17 out of 5 we were happy to see people enjoying the new system.

We collected a variety of different metrics about each mission, and which people played the most.

Blue: Collect Target
Green: Collect Tuna
Yellow: Destroy Target
Red: Destroy Turrets
Purple: Kill Enemies
Gold: Survive

Missions seemed to be evenly rated across the board, so we had no real standout popular or unpopular mission type. What is interesting is looking at the number of mission fails comparing “Collect Target” to “Collect Tuna”. These missions are both the same archetype, but with different collectables being spawned. They shouldn’t be that different.


Same mission type, different fail rates

After taking a closer look at the “Purple Ball” vs the “Tuna”, we noticed that the collider on the ball was about 75% the size of the tuna. While not a massive different, it seems to make a significant impact on the mission. The ball is also visibly smaller, and more abstract. It could also be that people were simply confused by the objectives.

The ball mission is only placeholder and won’t ship in the final game, but it was interesting to see the impact that changing the properties of the collectable can have such a huge impact on the difficulty of the mission.

Take Away:

  • People rated each mission pretty equally (all > 4)
  • There was a preference towards the “Collect Target” mission, but it also had the most fails. Maybe the purple ball is too hard to see (placeholder anyway)
  • Collect tuna was the most completed mission
  • Survive mission type is too easy

Player Feedback

We had less written feedback for this build, but it was generally positive. The one standout request was for additional flying and shooting controls. This is something we may add in the future, but for now we’re seeing how far we can push simple one-touch controls and keep the experience feeling engaging.

Summary

Even it a very simple and rough form we’ve seen players levelling up, unlocking new skins, and playing plenty of missions. While we haven’t seen many players move through to the max level yet, it’s also a very early version. We now have a baseline to work towards improving.

Next update we will be giving missions some more love, and working on the procedural generation and difficulty scaling.

Interested in joining our squadron of Test Pilots? Sign up here: http://bit.ly/skysquadronsignup

Leave a Reply

Your email address will not be published. Required fields are marked *