Sky Squadron Mission Report #5 – Wingmen

At ease, pilots! You’ve done an another amazing job keeping the enemy at bay. Don’t get too full of yourselves, because you never would have done it without the help of your wingman.

Make sure to shout them a beer at the officer’s club!

Last update we saw the introduction of the new wingman system, where AI allies join the mission, assisting where they can, and occasionally needing a helping hand themselves.

What’s changed

  • Wingmen join the squad! Work together to fight back the enemy.
  • Fly the brand new Wisp! Sleek and agile, they’ll never be able to hit you.
  • New plane colours! Watermelon is so hot right now.
  • New mission intro and outro sequence!

The Short Version

The wingman system has integrated well with the rest of the game, but hasn’t caused a significant change in player behaviour. While the wingman system is still early in development it,  will give us the opportunity to add more characters, story and aircraft to the world.

The Long Version

General Performance (and boring numbers)

Every update we have a look at our leading metrics. These values show the health and growth of the game based on real player usage. With a limited sample size we need to take everything with a grain of salt, but it does give some hints at how the game is resonating with players.

New Users: 25 (+11%)
Daily Active Users (DAU): 4 (+24%)
Weekly Active Users (WAU): 14.50 (+20%)
Weekly Stickiness (DAU/WAU):  0.27 (+4%)
D1 Retention: 8.7% (-23%)
Average Session Length: 97s (+2%)
Average Overall Feedback: 3.7/5 (-3%)

The stand out change here is the sharp drop in our Day 1 retention. This says more about the last update than it does the wingman update. Last update we focused on unlocks and player progression, and that seems to have had a much bigger impact on player retention than adding a wingman system.

The player progression system is still in the latest build. It seems that without directing the players to test the system we won’t see a lot of engagement with it.

While we didn’t see any negative impact from the wingman system, it alone was not enough to make a huge impact on player behaviour.

We’ve seen a good increase in the number of users in our test group, which will increase the accuracy of our data. We did see a good bump in our weekly and daily users, but the stickiness (the ratio of daily to weekly) remained largely unchanged, again suggesting that the update didn’t have a huge impact on player behaviour.

Take Away:

  • We’ve had a steady increase in new users, which will keep increasing the accuracy of our data.
  • The wingman system alone was not enough to increase retention.
  • We did not see a huge change in the session length from adding wingmen.
  • The rating dropped significantly from the previous week, but did not move much from the average. This says more about the last update then it does this update.
  • We seem to be making more improvements with meta features than we do with gameplay features.

Wingman System

To measure the impact of the new wingman system, we looked at a few metrics. We wanted to see an overall rating, and also the level of engagement with the wingmen during play.

Our target rating for the wingman system was to receive an average of at least 3.5 out of 5. The average for the wingman update came in at 3.6 out of 5, just meeting our target. This rating puts the wingman system on par with the average rating for our other systems (missions/combat/dodging), so we can say that it’s fitting in nicely with the systems we already have. While not a stand out result, it at least works well with the other parts of the game.

Wingman Events in Mission

Blue: Assist
Green: Rescue
Yellow: Rescue Success
Red: Rescue Failed
Purple: Number of Missions

When looking at how the wingmen were used in the game we can see that they featured heavily as part of the core experience.

There are two kinds of wingman encounters:

  • Assist: The wingman will join you in play and help you shoot enemies
  • Rescue: The wingman is in trouble and needs you to help them

Our criteria for testing the wingman system were to look at how many times players were either assisted by, or saved, wingmen.

The criteria were:

  • At least 50% of people will use a wingman when given the opportunity
    • Assist occurred at least once per mission – so this was a success.
  • At least 50% of people will save a wingman
    • Wingmen had a death rate of 6%, with 1.3 rescues per mission – this was also a success

However in hindsight we can see that the build didn’t cover testing these criteria particularly well. The player was never given the option to call in a wingman, and the difficulty was not tuned to put the wingmen in much danger when requiring a rescue.

Take Away:

  • We met both of our success criteria, but our experiment did not cover the test case very well. We did not end up giving the option for players to summon a wingman, meaning that every player would automatically get a wingman assist.
  • While saving a wingman was not difficult, the low death rate of wingmen show that players did engage with rescue events.

New Colours

The new “Wisp” with it’s juicy watermelon paint scheme

We saw some fancy new plane designs in this build getting closer to the art direction we want in the final game. Even though we didn’t direct players to engage with unlocking skins and selecting them, we still saw people configuring their aircraft.

Number of Colour Selections

Blue: Level 3 Unlock Colour (black)
Green: Level 2 Unlock Colour (purple)
Yellow: Non-default free color (blue)

None of the new players unlocked all of the skins, but a few did play long enough to unlock the level 3 paint schemes.

Written Feedback

We had some more written feedback, which is a fantastic way for us to get insights into the game we can’t get from just watching player behaviour.

What people liked

  • People are still enjoying the core shooting mechanics
  • The wingman system adds a lot to the missions
  • The new plane designs look good
  • The performance is generally smooth

It’s great to see people are enjoying the core shooting experience, it means we’re on the right track. Smooth performance is always a plus, but we know we’ll need to keep an eye on it as we add higher quality art and effects.

What people didn’t like

  • There isn’t much of a story
  • The game feels like it’s aiming for a young audience
  • The wingman was the same colour as the enemy, and that was confusing
  • The placeholder assets

We’ve been hard at work on adding more story and character into the game for the next update. There is a new mission and radio chatter system that we can use to create a more substantial experience. It’s interesting to hear feedback about the tone of the game seeming too young.

While we want to keep the game friendly for younger audiences, we still want it to be accessible to older people too.

Getting art feedback on colour issues regarding the wingman was an excellent point. We’ll be more mindful of our colour choices to make sure the game isn’t confusing.

What people want to see

  • A mission restart button
  • More of an opportunity to practice flying
  • Time limited unlocks

We’ll be adding new features like tutorials and pause screens (with restart options) as we go! While we haven’t decided on time limited unlocks, cosmetic upgrades are something we’re very keen on pursuing. Time limited unlocks would fit well into a cosmetic upgrade system.

Summary

Our first pass on the wingman system did not have a major impact on player behaviour, but did not seem to lessen the experience. We heard first hand from our testers that they enjoyed the wingman system in the game, and we see it as an opportunity to deliver the story and personality we need to make the world of Sky Squadron fun and engaging.

Now we’ve started to get a good collection of enemies, allies and mechanics we can start building out new and exciting missions!

Interested in joining our squadron of Test Pilots? Sign up here: http://bit.ly/skysquadronsignup

Leave a Reply

Your email address will not be published. Required fields are marked *