Behind the Release of Axis Descending: Iteration & Testing

I have spent many hours running Axis across the deck of this ship, testing and iterating.

For the second part of my 12 part series, documenting the journey behind finishing the most personal project of my life, I want to discuss some of the most important methodologies I aim to teach my students. Through iteration and testing, you can find success. Through these two terms: iteration and testing, numerous subtopics can be explored to break down some of the most fundamental lessons that have been bouncing around Gamasutra for years. For seasoned developers, they both serve as reminders and a metaphorical reflecting pool for current and future endeavors. For students, they provide the guidance and structure necessary to reinforce a productive learning environment.

One of the biggest lessons I have for my students is understanding that game development takes time and it isn't as glamorous as some commercials make it out to be. Failure and self-teaching are integral to the process. You will create work that is bad. You will make mistakes. You will fail. All of this allows you to grow as a maker. I regularly meet young folks looking to dive into the world of game development. Discussing how to get into the industry, why students should participate in Lawrence Tech's Game Art program, or simply trying to point talent in the right direction leads me to a common question. "What is the one thing you would tell an aspiring developer?". To that, I say fail hard, fail fast, and you'll succeed furiously. The last bit may be a bit overzealous, but when I say this I like to imagine those moments when someone resolves a problem in code or puts that final touch on an art asset. After struggling with something for so long, which I see often in my own process and the process of my students and friends, there is an air of defeat and silent determination.

I recently spoke at this year's Game Arts Conference about getting into the industry.

At times, you hit a wall in a project. We've all been there. It happens, and how we react differs depending on the day. Maybe the struggle is more frustrating than it really needs to be. Maybe the difficulty is veiled by something else in the work environment or in your own problem-solving state of mind. So you take a minute, you google the answer, you phone a friend and hope that this determination pays off sooner rather than later. You do not stop. You may take a break. You may take a break for far too long in retrospect, even, but you should never stop. The solution is there. Figure it out. Consider new angles. Considering the constant flux of Star Wars media and my family watching the original trilogy it is hard not to quote Yoda. Do or do not. There is no try. I am not a fan of organizing things mentally in a binary way, but in this case the only answer is to continue. Insert the coin, hit Y, and don't let the counter time out.

After resolving the issue that determination pays off, all is well in the Mushroom Kingdom, and you move on with a rush of dopamine. Great! Good job! Only later on in the project you realize all that work you did may be going to waste. Hypothetically, your art asset is getting scrapped. Your mechanic your were implementing? Too complicated for your player. This moment finds us all, often, and is entirely necessary to success as furiously as possible. Like the light and dark sides of the force providing a balance in the universe (if you believe it exists only in a binary way), both the successes and failures of the craft have to be experienced to grow and create polish. Do not be afraid of these moments.  By all means be frustrated and vent if need be, but never disregard the fact that this is entirely natural in design, in business and in making.

You will fail. You will redo things. You will make mistakes. Good for you! You should be!

Discovery is learning. Good or bad, the experience provides supplementary knowledge for future design choices. Through the iterative design process and project management like Agile/Scrum, all of this is inherent. Want to make games? Be prepared to make some terrible ones first. Many, many times. This same logic can be applied to most trades and careers, really. My Father is involved with heating/cooling/electrical. Every system he worked with had a design that he needed to understand enough to break apart, restore, and improve. Through a diagnostic approach, he would "try it" to see if his fix worked. Did that work? No? Try something else. Get it validated. Push it.

You can always try Wikipedia too. I found this image there.

Iteration is key. It encompasses and guides the intent of the designer the whole way through a project. Axis' run cycle has had numerous updates and tweaks over the years. His ground attack combo is an even better example, undergoing constant revisions, timing tweaks, adding and removing the ability to chain into other attacks, and more. Like many developers, I spent an immense amount of time in the last few years just getting the feel down for certain things. After so many modifications, I feel good about the decisions I made and the time spent on it. If time allowed, I would work on it for another year just to improve it but, like my Grandfather always said, a good artist knows when to stop. Prioritizing what you spend so much time cycling through the process again and again is important. All of this time and energy spent on the look and feel based on my own perspective is almost null. None of it matters unless we have the validation of the user experience to light our path.

I'm pretty sure this isn't even the latest version of it.

Test as soon as possible. Test individual mechanics if possible. Test when possible. Your goal is validation and observation.

Through testing, you will have decisions acknowledged and supported or disputed. Through testing, you will observe moments where players are confused, challenged, entertained, or immersed. Whatever decisions you have made about your primary and core mechanics, they need to be backed by the user. If players are having a difficult time jumping in a platformer you have a problem and the earlier this is resolved the better. It provides the chance to make that jump function and become a part of that entertainment that will lead to your intended experience. Make the jump feel great. Make the weapon feel good to use if that is your core mechanic. Whatever it that the player does frequently and often, you don't want it discouraging the player from continuing to play in any way. Focus on the feel, but not just the feel you experience. Game Design is human research. Testing is where your hypotheses are explored.

The first alpha test with Axis was ill-conceived. In many ways I wouldn't consider it an actual alpha test. I originally set out to fix bugs, not validate and observe. I submitted a Flash .swf embedded in this blog as a way to play and test the build I had, which offered little to no gameplay. It was a series of mechanics that simply did not play off of one another and provided zero context for who you were, what you were supposed to do, and how to do it. A student of mine, who I knew from my PSP homebrew days working with SmashGP, gave a review on his own blog that perpetuated the link and offered some insights. Like his post, other testers noted bugs. I failed by not asking the right questions. What made this fun? What are you supposed to do? Who is the player character? What 3 things make this great? Nothing from this "test" provided me with the knowledge I needed to accurately plan ahead and tackle my user experience challenges. After dabbling with this build for a period of time, a conversation in passing with a less-than-friendly ex-colleague proved to be more beneficial. As salty as it was, his disapproval and dismissive comments made me change my perspective and lead me to explore a few things that proved quite beneficial.

The game changed dramatically for the better.

The first beta was handled more appropriately. By embedding the game in HTML available for anyone with a Flash-friendly browser and a computer, I made the venture more accessible. Additionally, I provided a few links to a questionnaire and a bug report system through Google Forms that created quantifiable data, visual graphs, and a means for players to communicate with me regarding the game at any time, unlimited by a site-specific focus session. Updating these forms and providing new versions of the game build with additions, bug fixes and more keeps the feedback loop going. It also helped to build a small following and interest in the game. Players felt connected with the development process and were excited to see developments happening each and every day.

I cannot recommend Google Forms enough for your feedback questionnaires.

My questionnaire was trying to create a conversation with the user. I may have asked them to give a 1-5 rating, but the most important written questions related to what they took away from the experience. What were three things you loved about the game? This question alone tells you most of what needs to be known. Even if your test only gets a dozen or two responses, if the majority of that small group are saying you should increase your jump height then you should increase your jump height. That response will be there, most likely, whether you get 12 responses or 12,000.

The bug report form had only seen a handful of responses, but if major complications arose they could be addressed, transferred to a bug spreadsheet to prioritize and organize the information, then resolved if they were something I could reproduce.

Moving forward, I am aiming to take a few notes from Dan Felder's Design 101 entry on Testing. Specifically, Scattershot Testing.

Beta 2.0 was intended to be a more finalized "this is the game" experience, but was operating under the structure of a Masters course project. One of the biggest hurdles in the development of the game, appropriate screen scrolling, was simply ignored to finish up all of the content I wanted to create for the course. I spent much of my allotted time just building 30+ levels without justifying any of the choices I was making. Each level was split up into 10+ non-scrolling screens, making the fast-moving Axis jump between screens rapidly. 2.0 is jarring, to say the least, and the way the environment was implemented does not encourage the sense of exploration I want to achieve. Despite rushing under the restraints of time, the iteration provided me with an immense amount of insight into the process I will be using in 3.0's scroll-friendly environments.

2.0's rooms often felt like stunted corridors with forced encounters. Many of them, like this one, were in need of clutter or purpose.

Some of the areas in 3.0 are still static, but only if they are appropriate. The tight and cramped barracks for Axis' crew makes sense given the context, but also provides a great point of view for sparring with our friend Huey here.

Through Scattershot Testing, the next beta will be focused on the mechanics, not so much the final intended product. Once I have my validation and observations, I will have the tools I need to get that final polish. I am currently working on a tiling system for level generation to save time, avoid bugs and oversights, and establish a clear sense of "this is ground, this is a wall, this is a clingable wall" and so on.

If you take anything away from this post, trust that failure is a natural part of design, growth, learning, and success. Embrace it. Strive to learn from your experiences Make. Focus on the feel. Iterate. Fail hard. Fail fast. Know when to stop.