100% agree. If you don't account for all of this then you're in trouble. Data analytics done correctly, coupled with a solid understanding of the underlying fundamentals can yield quite a bit of useful feedback.

As a sidenote, I find the whole "I know the answer but won't tell you because it's such an amazing idea that I'm going to keep it secret and make my own game with it" to pretty much be the Godwin's Law of game design discussion. You either have an argument to contribute or you don't. If it's an amazing secret that you don't want the wider world to know, great then keep it a secret and don't mention it. If you want to use it to claim something then reveal it. But you can't have your cake and eat it too.

Understood. The converse is when folks like Forgottenlore affirm an absolute negative "it can't be done!!!". In a debate, affirming an absolute negative amounts to philosophical suicide, especially without even knowing what "it" is.

To lead in the direction of your answer without getting into specifics: at a minimum, playtester ELO rankings and full game tape is a fundamental prerequisite. Running data analytics on all that raw data is very powerful, but the how and why is where the secret sauce starts getting interesting. Getting a reasonable sample size is hard, as you pointed out, so normalizing and pruning the data is important. I hope that helps a little, or at least gets your imagination going. There are also design side equations, the MathWing stuff I have posted obviously being one of them.

You simply can not achieve the level of balance you guys are talking about before a game is released and played in the wild.

You absolutely can achieve that level of balance,

No, you can't.

Even you have admitted that your math wing dies not take into account all factors (such as the value of a ship's dial) and you've had 3 years to work on it. I've also seen you comment that you tweak your model based on major tournament results, which is not something that can be done until a game has been released.

With respect, you don't know what you don't know.

There are multiple models and processes that can be used. I have only published one. Maybe I don't want my competition getting too many good ideas from me if I start my own game company or do some consulting someday.

Wayyy off topic, but I think a better question is "Would you want that level of balance?". MtG has benefited greatly off of alternating ridiculous OP cards (which is exciting to players) and really mediocre cards (which helps settle out the field after omg OP seasons) for a while now. In that way the game always feels exciting and that you have something to beat somehow.

I have played games that felt very well balanced, but quickly became very boring, since it felt like many of your decisions at a given moment were made arbitrary because of the balance.I think players don't really want perfect balance. I don't enjoy those games as much as the ones where "so and so is broken, lets dogpile him".

Really psychological here: but it's very akin to the argument made in the Matrix. People don't want and can't accept Utopia. I have found that to be true. So while a mathematical model for balance may be possible, I think the real question is would anyone enjoy it and want to buy it?

MTG gets over that problem with their ability to retire sets, you don't have the same kind of mechanism here. I'd prefer the game to be as balanced as possible.

Not to mention the issue of a meta developing within that small group, that in no way resembles the meta when released to a larger audience, which further skews things.

I remember Mark Rosewater talking about that, years ago. How some interactions were completely missed in the MTG Future Future League (the developers playtest group) and just never came up.

There's a lot about their playtesting that's interesting reading over at wizards.com.

As developers, you're working a couple of waves ahead, so it takes a bit to steer the ship.

The worst thing about trying to steer the ship is that its hard to know if you've already corrected course. Say they're playtesting Wave 10. Attempts to react to the Wave 7 meta is pretty futile when it assumes that Wave 7 will remain stagnant throughout Wave 8 and 9.

I'm a huge math it out guy when it comes to game balance, but there are absolutely factors that just don't math well. The base value of a model simply having an attack and being a target to attack is the most difficult thing to account for. Some games do it much better than others, but it, along with the huge X-Factor that is opportunity cost (if I have 20 EPT's in my game and a list can only field 2-3 of them... there's going to be some that don't see the table regardless of balance), there's a grey area that coupled with meta (assuming any degree of natural RPS exists) that can only be worked out through testing.

As for balance itself; no evolving game gets it right 100% of the time, but its worth recognizing that multiple levels of balance exists and deciding what level is really important for the game to be good. For example, in X-Wing there's a number of levels:

• Faction Balance - In most games I consider this the most important metric to the point I feel it may be the only real indication of a healthy game. It probably is still in X-Wing, but slightly less so due to the relative lack of faction diversity and the tendency to need to own everything anyway. That said, aesthetic diversity makes this important enough it should probably remain the most important metric, despite being a teeter totter with the new guy running around on the axle these days.
• Ship Balance - This is what I really look to to see if the game is healthy myself. How many unique ships can I build a competitive list out of? The higher the number, even if my personal list only contains 1 or 2, the more unique lists exist in the meta. This will probably never be 100%, but it seems right now the game is at a pretty healthy percentage overall.
• Pilot Balance - I almost consider this one to only matter at a popularity level. I don't really, honestly care if Zertik Strom is playable, but it does matter if Vader has no place in the game. I used to really care about this actually when the game was young and this was the majority of its variety, but there are enough ships these days I consider it somewhat negligible.
• Upgrade Balance - Personally could not care less, though I'll again nod to popular characters being somewhat valuable to have viable; particularly if they generally go with a thematic ship. I consider these a tool to adjust far more important vectors, though I would like to see munitions fixed; as they are a unique gameplay mechanic that is completely missing from the game.

Complaining that those ships weren't balanced for 100 points, deathmatch, no objectives, competitive play is like complaining that Descent monsters and heroes aren't balanced for 100 points versus, deathmatch, competitive play.

I know, that's why they ran a scenario-driven event for the release tournament at GenCon in 2012, rather than a 100-point deathmatch with no objectives. Oh, wait... they DID run a 100-point Deathmatch with no objectives as the major release event? Unpossible!!

I'll agree entirely that FFG didn't expect X-wing to turn into the competitive juggernaut that it is, but trying to pretend they never meant for the game to play in this structure is fanboy revisionism. They ran what has always been the standard tournament structure as part of the release, so they rather obviously had it in mind. Pretending otherwise is silly, and that's even before you get to DR4CO's point that the game system simply doesn't have anything in it to support meaningful objective play.

MTG gets over that problem with their ability to retire sets, you don't have the same kind of mechanism here. I'd prefer the game to be as balanced as possible.

There's nothing inherent in the game that makes retirement impossible. You'd need to add new iterations of popular pilots, but that's nothing new for FFG's games and would work fine. The real problem would be models, and the X-wing sales model - you could retire pilots and upgrades and issue new ones, but people won't want to re-buy the models. So it's the same corner FFG's painted themselves into in regards to a v2. Cards sell plastic, you can't do a v2 (or a retirement system) without new cards, and making cards cheap/free means you don't sell the plastic.

Complaining that those ships weren't balanced for 100 points, deathmatch, no objectives, competitive play is like complaining that Descent monsters and heroes aren't balanced for 100 points versus, deathmatch, competitive play.

I know, that's why they ran a scenario-driven event for the release tournament at GenCon in 2012, rather than a 100-point deathmatch with no objectives. Oh, wait... they DID run a 100-point Deathmatch with no objectives as the major release event? Unpossible!!

I'll agree entirely that FFG didn't expect X-wing to turn into the competitive juggernaut that it is, but trying to pretend they never meant for the game to play in this structure is fanboy revisionism. They ran what has always been the standard tournament structure as part of the release, so they rather obviously had it in mind. Pretending otherwise is silly, and that's even before you get to DR4CO's point that the game system simply doesn't have anything in it to support meaningful objective play.

I get the impression that Jason Little did not exactly have much experience with a tournament game. And neither did FFG at that time, quite frankly. Which is quite evident in their fairly loose rules writing at the time. Even their LCGs were a fraction of of how popular they are now.

As for a lot of the other arguments going on. It is a lot of Euro vs Ameritrash arguments. Personally, I feel some slight imbalances are fine. As it scratches the itch of the Dany and Ned players (I completely forget the original MTG names for the three type of players).

I get the impression that Jason Little did not exactly have much experience with a tournament game. And neither did FFG at that time, quite frankly. Which is quite evident in their fairly loose rules writing at the time. Even their LCGs were a fraction of of how popular they are now.

This always comes up, and I always shoot it down, but it keeps coming up. It's like a bad urban legend. I really wish people would stop.

The LOTR LCG has much, much tighter rules than X-wing, and it released a year and a half before X-wing. The Star Wars LCG released around the same time, which means they were in concurrent development. It, too, has a much tighter rules system than X-wing.

The idea that FFG just didn't know how to write good rules when X-wing was made is simply wrong. You can argue that they were lazy with it - that X-wing was intended as a light beer-and-pretzels game and they just didn't bother making a good rules system. That's reasonable, and IMHO what happened. But there are a number of example cases which disprove the idea that they never wrote tight rules during that time period.

If you want to go back and explain the issues for AGOT, which launched in 2002, then sure, there's a case there. But too many of X-wings siblings share quality rules for it to be some broad issue with FFG's lack of ability.

affirming an absolute negative amounts to philosophical suicide

Which is, completely unrelated to the topic at hand, one of the best descriptions of Sith I have come across.

I get the impression that Jason Little did not exactly have much experience with a tournament game. And neither did FFG at that time, quite frankly. Which is quite evident in their fairly loose rules writing at the time. Even their LCGs were a fraction of of how popular they are now.

This always comes up, and I always shoot it down, but it keeps coming up. It's like a bad urban legend. I really wish people would stop.

The LOTR LCG has much, much tighter rules than X-wing, and it released a year and a half before X-wing. The Star Wars LCG released around the same time, which means they were in concurrent development. It, too, has a much tighter rules system than X-wing.

The idea that FFG just didn't know how to write good rules when X-wing was made is simply wrong. You can argue that they were lazy with it - that X-wing was intended as a light beer-and-pretzels game and they just didn't bother making a good rules system. That's reasonable, and IMHO what happened. But there are a number of example cases which disprove the idea that they never wrote tight rules during that time period.

If you want to go back and explain the issues for AGOT, which launched in 2002, then sure, there's a case there. But too many of X-wings siblings share quality rules for it to be some broad issue with FFG's lack of ability.

SWLCG was launched a year after X-wing. It is also important to remind people that the initial co-op design was scraped after the initial Gencon demo.

To me, it feels that there was bit of shift with the release of X-wing that they started to focus more on tournament play and official play. Hard evidence, no. Just my impressions, as I was getting into FFG games at that time.

SWLCG was launched a year after X-wing. It is also important to remind people that the initial co-op design was scraped after the initial Gencon demo.

Again, this is incorrect. X-wing launched at GenCon in August 2012. SWLCG launched in December of the same year. The announcement of the redesign was in February 2012, and that initial co-op demo was a year earlier at GenCon 2011. Assuming the same lead time for initial design as we hear about for new waves, the two games would have solidly overlapping design windows. At the very least, the 3-4 month window between the two releases isn't anywhere near enough for them to take live, in-the-wild experience from X-wing and start updating how they approached the rules, or the tournament play environment. The SWLCG environment got attention from the beginning. As did Netrunner, which has one of their more solid OP systems, was announced in May, and released at the same GenCon as X-wing.

To me, it feels that there was bit of shift with the release of X-wing that they started to focus more on tournament play and official play. Hard evidence, no. Just my impressions, as I was getting into FFG games at that time.

There really are no impressions required here, because we actually do have hard evidence - all the news articles are still available. The dates are there. LOTR beat X-wing by a year and a half, with very solid rules. Netrunner, SWLCG, and X-wing all released at roughly the same time, all with similar tournament systems. X-wing did not provide some shocking renaissance of competitive play from FFG, or motivate any shift in that direction. Honestly, FFG already had solid competitive systems in place for Game of Thrones for nearly a decade before X-wing even came around. If there was a change it was only that they started producing more games that fit well in those formats.

So again, if we look at the actual evidence, there's nothing to indicate that X-wing was some leading edge that couldn't know better, either from its rules or its tournament structure. There's nothing to indicate that it was designed solely as a scenario game, or that it was never intended to be used in tournaments. There is, in fact, an abundance of evidence that disproves all of that. Not that I expect it to kill the urban legend this time any more than it has the last dozen times it's been offered up as an excuse for why It's Really Not That Bad.

They also have done cards that were fairly costed at the time, but then became undercosted later when better ships could use them.

Best example probably is R2-D2 astromech. Originally you could only take him on a 1 agility/3 shield/2 green move ship or a 2 agility/2 shield/3 green move ship. At best you could take him with Luke and shield upgrade which was powerful but not OP.

But then they came out with the e-wing, a ship that has more shields, more agility, and more green moves than the original ships, plus it can also take the evade and barrel roll actions to make it even more defensive.

MTG gets over that problem with their ability to retire sets, you don't have the same kind of mechanism here. I'd prefer the game to be as balanced as possible.

There's nothing inherent in the game that makes retirement impossible. You'd need to add new iterations of popular pilots, but that's nothing new for FFG's games and would work fine. The real problem would be models, and the X-wing sales model - you could retire pilots and upgrades and issue new ones, but people won't want to re-buy the models. So it's the same corner FFG's painted themselves into in regards to a v2. Cards sell plastic, you can't do a v2 (or a retirement system) without new cards, and making cards cheap/free means you don't sell the plastic.

Just look at the problem they've had retiring the damage deck in favor of the clearly superior new deck.

Granted, part of the difficulty they had was bundling it with ships around which opposition to the new canon crystalized.

Bah! Anyway. Yes. Getting people to accept a new tournament version of certain cards is very difficult.