S&V 60: Developers Interview with Max Brooke, Frank Brooks and Alex Davy

By Kelvan, in X-Wing

9 minutes ago, sozin said:

RE: alternate formats, I'll lead by example here. Some Vassal league guys and I took the second place winning idea from our Alt Format competition and fleshed out out. I give you version 0.3 of A-Wing Soccer (imagine: Rocket League and Blood Bowl had an X-Wing baby). We're currently playtesting on Vassal! If you'd like to contribute please shoot me your email and I'll invite you into our playtest channel.

Why not put it into X-Wing Mission Control.

https://tools.fantasyflightgames.com/xwing/

It's a way to teach computers how to solve problems without explicitly teaching them how to solve the problem. Here's a good one minute video explaining the big idea.

For X-Wing, we'd give the computer a bunch of "labelled" matchup results -- for example, Triple Jumps will beat Four X-Wings --- and if you feed it enough labelled matchup results, it will be able (given enough training examples) to predict for new lists -- lists it hasn't seen before -- which list will win in a matchup. In kind of sounds like magic, but it works very similar to the way our own brains work.

Another way to think of it is that a neural network is like Tom Cruise in the movie Edge of Tomorrow. Every day it gets to go back and figure out what it got wrong -- based on the labelled matchups that the humans provide it -- until in the end it, like Tom Cruise, can ace all the right steps it needs to take.

EDIT: here's a beautiful picture that walks you through the sort of problems you can solve with ML, and the tools you use to solve them.

Edited by sozin
13 minutes ago, Transmogrifier said:

@sozin For those of us who aren't coders, what exactly is machine learning/neural networking?

I'm not a pure coder, but I am in IT so I'll take a stab at it...

ML is basically glorified statistical analysis where a machine says "given X thousand data points, and based on the knowledge provided, then Y is true". A classic example is an email spam filter - you "show" the filter what spam looks like and if you give it enough data points, it can begin to rule new messages as spam or not spam. The more data you "show" it before turning it loose, the more accurate it becomes.

I'm probably over simplifying it but hey I just work in infra/network and keep those pesky apps running for the devs.

Edit: beaten to it.

Edited by ScummyRebel
23 minutes ago, kingargyle said:

Why not put it into X-Wing Mission Control.

https://tools.fantasyflightgames.com/xwing/

Hmm, I logged in, and it looks like they haven't updated it in a while. Newer ships aren't there, newer rocks and debris aren't there, token set is incomplete, etc. Could work still, but maybe Vassal + Photoshop is good too.

33 minutes ago, PhantomFO said:

Yep. The JumpMaster was probably the worst offender here. They've tried every small nerf they could to fix that ship. First came the Deadeye nerf, and the change to R4 Agromechs, which didn't matter as people switched to Dengaroo and Parattani. Then came the Manaroo nerf, but people just switched to mindlinked Scouts. Now rumors abound that they'll be deploying the nuclear option to kill it for good, since none of those small nerfs addressed the fact that the chassis was simply too cheap for all of its bonuses to begin with.

Same goes for the Palp and x7 nerfs. Both were intended to bring balance to the game, but all that ended up happening was that they've almost completely pushed Imperials out of the competative meta.

Yes, the Jumpmaster is a grand example of a failed design FFG and badly aimed, failed corrective measures. The Palp errata was a hand grenade thrown to late on account of political pressures.

The more I ponder this incredible interview (thanks again guys), and recall the continual nudges over the year from mates --who had an ability to somewhat glimpse behind the Mouse's curtain a bit-- that the rest of FAQ 4.3.2 was surely set and ready for publication, I have gone back and read between the lines. It fear the FFG team has shelved the rest of the FAQ and are now resistant do to being angry at their apparent embarrassment from acting based on pressures from customers. Listen to this exchange again:

"We definitely keep an eye on the meta. We keep an eye on what has become dominate. We're just taking our best guess about what is going to go down." -Alex

"When things become really egregious we can step in with errata. We've done it in the past. We've certainly might do that again in the future, but until something is demonstrably a major problem, we definitely prefer to let things play out. Um, there's a bunch of product in the pipeline that's going to be coming out, uh, fairly soon that's probably going to be changing the equation again. So, there's a danger to being too reactionary. Um, even some of the nerfs that we have implemented, which were, you know, uh, people were howling at the moon for something to be done about Palpatine, for example. And now people are calling for Palpatine to be UN-nerfed. So....." -Alex

"You're just like rubbing, you're just rubbing every note in Tyler's face right now and I love it" -Richard

"Hahaha. Cause that's the thing, I mean you never know how the next wave of stuff is going to effect the meta. You never know if it's going to be a nerf too far or if something that is a problem will no longer be a problem. Uh, certain patterns take shape of a long enough period of time where you can be like, 'yeah, this is, this is a problem and needs to be addressed.' But it's dangerous, it's a very delicate, uh, art form changing the game enough to stop the really abusive stuff but not to throw it into total chaos. We're doing the best we can." -Alex

"We're happy as long as the meta is relatively diverse." -Alex

Reading this, I have surmised the following --albeit personal-- from Alex's statements:

They keep an eye on the meta. They watch the competitive builds and power cards; they understand the power of JMK5s, Sabine, Biggs, Nym, etc.

When things become really egregious, they step in with errata. They haven't stepped in, they clearly see nothing is egregious.

They feel their last errata was a nerf too far, a mistake. They will not fall into the 'errata due to cries from customers' again.

They feel the next wave of stuff will change the equation of the meta and fix current problems. They now rely on new design for balance.

FAQs are dangerous and can easily cause total chaos. They will not be implementing the rest of "Bringing Balance" FAQ 4.3.2.

31 minutes ago, Kelvan said:

Coincidentally this is why we didn't ask about the FAQ. They can't or wouldn't say anything. It also puts them in an awkward position. There wasn't any reason to waste the time when time is limited.

You didn't need to ask @Kelvan, they probably already answered your question, see above.

Just now, sozin said:

Hmm, I logged in, and it looks like they haven't updated it in a while. Newer ships aren't there, newer rocks and debris aren't there, token set is incomplete, etc. Could work still, but maybe Vassal + Photoshop is good too.

Yeah but in this case, you just really need the basics. It has A-Wings, and rocks.

41 minutes ago, Transmogrifier said:

I think this is the crux of the issue - designing a game like this requires a huge set of skills and 2-3 people aren't going to have that complete set. One of those skills is understanding the math underlying the game mechanics and obviously they don't have someone in-house who is doing that. I'm not saying "hire Bob Randall", but they certainly need to incorporate *someone* with that sort of knowledge base/skill set into their team, even if it's just a contract consultant or whatever. Math isn't going to magically solve all of the games problems, but having someone who thinks mathematically would really help round out the skill set of the development team and eliminate one of their major blind spots. .

I will be the first to say that any mathematical model has its limitations and you need to know and understand those limitations.

But the real problem is that in the case of board game design and miniature combat games in particular, those models don't even publicly exist. While "understanding math" is certainly a prerequisite, that's actually the easy part. The hard part is figuring out what the model needs to be to begin with. You don't need to already have a PhD to derive those models from scratch, but it does fundamentally require "making a scholarly contribution to the field of interest" as is required for a PhD, except in this case your employer probably wouldn't want you to publish your results. So to me, that's the minimum bar for the sort of person that you would want to hire for this position. Later on down the road when the "how to" approach has already been made public knowledge through academic journal articles, you can reduce the job requirements to someone that "knows math and can leverage an existing published theory". Also bear in mind that for a large company like FFG, you're not going to be working on just one game. You would be balancing multiple games.

Re: Neural Networks: I think there's two different problems here:

  1. Identifying ComboWing
  2. Identifying real ship values

It would be nice if an AI had found Dengaroo during playtesting. We will all be sad when Jeff Berling is made obsolete, but it will be for the best long term. The difficulty though, is I don't think you can really separate either #1 or #2, they are interwoven. So I think you actually need an AI that can play the game at the micro level, and not just analyze macro-data. This is a pretty difficult problem, for example the best Starcraft AIs are still inferior to human players. X-wing is probably a much easier game to design an AI for than Starcraft, but it smells like a lot more than 100-200 lines of code.

Yeah, you are talking about a semi-supervised or reinforced ML, which Google's AlphaMind (WC beating Go program) was. That is MUCH harder to do. My "hot or not" neural net is going to be a pretty straightforward stochastic gradient descent implementation, easily 100-200 lines of code (probably 50-100).

The harder problem is getting the community to label all the data for us. But @haslo and I might start baking that into Meta-Wing soon (when you hit the home page, it asks you to weigh in on a random matchup).

Edited by sozin
10 minutes ago, clanofwolves said:

Yes, the Jumpmaster is a grand example of a failed design FFG and badly aimed, failed corrective measures. The Palp errata was a hand grenade thrown to late on account of political pressures.

The more I ponder this incredible interview (thanks again guys), and recall the continual nudges over the year from mates --who had an ability to somewhat glimpse behind the Mouse's curtain a bit-- that the rest of FAQ 4.3.2 was surely set and ready for publication, I have gone back and read between the lines. It fear the FFG team has shelved the rest of the FAQ and are now resistant do to being angry at their apparent embarrassment from acting based on pressures from customers. Listen to this exchange again:

"We definitely keep an eye on the meta. We keep an eye on what has become dominate. We're just taking our best guess about what is going to go down." -Alex

"When things become really egregious we can step in with errata. We've done it in the past. We've certainly might do that again in the future, but until something is demonstrably a major problem, we definitely prefer to let things play out. Um, there's a bunch of product in the pipeline that's going to be coming out, uh, fairly soon that's probably going to be changing the equation again. So, there's a danger to being too reactionary. Um, even some of the nerfs that we have implemented, which were, you know, uh, people were howling at the moon for something to be done about Palpatine, for example. And now people are calling for Palpatine to be UN-nerfed. So....." -Alex

"You're just like rubbing, you're just rubbing every note in Tyler's face right now and I love it" -Richard

"Hahaha. Cause that's the thing, I mean you never know how the next wave of stuff is going to effect the meta. You never know if it's going to be a nerf too far or if something that is a problem will no longer be a problem. Uh, certain patterns take shape of a long enough period of time where you can be like, 'yeah, this is, this is a problem and needs to be addressed.' But it's dangerous, it's a very delicate, uh, art form changing the game enough to stop the really abusive stuff but not to throw it into total chaos. We're doing the best we can." -Alex

"We're happy as long as the meta is relatively diverse." -Alex

Reading this, I have surmised the following --albeit personal-- from Alex's statements:

They keep an eye on the meta. They watch the competitive builds and power cards; they understand the power of JMK5s, Sabine, Biggs, Nym, etc.

When things become really egregious, they step in with errata. They haven't stepped in, they clearly see nothing is egregious.

They feel their last errata was a nerf too far, a mistake. They will not fall into the 'errata due to cries from customers' again.

They feel the next wave of stuff will change the equation of the meta and fix current problems. They now rely on new design for balance.

FAQs are dangerous and can easily cause total chaos. They will not be implementing the rest of "Bringing Balance" FAQ 4.3.2.

You didn't need to ask @Kelvan, they probably already answered your question, see above.

They managed to poke @Starslinger72 on:

Swarms
The Phantom Nerf
Not Nerfing Nym/the Silencer
The Palp Nerf
*EDIT*
And they threatened to Nerf Kylo

I can't believe he's not in a Minnesota Jail for manslaughter.

Edited by Kelvan

Could someone please explain to me why the devs are so lothed to change point costs on cards ? They have changed text on a card which makes it function completely differently but changing a number is a complete no no, I don’t get it.

1 hour ago, MajorJuggler said:

This is definitely another topic, but for now I am taking the Blizzard approach of "when it's done". I have actually been debating opening it up from closed alpha to open beta, but without a squad builder ready to support it, building lists is a bear.

I am ready. My body's ready.

1 minute ago, Thepreacher said:

Could someone please explain to me why the devs are so lothed to change point costs on cards ? They have changed text on a card which makes it function completely differently but changing a number is a complete no no, I don’t get it.

Because changing text doesn't make a list illegal, generally. Changing point costs, so your list that you though was 100 points is actually 105, or something like that, does. I don't know where they've stated that off the top of my head but that's the reasoning.

One thing I would have asked them about was the last faq they released, it was a joke it has 3 fixes I think

one made them look stupid.

another wouldn’t have matter if they had allowed it

and the other no judge would allow to happen

but they ignore a lot of things that are crying out for clarification

31 minutes ago, Kelvan said:

They managed to poke @Starslinger72 on:

Swarms
The Phantom Nerf
Not Nerfing Nym/the Silencer
The Palp Nerf
*EDIT*
And they threatened to Nerf Kylo

I can't believe he's not in a Minnesota Jail for manslaughter.

They really do seem very concerned about Imperial ships being completely balanced. As a primarily Imperial player im still kinda ok with that.

If you only play Imperials vs Imperials the game feels perfect in every way.

It places an emphasis on 90 degree arcs.

Its turret ships explode very quickly under concentrated or multiple fire.

Its Aces even at PS11 can be driven back down to PS0 with Kylo which is easily countered with Draw Their Fire or Determination.

Its Best Ship ( RAC ) can be destroyed by 2-3 Missiles or 4-5 ships very easily.

Swarms are 100% relevant.

Whisper can be out flown and countered.

Bombs are all "simple" and "skill" based to land.

The best lists generally involve limited arcs.

Even its 4x TLT list is more reasonable with only 5 hp on 2.5 green dice per ship.

I want a new core set that is just called Imperial Civil War.

The rest of you Rebel Scum can have the Galaxy.

The Empire is moving to Jakku for some gravity training.

Edited by Boom Owl
10 minutes ago, Otacon said:

Because changing text doesn't make a list illegal, generally. Changing point costs, so your list that you though was 100 points is actually 105, or something like that, does. I don't know where they've stated that off the top of my head but that's the reasoning.

It a reasonable argument, but you could have someone turn up to a tournament with a dengaroo list thinking I’ve come up with the best combo ever and to find out their list is now useless, or if the leak faq is right and upgrades are taken off the jumps someone could turn up with an illegal list.

6 minutes ago, Thepreacher said:

It a reasonable argument, but you could have someone turn up to a tournament with a dengaroo list thinking I’ve come up with the best combo ever and to find out their list is now useless, or if the leak faq is right and upgrades are taken off the jumps someone could turn up with an illegal list.

Dengaroo would still be a legal, albiet weakened list.

The leaked FAQ was in fact a very big deal because they were making changes to cards that could result in an illegal list. It has not happened yet.

Edit: Well, not exactly; they had made some changes to cards that would require someone to remove them from a list if they showed up to a tourney with them, like say Deadeye on a large base or multiple Tacticians.

Edited by kris40k
1 minute ago, Thepreacher said:

It a reasonable argument, but you could have someone turn up to a tournament with a dengaroo list thinking I’ve come up with the best combo ever and to find out their list is now useless, or if the leak faq is right and upgrades are taken off the jumps someone could turn up with an illegal list.

Leaving a list as worse is likely considered preferable to making it illegal. As for the upgrade slot argument, it's a very simple matter to just remove the upgrade from the list. It might make other upgrades pointless but it's effectively the same situation as just changing card text to be less effective; the list is still legal, it just might not be any good anymore.

1 hour ago, Jeff Wilder said:

Okay, just so I can reassure myself I'm not just being Mr. Doom-and-Gloom ...

... that was not a reassuring interview for the future of X-Wing, right?

I listened, squinting for a glimmer of hope, ruthlessly mixing metaphors, and I came away from it more convinced than ever that the game is effectively dead for anybody that cares about firing arcs and maneuvering duels.

Right?

All the following quotations are paraphrases, I don't recall the exact wording, but they capture the spirit of what was said.

"We nerfed the Phantom because Whisper could just decloak the other way and took away a vital piece of the game, which is predicting what your opponent is going to do... nah, we don't expect PS9 AdvS PtL boost/barrel roll Kylo to be much of a problem" except it's literally the exact same issue, how can they not see that? For that matter even single action Nym has a similar 'nope' button with AdvS, and even SLAM on PS8 Miranda can have a similar effect. Acknowledging that the one use of Kylo Ren seen today is one that didn't make a big splash in playing (and didn't like, but didn't care enough to change it ), because it "wasn't that good", they predict Kylo not to be a big deal... because evidently playtesting is going to do a great job of showing what's broken!

Ugh... and then Frank's comment about harpoon missiles being the last nail in the coffin for swarms, "well, if a list is already competitively dead, what's the harm in releasing a card that kills it even further?"

I dunno, Frank, maybe because we'd like to see it on the tables again? Maybe because we don't like having useless ships and useless cards? I know that was at least somewhat sarcastic but the problem is there's real truth behind it, and that they aren't taking it seriously is a huge red flag for me.

Pretty soon, if they want to see play, the Imperials are going to need a pilot with the ability, "Enemy ships at Range 1-3 cannot discard upgrade cards. If a card equipped to an enemy ship at Range 1-3 has text instructing the ship to discard it, it cannot be used and none of its effects trigger."

Still does nothing for bomblets.

That being said, silver lining is that there's likely a Sabine nerf inbound. When describing the B/SF-17 they talked about its lack of crew to avoid combo with Sabine, which would be over the top - but it already has 2 bomb slots (bomblets) and Sabine would help it out even if she's literally just on VI Ahsoka. So, sounds like she'll be getting a nerf to her phrasing, probably "your bomb token" instead of "friendly bomb token", and that's probably something they've already incorporated into their playtesting so they thought nothing of it to say that.

The Kylo crew discussion was frustrating to me since their argument was basically "In most builds Kylo's ability is fine since it can be counter-played and, assuming a 3+ ship meta, it's not all that powerful". So they are basically looking at the ability in general and saying "Most of the time this is worth about 3 points". Instead of "What the most min/max thing you can do with this card? What's the value of this ability in that context?" Is playtesting focusing on general builds or is it focusing on the most exploitative combos that can be built with cards? It seems to be missing a lot of the min/max combo stuff and I'm wondering why that keeps happening? A lot of these combinations are thought up within hours of the spoiler articles posting. Why is this continually a blind spot?

57 minutes ago, Sparklelord said:

That being said, silver lining is that there's likely a Sabine nerf inbound. When describing the B/SF-17 they talked about its lack of crew to avoid combo with Sabine, which would be over the top - but it already has 2 bomb slots (bomblets) and Sabine would help it out even if she's literally just on VI Ahsoka. So, sounds like she'll be getting a nerf to her phrasing, probably "your bomb token" instead of "friendly bomb token", and that's probably something they've already incorporated into their playtesting so they thought nothing of it to say that.

I wish I had you're optimism, I heard that Max thought since they made sure Sabine can't ride on the BFF, then throwing bombs out all over with a 5 template is wholesome. I think he just runs the ships and possible upgrades in "play-testing" and never thinks about squad interactions; I mean, how can he and say that?

Edited by clanofwolves

I listened to this at work today so missed bits, luckily some kind folk have transcribed the important bits that concerned me in this thread.

Im somewhat confused by all this. I'm sure there was an interview with Alex and Frank on S&V a couple of years ago where they said their overall aim was to have every ship remain relevant and playable. IIRC; they talked about how the TIE bomber and defender needed some love (we got imp vets), how the m-3a and star viper needed a little something and they'd love to do a scum epic (we got the c-roc and GFH). They talked about how the x-wing and e-wing needed a boost but Biggs and Coran were problematic doing this. Whilst I feel it's a bit lax for them to constantly release ships which need another pack to bring up to competitive levels; I respected the fact that they were doing things to reinvigorate the barley seen ships. I loved the idea that everything remaining viable was one of their main goals. Was I dreaming? Has booze and weed addled my mind to the point where I just imagined this?

I ask because that seems like a world apart from this interview. It feels like they're chasing their own tails here. The comment about harpoon missiles killing swarms not mattering is the total opposite of where I believed their core values to be for the last 2 years. Don't get me wrong; I get where the difficulties lie but it just really felt like a whole new approach of "I do what I want!". I'm honestly not feeling too optimistic about the future of the game right now, I'm not quitting but... I'm a lot less confident everything will work out in the end now.

tenor.gif

Edited by Smutpedler
auto-correct is rubbish
On 10/16/2017 at 4:44 PM, Azrapse said:

I don't think the Missile Boat is ever going to come to the game. That ships was just too broken and specialized. Also, it's probably one of the least known ships in the Empire perhaps just above the TIE Punisher.

Given that, I can buy the idea that the Missile Boat is a customization of the Starwing, so the SLAM action goes to the Starwing.
Since the Starwing doesn't even carry bombs, it doesn't bother me that they use the SLAM action more like the actual Missile Boat used to do: to release missiles and torpedoes. That is what the OS-1 title seems to do and to me it's fine.
The presence of SLAM on a non-Missile Boat Starwing is harder to explain, but I am not going to complain. It's just an intermediate hybrid model. There, explained. :)

As long as it doesn't hurt the dial to bad.

7 hours ago, clanofwolves said:

I'm leaning towards this too.

The Mouse just wants his movie and cartoon ships out there, and they are pretty much all balanced. I think it's these other ships plucked from Star Wars obscurity, pushed up from some dark recesses, completely internal to FFGs studios.

Did you listen to the podcast?

It's not explicit, and it's not entirely clear, but my takeaway is that the decisions about which ships to make were made above them at FFG. Shortly after the 50:00 mark they talk about creating a list of proposed ships to send "somebody", getting approval, and then running it by LFL. And the kind of feedback LFL would give was characterized along the lines of "Hey, we're releasing a book soon that will invalidated everything you know about that ship, maybe use a different one".

It ain't the Mouse, fun though it is to tilt at that windmill.

20 minutes ago, Transmogrifier said:

The Kylo crew discussion was frustrating to me since their argument was basically "In most builds Kylo's ability is fine since it can be counter-played and, assuming a 3+ ship meta, it's not all that powerful". So they are basically looking at the ability in general and saying "Most of the time this is worth about 3 points". Instead of "What the most min/max thing you can do with this card? What's the value of this ability in that context?" Is playtesting focusing on general builds or is it focusing on the most exploitative combos that can be built with cards? It seems to be missing a lot of the min/max combo stuff and I'm wondering why that keeps happening? A lot of these combinations are thought up within hours of the spoiler articles posting. Why is this continually a blind spot?

Because they're right? It's impossible to design a game where things are equally valuable against 2-ship builds and 4-ship builds. Kylo is certainly a monster against low-agility 2-ship builds, but swarms could care less about him.

I play a ton of Imperials. RAC is basically the only place where Kylo is even a thing, and even there he's not exactly lighting up the charts.

14 minutes ago, clanofwolves said:

I wish I had you're optimism, I heard that Max thought since they made sure Sabine can't ride on the BFF, then throwing bombs out all over with a 5 template is wholesome. I think he just runs the ships and possible upgrades in "play-testing" and never thinks about squad interactions; I mean, how can he and say that?

Ehhhhhh... it's possible you're right, but that's a really big oversight.

Or, maybe there's a smaller oversight responsible for that comment: it almost makes sense that, since by their own admission they're playtesting a different game than the one that's on the tables now, it slipped their minds that the current Sabine can affect every friendly bomb instead of just the ones coming from the ship she's on.