*Disclaimer: I am not mad, merely a Physicist...*

So for those of you not scared of the title, I am beginning an effort to generate a set of statistics to help judge Imperial Assault unit cards better. This obviously isn't easy but I am beginning small, first by taking into account simply attack power without rerolls and most likely rolls, health as a fraction of points and speed as a fraction points. Then onto categorising surges before moving onto rerolls and working when one should reroll and the best dice to reroll. There will need to be some ham-fisting of scores, for instance it is very difficult to simulate Battlefield Leadership, so some human judgement will be required. I intend to make all this data available to the community.

"But, Fez, what does this all mean?!" I hear one of you cry. Well the intention is to rank each unit, then using very crude heuristic analysis and many days, have a computer generate the best Imperial Assault list. Simple really? The ultimate form of net-decking, a list literally built using the internet. I intend to log my progress in this thread. All those interested are more than welcome to follow and I will be releasing any source code as and when it is in a fit state for people read.

*Disclaimer #2: Being a physicist, I have little to none coding training, although it is my day-to-day job...*

So there we have it! This is a call for interest and anyone who would like to discuss these ideas. After all, scientific discovery is only advanced by the work of the many!

Fez

This sounds great! Some initiatives of this nature have been taken before, but I don't recall hearing about any of them being completed. I'm really looking forward to what you come up with!

I'll offer a couple of suggestions, if you're so inclined:

--While I can see the role of guesstimation at various points (ie, on Battlefield Leadership), I think it would probably be best to also keep some scores completely separate from any semi-forced calculation. That is, I'd rather see Leia (who has Battlefield Leadership) ranked according to her YYB attack dice, rather than trying to artificially add some random number to account for her partner's dice. I know that Leia's damage output will be a little bit lower on paper, but I also know why that is, and am therefore able to make my own decisions about her relative power or effectiveness for her cost. I would personally think that is better than giving Leia an artificial score and then having to wonder if she's deserving of that score in a battle.

--Statistics like Avg Dmg vs 1 White die and Avg Dmg vs 1 Black die, and then another column which includes "while Focused" (or maybe even "while Hidden"??) stats would be valuable. For instance, eSaboteurs have a much better chance of triggering Blast when they're Focused...because of the extra base damage to punch through defenses, and also the better chance of rolling a surge...eRebel Troopers, on the other hand, don't gain quite as much benefit from their Focus since their surges are much weaker.

I have a simpler version of this. It's pretty straightforward: I use average damage and then determine the percent chance of number of surges, and figure out how much damage a unit can do as a result of that. I then divide it by the point cost of the unit to determine efficiency. I ignore reroll effects because they are too hard to do simply.

I assume the best conditions are always active: there's always someone to cleave to, brutality finds two targets, etc. Then you can kind of tell interesting stats: "the Rancor needs to be doing 2 attacks 70% of the time to be as efficient as the elite Gamorrean." Stuff like that.

It also tends to weigh towards multi-unit activations, since I'm not modeling any defense dice. One of my future projects is planned to be adding columns for different defenses.

I then tried to somehow tie it into health/point cost, but I'm not sure what the weighting of that would be. I don't think it's a simple attack efficiency * health efficiency = total efficiency, but something like that seems like it'd be worthwhile to compare units.

So in the end you want to calculate the best probabilities and turn the game into mathematical data? Where is the fun in playing the game based on pure math?

This sounds great! Some initiatives of this nature have been taken before, but I don't recall hearing about any of them being completed. I'm really looking forward to what you come up with!

Thanks for the feedback! I am always open to suggestions. My main drive for this is the heuristic analysis and list building but if the stats on the way can be helpful then that's great! Ideally, a lot of information for 'weird units' (Leia et al.) can be balanced by also ingesting tournament lists into a learner which will then understand where a unit might've been underrated due to an ability not accounted in the original scoring.

I have a simpler version of this. It's pretty straightforward: I use average damage and then determine the percent chance of number of surges, and figure out how much damage a unit can do as a result of that. I then divide it by the point cost of the unit to determine efficiency. I ignore reroll effects because they are too hard to do simply.

Average damage is inadequate for the level of detail I want. Using standard deviation means not only can you work out the most likely but also how tight are the results around that. A large standard deviation means that if you don't get the average, the next most likely result is potentially a lot worse or a lot better. This can be used to contribute to picking a more consistent unit over a more risky unit. Rerolls be selected for also by rating each side of each dice out of all potential dice faces and rerolling the lowest score. In most cases this is fine and only corner cases will be ignored, for instance where a surge is searched for instead of maximum damage.

I'd be interested in your efforts made to get a score. I agree it's not that simple as each stat has a different value. I was going to measure health and speed as a fraction of a unit's health. Basically calculating how many points a unit pays for each health and each speed against the units total points cost, this gives a better comparison than simply plotting speed and health against cost. The obvious example being C-3PO and R2-D2. C-3PO pays only 0.5 points per health...

So in the end you want to calculate the best probabilities and turn the game into mathematical data? Where is the fun in playing the game based on pure math?

I think if you take this approach you are only viewing one view point. I understand how this could be viewed by some but I prefer this view:

This game is all about tactics and strategy with a good smattering of luck thrown in. This tool/maths doesn't aid at all in the playing of the game, simply simulating the maths which humans do instinctively anyway e.g. how many hits can I get a on red dice, What'd the most likely range on a red and green? These are the questions I want to answer. These are the questions I often ask during the game. It is not as if I intend to take this table into games, but if I can use it to aid in my pre-game decisions that is very helpful!

If you have any other questions, please feel free to ask. There are many more points of view I'd like to discuss and hear from the community, especially those who disagree. Not with the aim of converting, but simply of hearing.

Apologies if I am not clear, it is late and I have been at work...

I was going to say pheaver talked about having made some kind of spreadsheet way back during the 2015 Worlds (or Nationals, can't remember) in an interview, but he beat me to it on here! I'd love to see that kind of thing sometime - this really intrigues me.

And I totally agree that you can't reduce this game down to a simple formula with all the different abilities that are thrown in on characters and command cards, but you can definitely get a sense of how naturally efficient certain things are and use this to make some educated decisions and see where you can find good synergy/value! Looking forward to what you find!

Nice! And best of luck on the work! I also have an excel version of a ranking that mainly values expected offensive output efficiency and expected offensive soak efficiency. Some interesting things I've noticed:

• In creating a "score" per unit and then ranking the units, I have found the model that fits my "prior distribution" best is not a linear combination of the raw scores, but exponential.
• It's awfully tricky to incorporate the impact of good command cards into a deployment score. Tried a couple versions of this; as of yet none have panned out satisfactorily.
• I use the crutch that thereisnotry pointed out, I estimate the impact of certain harder to quantify abilities into expected offensive output and expected offensive soak categories.
• I have not done much variance analysis. It's all been expected value work.

​The other IA "coding project" I just finished is an attack/defense calculator, where you can plug in dice pools, surges, attributes, some abilities (like re-rolls) and it outputs the expected value of the attack and damage CDF. Again, the caveat here is that it's maximizing for single target damage, so surges like blast, cleave, and recover are not included. This was a fun project in python and I learned a lot working through it.

I've found that analyzing games I enjoy is a great way to make learning more job specific skills interesting, and it keeps me from abandoning the process after about a week. The skills development, above the mostly academic insights generated by the results, has been real value of working on those kind of ranking/calculating projects for me and I hope you find similar value in the project even if it does not revolutionize your game.

Wow guys! Thanks for the feedback! The initial scores are in! I have generated a google doc containing my very very very first initial findings. There will be more data added soon, for instance some of the stats I was talking about (health/point vs points etc.) as and when I get the chance. In the meantime, the spreadsheet is locked but feel free to look over the data, manipulate as you will and I am interested to see your opinions of what it shows!

Because I am a scientist, I must provide background on my data:

• Each row contains the name, points, attack, health and speed of each unit, currently none of the roles take into account deployment (multiple attacks per activation etc.)
• The calculated columns are in beige show the average hit, surge and range score rounded to 1 decimal place and the standard distribution rounded to 2. Anymore and the numbers become meaningless anyway. For useful information on why standard deviation is important, see this: http://www.mathsisfun.com/data/standard-deviation.html.
• The data was created using a Python program I have created, reading data from a json file, parsing the information from the resulting dictionary into 'Cards' and then performing the sampling.
• Each data point is 1,000,000 roles.
• The average time per million roles was around 18 seconds

Without further ado, the data is here: https://docs.google.com/spreadsheets/d/1PU1DBKn-JmXmJeRFwkf9Au1Gu5t28IETeNG3SOFb0xQ/edit?usp=sharing.

Please do provide feedback! The overriding intention for me is to generate a list-building algorithm but the data produced along the way is useful for all to understand their list creation better.

Thanks,

Fez

Edit: Just got back from Imperial Assault, I will reply to specific comments tomorrow. GO SEE IT!!!!

• The data was created using a Python program I have created, reading data from a json file, parsing the information from the resulting dictionary into 'Cards' and then performing the sampling.
• Each data point is 1,000,000 roles.

See this is why these projects are so neat: I didn't approach this problem stochastically with simulation, I did it combinatorially. I would love to see the simulation code. Did you use SimPy, or some other lib?

The calculated columns are in beige show the average hit, surge and range score rounded to 1 decimal place and the standard distribution rounded to 2.

I'm not seeing any decimal places in the hits, surges, or ranges columns. Are they there and my view just goofed?

I also didn't really know how I wanted to approach IG-88 or Wiess, but because I'm not super interested in playing them (they don't pass the "eye test") I let the matter drop.

Are you planning to do anything to analyze defense in a similar manner or simulate versus defense dice?

Also on variance or std dev, how are you using that decision making wise? My initial thought was high expected value, low variance was the way to go. High levels of consistent performance. I haven't put any more thought into it than that and I could be 100% wrong on what we desire the variance to be if spikes in damage are more useful than consistency.

Thanks!

• The data was created using a Python program I have created, reading data from a json file, parsing the information from the resulting dictionary into 'Cards' and then performing the sampling.
• Each data point is 1,000,000 roles.

See this is why these projects are so neat: I didn't approach this problem stochastically with simulation, I did it combinatorially. I would love to see the simulation code. Did you use SimPy, or some other lib?

The calculated columns are in beige show the average hit, surge and range score rounded to 1 decimal place and the standard distribution rounded to 2.

I'm not seeing any decimal places in the hits, surges, or ranges columns. Are they there and my view just goofed?

The sim code is actually all based on standard Python library, at work I have limited access to Python and can't install any libraries really, for this reason I have to do such stupid things as write my own standard deviation function... I hope to upgrade this code and even host on an Amazon Web Server so these results can be constantly updated and get more and more accurate. Certainly for heuristic analysis, I'll have to use libraries but I am currently enjoying the experience of figuring things out 'the old fashioned way'.

To your second point, It just so happens that when you simulate something that only has about 4 or 5 possible outcomes a million times, you tend to get zero. Anyone interested in the full results, I am happy to share but the reason there is no decimal is because of the Google sheet rounding removing the obsolete zero. In hindsight this is rubbish and bears checking my data again, stay tuned sports fan!

I also didn't really know how I wanted to approach IG-88 or Wiess, but because I'm not super interested in playing them (they don't pass the "eye test") I let the matter drop.

Are you planning to do anything to analyze defense in a similar manner or simulate versus defense dice?

Also on variance or std dev, how are you using that decision making wise? My initial thought was high expected value, low variance was the way to go. High levels of consistent performance. I haven't put any more thought into it than that and I could be 100% wrong on what we desire the variance to be if spikes in damage are more useful than consistency.

Thanks!

To deal with the remainder of your points, IG-88 and Weiss can use any 2 or 3 dice combination. This makes them impossible to check for but always gives them the best possible outcome. As humans we might not always pick the correct colours but it is worth noting that a computer probably could given any situation. In order to select for this, I intend to look at damage against range. So for instance in order to hit a figure 5 squares away, there are better and worse dice just to get the damage rather than to guarantee the range.

Defence results are valuable but mainly as a test for how damage rolled translates to damage done. This means that as defence is rarely changing, usually only a black or white dice, it is better to look at the damage a model can cause rather than it's survivability. In the near future, I hope to be able to generate a matrix of tipping points, relating the amount of health a model has and whether it is significantly more or less survivable with a black or white dice. My hypothesis is that a low health figure with a white dice survives longer than a low health figure with a black dice, but a high health figure with a black dice survives longer than a high health figure with a white dice. There is clearly a tipping point where your defence dice colour doesn't matter but it would be useful to find that.

On Variance and Standard Deviation: I foresee this topic coming up very often and I feel that is it basically down to personal preference. Would you prefer a figure with an average damage but a large deviation, meaning that occasionally it will out-of-the-blue one shot a guy, or would rather a steady, consistent damage output? All of this is also related to surge variance and averages, if surges are higher then this traditionally, but not in all cases, increases the damage result, but not the damage rolled. This game is all about the interactions and as humans it is difficult to imagine all the possible interactions, with computers we can consider most if not all of these and come to some kind of conclusion. And this, in it's heart forms why Standard Deviation is equally important to the average roll. If you absolutely have to get that last 2 points on Leia with your HK Droids because otherwise she'll activate and shoot Bossk along with Luke, then you want a lower variance. But if you just want as much damage on Obi-Wan as possible because everything has activated and you have the initiative then actually, maybe a higher variance is better.

To @Sevenstep, @Masterchiefspiff, @robertpolson, @pheaver and @thereisnotry, thank you for your initial feedback and I hope you will remain with me for this journey. If any of you are playing in the Hague Star Wars Day tournament, I hope to see you there!

Thanks,

Fez

Wow, this is great!

At some point, I'd love to see a version where I could make cosmetic adjustments (ie, adjust the column widths, add background colors to represent factions on the Names, etc), but already this thing is so interesting!

I can follow everything, except for this: I'm not sure what to do with the "sig hits" or "sig surges" or "sig ranges" values. Should those "sig" values be added to the basic values (ie, the "hits" and "surges" and "ranges"), or are they already included in those basic values?

Anyway, thank you so much! I've been wanting to find a resource like this for a while, but I just don't have the programming or statistics background to be able to handle it well.

For everyone's general reading and in the interests of full accountability, I have redone the data. It looks there was an error in my first code so it reported the wrong result. The latest data has been uploaded and checks out. 5 runs at 1,000,000 rolls per unit.

I can follow everything, except for this: I'm not sure what to do with the "sig hits" or "sig surges" or "sig ranges" values. Should those "sig" values be added to the basic values (ie, the "hits" and "surges" and "ranges"), or are they already included in those basic values?

The "sig" values are the standard deviation values. Sig is short for sigma, the greek letter meaning standard deviation in maths. Standard Deviation, or "sig" as I have called it, is a measure of how varied the results are. The larger the standard deviation, the larger the spread. A smaller standard deviation implies that you can more heavily rely on the average score, as a high proportion of results are near the average.

An example, consider 3 sets of numbers:
Set 1 - {0, 0, 14, 14}
Set 2 - {6, 6, 8, 8}

Set 3 - {0, 6, 8, 14}

All have an average of 7, but the standard deviation of set 1 is 7, the standard deviation of set 2 is 1 and the standard deviation of set 3 is 5. This is an extreme example, and it implies the maths is more simple than it is, but it is clear that a result with a lower standard deviation shows that the average is a better measure of a typical result.

The next set of results I release will contain some graphs. These graphs will show speed/cost vs cost, health/cost vs cost. I will then move onto defence analysis and comparing each attack of each unit against a white, black, white/black and black/black defence. After that I would like to ask the community for some help. I am looking to go over some ratings for different surge effects.

If there is a desire for flashier tables/spreadsheets, I will create some buttons on the existing spreadsheets to allow highlighting/colours.

Thank you all for your continued interest,

Fez

So for those of you still following my madness, I have changed the layout of the tables... Again! There is method in this however, I have added a Monte Carlo simulation to my code. What this does, is to count the "baskets" of hits. For instance, I am already checking how many hits are scored per role, working an average and a standard deviation but what if you a gambler? You are nothing like Han Solo and you need to know if there even a slight chance Trando's will roll enough to do the 4 damage required to Leia? This is where Monte Carlo sampling comes in. Monte Carlo sampling is not a predictive modelling technique and is never 100% accurate but it provides a better world view simply trying to work out the averages of each dice and adding them together. The graphs produced are often cluttered and hard to read however, I am looking into a better way of hosting them but for now, they are in the the spreadsheet here. To make the graphs more readable and also the spreadsheets more manageable, I have split them all by Faction. One each for Rebel, Imperial and Scum( ).

The new data reflects the number of roles in a million that rolled 1, 2, 3, 4, 5, and 6 or more hits. This is reflected as a percentage to keep the numbers manageable. So if you only need 1 hit, you would add the percentages from all columns, which is basically 100% apart from in rare cases. To get 5 hits, add the 5 and 6 or more values and there you have it!

Next Steps

• I have begun work to add any printed bonuses into the attack results. This will provide a more realistic view of offensive capability.
• Rank each surge so they are chosen intelligently getting the most damage each time for each number of surges rolled.
• Calculate damage averages after defence dice against all current combinations printed on cards (none, black, white, black + black, white + black)
• Achieve some kind of understanding of how much movement and health means in determining points cost.
• Generate a ranking system for each unit to enable heuristic list building

Easy really! Until next time,

Fez