Testing Dice - Or, do my dice really hate me

By Hastatior, in Star Wars: Armada

Star wars dice app. Problem solved.

Sure I'm not 100% that the program coding is perfect but at least ffg can patch it without us having to buy more dice.

On that note though, this may say something about tournament scenes where there is the possibility where players can buy packs of dice to find those weighted in their favour. And it's still 'official' product. Doesn't affect me as I'm a casual commander, but I do like my product as accurate as possible too to get the game's full experience.

Edited by Muelmuel

Once upon a time when I was playing Flames of War, I would bring about 4 different dice tins. I had such terrible streaks of luck with dice that I would change them out mid turn at times. Maybe I need to do the same with this?

This has been a problem with board game quality dice for 40 years. and unless you want to pay $4 plus per die its always going to be a thing. Just know it there and don't let it bother you.

big problem for a time with D6s rolling a 6 33% of the time. because the 6 pips were lighter then the other sides. heavier pip paint was the fix.

Edited by ouzel

He's not rolling the dice, he's checking the weight distribution. If he was rolling the dice he'd roll them on a table thousands of times. He's checking the weight by floating the die. If the die is heavier on one side (more likely to be on the bottom) that side will roll to the bottom of a floating die.

An unbalanced die will have a tendency, however small, to roll with the heavy side on the bottom, which can skew dice results significantly.

Oh really, can skew dice results significantly? I'll repeat,

"Without any performance data, there is not enough to be concerned about your results. "

Clearly you don't know a lot about physics, or dice.

Let me ask you this way then:

If you had a choice between a red die you knew via buoyancy testing was well balanced and random and a die that via the same testing showed a clear and egregious weighting towards rolling blanks, which die would you take to regionals or nationals?

Unless you are seriously just in here to troll and cast aspersions for no other reason than to argue on the internet like a real keyboard hero your honest answer would be the true random die.

If you are too lazy to check your own dice or just don't care enough to do so, I'm not going to tell you to go do it. I'm also not in here posting speculations about how much of a skew it is for a given die accross a thousand rolls because thats pedantic and I don't work for FFG QA.

Fact of the matter remains, Physics is a real thing and so is gravity, if a die is poised on the edge of 2 faces an imbalance will cause it to skew. For myself, and my game, I now know that the dice I kept are the most true and random I had. The red die that skewed double hits went into the trash right alongside the dice with the blank skew.

You can do what you want!

Although now that more than one doubter of basic physics has said similar things I think I might take a die that skewed and one that showed true and roll it a bunch of times, but what might be equally useful is a quick video showing how a bad die rolls underwater like an alligator to show the same face over and over and then floats straight up when you hold the favoured face up, and if you don't think that kind of weighting affects the roll significantly, well, good luck to you my friend!

Dude, no need to get rude!! Please let's keep things civil!

All he is trying trying to say is without empirical data to show a relationship between your good dice and bad, you have no idea how the misweight affects your rolls. Yes the die is weighted, and yes it will cause a skew; how much really needs data to determine. If the weighted die causes a small enough deviance it could still be usable. But that's really up to you.

I think it's very admirable for you to do your best to make sure your dice are as random as possible. For me however, I can't afford to test my dice and throw numbers away. For one, I like the look of my regionals dice, and I can't just get more of them. Secondly, if I tested my dice I'd be obligated to remove faulty dice. By not testing them, I have a "random" pool of dice to select from. Every time I roll I grab one, it could be weighted to hits or blanks or not at all, I don't know or care, it's "random" in that sense to me.

Edited by JJs Juggernaut

I might actually try this.

I hate my black dice. 25% chance to miss and you would not believe the blanks I roll.

One game 10 Y-wings. I rolled 9 out of 10 blanks.

another game, 5 A-wings attacking ships. over 2 turns I rolled 10 black dice...again 1 hit.

I will rarely take anything that rolls black dice anymore.

"All he is trying trying to say is without empirical data to show a relationship between your good dice and bad,"

How about 100+ years of testing and making dice for casinos back up his findings about balance testing the dice?? A spin balancer will come up with the same data as his water test.

its a tried and proven way of testing dice. granted you can't tell from the test how bad the die is off but it does tell you its off.

and in a Game with money or prizes riding on it that is all that should matter.

Now if you want % of badness you need to do the 100 or 1000 rolls test. or use a computer spin balancer. :)

All i know is my regional dice love me and hate my friend. I'm pretty sure because the dice know that he didn't earn them and shouldn't be allowed to touch them.

You gotta respect the heart of the car--dice.

I might actually try this.

I hate my black dice. 25% chance to miss and you would not believe the blanks I roll.

One game 10 Y-wings. I rolled 9 out of 10 blanks.

another game, 5 A-wings attacking ships. over 2 turns I rolled 10 black dice...again 1 hit.

I will rarely take anything that rolls black dice anymore.

This is how my regionals dice are. My standard black dice roll superbly, the clear ones, not so much...

Bomber command center time!

"All he is trying trying to say is without empirical data to show a relationship between your good dice and bad,"

How about 100+ years of testing and making dice for casinos back up his findings about balance testing the dice?? A spin balancer will come up with the same data as his water test.

its a tried and proven way of testing dice. granted you can't tell from the test how bad the die is off but it does tell you its off.

and in a Game with money or prizes riding on it that is all that should matter.

Now if you want % of badness you need to do the 100 or 1000 rolls test. or use a computer spin balancer. :)

Humm I hope people can see Vegas is a tad bit different then a table top miniature game. I am sure there are much higher tolerances and much more regulation then the plastic FFG uses. If FFG thought it would be a problem, I am sure they would have higher tolerances, but they dont so this should be a clue to significance of slightly off die to the game. Maybe is is simple not worth the extra cost to make the 99% of produced die perform statistically correct at 1/10000 level vs 1/1000 level. We are taking about statistics and results being skewed from expected results.

Some shocker, some die are not perfect, so what?

Does this affect the odds in 1/10 die rolls for an extra blank on black? or

1/100?

1/1,000?

1/10,000?

1/1,000,000

Makes a bit of a difference dont you think rather then simply showing the die are not perfect. Suggesting your testing would have practical relevance to your results would be inaccurate without the data. You may confuse or alarm people needlessly.

Edited by Amraam01

while waiting for the AI to process their turn, (Triple-A star-wars map) I've done some limited testing of my dice. The worst offender (black die #4) rolled hit/crits approximately 40% of the time, while Black die #2 rolled hit/crits and blanks 30% of the time. So far, I'm only through testing black dice, and only up to 70 rolls each. (the two blue dice I've tested were marginally biased in offsetting ways - one likes crits & one likes accs)

Later, I'll monte-carlo some dice in excel to see what the expected variance is on such a low number of rolls. Then I'll decide if I want to keep rolling the plastic.

Finally, if I find a couple of biased dice, I may decide to give everyone the salt-bath treatment, and see which dice are also weighted. If weighting & biasing correlate, then I may switch to that as a test going forward.

Assuming the results I have are representative, I'll probably retire black #4. Black #2 is biased towards "drama" as opposed to winning.

Hi again, sorry I have not had the chance to read all the posts up to this point I just wanted to chime in that I have done roll tests and statistical analysis with a friend better equipped to the the chi equations and analysis crap.

We did a significant number of roll trials after uniquely identifying each face, recorded each roll and then analyzed.

Turns out the die had approximately 6% less chance to roll a damage face than expected.

after doing the salt water test we found that the die was weighted towards a blank face at medium strength (about average of what I found in the blank face preference dice).

After cutting the die open we found 2 big old air bubbles sitting in the area behind the blank face...

In conclusion:

-Buoyancy test show die is bad

-Statistical analysis shows same die is bad

-Physical examination shows the die is bad

You can choose not to be concerned by my findings if you like, but I can conservatively and safely say that around 50% of the dice I tested are of Average miss-weight or worse. And if an average miss-weight can cause a 6% impact in your damage output (which I consider significant) I can't imagine if your dice are weighted heavily (either in your favour or against) It might hit as high as 15% variance per die.

Thank you very much for going thru the work to do this testing. I must say that it is disappointing that they have the problems that they do. I can expect imperfection in 'cheapish' gaming dice but these seem to be more so than what would be acceptable.

I have seen those acrylic dice on Ebay for like over 100 dollars, but to hell if I'm going to spend 100 bucks for acrylic dice ...

Other than a dice rolling app (which I prefer the dramatic act of rolling physical dice tbh) I wonder if there is a future fix for this ? (higher quality dice from a 3rd party?)

I just want to chime in that I tested one of my own d20s somewhat recently using the salt water test. I found that it had a bias as to what side was facing upward. I then did 1000+ rolls with a chi-squared test and I found that one of the faces was slightly less likely to appear than the rest (but it wasn't significant). And it was not the face you would expect from the salt water test. So it can be a bit tough to conclude if your die is bad simply from the salt water test.

I know the GameScience dice company was very concerned with non tumbled edges on their dice. I think he knew that dice tend to have production issues like the ones found in the salt-water test, and having sharp crisp edges prevents it from finding the heavy side as easily. A d20 would have more issues than a d8, as it takes less energy to move to an adjacent edge (shallower angle).

Anyway, I wouldn't be surprised however, to learn that some of our dice aren't as fair as we would expect. The crappy part is -- we could buy a bunch of official dice and pick out the least fair ones (in our favor) to use. I guess that's why all tournaments allow you the option to force the sharing of dice.

I just want to chime in that I tested one of my own d20s somewhat recently using the salt water test. I found that it had a bias as to what side was facing upward. I then did 1000+ rolls with a chi-squared test and I found that one of the faces was slightly less likely to appear than the rest (but it wasn't significant). And it was not the face you would expect from the salt water test. So it can be a bit tough to conclude if your die is bad simply from the salt water test.

I know the GameScience dice company was very concerned with non tumbled edges on their dice. I think he knew that dice tend to have production issues like the ones found in the salt-water test, and having sharp crisp edges prevents it from finding the heavy side as easily. A d20 would have more issues than a d8, as it takes less energy to move to an adjacent edge (shallower angle).

Anyway, I wouldn't be surprised however, to learn that some of our dice aren't as fair as we would expect. The crappy part is -- we could buy a bunch of official dice and pick out the least fair ones (in our favor) to use. I guess that's why all tournaments allow you the option to force the sharing of dice.

Ironically, we found that faces adjacent to the favoured face (and actually as has been the tendency I found, in D8s its a pair of faces that are favoured) tended to have an extreme non-statistical result and THE EXACT oposing face had as extreme an unlikely amount of results as the favoured face. This face was the double hit...

So how many dice roles do you need to get regression to the mean ? I put a couple of reds to the test last night and it was interesting ,but not sure I got an appropriate number of data points?

Did a hundred rolls on each:

1) double 9%, hit 23%,accuracy 7%, crit 29%, blank 32%

2) double 15%, hit 17%, accuracy 9% crit 28% blank 31%

That's a bit more a a swing towards blank that I would like, still 100 may be to small a sample.

simulating 100 runs of 100 rolls, there were only 7(die 1) & 11(die 2) runs that had more blanks than you. The extra blanks seem to have come largely from accuracy, as only 3 & 12 runs had fewer accuracies than you.

So, we can't be totally sure that your dice are biased towards blanks, but I'd consider it a fairly safe bet.

As far as how many rolls is the right number, it comes down to how confident do you want to be that your dice aren't biased more than X percent? Judging by what I'm seeing in my simulated rolls, 100 rolls should leave me fairly confident that a given die isn't biased by more than 10-20%. But it's probably not safe to say anything more than that, especially for red dice. (more possible outcomes, more rolls needed to spot bias)

Also, it's very important not to introduce bias in your testing procedures. If you only check your poorly-rolling dice for bias, and not the others, you are going to increase the overall bias of your dice pool.

So, this is an eight sided die, so each face should have a 1/8 chance on any given roll in theory or 12.5%X 2 (25%) of the time we get blank on red.

How can we then tell if our die is biased? That is the question ehh?

Obviously, the more samples we take, we should approach the mean- 12.5% for each side, so if we keep it simple and only look at blanks, to have a nice normal distribution and get 30 samples for each side, I would like to see the results for 240 random rolls in a dice tower and set you alpha error (significance) at 5%- meaning if after the rolls, we can possibly show a significant difference or 95% confidence that we have a nonrandom die.

The thing is you have to set up your experiment and run and see if you prove or disprove your null (You have a random die).

Edited by Amraam01

.I appreciate people looking into this running actual experiments as it is interesting to look over. If the bias is very big, you can see it with a smaller sample size, if it is very small your tests would need to be powered (Enough samples) adequately.

Edited by Amraam01

found a nice site to analyze the results. You have to enter each die's results separately. 100 rolls are not enough for the levels of bias I'm seeing to be statistically significant, but 150 is enough for the worst of my dice, and 200 detects the 3 worst for each color. (ie, which would get replaced if I bought a new dice pack... Assuming that the dice pack is all fair, which seems optimistic)

graphpad.com/quickcalcs/chisquared2/

This is fascinating. I simply pray to the dice gods, sacrifice some chickens and roll the dice.....

Mark each side with permanent marker in a different way or color if you want to see which sides pop up more. Well at least for the sides with duplicates.

Edited by Lyraeus

Ok, so as was discussed in another thread, I have decided to start testing my Armada dice

Here is the methodology:

Materials:

1 - Dollar Store chip Bowl

2 - Cheap brand Epsom Salts (4kg)

3 - Many dice

Fill bowl with water, salt until die floats.

Now, put a die in the water and flick, spin and roll it at least 10 times and try to observe any tendency.

Edit: once a tendency was suggested, I held the die with that face to the bottom of the bowl. Bad dice would flip as they rose to show the prefered face, good dice rose straight up regardless of orientation. Several orientations while being held to the bottom were tried with each die and the difference between good dice and bad dice was immediately apparent.

So far I have tried 1 black die and something becomes quickly apparent:

In an 8 sided die, the "spin" tends to favour not 1 face, but 2 adjacent faces (the balance axis is like a rod through 2 pyramids joined at the base)

After testing 1 die (black) I have found it very clearly favours a pair of adjacent faces (one is a blank and one is a Hit/Crit) out of 10 spins of various descriptions, these 2 faces came out 7 times and a regular generic hit came out 3 times. Since we know a regular hit has a random chance of 50% and a Hit/Crit 25% and a Blank 25% this already points to a skew.

Second black die spins far more randomly and has a tendency to surface on a Point or Edge before settling on a face. the difference from die #1 is striking

Third black die is basically set to blank. You could just erase all the other faces for how often they come up

Black Dice:

8 out of 12 black dice presented as significantly random, 4 are clearly miss-weighted. It really isn't rocket science to see which ones are bad and good, when you see the bad dice do underwater acrobatics to present the light face while the good dice surface on a point before rotating to a final face.

Blue Dice:

Blue dice were utter garbage 3/12 are good. a majority of my blue dice are weighted towards crits (6) 2 weighted towards hits and 1 weighted towards accuracy. I guess this explains why I can't seem to buy blue acc when I need one from my ISD 1s. I literally have to go buy another pack of dice to play now because of this. Found one that was so aggressively weighted to one half that it sat upright in the water like a top. The half it was favouring was a 2hit/2crit, probably almost never see a natural accuracy on that thing.

Red Dice:

Well, I was hoping the blue die were an outlier, but it looks like my black dice were the oddballs. My reds are as bad as my blues. 3/12 good again. a full 5 out of 12 were weighted towards blanks, one of them notably aggressively, 3 of them quite heavily and one shared the weighting about half with a hit face. the other bad ones represented the other faces (1 crit, 1 hit, 1 2hit and 1 accuracy).

In conclusion I am extremely disappointed with the balance quality of these dice. out of 36 dice 14 are random. 38% of a product performs as it should...quite sub par

Not sure if your experiment has any relevance other then saying it is not perfectly balanced- we know the paint and die cut outs may change the symmetric properties. Roll it a thousand times and record the results- that would be interesting and a natural progression of your experiment. Will a slight mis-weight effect performance? meh. Without any performance data, there is not enough to be concerned about your results.

He's not rolling the dice, he's checking the weight distribution. If he was rolling the dice he'd roll them on a table thousands of times. He's checking the weight by floating the die. If the die is heavier on one side (more likely to be on the bottom) that side will roll to the bottom of a floating die.

An unbalanced die will have a tendency, however small, to roll with the heavy side on the bottom, which can skew dice results significantly.

It's not necessarily the case that a die with a heavier side will roll with that side on the bottom. A professor of probability once did a very famous experiment rolling a statistically significant amount of small D6s (the kind that come in GW games) a statistically significant amount of times (using machines and students to actually do so).

What he found was that the 1's side was over-represented, which was counter to the hypothesis that the 6's side should have been. Because the die had pips grooved out of them, the face with the single pip was heavier than the face with six pips, yet the heavier 1's face came out more often than statistically expected (1/6 times). [see: http://www.dakkadakka.com/wiki/en/That's_How_I_Roll_-_A_Scientific_Analysis_of_Dice]. So heavier sides do not necessarily "roll down" more than lighter sides. At least this experiment was testing die performance and not weighting, which makes it a better assessment of dice.

Either way, there is not necessarily an equating of imbalance and performance. A small weight imbalances in a buoyant salty water environment with even distribution of water pressure on all faces of the die are not necessarily at all influential to the result of an eight-sided die that is shook up and thrown from the hand 1.5 feet above a table surface.

Edited by AllWingsStandyingBy

sure, fine. Great. But if your choice is to roll balanced or nearly balanced (weight-wise) dice v.s. rolling aggressively poorly weighted dice which would you prefer?

Sure, it's a personal choice as the FFG dice as they sell them (IMO with crap QC) are all legal and theoretically within the bounds of "fair" and you are welcome to take your chances and literally roll the dice.

I however, will continue to choose the least biased dice I can, regardless of which side (heavy or opposite-heavy) it's biased towards. Maybe it makes little to no difference, so what? What if it does?

Also, D6 do not roll the same ad D8s do not roll the same as D20s (as several experiments have also shown) and further experiments have shown that depending on several factors (rounding of edges, weighting etc) a die can and will show a statistically significant bias.

If the balance of a die had no effect on the result you should notify casinos immediately cause they are wasting a lot of money on balanced dice.

Had someome due with with a d20 swearing my d20 would float 20 up based on how often it rolls 20. This test show that it should roll 3. And the dice that the test should should role high doesnt.

Your better off setting up a dice rolling sstem ie drop and recording the results of roughly 50 to 100 rolls and see what the range is