So for us, 'midichlorians' exist, but don't measure anything that scientists can be certain of, an inexact science that is mostly dismissed as quackery. Hutts and Toydarians have no Force immunity beyond a higher Willpower score.
Of course the First Rule of Canon applies - your table, your rules and Lucas be damned - but I've actually made peace with the Midichlorians. I came around to being okay with them once I realize that the Midichlorians were not what caused the Force, but a byproduct of the Force. Kind of like having dark skin pigment does not mean you live in Africa. Living in Africa gives you dark skin pigment.
Same thing - Midichlorians don't necessarily "cause" the force in people, rather that perhaps they are tiny organisms that gather near the force. Therefore, you could measure them to get a general idea of how much Force is generated in a person at that time, but it isn't as though people have the ability to control tiny beings to lift X-Wings and stuff.
Also, just because the Jedi said it was so does not necessary make them right. Midichlorian readings could easily be the high-tech equivalent of reading tea leaves.
Being in Africa doesn't make you black, being in Europe drains your melanin and makes you white (over the course of thousands of years, anyway). But I see what you mean, and you're absolutely right; midichlorians are merely a guideline, a curiosity, even. In-universe characters don't even fully understand them; they've merely noticed that high Force sensitive people tend to have this strange bacteria in their blood.
Being in Africa doesn't make you dark. Being in Europe doesn't make you white. Even after thousands of years. What makes your skin have the pigment (or not) is a combination of survival and mate choice. With the right diet we can survive in northerly climes and have dark skin to reduce the UV damage. But without that diet, those that have slightly lighter skin are more favored to survive to reproduce. Clearly we can survive in Africa and not have dark skin. Just look at all the white folks that are there. A few hundred years later and their skin hasn't gotten any darker just from being there.