Ok, looking through an Eye for an Eye, it seems like there are lots of calls for observation checks, and this provides some meta-knowledge. In theory, the players should ignore this meta-knowledge but when there is a mystery to solve, the temptation to use it is hard to resist!

I mean, let's say they are searching Aschafenberg's bedroom. One player wants to examine the door. I don't know what difficulty that should be since it isn't mentioned in the adventure and there is nothing unusual about it. So I say make an observation with no purple and the player knows even if he fails, there is nothing to find there. Or even worse, I just say 'it's a normal door' and don't even call for a check. Now he searches the bookcase, and I say make an observation with 3 purple, even if he fails, he knows there is something there!

One thing that immediately comes to mind, is that the GM could roll the purple dice in secret, and if even if it is a simple test, roll a few dice and ignore them. Now the players don't know how many dice you rolled, and don't know how many crosses you generated, though they do still know how many hammers they rolled.

Obviously, the GM should discourage metagaming and prevent other players searching the same area if they are doing so because they suspect the first pc missed something!

Has this been an issue for anyone else?

I have this too and I wasnt able to come up with a good solution.

The second you ask for an observation check everyone listens up and searches the **** out of every corner of the room.

Here's a dozen possible options you could pick from:

1) Only allow one check per location or clue, ever . Let the PC with the best Observation check, and all others in the room add white dice for assisting. If they ask to check there again, explain that the consequence of failure was that their characters are now convinced there's nothing there. Just like how failing a Charm roll might actually upset the NPC you wanted to charm. Or like how failing an attack roll means the foe gets to counterattack before your next action.

2) Only allow one check per day or per act. This is the less extreme version of the above method. You're temporarily convinced there's nothing there, but you can come back later with a fresh perspective after thinking about it for a while.

3) Allow additional attempts on the spot, but make each subsequent roll have escalating difficulty. Ever have that experience of looking for your lost keys and you search the same room again and again only to eventually find it in plain sight somewhere you just could not think to look at? So frustrating, right? You can simulate that by adding +1 purple per check that preceded it. Eventually the banes and chaos stars will start to look daunting.

4) Allow the rolls, but award Stress and/or Party Tension for each one after the first. "I all ready looked over there, Frederick, I'm tellin' you it's not there!"

5) Make each subsequent search take longer, or make more noise. Eventually an NPC will hear them or walk in on them. Can be combined with methods #3 or #4. These are great bane and chaos star consequences, for example.

6) Make your rolls per room instead of per sub-location within the room. The scenario says there's a 3-die test for checking the shelves. Don't roll when they check the shelves, roll when they check the room in general. Difficulty is 4 dice if they don't go into detail, reduced to 3 dice (or even 3 purple plus 1 white) if they specifically mention the shelf.

7) Use the die roll only if the PCs search the room but fail to mention the specific part of the room the clue is in. If they specifically check the shelf, they automatically find the thing that's hidden there. If they just toss the room, they roll the 3-dice.

8) Reduce all search difficulties everywhere to some standard number you're okay with, probably 1 or 2 purple. Use that same difficulty whether the hidden thing is plot-critical, trivial, or completely nonexistent. It's a bonus to the GM in that the PCs can't metagame the difficulties, but also a bonus to the PCs in that the hardest difficulties have been reduced. Everybody wins.

9) Make all searches have a difficulty of exactly 1 purple, but with a variable (hidden) number of successes needed to reveal the clue. Similar to the mechanics the game uses for First Aid checks, and the recovery rolls from Diseases/Insanities/Criticals. The 1 purple is so there's a chance of stress or location-specific consequences. If the scenario notes call for a 3-difficulty die roll, translate that to mean they need a net of 3 successes to actually find the clue. Tell the players that this is the way you're handling it from now on, but never tell them the actual number of successes needed to find a specific clue in a specific place.

10) Base the difficulties on who hid it, not how well they hid it. An NPC who's very careful and secretive sets a 3 or 4 die search die difficulty on his bedroom and office even if there's nothing there. Let the PCs know that's how it's going to work. They still get metagamy info, but it's more about the personality of the NPC instead of revealing "the clue is on this shelf somewhere". It's worth noting that "NPC is private and detail-oriented" does not mean "NPC is hiding an evil secret" but either could result in high purples on the search.

11) Take the approach that the Gumshoe RPGs use: Clues are meant to be discovered, they exist to advance the plot. Make clue discovery automatic if the player's search the right places. No rolls at all. Fast and simple, and it favors the players so most of them won't complain. One small problem is that Intelligence and Observation are slightly devalued this way, so it might not be an ideal method if a PC has invested heavily in those stats. (As those with low Int + Obs will still his spotlight moments.)

12) Make clue discovery automatic so the plot doesn't stall out, but still roll the normal number of dice just for the bane & chaos-star results. If you technically failed the roll, you still get the clues, but it means you've left evidence of your tampering that the villain will later notice. This keeps the plot from stalling out (which can often happen if the PCs miss a vital clue) but it still rewards players for investing in Intelligence and Observation.

Hope at least one of those (or a combination of a few of them) is helpful to you. Personally, I've been using method #1, but I'll probably swap to #9 or #12 now that I've thought more about it.

r_b, that was a rich source of ideas! thank a lot for stretching my mind with possibilities!

Yavathol, just a detail: when the scenario doesn't specify difficulty, I would advise against automatically assuming it is as simple difficulty (no purple). I usually assume it's easy (1 purple), but I tend to base my decision on how the situation (and the narrative tension) is. It is rare indeed that I ask for a simple difficulty test; when it's simple, I usually don't ask for a test at all.

Observation is for sure one test that is worth to have a discussion about. I've seen (and GMed) a lot of adventures in which about half of the testing was perception-like tests. That ends up being too much, even for an investigative or scouting adventure: you'll get every player wanting to train their perception skills as much as they can, even if their characters shouldn't be perceptive at all.

Other thing I have seen a lot (and also GMed once or twice) is to ask for a test when you want to give the players an information. And then they fail... What do you do then? I think that, if something is crucial for the development of a story, you shouldn't ask for a test, just give the information.

Personally, I tend to use two strategies regarding Observation checks: I keep them at a regular pace. Like, when investigating a scenario like An Eye for an Eye, I ask for one test for each room (and the players will be waiting for them and not surprised). When travelling, I ask for a test each night. And the other thing, really important to relate to the first one, is that I don't ask for tests only when there is something. So the players will find their characters passing one of the paced tests to discover nothing special.

I also like to give some time (ever changing) from the test and the situation that it should apply. So I ask for an Observation test, the character passes, I describe something ordinary, or some minor thing he finds (and the player will either consider there wasn't anything to find, or he things the difficulty was harder and maybe he will want to try again; I usually deny). Afterwards, I just describe what he finds, considering the test previously done.

One thing I find is important is to try different things and keep your strategies changing. That when you are experiencing asking for a test when there is nothing to be found, or when you are spacing it out. When you try some of the ideas r_b gave us, I suggest you warn the players as he suggests (because some of them gives new perspective of how the mechanics of the game are being dealt). But when trying things regarding narrative, don't let the players create a picture of the way you always deal with things, because if it happens you are allowing a situation where they feel you and them are playing against each other.

The last thing that has crossed my mind (and one I never did try out) is to have some kind of hidden spreadsheet at the start of a session with a couple os Observation tests that are really important to the story. Ask the players to roll them before the session even begins, and write the results for each situation. They could even have different difficulties. Then, if the situation do comes to happen, you already have the results. And if they don't, the players can always feel al little healthy confusion, or think they did apply to some part of the adventure. But I imagine that doing this with more than 3 tests might be a little too confusing...