It isn't like he's a vital part of the third quest either. I'd play him and just take out the encounter card and wouldn't feel too bad about it. Grima leading the heroes through Orthanc to find Saruman is a fun thought too.