I'm still really new to the game, but when making a horror check, do I roll dice equal to the investigator's Will level?

I believe you roll what the app tells you to roll. (<symbol> negates)

I haven't played yet, but I suspect Will will feature prominently.

After a monster spawns on the board the app will tell you how to activate to move and attack, and then it will ask if it is in range to trigger a horror check. If it is, you pick the monster's icon at the bottom of the screen, and hit yes to do you want to resolve a horror check, and then a text box will pop up with some flavor and a check that you need to do.