The other day I grappled a building with Leia and went inside through a window. Is this ok to do?

Is this a window on the second floor? So you’ve performed a clamber to go from outside the first floor to inside the second floor?

I think they’ve (rather intentionally) left things vague for us to determine, and do cool stuff. From page 9 (bolded the parts I think are most relevant):

Quote

For the sake of simplicity, most buildings are generally
best treated as large pieces of impassable terrain, but sometimes
players may wish to incorporate more nuance.

In particular, buildings will sometimes have parts that are
impassable while the rest is open or difficult terrain—a mini
may be able to move through a doorway or a large window as
open terrain, but the building’s walls are impassable. When
using a piece of terrain with mixed types, players should clearly
define the terrain so that there is no ambiguity.

So I’d say just come to some agreement with the other player with regards to the nuances of certain buildings. I, personally, wouldn’t have a problem with what you’ve suggested.