How does caltrops affect large monster movement? Specifically if a large monster shrinks, enters an empty adjacent space to a hero with caltrops, test is successful, tries to expand but cannot. What is the result?

Sounds like a good question to ask FFG via the Rules Questions link.

Personally, I'd say that any time a figure is forced to stop in an illegal position, it gets moved to the closest legal position (of the opposing team's choice, if there are several "closest" legal positions) and stops there.