If you use the axe to build a raft at the start of your turn, can you ditch the axe?

According to the rules you may ditch an object at any time so I would assume you could leave the axe in that space. I can see this not being a real popular idea among those playing though......