I hated this phrase too, until I learned where it comes from. And I can therefore inform you all that the box referred to is, in fact, a square.
The phrase originates in a rather clever puzzle that was thought up in 1914, which I shall replicate here. Try and join all nine dots in the picture below using only four straight lines and without taking your pen from the paper (or computer screen).
Simple, isn't it?
Well, try thinking outside the box.
Pandora thinking outside the box