You won’t bother building a Wall until you have troops to defend your town, right? Everyone thinks that way. Walls add a % boost to a city’s garrison’s defensive strength. No garrison, no defensive strength, so no point building walls.
But Walls have one other property: they are the only structure that actually shows up on the map. No other buildings change the town’s graphic on the map. But as soon as you build a wall – even a dirt-cheap-to-build level 1 Wall – it changes how people browsing the map see your settlement.
So, here’s a thought: rather than waiting til you have troops before you build a Wall, why not build a Level 1 Wall long before you bother with troops? That way people looking at your settlement will see the Wall and naturally assume that you have an army to man it. It will make you look tougher than you actually are.