The A4 is to be closed on two occasions in the next three weeks – alongside temporary traffic lights – to allow contractors to work on the road.