Police said the four men aroused suspicion due to their behaviour and made residents feel uncomfortable