A council has told a cafe owner to remove a multi-coloured sign from his shopfront after a single complaint was lodged against it.