Almost half of the public avoided or delayed contacting their GP about a health concern last year, a survey suggests.