Only men with a specific gene variant and a family history of certain cancers should be screened, experts have said.