A row has broken out over a proposed beachfront restaurant in the luxury holiday enclave of Hermanus.