The planned renovation of a 19th Century Grade II listed residential building has been refused over its ‘unacceptable’ impact.