The Gavin & Stacey star bought the horseshoe-shaped Templecombe House near Henley-on-Thames back in 2020 with plans to make it his family home