A Lisburn City plaque dedicated on an Irish Yew tree planted by the former Prince Andrew has been “temporarily” removed to avoid “vandalism”.