After becoming a hotspot during the pandemic because of its tempting tax rates and persuasive property prices, the southern city is falling out of favor.