A Japanese island that was once filled with wealthy residents has been sat abandoned for over half a century.