Hopes of halting the sale of a much-loved library building have been dashed, with the site now expected to be sold at auction in the summer.