Firefighters have revealed that buildings connected to a culturally significant WA site have been lost in a raging bushfire.