Insiders said that Iran refused to end enrichment during talks this week, raising fears diplomacy may fail