Americans of all ages are facing affordability concerns, with retirees on fixed incomes in particular feeling the strain.