After labeling one retired actor majorly "unpleasant," the talk show host admitted she "hate[d] to say" so since the A-lister is "struggling now."