Shares of Broadcom rose about 5% in extended trading after its earnings report underscored the strength of its custom chip business.