Brown has had a season-long running feud with the league's officials, culminating in a massive fine for saying the refs had an agenda against him