Hearn said he had “never” previously allowed sentiment to be such a big factor in his business dealings