Money managers slashed their bullish wagers on silver to the lowest in nearly two years before the White House refrained from putting import tariffs on critical minerals that include the white metal.