Despite years of solid returns, direct lenders can't shake concerns about the once-niche corner of global finance.