“I think that was what upset him the most," the New York Times reporter told "Late Show" host Stephen Colbert.