The former NFL receiver said Tom Brady Sr. once expressed concern his son wouldn't retire, but the elder Brady said he didn't recall the conversation.