An influencer with over 1 million followers was dropped by brands after a racist voice memo leaked