Maybe Dean won't actually say it like that again, but he already made it pretty clear this episode that, in his mind, Sam leaving hunting behind is pretty much the same thing as Sam leaving him.