When you are INSTRUCTED to go with out porting to a friend. I forget the exact level, but if you do all the storyline quest, you are then told to report to DS.
I say any level at which you are given the quest to go. You may have skipped a few sidequests but I see no reason why if you get quest that you shouldn't go.