If you keep your wizard pointed diagonally off towards the outer edge of the sidewalk (i.e. the wall or building) and run forward, you shouldn't be pulled in. This is generally true of any area with sidewalks.
Note that in many areas of Wysteria, the sidewalks disappear at certain points and merge into street area. In those places, you have to wait until the coast is clear before moving forward.
Do NOT take the right sidewalk, though the left seems unsafe. Take the ramp, walk into Burble's house (if you have/have done that quest) and take his downstairs door. You will be closer to the passage to Bramble's tower if needed.