If you get too close to the edge of the ponds, even though they are raised from the water, the fish get scared away. This just happened to me twice in the Ancient Burial Grounds. In the case of mooshu, or really ANY area where you cannot physically touch the fish with your wizard (dragonspyre is another example of this), even if the fish "run into you", they should never be scared away if you're not actually IN the water
