There are times when, frankly, you decide you would like to avoid certain quests, simply because you consider them a waste of time, such as fishing and crafting.
Yet, every time you pass through Wizard City, there they are, with the exclamation points over their heads.
How about this: When you go to them and they get to the point where the offer the option to accept or reject, there could be a third option: turn off. This would do two things, 1) Put the quest in the same state as reject, except the exclamation point does not show, and 2) Makes it so the Quest Finder does not find it. You can still approach the character and talk and they will offer the quest if you want to try it at a different point in the game, but you would have to remember it is there.
An alternative for #2 would be for the Quest Finding to find any quest in an area, but when it has already found one, if you click it again, it finds any other quest in the area.