So, the more time you spend in your house, the faster plants get pests?
So, if I need to access my shared bank, if I go to my now empty dorm room, will that impact the plants at my house?
It shouldn't. The plants at your house should only develop needs when you visit the house. As long as your plants don't have needs when you leave they should be fine no matter how long it takes you to return. At least that's the way I understand it. Visiting your dorm room should have no effect on the plants you might have growing at any OTHER residences.
Barley the Gardener in Khrysalis>Silent Market has the Itsy Bitsy Spider spell that will get rid of the level 6 pests. You have to be at least a Level 15 Gardener and it costs some gold.