They can be bought at the Bazaar when available. May take some time farming for them though.They are also kinda rare drops from harvests of Cat Tail plant. You can also use a Transmute spell and turn 15 Cat Tails into 1 Water Lily. The transmute spell is sold by Avery Templeton in Celestia Base Camp. I think the spell costs about 200 gold. You must be an apprentice crafter to use this spell, and have a card crafting station. Best of luck
A really good way to make a large amount of rare reagents is to find a transmute card for that particular reagent. This "Card" is actually a recipe that allows you to craft that recipe out of other reagents. This particular reagent transmute can be found from Avery Templeton in the Survey Camp in Celestia. Good luck Wizard!!! ~Wolf IronCaller~
You can find water lilies sometimes when you harvest cat tails around the spiral - the water lilies are a "rare" drop. More info is on the Wizard101 Wiki - under "Crafting > All Reagents > Harvested > Rare Harvested."
You may also be able to buy some in the Bazaar in Wizard City's Olde Town, if gold isn't a problem and if there are any there. Keep refreshing the screen for the reagents for sale in case someone sells them to the Bazaar, and you might be able to scoop some up.