Whenever I am in a battle I always have my trusty calculator ready to keep track of any buffs that I am adding so that I can be ready to strike when I know I will have enough damage to kill. I have gotten pretty good at this, and I have gotten to the point where I can account for every variable that effects the damage calculation and get a pretty good idea of the value for the hit. However, no matter what I do I am always just a little bit off.
I will give an example. I am currently in Azteca using my Death wizard to fight Venomous Quetzals. My wizard has a 77% death damage boost from gear, and I know from the wiki that the Venomous Quetzals boost 40% to death attacks. I am using a Scarecrow (enchanted with Colossal) for a base damage of 583, plus a 50% Deathblade (enchanted with Sharpened Blade), a regular 40% Deathblade, and a 45% item Deathblade.
So to get the total damage the math should look like this:
583 • 1.77 • 1.5 • 1.45 • 1.4 • 1.4 = 4399
However, when I actually use the attack it only deals 4387 damage.
That means my calculations are off by 12. That is not very much, but I still would like my math to be as close as possible. That is a margin of error of less than 1%, so it doesn't seem like the numbers I am using are incorrect. Instead it appears to me as maybe some kind of rounding error.
Would it be possible for someone to explain to me a little bit more about the way the game processes this kind of math in combat, and what I might do to eliminate/account for these small errors I am seeing in my calculations?
There are two main things that make calculations seem a little off. The game displays whole numbers but seems to calculate damage in fixed-point binary, so it doesn't round quite the way you'd expect. Stat boosting pet talents show as whole numbers, but count at their true value (derived from pet attributes) so they can be up to a half percent higher or lower than shown.
