As far as I understand it, over the entire time period of your stack (lets say its 2 years for example), the velocity is calculated by looking at the absolute difference between the first and last point (lets say for one point that is 4mm) then you have a 2mm/yr velocity.
However, there might be other ways to calculate that. e.g. calculating the velocity for the period 1st image to prime and a second velocity for prime to last image and then the mean of both results. But as far as I remember (I looked into this in 2017/2018) It was done as described in the first option