Am I the only one who thinks this? - the internet slows down when you look at it?

Have you ever been downloading a torrent or something, and you open your torrent program to look at the speed, and see it flogging along at a nice 700kb/s, then 2 seconds after you see this speed, it goes down to something like 400kb/s? Am I the only one who thinks the internet slows down when you look at it?