Mechanisms in your eye and brain detect motion in various directions. For example, regions of your brain fire nerve impulses when your eye forms images that are rotating in a clockwise direction. Other regions respond to counterclockwise rotation. When something is stationary, both of these motion detectors still fire, but their firing rates are equal. The two signals balance each other out, and you see no motion.
As you stare at the spinning disk, the set of motion detectors that respond to its rotation adapts to the motion of the pattern. These motion detectors start out firing rapidly, and then slowly decrease their firing rate. When you look away from the rotating pattern and stare at a stationary object (such as your palm), the motion detectors that were already active fire less rapidly than the ones that have not been stimulated. As a result, you see motion in the opposite direction.
You also have sets of motion detectors that respond to upward and downward motion. Adaptation of these upward and downward motion detectors causes a similar effect (aptly known as the waterfall effect) that you notice when you stare intently at a waterfall.