When the bottom block is knocked away by the "whacker" force, the rest of the stack resists moving horizontally along with it. This is due to a phenomenon called inertia—a resistance to change in motion. Inertia is the subject of Isaac Newton's first law of motion: An object remains at rest, or in uniform motion in a straight line, unless it's forced to change its state of motion by forces acting upon it.
Mass is a measure of inertia, as shown in Newton's second law: Force = mass x acceleration. In other words, the more massive something is, the harder you have to push or pull it to change its state of motion. The stack of blocks is fairly massive, so it has quite a bit of inertia.
You may notice, though, that the stack moves horizontally just a bit, especially as the stack gets smaller and therefore has less inertia. That's because there's a frictional force between the bottom block and the stack above it. If we could turn friction off, the stack wouldn't move sideways at all.
The stack doesn't stay in midair, of course, but falls to the tabletop. That's due to a different force, the force of gravity. (But you knew that, didn't you?)