Yes, I’m unsure whether he’s just guiding both bodies of if he actually pushes them, but if, it’s just slightly. The vast majority of the additional momentum of the hammer comes from the ball. You can see just how low the ball bounces back. Not even half of its initial height. The ‘missing’ momentum (or energy) is mostly transfered to the hammer, the rest is dissipation.