Per app volume control arrived in Vista. Previously you had a single master volume control in the notification area which if you double clicked would open the sound mixer control panel.
That panel on XP (and before) only permitted per-device volume control.
For reference see [1] at around 04:00 and Larry Osterman's posts from September and December 2005 [2][3].
JACK is a huge amount more than just a per-app volume mixer, it's basically a virtual audio cabling system. Very handy if you've ever had a need to pipe audio between applications or manipulate audio streams from arbitrary apps.
It's also a complete pain in the arse to get running on Linux, due to the ALSA+Pulse stack being in the way.
If your goal is to use Linux for audio production (or some other use case where JACK would be necessary), it helps to use a distro like Musix or Ubuntu Studio that already takes care of that "get running" part for you.
JACK is pretty complex, but infinitely useful; one of those things I end up installing long before I decide I actually need it.
An app called Jack OS X does the job on Mac: http://www.jackosx.com/