In any environment you need to license per PHYSICAL CPU cores. With the ESX environment you have an additional option, making use of the sub-capacity, if you choose. It simply allows one to license a sub-portion of the over all physical ESX CPU cores if the VMs you back up are not using all the ESX CPU cores.
Due to the nightmare of how it worked when I last looked into it, I do not muck with the sub-capacity. (When I first looked into it a report would have to be submitted quarterly stating the number of physical cores your VMs are using. We have way too many VMs for me to waste time on that!)