Well, it's a bit ... if-then-else in such a situation.
If on a 32 bit OS, the 3.5GB addressable memory would be shared between OS, programme and graphics card.
If on a 64 bit OS, the limit is 16GB ram on "Home Premium", but that's just hardware on motherboard... and the OS can still address more memory. Sooo, the 2GB video memory doesn't subtract from the RAM.
In your example, the 64bit OS sees 4GB+2GB=6GB. The complete 4GB is available for windows and programmes. Suppose Windows uses some 700MB of that, you have 3.3GB remaining for programmes.
As you said, a 32bit programme is limited to 2GB... unless it is LargeAddressAware. In that case, a 32bit programme will use up to ~3.5GB of memory. In the example, it would therefor be able to use all the remaining free memory.
Would you have had 8GB of RAM, two 32bit programmes could have taken up ~7gb of memory and there would have been 1gb for the OS to play with.