Admittedly I don't use this machine much, and certainly not for anything that could be called number-crunching, but I haven't noticed any real difference between the two systems.
I doubt anyone who only uses their machine for average desktop type things (email, web etc.) would notice a difference - If their machine is new enough to have a 64 bit chip in it, then its probably quite fast anyway. Average desktop usages doesn't stress the cpu much at all, so a little gain in speed isn't really going to be noticed. Maybe if you do a lot of audio/visual stuff (CD/DVD ripping encoding etc.) you might see a difference...
In fact, if you are more interested in multimedia type stuff than number crunching, you probably are still better for (for now) stucking to the 32 builds, as there are still some issue around with 64 bit (web plugins usually).
The situation seems to be improving all the time though, and I expect quite soon these issue will all go away, at which point so will the discussion as to whether its best to install 32 or 64 bit OSes on a 64 bit chip. There will be no reason not to go 64 bit, as it is that little bit faster (plus all the other avantages like memory addressing etc.)
Chris