But now i am wondering if it is worth doing that. Will "rpmbuild SRPM" lead to performance advantage over i386 RPMS that are available on the site?
Perhaps, but I doubt it would be significant. Depends on what you're trying to tune. "Performance" can mean many things. I wouldn't rebuild everything just in hope that there would be a noticeable speedup. Determine what's actually slow and optimize just that, if it's possible. Blind optimization is a bad investment.