After all your hardware building, that's not a pleasant outcome! But have you tried dumbing down your graphic card settings in Nvidia's control panel? Try everything as application controlled or off. If the driver is so old it doesn't have all the settings that might help, try Riva Tuner.
I've never seen these issues, so not clear on specific tweaks. But you know there are quite a few options in the wrapper...one is the "closer to real hardware" option. Also lowering the texture memory selection size if you have that maxed might help. And you might experiment with enabling the vertical synch option under Refresh rate.
