So it is so far not related to 3d issues.
My first guess is that it is related to memory handling. Have you altered your system memory or used the CA.ini fix mentioned on PositiveG's site ? For me, only the CA.ini entry was necessary, but the maxmem method worked also.
Have you made the
"[Backdoors]
Heap_Percent=0"
entry into your CA.ini ?