Well, Mike. If the DVD playes on your computer it isn't the ripping or compiling software. And you say that you are confidant in your burning software. Let me know when you're done banging your head against the wall.
Try setting Nero to burn at 1x only after using DVD Shrink. Sometimes if you go above this with compressed files you have problems playing back.
The DVDToolbox approach didn't work for me. I'm beginning to get good results by first using ImageTool to convert the shrunk fileset to an image, .img. The image file then burns successfully with RNM. I confess to having this problem with RNM with other kinds of filesets. See: http://supportforum.stompinc.com/showthread.php?s=&postid=5770#post5770