You need to make sure the subtitle file has the exact same name as the video file except for the extension of course. Also, TMPGEnc cannot use AC3 audio, so that might be the problem with no sound.
for video playback i use media player classic. it can play those subs. or i would use directshow vobsub
Yeah don't use Windows Media Player, it sucks for almost everything. I use Winamp for media playing instead because it is good.
you may have to split the audio and the video into separate streams. this is called "demuxing." you can use the guide here on converting divx to avi to learn how to demux with vdub. you will then have to convert the audio to something different, like wav or mp2. then just select your original video as the video source and the demuxed and converted audio as the audio source and you should be able to hear it in the preview and once it is converted.
are you sure you have both BeSweet and the GUI? You need both the BeSweet package and the BeSweetGUI one in order for it to work. Then point the GUI to BeSweet.exe. I know this because I also was trying with just the GUI, and no, it didn't work.
ffdshow supports subtitles. If you enable it then it should playback with subs for wmp, TMPGEnc, etc. Your AC3 file? I thought that you had mp3 audio? If you were to frameserve the file to TMPGEnc via AVISynth then you can use VSFilter to add the subs and the audio will be decoded to PCM on the fly. You might want to try BeLight instead of the default GUI, it is a little simpler.
Hm, well I don't know what to do, so you should try BeLight. Here is the website: http://corecodec.org/projects/belight/ Also, make sure you have the newest version of BeSweet by downloading it here: http://www.dspguru.doom9.net/ Hope it works