Keep movies in VOB format or convert to another format?

Nope to both. try ps3 media server it will encode on the fly so hardly any need to convert at all first.

New Divx (Divx7) has support for mkv so if sony upgrades to that from whatever version they are on now it might (bear in mind mkv is just the container)