For that setup i used mediatomb. It is a ver simple program that says it can do transcoding but I was unable to do it. I think debian has it in the repos, but if not pretty easy to compile. One it is set up and the config file has the correct info in regards to databases (it can us mysql or sqlit as the back end) the the rest is through a web interface.
Nadim Hoque