For a hosted math Mimetex is a proven open source solution (http://www.forkosh.com/mimetex.html).

Mimetex is a web-based program that generates LaTeX images (math) on the fly, if you use it like this: <img src="http://www.forkosh.dreamhost.com/mimetex.cgi?c=\sqrt{a^2+b^2... alt="" border=0 align=middle>.

To do a plugin that does [tex]\sqrt{a^2+b^2}[\tex] is simple (PHP).

   $text = preg_replace('/\[tex\](.*?)\[\/tex\]/ie',
   "'<img src=\"/cgi-bin/mimetex.cgi?'.rawurlencode('$1').'\"          
   align=\"middle\" />'", $text);

To do something like this with JavaScript is also trivial.

Mimetex is great beause A) you don't need to have a full TeX installation on your server, the program generates it internally B) you don't need to worry about scalability - mimetex can be compiled with caching enables, which makes it the same as static images