The situation you are in isn't a good one. Going out with a friend's ex is "breaking girl code", unless your friend approves (and even then it might secretly bother her). It's interesting how he liked you, but dated your friend first--not that I'm trying to imply anything. I just think that you wouldn't be in this dilemma if he actually explored his interest. Anyways, I would talk to my friends, and talk to him, because he has to understand that he is your FRIEND'S ex, and a relationship with him can become a complete disaster because of hurt feelings or bitter feelings, etc. Elsi Flores is right though--it all comes down to, is it going to be your friends, or this guy? Perhaps your friends wouldn't mind, but that's a scarce reaction to such a situation. So think about this very seriously before you choose anything. You can't help who you like, so it's not like you're a terrible person :)
I hope that everything works out! Things like this happen, where you have to make big decisions, so just think on it, talk to your friends if you still want to pursue a relationship with this guy and need their blessing/okay, or talk to him and your friends or--however you want to do it, as long as you communicate to get it all sorted out! Good luck! :)