Definitely have a talk with him. Not a coffee date, a serious talk. He needs to understand that this is something that is bothering you, and could potentially be a make-or-break decision. You need to know what his priorities are, and if you beside him is one of those things.
On a completely different note, I get the sense that besides this boy, there isn't a lot of time spent with your own friends, especially if you are worrying about how long it takes for him to respond to your text messages. The boy has a life, for God sake! If you're going to be in a relationship, you need to have a life outside of him too. And I'm telling you right now, if you don't have that time set aside to spend with your friends, family and people outside of him, your relationship is doomed, end of story. Reason: There is no 'I-Miss-You' factor. This was SUCH an important part of my particular relationship; you become so dependent on him that you can't make any decision without thinking about him first. He's never going to miss you because you're AAAAALLLLWWWAAAYSSS there! When was the last time you went to the movies with you and your girlfriends? Shopping with your mom? Or fishing with your dad? Your man should take up 40% of your time, at the most. The rest is divvy'd up between friends, school, work, family, and just about everything else. You'll have fun stories to share with him when you do see him, and your relationship will grow.
I would strongly recommend reaching out to other GIRLS, and spending time away from your guy for a while. Don't break up; I didn't say that. Just stop responding to him right away, and when he asks what you're up to, message him 30 minutes later telling him you're thinking about going to a party that your girlfriend invited you to, and actually make sure that is happening! Enjoy your life, and surround yourself with an even playing field of PEOPLE. Remember: if you have to 'flirt' with other guys to make up for the time your man is not with you, then you're not ready to be in a relationship. Think about it.