I've been drinking coffee since I was five. I hated milk and would only drink it if my mom put a couple spoonfuls of coffee in it. I wanted more and more, and by the time I was 10 I was drinking coffee with a little milk in it! Never noticed any ill effects from it.
My opinion: The coffee is fine, but not the sugar! Put some half-and-half in it, and some stevia if you want it sweetened.