User blog comment:SpookyBoogie27/Light Novel Spoilers/@comment-82.178.115.57-20140207040435/@comment-16281343-20140208012812

That's the difference between Issei and other harem MCs that are typically dense. He's not making moves on the girls because of the traumatic experience he gained from Raynare.

There are several instances in the series where when Issei is thinking about whether the girls really liker her or not and he nudges that idea because he remembers what happened to his first relationship with a girl.