User blog comment:B214/Talk About What You Want! 2/@comment-4547267-20140618222838/@comment-1359322-20140619000703

It depends on which type of family. For starters do you think the Gremory has any right to complain especially since Lord Gremory himself supposingly has a harem of his own. It'll just make the Gremory look like hypocrisy if they say Issei cannot form a harem of his own. Plus depending on the status of the girls, it might bring some good to the Gremory too. So its kinda like a win-win situation.