User blog comment:B214/Volume 23 discussion blog/@comment-202.83.172.186-20170722150415/@comment-10330822-20170722193412

Since the Rias peerage is closest to Crom, it makes since Crom may have come to them simply because he isn't necessarily interested in fighting them, but he is interested in fighting their eventually opponents. This is something Issei's group couldn't do because Crom wants to fight Issei, even though his team has Asia.

Rias probably sweatened the deal with a bribe, which she is known to do, of bananas. In addition, there are probably battle requirements, similar to Fafnir, such as Crom getting to choose who he faces, hence why he doesn't participate in every match, and the freedom to fight alone in said matches.