The only problem with this is none of the orphans ever get to leave.

Well no, when they grow up they are no longer children. They have to leave the horde. Its efficient too cause once his former orphans have a few children he can kill them for more orphans

the fuck kind of dragon just lets their horde walk away?

Its an investment. Previous orphans have kids. Dragon kills parents. Now he has more orphan kids than he started with.

Ok but now I'm imagining some sort of local folklore developing of "don't have more than 2 children or the dragon *will* come for you

Incredibly stable population under this one mountain - Paper