When a crystal gem has a child, that child inherits their gem stone and the parent gives up their physical form for their child. (We already know about this because of the “Laser light cannon” episode)

So each crystal gem gets a gem stone from one of their parents so that means every crystal gem has to lose either their mother or father to be born.

However Garnet has two gem stones which means….

She lost both of her parents.