There are two words that rhyme with orange. The botany term ‘sporange,’ which means ‘a sac where spores are made,’ and 'Blorenge,’ which is a mountain in Wales. Source
Also there’s like a half-dozen half-rhymes. Lozenge and door hinge come to mind.
Can’t think of any rhymes for Silver, though. Purple is also supposed to be rhymeless.
I bet there are old obscure words that rhyme with both, though!
Silver rhymes with Wilbur if you want to make a poem about the Wright brothers. And who can ever forget the classic purple nurple?