I think that if voldemort really wanted to kill harry potter the night the spell didn’t work on him he could’ve just picked him up and thrown him out a window given the fact that he was a one year old infant
first off, as soon as he killed lily, he could no longer touch harry. That’s why harrys touch killed quarrel and him in the first year and why he needed Harrys blood during his resurrection in the fourth year.
Second, as soon as the spell backfired, he was as killed as muach as someone who shattered his soul into seven pieces could be. That’s why he needed to cling to a snake, quarrel and needed to be resurrected in the first place.