The Europeans needed to win by three goals to overtake the Black Sticks and then hope for a positive result in the final match of the day between Argentina and Australia.
However, they barely raised a challenge and when Celine Wilde missed an open goal from two yards early in the second half it looked like Germany, who were fourth in Beijing four years ago, were not going to get lucky.
New Zealand hit the post in the opening 30 seconds through Charlotte Harrison but otherwise the first half was evenly contested.
But that was not what Germany wanted as they needed goals and they should have made the breakthrough soon after the interval.
Natascha Keller's shot was deflected up off Kayla Sharland's stick but Wilde somehow diverted the high ball over from almost under the crossbar.
A diving deflection from New Zealand's Katie Glynn forced Yvonne Frank into a save and resulted in a penalty corner but Sharland's strike was blocked.
Germany thought they had finally scored when Julia Mueller hit in a high reverse-stick shot from a penalty corner but the ball had not travelled five yards out of the circle for her to be able to do that.
The Europeans protested but the video referral backed up the decision to rule out the goal and Germany ceased to be medal contenders as they went out with a whimper.
New Zealand's first Olympic semi-final is a huge step forward for a women's side who finished last in Beijing four years ago and whose previous best was sixth, according to coach Mark Hager.
'It's huge to get into the semi-finals,' he said. 'I don't think anyone expected us to, although we are always a threat if we play well and we started the tournament well and got on a roll.'