Bangladesh looked dead in the water at 33/4, but hundreds from Shakib Al Hasan and Mahmudullah scripted a miraculous five-wicket win that resulted in New Zealand being eliminated from the Champions Trophy.
Shakib hit 11 fours and a big six in his innings, while Mahmudullah smashed eight fours and two sixes in different parts of the field.
Tamim's opening partner Soumya Sarkar was also trapped leg before by Southee in the fifth over, and when Adam Milne bowled Mushfiqur Rahim with the score on 39, Tigers fans were fearing a humiliating defeat.
Both teams needed to win this match to maintain their hopes of reaching the semi-finals of a tournament featuring the world's top eight one-day worldwide nations.
The five-wicket win on Friday at Cardiff Wales Stadium meant that the whole of Bangladesh would be praying for England, the tournament favourite, to beat the old enemies Australia on Saturday, which will put Bangladesh in the semi-finals of the tournament, the furthest it would have ever gone in an ICC event.
Ross Taylor top-scored with 63 and New Zealand captain Kane Williamson made 57.
Besides Mosaddek's three wickets, Taskin Ahmed, who was playing his first Champions Trophy clash, bagged two wickets while Rubel Hossain and Mustafizur Rahman chipped in with a wicket each.
At 152-3, New Zealand still found themselves in a healthy position before Taylor added to the score with a partnership of 49 alongside Neil Broom.
New Zealand were unchanged from the side that lost to England by 87 runs at Cardiff on Tuesday.
Trent Boult (1-48) had lamented a struggle to find swing at the tournament in the build-up but his bowling partner Southee caused Bangladesh significant problems, accounting for the in-form Tamim Iqbal for a second-ball duck. Both the teams went into the match without even knowing whether a win will guarantee them a place in the semi-finals.
However, that was all for New Zealand in the game as Shakib Al Hasan and Mahmudullah stitched together a record partnership, taking Bangladesh past the target quite nonchalantly.
Shakib had a quiet tournament until Friday but against New Zealand, when he needed to stand up for the team, he did.
"We knew it was going to be hard because conditions weren't that easy". The desperation for a win aside, both New Zealand and Bangladesh would hope for an England victory against Australia, assuming rain does not disturb their calculations again.
When will the New Zealand (NZ) vs Bangladesh (BAN) ICC Champions Trophy 2017 Match be played?
Earlier, Mosaddek Hossain shocked New Zealand with three wickets in three overs as the Kiwis were restricted to 265 for 8 despite a great beginning. Shakib was finally dismissed in the 47th over for a 115-ball 114.
"It was frustrating to be on the other side of it, we wanted to come into this game firing but we didn't play our best cricket, we left a few runs out there", Williamson said.