Gazi’s hat-trick, which gave him his sixth wicket in the innings, saw the Black Caps slump from 260 for four to 260 for seven in their second knock after lunch on the fifth and final day.
The tourists declared the innings closed at 287 for seven, leaving Bangladesh a target of 256 runs in a minimum of 45 overs to win the Test and take the lead in the two-match series.
The hosts showed no appetite to chase the stiff target and ended play on 173 for three in the 49th over, with Shakib Al Hasan unbeaten on 50 and newcomer Mominul Haque on 22.
Gazi, who hit an unbeaten 101 on Saturday, joined Alok Kapali as only the second Bangladesh bowler to claim a Test hat-trick.
Kapali had achieved the feat against Pakistan in Peshawar in 2003 when he dismissed tail-enders Shabbir Ahmed, Danish Kaneria and Umar Gul in three consecutive deliveries.
The draw loomed after New Zealand made 469 in their first innings and Bangladesh replied with 501 on a newly-laid pitch that provided no assistance to the bowlers.
Gazi, playing only his seventh Test, claimed all the six wickets to fall on the final day to finish with six for 77 in 26 overs.
Bangladesh have yet to win a Test against New Zealand, having lost eight and drawn one of their nine previous matches against the Black Caps.
The second Test will be played in Dhaka from October 21 to 25, followed by three One-Day Internationals (ODIs) and one Twenty20 match.
New Zealand 469 (Kane Williamson 114, BJ Watling 103) and 287 for 7 decl. (Peter Fulton 59, Kane Williamson 74; Sohag Gazi 6 for 77) draw with Bangladesh 501 (Mominul Haque 181, Sohag Gazi 101) and 173 for 3 (Tamim Iqbal 46, Shakib Al Hasan 50*).