The Black Caps, ranked eighth above only Zimbabwe and Bangladesh, helped themselves to 117 for one in their second innings before heavy rain forced play to be called off at the Zahur Ahmed Chowdhury Stadium.
Peter Fulton was unbeaten on 44 and Kane Williamson was on 28 with the Black Caps going into the final day’s play on Sunday with a lead of 85 runs and nine wickets in hand.
Hamish Rutherford was dismissed for 32, but both sides face a tough task to force a result on the slow, newly-laid pitch that has offered no assistance to either seam or spin bowlers.
Gazi, a 22-year-old off-spinner whose highest score in six previous Tests was 32, remained unbeaten on 101 after coming in to bat at number eight. He hit 10 boundaries and three sixes.
Gazi and Robiul Islam exposed the limitations of the Black Caps’ attack by adding 105 runs for the ninth-wicket to help Bangladesh grab the first-innings lead.
Robiul hit a career-best 33 before he was dismissed soon after lunch, caught off seamer Doug Bracewell, who returned as New Zealand’s most successful bowler with three wickets for 96 runs.
Frontline spinners Bruce Martin and Ish Sodhi both conceded over 100 runs, while part-timer Williamson went for 1-83 in 27 overs.
Bangladesh added seven runs to their overnight score of 380-7 when they lost Abdur Razzak for seven, leg-before to seamer Trent Boult.
Bangladesh are looking for their first Test win against New Zealand, having lost eight and drawn one of their nine matches against the Black Caps.
The second Test will be played in Dhaka from October 21 to 25, followed by three One-Day Internationals (ODI) and one Twenty20 match.
New Zealand 469 and 117 for 1 (Peter Fulton 44*, Hamish Rutherford 32; Nasir Hossain 1 for 20) lead Bangladesh 501 (Mominul Haque 181, Mushfiqur Rahim 67, Sohag Gazi 101*; Doug Bracwell 3 for 96, Corey Anderson 2 for 34, Trent Boult 2 for 50) by 85 runs.