City made light of an injury crisis to race into a three-goal lead but they almost threw the game away as Ronnie Moore's men staged a dramatic fightback.
Substitutes Richie Barker and Mark Robins fired home a late double to set Bradford's nerves on edge but skipper Ashley Ward sealed their first win at Valley Parade with the decisive fourth.
Rotherham midfielder John Mullin wasted a great chance after 19 minutes when he drove a shot straight at keeper Aidan Davison after a missed kick by Mark Bower and Ward made him pay dearly with the opener four minutes later.
City gained possession through a disputed free-kick but when Danny Cadamarteri swung over a cross from the left, Ward rose highest eight yards out to head over stranded keeper Mike Pollitt.
Bradford took the game by the scruff of the neck within a minute of the interval when fullback Gus Uhlenbeek opened his account for the club with a stylish finish.
Midfielder Andy Gray opened up the Rotherham defence with a precise through ball on the edge of the box and Uhlenbeek showed great composure to flick a right-foot shot over Pollitt from eight yards out.
After 56 minutes, Ward shrugged off two tacklers before sliding the ball into Michael Proctor's path and he side-footed it home from six yards.
Barker was left unmarked to head home Paul Warne's left wing cross after 71 minutes and with ten minutes to go experienced striker Robins pounced again to set up a tense finally.
City keeper Davison did well to push away a vicious strike from Martin McIntosh but Robins was on hand to fire in the rebound from close range.
With seven minutes to go Barker was inches away from meeting another superb Warne cross but Ward had the final word.
Pollitt made a mess of a Cadamarteri shot and when the ball slipped out of his grasp, Ward forced it under his body before slamming it home from a yard out