The O's looked full value for their early lead after midfielder Jason Demetriou curled a superb shot beyond Bradford keeper Rhys Evans in the 14th minute and with dangerman Adam Boyd showing plenty of neat touches, it looked only a matter of time before they would bag a second.
Martin Ling's side went into the game without conceding a goal in their previous five games and Bradford rarely tested their back four in the opening stages.
Debutant Steve Jones almost set up a Bradford equaliser when his 60-metre run stretched the O's defence after 23 minutes, but when his cross fizzed across goal there was no-one able to add the finishing touch.
Just after the half-hour, Orient striker Dean Morgan got the better of Thomas Moncur inside the City box, but his shot drifted just beyond the far post.
Seven minutes before the interval, Orient keeper Glenn Morris had a lucky escape when Michael Boulding's snap-shot from 30 yards bounced beyond him and rebound off the foot of the post.
City started the second half at a higher tempo and centre-half Matt Clarke had a strong penalty shout waved away after his shirt appeared to be tugged as he raced in at the far post.
Just before the hour, Bradford finally ended Orient's proud defensive record when Boulding got the better of his marker Alton Thelwell and calmly slid the ball past Morris from 12 yards.
City had their tails up, but their bubble was burst by former Leeds United man Granville, who couldn't believe his luck when he stooped to send a looping header into the Bradford net after being picked out by Dean Morgan's 65th minute corner.
Winger John Melligan should have wrapped up the game when he was put clear by Boyd's excellent back heel, but the O's man seemed to hesitate and Evans did well to block his shot.
The Bantams, who went out of the cup at the same stage last year, mounted a late rally, but Moncur saw his long-range shot pushed out by Morris and substitute Peter Thorne saw his header go wide.