The visitors went ahead through Dave Syers' goal from a corner but Steve MacLean forced the ball in for a 77th-minute equaliser and Tom Craddock won it with a header with seven minutes left.
Taylor said: "We didn't keep hold of the ball.
"I felt at half-time we needed that second goal. We had a brilliant opportunity to score a second when Omar Daley went through. He had an easy one really.
"But Oxford played very well. They are a team that get a lot of bodies forward.
"I can't fault my players' effort, but we are having to work hard because we don't keep hold of the ball."