Indeed it took a hotly disputed penalty by Chris Sutton early in the second half to give Celtic all three points.
John Hartson had opened the scoring with a trademark header, but the home crowd were stunned when Jim Hamilton equalised for the visitors shortly before the interval.
In the last minute of the match Celtic keeper Rab Douglas was called upon to block Gus Bahoken close-range shot as Livingston came so close to snatching a draw.
It was low-key start to the game, but from Aiden McGeady's corner, Sutton touched the ball on only for Stanislav Varga to head over.
Celtic took the lead through a set-piece move. After Rubio pulled back Hartson, Alan Thompson curled in the resulting free-kick which the Celtic striker glanced into the bottom corner.
Livingston Colin Meldrum keeper then came off his line quickly to save the at the feet of Sutton. From Varga's long diagonal free-kick Hartson nodded down but Sutton was eventually crouded out.
A snapshot from Bahoken wide on the right nearly caught Douglas by surprise, but the keeper recovered to touch the ball over. From the resultant corner Livi drew level.
From Burton O'Brien's cross, Goran Stanic mishit shot fell kindly for Hamilton who slid the ball home under Douglas.
Celtic laid siege to the Livingston goal at the start of the second half and forced a series of corners. After Sutton was fouled just outside the area, McGeady's free-kick found the sidenetting.
Meldrum then produced an outstanding save to block Hartson's effort after Thompson's corner found him in space. The keeper did well to hold Ulrik Laursen's deflected shot before producing another incredible save to deny Hartson yet again.
From Sutton's cross Hartson had time to chase the ball down, but somehow Meldrum saved his well struck shot before Celtic finally made the vital breakthrough.
From McGeady's corner, the ball appeared to break off Hamilton's arm in the resultant scramble with referee Craig Thompson immediately pointing to the penalty spot. Sutton made no mistake with his spot-kick, sending the keeper the wrong way.
Meldrum again saved well to keep out Sutton's header at the back post. Anxiety crept into the Celtic defence at their inability to score another goal and in the last minute Livingston so nearly equalised.
Hamilton touched on a cross to Bahoken at the back post but from no more than six yards Douglas bravely parried his powerful shot.