It was the visitors who went ahead through Stephen McGinn and it was only a goal five minutes from time by substitute Derek Riordan which spared the home side's blushes.
It was a lacklustre display by Celtic who, in the end, were grateful for a share of the points.
For the visitors it was a marvellous result, especially after their five-goal defeat at home last week.
Celtic went on the attack early on but Paul Hartley's shot was gathered by keeper Chris Smith. Aiden McGeady then cut in from the right but his effort was deflected for a corner.
Keeper Smith came off his line well to gather Gary Caldwell's cross and Celtic found it difficult to find a way through Saints' five-man midfield.
On a rare break, Mark Corcoran's shot was blocked by Steven Pressley.
A slack pass by Gary Mason was intercepted by Jiri Jarosik, but Smith turned his strike behind for a corner from which Celtic should have taken the lead. Hartley's cross found Jarosik on his own in the six-yard box, but he headed weakly at the keeper.
Just before half-time Celtic won a free-kick at the edge of the box, McGeady curled his shot round the wall, but it struck the side-netting.
Celtic stepped up the pressure after the break and went close to taking the lead. From Hartley's free-kick Jarosik headed against the post with keeper Smith helpless.
McGeady then tested Smith with a 20-yard shot before Corcoran went down in the Celtic penalty area, but his claim for a penalty was dismissed.
In another break, Franco Miranda fired in a shot which stung the palms of Mark Brown as Saints started to threaten.
Amazingly, the visitors took the lead in the 74th minute. Billy Mehmet forced his way past Hartley and when Brown could only weakly push out his powerful shot, McGinn stepped in to shoot into the empty net.
Smith then made a superb save to touch over McGeady's deflected shot which looked as though was creeping in under the bar.
It was the Celtic substitutes who combined to earn a point for the home team.
Evander Sno's cross to the back post eluded Ryan McCay and Riordan controlled the ball with his right foot, before sending a left-foot shot high into the net.