The assistant manager announced his departure during a pre-match press conference at the club's Lennoxtown training ground ahead of the trip to St Johnstone on Wednesday night.
The former Celtic and Scotland midfielder returned to the club two years ago and helped Deila's side win two Ladbrokes Premiership titles and a League Cup.
He said: "I will be going as well
I came with Ronny so I think it's only right and correct that I leave as well.
"The season will end at the weekend and I would imagine I'll be going on to pastures new.
"Obviously it's disappointing, but a lot of people would be delighted with what I've had.
"I've managed to play for the club and been part of the coaching team and I've got two league titles and a cup
History will dictate if it was good or bad.
"There's been lots of highs and obviously there have been lows, but that's the nature of the business we're in.
"Of course, we all want better, we all want more but we got the title, we've got five-in-a-row so we pass the baton on to the next manager and see what he does.
"The new manager will come in and bring his own team with him
That's normal
I'm going to have some summer holidays, some sunshine and then we'll see what happens."
Collins warned the new manager, whoever it is, will face exactly the same high demands that he and Deila have encountered.
He said: "It's a great challenge to be in at the top end, in the firing line
Every week is a challenge.
"You win and people are still not happy
You lose and everyone's very unhappy, but that's football, especially at big clubs.
"At big clubs there are high demands and Celtic is a big club, up there with the biggest, and whoever comes in will have the exact same demands that we've had and the previous managers have had."
The former Everton player also stressed that, with the Champions League qualifiers beginning in the middle of July, the next Parkhead management team will have to hit the ground running.
He said: "You can't get the new manager in quickly enough.
"He's got to assess the squad, prepare for the Champions League games which come straight away and we've got a lot of players away on international duty.
"So it's not an ideal scenario for a new manager to come into - we had it two years ago - it's not easy to manage, but it's not going to change so you just have to deal with the situation as best you can.
"There have been lots of names mentioned, lots of good managers and coaches, so fingers crossed we get the right one and he has success."
Source : PA
Source: PA