Pichai is largely expected to make major changes in some of Google's core businesses. "If you make a change like this, the idea is to bring new ideas in and make changes to keep the company on top," said Kerravala. "If he just keeps the trains running, what value does he bring?"
Patrick Moorhead, an analyst with Moor Insights & Strategy, expects Pichai to more closely integrate Google's products, giving users a more connected experience across services.
Dave Besbris, who has had a short run in charge of Google+, is one of the executives being moved aside. Besbris took over Google's social network after Vic Gundotra, the well-regarded face behind the creation of Google+, left the company this past spring.
With Pichai now in charge, the social network, which has been quietly existing without many updates or feature additions, could get a new burst of energy
"Google+ seems to be the weak link," said Kerravala. "Perhaps Pichai will try to improve Google's social position and make Google+ a stronger social networking tool."
Olds, on the other hand, doesn't think Google needs to make any big changes right now.
"I wouldn't expect to see any changes in Google-branded products right away, or at least any changes that are spurred by having Pichai as the new boss," he said. "I think a lot of this is more about Page wanting to do what he does best -- figure out new opportunities for Google."
Sign up for CIO Asia eNewsletters.