Patrick Moorhead, an analyst with Moor Insights & Strategy, said, Rubin is known as someone well-suited to getting an idea off the ground and that Google may now want someone else to keep Android moving forward.
"Rubin is a classic entrepreneur, and Android, while far from mature, is in growth mode. Others are better-suited to take Android to the next level," Moorhead said. "It's uncertain what happened exactly to Rubin, but given his next role isn't defined, this probably means he was pushed aside."
Another interesting part of the announcement, according to both analysts, is that Google chose to replace Rubin with someone who has been leading, and will continue to lead, Chrome.
"We will see more of a maturing of Android, which is exactly what it needs right now," said Moorhead. "Chrome and Android will be coming together, something the industry knew had to happen ultimately. Giving the top job to the Chrome leader nearly ensures this is the case."
Gold noted that he doesn't think Android and Chrome should have been developed separately and it's a natural move to bring them together.
"I think as Chrome becomes a richer OS, I think you'll see some of the underpinnings of Android move in there," said Gold, who added that the integration may not happen immediately. "I think you'll see some of the kernel of Android, not all of it, move over. The Java machine, some of the driver capabilities and some components, will move over, and maybe some of the Android apps, like Google Play, could make it into Chrome and the Chromebook as native apps."
Users, though, shouldn't notice much of a change under Pichai's leadership - at least not in the short term.
"I don't think anyone will even notice to be honest with you three months from now, six months from now," said Gold. "I think for the market, this will just be a minor glitch. But a year or two out with a new head, you don't really know what is going to happen."
Sign up for CIO Asia eNewsletters.