Ballmer has also shouldered the blame for the controversial and so far not very successful decision to have Microsoft manufacture and brand its own tablet, the Surface, an attempt to mimic the model popularized by Apple with its combination of iOS and the iPhone, iPod and iPad devices.
The move has upset the company's hardware partners, because they view it as unexpected competition from Microsoft. Beyond that, the first Surface models haven't sold well.
In its fourth quarter, which closed in late June, Microsoft missed Wall Street's revenue and profit expectations while taking an almost $1 billion charge related to the dismal sales of the Surface RT, the model that runs Windows RT, the Windows 8 version for ARM chip devices.
The other Surface model, the Pro, which runs x86 chips, has been criticized for being too expensive and for being a battery hog.
In mid-July, Ballmer shook up the company's executive ranks with a broad reorganization billed as necessary to reinvent Microsoft as a devices and services company, and evolve from being a provider of packaged software.
The goal is to make Microsoft function more cohesively and be more efficient and innovative so it can better compete against rivals like Apple, Oracle, IBM and Google.
The reorganization, which is being implemented now, dissolved the company's five business units — the Business Division, which housed Office; Server & Tools, which included SQL Server and System Center; the Windows Division; Online Services, which included Bing; and Entertainment and Devices, whose main product was the Xbox console.
Those business units are being replaced by four engineering groups organized by function, around OSes, applications, cloud computing and devices, and by centralized groups for marketing, business development, strategy and research, finance, human resources, legal and operations.
However, the plan has also met with skepticism among those who believe that it will lead to less accountability, less clarity and ultimately less agility.
Others maintain that the "One Microsoft" mantra at the center of the reorganization is misguided because the opposite approach is needed, namely to reorganize it into more independent operating companies because it now houses businesses and products that are too different — like the SQL Server enterprise database and the Xbox console.
Ballmer's departure will be a historic turning point for Microsoft. Ballmer, who is 57 and joined Microsoft in 1980, has been CEO since 2000. In a statement, he said the decision to step down wasn't easy, but that he believes it is the right one.
In particular, he's convinced Microsoft should have a CEO who is on board for the entire "transformation" process set off by the reorganization announced last month.
"Our new senior leadership team is amazing. The strategy we have generated is first class. Our new organization, which is centered on functions and engineering areas, is right for the opportunities and challenges ahead," Ballmer wrote.
Wall Street's response has been enthusiastic, with the share price up almost 7 percent in late morning trading.
Sign up for CIO Asia eNewsletters.