* Alerts – A system should alert users to repeated error occurrences, system anomalies, or the absence of typical events. Make sure alert settings can be adjusted to different delivery methods, time intervals and priority levels.
* Automation – In most log analysis systems, the above features are dependent on manual operation and programming by users. But the growing scale and complexity of log data has pushed the most advanced analysis platforms to embrace machine learning. A productive platform should have the ability to:
- Intelligently learn a business’s routine log flows
- Use that intelligence to automatically catch anomalies and errors as they occur in real time
- Generate alerts independently
- Provide a blueprint for devs to resolve identified issues
- Present data trends in intuitive visualizations
Technologies may change dramatically from year to year, but time still equals money. Less time spent manually hunting bugs and system events leads to earlier and more rapid releases, more time for innovative feature development, and happier users. Those are the essential benefits from the DevOps perspective, but log analysis is a diverse art, and has a surprisingly broad range of uses.
Who does it help beyond DevOps and IT?
A flexible log analysis system could conceivably become the singular source of analytics for an organization, across departments. Here are three (among many) examples:
- Security Engineers can exploit an Analysis platform’s search tools to find suspicious actions or breaches. The ideal platform should save search queries for repeated use, and automate the searches for a regular interval of security checks.
- Compliance – Log analysis can help keep your product in line with a range of compliance laws, and a strong automation component can speed up the process of auditing and review.
- Marketing and UX – Tools like Google Analytics and HubSpot might generally be considered irreplaceable for usage and engagement analytics. However, log analysis provides the same essential data: user traffic, referrals, usage time, click rates – they’re all in the log.
Though some departments will be resistant to experimenting with what they perceive as a dev-only tool, the C-suite’s collective ears might perk up at the mention of paying for just one multi-purpose analytics platform. This cuts down operational costs associated with buying a litany of analytics softwares for different teams.
To build or buy?
The market is flush with out-of-the-box log analysis systems, varying in levels of price, quality and versatility. Be mindful of the level of technical support offered by each, particularly if you plan to use the platform for departments without a dev background.
For those larger organizations that have the time, human capital and financial resources necessary to build in-house, this approach allows for made-to-measure customization for your particular industry and your business’s tech architecture. If this is your first time building in-house software, ponder this statistic during your cost-benefit analysis: the industry of data analytics enterprise software will reach nearly $200 billion in sales volume in the next two years. This is indicative of a general (though not exclusive) preference toward commercial software as opposed to in-house.
Sign up for CIO Asia eNewsletters.