Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

The key functions to consider when building or buying a log analysis platform

By Ariel Assaraf, co-founder and CPO, Coralogix | Feb. 7, 2017
Adoption of the wrong platform might make matters worse

predictive model data user magnifying glass graphs man

This vendor-written tech primer has been edited by Network World to eliminate product promotion, but readers should note it will likely favor the submitter’s approach.

“Life is really simple, but we insist on making it complicated.”  The immortal words of Confucius resonate with anyone who has ever tried to glean useful information from log data.

There are consensus-driven definitions of what exactly log analysis is, but a simplified, accessible explanation might be: to organize log entries into a human-friendly display and make business decisions based on what you learn.

1GB of log data is the equivalent of nearly 700,000 pages of text. Sifting through this manually would take nearly three years, or about the same amount of time it would take to read War and Peace 571 times consecutively (debatable as to which fate is worse). And if you consider that at least a few gigs of log data are produced per day even in small organizations, well… cue the data-parsing apocalypse.

Suffice to say, attempting to make sense of all your logs without some kind of automated assistance will lead to information overload, wasted time and serious staff demoralization.

Investing in a dedicated log analysis platform, be it an in-house build or 3rd-party software, should be seriously considered by the C-level powers.

So the solution appears simple: just find or build an automated analysis platform, right?  

Just be careful. Adoption of the wrong platform might just exacerbate the issue. Non-intuitive visualizations and statistical views can lead to hours of screen staring and head scratching.  

Log analysis platforms should solve the issue of information overload by breaking down the mass of log entries into digestible, relevant groups, and ignoring non-relevant data. Entries might be grouped according to common characteristics or patterns, e.g. the action performed or users performing them. Keep the following functionalities in mind when building your analysis platform or choosing an out-of-the-box solution:

* Broad collection – Make sure that all the log types you need are supported, including logs that originate from different operating systems, device types, and languages.

* Aggregation – Log entries are structured/written in a wide variety of ways, depending on their point of origin. Aggregation helps clear up this chaos by correlating similar logs together based on their shared patterns or characteristics, e.g. the action performed or users performing them. An aggregation feature should have some level of customization, such as saving logs to user-defined groups for ongoing reference.

* Search – A useful search function goes beyond a mere text box that returns matching log data. It’s important that queries return not only search-specific data results, but also allow users to easily access the context of those results, i.e. what occurred immediately before and after in the log.  


1  2  3  Next Page 

Sign up for CIO Asia eNewsletters.