With Internet connections to cloud services growing rapidly and cyber attacks becoming craftier and more sinister, the need for improved traffic visibility is in high demand. In the past, both layer 7 application awareness and malware detection capabilities have been major separators when choosing between flow capture and packet capture for traffic analysis, but today the decision is most often NetFlow in lieu of packet capture.
Until the release of NetFlow v9, flows were limited to roughly 20 common fields. The rest of the packet contents were discarded. NetFlow v9 can be used to export any details found within a packet including the entire datagram. What makes flow technologies attractive over the raw packets is the ability to turn existing routers, switches and servers into distributed collection points.
Shortly after the introduction of NetFlow v9, router and firewall vendors started performing Deep Packet Inspection (DPI) to identify the applications hiding on ports such as TCP port 80. After DPI identifies the correct application, this additional information is included in the NetFlow export. Prior to DPI and the introduction of NetFlow v9, identifying the correct Layer 7 application (e.g., Skype, Citrix, Facebook) was not possible. Today several vendors including Cisco, Dell-SonicWALL, Palo Alto and nBox all provide layer 7 visibility in their flow exports.
Because of the enormous amount of detail available in today's flow exports, major router and firewall vendors have made packet capture less necessary. Now vendors are moving away from NetFlow v9 and as of July 11th, 2013 a standard for NetFlow has been accepted by the IETF which is called IPFIX. Several vendors are now supporting both NetFlow v9 and IPFIX.
In a 2012 analysis the Gartner group concluded that flow analysis should be done 80% of the time and packet capture with probes should be done 20% of the time. But some vendors also include flow details such as round trip time, packet loss, packet size, retransmits, jitter, HTTP host, URL and much more. Even packet sampling like sFlow is possible with NetFlow and IPFIX. These details allow network analysts to follow a flow and observe the hop by hop performance of a connection. Isolating exactly where a problem was introduced in the path (say packet loss) becomes much easier when the quality of the flow can be mapped out end to end. This is all done for NetFlow or IPFIX, not packet analysis.
Given that, the percentage of time flow analysis should be used might approach 90% or more. This is especially true when you consider the cyber threat detection usefulness of flow technologies.
Threat Detection with NetFlow
Cyber threat detection with flow technologies focuses primarily on behavior monitoring. Rather than performing Deep Packet Inspection (DPI) like a firewall and triggering events based on a single isolated signature match, behavior monitoring watches for odd behaviors over time. Odd behaviors trigger small increases in an index which can increase and decrease over time. If the index for any one host rises too fast and breaches a configurable threshold, action can be taken. Sophisticated malware is often identified with behavior monitoring - firewalls don't provide this intelligence.
Sign up for CIO Asia eNewsletters.