A month ago I was on a call with a hardware vendor that exports flows and he asked “What else should we be exporting with NetFlow v9?”. This is a great question and fortunately almost any information observed by the flow exporter or passing through it can be exported in NetFlow v9 or IPFIX.
Generally we recommend IPFIX to send the information over older technologies such as SNMP traps or syslog. A big differentiator is that with IPFIX, we can stuff multiple messages into the same datagram while keeping the data structured which leads to faster event correlation and improved network threat detection. I told him that the items we hear the most demand for from NetFlow are:
- Application Aware Exports: These tell us what the actual application is for example: Facebook, Skype, Webex, BitTorrent and not just TCP port 80.
- Usernames: This tells us the end user who created the flow. Having the IP address is a great start but, the user name is even better.
- Round Trip Time: This provides the response time the packets are seeing within the flow to and from the end users
The message I wanted to convey is that the sky’s the limit on what you can export but, it best to choose wisely based on customer demand. Also, unless the flow developer works for Cisco, the vendor should export the unique data as IPFIX as it allows for vendor unique elements and NetFlow v9 doesn’t. IPFIX is emerging as the preferred export medium for flow information. Even Cisco is starting to support IPFIX. Why? Because IPFIX is the proposed flow standard and it includes enhancements such as the ability to export variable length strings (among other things).
Variable length strings allow vendors to export details such as URLs. Just about everyone wants details on the URLs being visited as it can sometimes simplify the troubleshooting process (E.g. identifying certain malware).
IPFIX allows for variable length data exports but, the strings must be formatted and structured correctly to avoid costly query times.
NOTE: developers should always check for IANA defined standard IPFIX elements before they create a proprietary one. This helps ensure support across vendors. Developers can also request new Information Elements from IANA.
The Value of Flow Information
I’d like to digress a bit more on why vendors should start migrating away from SNMP and syslog technologies in favor of NetFlow v9 and IPFIX.
SNMP is a polling protocol whereby a reporting system must poll the device every X minutes to get a response regarding details about the device. Even if there is no change, the device is still polled. Given the potential for voluminous amounts of flow data, SNMP wasn’t designed to carry this much information. There is also considerable overhead when using SNMP.
Syslogs, on the other hand, are like NetFlow v9 in that they are a push technology – in other words, no polling is necessary. Syslogs have been used by vendors such as NetScreen (see figure below) to export flow-like information. However, due to their unstructured data format, querying the data is often cumbersome and slow.
When NetScreen exported flow like information in syslog as show above, the export was not as well structured as NetFlow. Although the flow details were delimited, its format was proprietary. It also lacked significant support by the company to build a reporting community around the difficult to read export. Although NetFlow is owned by Cisco, Cisco opened it up and shared the architecture with the Internet community enabling other vendors to copy it. Vendors who copied NetFlow’s architecture (or IPFIX) sometimes renamed it as NetStream, J-Flow or AppFlow. These are all an exact or near exact replicas of NetFlow v9 or IPFIX.
How can flow information help your company? We’ll it depends on the data provided. If the goal is network trouble shooting and the reporting tool is leveraging NetFlow v5, possible top bandwidth consumer reports include but, are not limited to: the top hosts, protocols, applications via common port, ToS, autonomous systems, subnets, interfaces, next hop and more. Flows can also be used for accounting and billing purposes, historical archiving for regulatory compliance, and traffic anomaly detection. All of this can be done with NetFlow v5 or v9 with exactly the same tuple. However, why stop here.
NetFlow v9 and IPFIX can and are both used to send more than just aggregated packet details. Details on interface names and speeds, routing decisions, URLs, intrusions, viruses, syslogs, caller ID, round trip time and more are all being exported today with NetFlow v9 or IPFIX. The benefits of this new type of structured data export are becoming much more obvious in the IT world and this of course will ultimately lead to greater savings for the businesses which implement them.
In summary, the savings come in the form of:
- The lack of polling results in less overhead on the network.
- The insight provided by flow data reduces the need to deploy probes in strategic locations. This means less investment, less maintenance and less vendors.
- The details in flow exports result in faster troubleshoots, greater insight and ultimately a finer tuned network.
- The standardized format of the data means that it can be more easily correlated with exports from other vendors. This makes traffic monitoring, reporting and anomaly detection – faster, easier and more thorough.
- The same technology can be used to export different types of information (e.g. policy violations, byte counts, latency readings, user names, etc.).
As flow technology grows for network traffic monitoring, additional savings will likely be recognized. If you are interested in learning more, read my book on IPFIX and NetFlow. In it I digress more on this topic and explain why NetFlow v9 is where it’s at and why IPFIX is even better for exporting details as flows.