I set out today to write about Cisco ART metrics as I was having trouble understanding the difference between it and technologies such as AVC, MACE and Performance Monitoring as all of them can be exported using Flexible NetFlow. As I read different materials found on Cisco’s Web site, I became even more confused. Hate when that happens.
I found stuff like this when searching Cisco’s web site:
“Application Visibility and Control (AVC) provides visibility for various applications and the network to central network management stations. MACE (Measurement, Aggregation, and Correlation Engine) provides AVC services by measuring metrics on a subset of traffic and exporting those metrics to a target. This enables the traffic to be measured and analyzed and the applications’ performance to be base-lined, monitored, and troubleshot.”
I’m not disputing the accuracy of the above paragraph. I’m sure it is spot on but, those 60 words above left me even more confused. They also hardened my resolve to investigate this even further and document what these terms are, how they relate to NetFlow and IPFIX as well as the benefit to our joint customers with Cisco.
Flexible NetFlow
First of all, Performance Monitoring, Performance Agent, ART, MACE, AVC, NBAR, etc. they are all configured using the Flexible NetFlow (FnF) interface. FnF by the way is not a new version of NetFlow. It is a method of configuring NetFlow v9 or IPFIX to export cool pieces of information like round trip time, URL, host name, TCP window size or counters such as retransmitted packets or dropped packets.
< This section to explain the differences between platforms and IOS versions>
ART metrics are a subset of Performance agent. MACE is not synonymous with Performance agent because Performance Agent exists under Performance monitoring on IOS XE and IOS 15.4(1)T+. It may be tough to articulate that. The hierarchy of these looks like this:
IOS pre-15.4(1)T
AVC
Performance Routing (biflow or uniflow)
NBAR
MACE (biflow ONLY)
Performance Agent
ART metrics
WAAS metrics
Performance Monitor
Medianet
RTP Metrics
Round trip Time
With IOS XE and IOS 15.4(1)T+ it looks like this:
AVC
Performance Routing
NBAR
Performance Monitor
Performance agent
ART metrics
WAAS metrics
medianet
RTP Metrics
Round trip Time
</end explain section>
The above is what I began to realize after speaking with one of our top engineers and reading about these technologies. During the research, I learned what Flexible NetFlow can be used to configure and ultimately export to your incident response system. What’s challenging for Cisco is that flow technology is evolving fast which is why it is a bit confusing at times to research. Notice above that I pointed out ‘biflow’ and ‘uniflow’. This is because the device can be setup to export source and destination IP address or client and server IP addresses. These are entirely different elements and are NEVER exported at the same time. I say never but, someone somewhere will dispute this. GO FOR IT!!!
Anyway, MACE only supports the export of biflows which means that it can only export client and server IP addresses. If you look under the hood, these do correlate directly to source and destination IP address respectively BUT, in some vendor implementations they don’t. By the way, the biflows exported by configuring MACE are not compliant with RFC 5103. Probably splitting hairs here but, wanted to point that out because to be compliant with the RFC you are supposed to export an initiator bit and Cisco doesn’t within MACE. That’s the geek in me is coming out….
If you read in the Cisco MACE configuration documentation you will see the snippet below.
<<< BEGIN >>>
It aggregates the flows based on the following Layer 7 (L7) information:
- Destination address
- Destination port
- Layer 4 protocol
- Segment ID
- Source address
<<< END >>>
The mistake above may not be obvious to most people. Basically the problem is with the words ‘source’ and ‘destination’ as these words have been replaced by ‘client’ and ‘server’ for the MACE export. I’ll contradict myself here and point out that unique to Cisco and in the case of MACE, the source IP address does correlate directly with the client IP address because the biflow logic used is stateful. The term ‘stateful’ is a whole other blog. Anyway, in other vendor implementations, the client could actually be the source or destination depending on who initiated the flow and the interface the flow is metered on. Are you confused and starting to pace?
Later on in the SUMMARY STEPS in the above document it states:
6. collect counter client bytes
7. collect counter server bytes
Source and destination IP address are not referenced in this section but, I did find something else that caused confusion in the Flexible NetFlow configuration example:
<<< BEGIN >>>
- collect art client bytes
- collect art count new connections
- collect art count responses
- collect art count late responses
- collect art count responses histogram
- collect art all
- collect datalink mac source address input
<<< END >>>
It is very important to keep in mind that client and source are the same values for Cisco MACE. Although this confusion needs to be cleaned up in the configuration, I think it certainly demonstrates Cisco’s continued commitment to develop new exports with better – richer traffic details. Remember, Cisco is a big company with lots of engineering departments and this technology is growing so fast that sometimes it evolves with capabilities that could not have been foreseen. Good news: there is an effort underway at Cisco to clear up all this jargon and simplify it.
If you would like to export one of these technologies from your Cisco router or switch, how you reference these technologies on the CLI depends on the version of the firmware you are running. For example, in IOS 15.2(3)T some terms like “WAAS (MACE)” were replaced by “WAAS(Performance Monitor) in IOS 15.4.(1)T because Cisco has simplified configuration by allowing all of these metrics to be configured under Performance Monitor.
This document titled Advanced-Netflow-Capabilites was created by one of our engineers and it helps clarify some of these terminology changes. I believe some settling out of terms is to be expected as Flexible NetFlow continues to grow in popularity. Reach out to our team if you have questions.
If all of this is making your head spin like it did mine, you are not alone. This is probably why Cisco has further simplified configuration of these features by implementing something called EzPM or Easy Performance Monitor. EzPM allows you to implement a “best practices” performance monitoring configuration in just a few configuration lines without having to learn everything there is to know about these features! End rant ….. Hope you found it informative.
