The need for more telemetry data is real.
The word observability is being used quite a bit nowadays. I see it being used in discussions about SIEM tools. My guess is that you’ve heard it and have a general understanding that at its core the term refers to the collection of measurements, better known as telemetry. Telemetry data is extremely valuable for network monitoring, but the million-dollar question is why?
One of the biggest benefits of telemetry data is that it helps you get a better understanding of the real-world performance of your network, its devices, applications, or the data sources a business decides to focus on. Raw telemetry data, like syslogs or NetFlow/IPFIX, is delivered to specific tools to be analyzed and turned into actionable insights (I.e., observability). Examples would be Splunk for general log intelligence or Plixer’s Network or Security Intelligence engine for network traffic intelligence.
So how does observability serve today’s IT teams? It gives them the ability to answer any question about your network quickly and easily. Ok that’s a lot of academic babble, I know, but the truth is the more diverse data sources you have the better you can understand what is happening inside a network, and how the internal state of the network impacts business objectives and user experience.
Obtaining a high level of observability is traditionally a challenge. And with the projected growth of network devices to be well above six billion in 2022, the job is likely to be a bit more difficult. Moreover, security and network teams regularly introduce new products and tools into their environment. Providing network streams to each one of these injection tools to gain observability is often not achievable. Sometimes the UDP streaming devices don’t export to multiple locations, or the exporter lacks the ability to filter the stream to your specific needs.
So, what can you do? How do you continue to grow your network to meet operational demand and process telemetry data for security and performance insight?
You need an appliance that can replicate those data streams and forward them to your injection tools. To go about that, first, you need to determine how you see your NetOps and SecOps tools evolving over the next five years. Also, start to think about how your team and their needs are going to evolve. Finally, ask yourself if the demands of those new tools are going to grow at the same level as your team and their skills? My guess is the answer to the last question is no, and that means manually or lower-level administration of your streams, like open source or core level configuration/IPTABLES, just isn’t scalable. Basically, managing all the UDP streams configuration is and will be a nightmare.
How to choose a solution?
When evaluating a UDP forwarder solution, there are several features that should be kept in mind. For example, does the UDP forwarder solution:
- Detect when the destination hosts are offline and stop forwarding traffic?
- Only send packets to management stations that are up and operational?
- Help simplify network configurations by providing a single IP to which all routers and servers can send their log data?
- Provide a way to measure performance and workloads?
- Is the solution easy to configure and does it scale for larger environments?
- Provide fault tolerance and redundancy in case of a failure?
An open-source solution, in all likelihood, won’t give you the scale you need. If that’s the case, then you’ll want to consider a commercial-level UDP replication tool, like Plixer’s Replicator, that makes new device configuration a one-time effort.
Instead of touching every network device again, simply update Plixer Replicator to forward a single UDP datagram to wherever you want it—be it to a SIEM, flow collector, big data platform, or analytics application. This means that the destination IP address is changed but the source IP is not. As far as your ingestion tool is concerned, the data came from the original UDP export.
A tool like Plixer’s Replicator offers additional benefits, such as
- Reduces the amount of traffic on the network
- Reduces the load on routers and switches as they only need to send UDP messages to one location
- Lessens the configuration workload when all ~500 routers suddenly need to send NetFlow, sFlow, IPFIX or syslogs to a different IP address
- Provides management station redundancy by sending logs to multiple destinations simultaneously
- Allows both network and security administrators to receive the same log messages while maintaining separate systems.
Conclusion
If you’re looking to increase your network observability but find that you are constantly modifying UDP configuration and find the process confusing, then it’s time to investigate a UDP replicator. By duplicating and forwarding network streams, you can then ingest that data with security and performance tools that can give you invaluable insight. Check out our recent whitepaper that breaks down how Plixer’s Replicator works, or reach out to us if you want to learn more or need help with setting this configuration up.