As the saying goes, “knowledge is power,” but no one understands the vast difference between information and knowledge better than a network administrator.
In networking, big data typically refers to the massive amounts of traffic, trunk, and device information collected from standard management systems and interfaces, gathered from probes deployed at various endpoints and from network-layer software within client and server devices. When this data is placed within standard management system interface frameworks, some information may reflect current fault, configuration, accounting, performance, and security (FCAPS) management practices, but most companies cannot correlate data from client/server devices with current operational activities. This is precisely where big data and big data analytics come into play.
The most critical factor in effectively leveraging network big data is ensuring precise event timing for all data elements. Networks are about the instantaneous conditions and juxtaposition of events; losing time synchronization means completely losing value when analyzing information. If all data collection is timed from a common source, time synchronization can be ensured. If not, you should introduce synchronization events into the big data collection points to correlate the timestamps of all records at regular points.
Building Mappings to Identify Network Problems
After ensuring that event times can be accurately correlated, the next step is to establish a mapping between this common timeline and network problems. Information about the source of network problems may come from current FCAPS processes, user complaints, or client/server telemetry. The latter may also help recover quality-of-experience information, such as response times, and network performance data measuring packet loss rates and latency (for example, from TCP window sizes). This mapping allows big data analytics to explore the relationship between these problem points and the metrics just before the problem first appeared.
This type of big data analytics can help analyze the root causes of network problems, which is often impossible through other means. Because network environments change very rapidly, administrators frequently chase problems from one place to another, yet never manage to find the correct cause when the problem occurs. Big data analytics can correlate thousands (or millions) of data elements with known problem points, identify correlations, and then pinpoint the root cause through data analysis.
Determining Normal Operating Conditions
Another strategy for using big data to solve network problems is to use big data to derive baseline data of the normal network environment. If the previous step (mapping problem points to the big data common timeline) is done correctly, we will know what the network looked like when there were no problems. Collecting and analyzing network data from these “well-running” periods will allow administrators to determine what constitutes normal network behavior and quantify this “normalcy” based on the volume of collected data.
The baseline normal behavior can then be used to analyze periods of network operation not considered problematic but not entirely confirmed as normal operational behavior either. Experienced network administrators know that networks sometimes enter an unstable state without actually experiencing failures or receiving user complaints. Conditions in the network, overall demand, or server resource status can also affect network operations. Baseline data can help identify the causes of such conditions.
Big Data Analytics Can Help Find Ways to Fix the Network Environment
We need to look for behaviors where analysis shows the network environment failed to generate problem reports, even when it closely mimicked a problem period. Here, the goal is to use analytics to explore what mitigated the expected problem; this may improve your root cause analysis or provide alternative ways to fix the environment.
Another area to examine is how resources respond to network events, application or server events, or changes in user traffic load. When these aspects change significantly, the network should respond in predictable ways. For example, a significant change in application traffic typically leads to a noticeable increase in response times and a rise in packet loss rates.
But if these behaviors occur without an accompanying significant change in traffic, it indicates that resources are already overloaded. Likewise, if a significant traffic change occurs without an accompanying increase in response time or network packet loss, it may suggest the network is over-provisioned. In such cases, some capacity could be reduced, helping to preserve a leaner operating budget.
Focusing Only on Actionable Intelligence
A final piece of advice: some administrators