JDiff for Network Engineers: Intelligent JSON Data Comparison
Introduction
In today’s data-driven world, comparing and analyzing structured data is a common task, especially in network operations and API-driven applications. jdiff is a powerful Python library built and maintained by Network to Code that makes this task easier by providing intelligent comparison capabilities for JSON data structures. Whether you are a network engineer, DevOps practitioner, or data analyst, jdiff can help you make sense of complex data comparisons.
What Is jdiff?
Powered by JMESPath, jdiff goes beyond basic equality checks. It intelligently compares JSON data objects, surfacing differences in ways that make sense for real-world use cases. The library is particularly useful when working with network device data, but is versatile enough to handle any JSON-structured data.
Who Is jdiff For?
Jdiff is particularly valuable for:
- Network Engineers
- Compare the device state before and after changes
- Validate network configuration payloads
- Check operational state changes
- DevOps Engineers
- Validate API responses
- Compare system states
- Use automated testing for structured data
- Data Analysts
- Compare JSON datasets
- Validate data transformations
- Check data quality
Installation
pip install jdiffExamples
1. Basic Comparison with exact_match
In the following example, we use the compare_interface_state() function to take two dictionaries and evaluate whether they are the same. In this example, we treat one dictionary as a pre-operation state and another as the post-operation state.
The exact_match CheckType object from jdiff is where we specify we need an ‘exact’ match in order for the evaluation to declare passing. In the example, the pre-operation state has eth0 “up” and eth1 “down” with the post-operation state eth0 “up” and eth1 “up”, therefore, the final comparison did not pass.
#01_basic_comparison.py
#!/usr/bin/env python3
"""
Basic comparison example using jdiff.
This example demonstrates how to perform a basic comparison between two JSON objects
using jdiff's CheckType class.
"""
from typing import Dict, Any, Tuple
from jdiff import CheckType
def compare_interface_states(pre_state: Dict[str, Any], post_state: Dict[str, Any]) -> Tuple[Dict, bool]:
"""
Compare interface states between two network device states.
Args:
pre_state: Dictionary containing the pre-change interface states
post_state: Dictionary containing the post-change interface states
Returns:
Tuple[Dict, bool]: Dictionary of differences and boolean indicating if comparison passed
"""
try:
# Create an exact match check type
check = CheckType.create("exact_match")
# Evaluate the comparison
result, passed = check.evaluate(pre_state, post_state)
return result, passed
except Exception as e:
print(f"Error during comparison: {e}")
return {}, False
def main() -> None:
"""Main function demonstrating basic comparison usage."""
# Example interface states
pre_state = {
"interfaces": {
"eth0": "up",
"eth1": "down"
}
}
post_state = {
"interfaces": {
"eth0": "up",
"eth1": "up"
}
}
# Perform the comparison
result, passed = compare_interface_states(pre_state, post_state)
# Print results
print("Interface State Comparison Results:")
print(f"Pre-state: {pre_state}")
print(f"Post-state: {post_state}")
print(f"Comparison passed: {passed}")
print(f"Differences found: {result}")
if __name__ == "__main__":
main()
Here is the output from the script above:
# python 01_basic_comparison.py
Interface State Comparison Results:
Pre-state: {'interfaces': {'eth0': 'up', 'eth1': 'down'}}
Post-state: {'interfaces': {'eth0': 'up', 'eth1': 'up'}}
Comparison passed: False
Differences found: {'interfaces': {'eth1': {'new_value': 'up', 'old_value': 'down'}}}
2. Threshold Comparison with tolerance
In the second example, we want to demonstrate comparison where small differences between values are acceptable. We can do so with the ‘tolerance’ CheckType object from jdiff.
In the example below, we specify a tolerance range of 10% for IPv4 routes and 5% for IPv6 routes. In our mock data, our IPv4 route decreased from 1000 to 950 (5% decrease) while the IPv6 route decreased from 500 to 450 (10% decrease). The result is IPv4 routes comparison passed because it is within the tolerance range, while the IPv6 result failed because the result exceeded the tolerance range.
#02_threshold_comparison.py
#!/usr/bin/env python3
"""
Threshold-based comparison example using jdiff.
This example demonstrates how to perform comparisons with thresholds,
useful for comparing numerical values within an acceptable range.
"""
from typing import Dict, Any, Optional, Tuple
from jdiff import CheckType
def compare_with_threshold(
value1: Dict[str, Any],
value2: Dict[str, Any],
threshold: float,
path: Optional[str] = None
) -> Tuple[Dict, bool]:
"""
Compare values with a specified threshold.
Args:
value1: First dictionary to compare
value2: Second dictionary to compare
threshold: Acceptable difference threshold (0.0 to 1.0)
path: Optional JMESPath to specific values to compare
Returns:
Tuple[Dict, bool]: Dictionary of differences and boolean indicating if comparison passed
"""
try:
# Create a tolerance check type
check = CheckType.create("tolerance")
# If path is provided, extract the specific values
if path:
# Simple path extraction for nested dictionary access
# This handles paths like "routes.ipv4"
def get_nested_value(data: Dict[str, Any], path: str) -> Any:
keys = path.split('.')
current = data
for key in keys:
if isinstance(current, dict) and key in current:
current = current[key]
else:
raise KeyError(f"Path '{path}' not found in data")
return current
# Extract values using the path
val1 = get_nested_value(value1, path)
val2 = get_nested_value(value2, path)
# Compare the extracted values
result, passed = check.evaluate(val1, val2, tolerance=threshold)
else:
# Compare entire dictionaries if no path specified
result, passed = check.evaluate(value1, value2, tolerance=threshold)
return result, passed
except Exception as e:
print(f"Error during comparison: {e}")
return {}, False
def main() -> None:
"""Main function demonstrating threshold-based comparison usage."""
# Example route counts
pre_state = {
"routes": {
"ipv4": 1000,
"ipv6": 500
}
}
post_state = {
"routes": {
"ipv4": 950, # 5% decrease
"ipv6": 450 # 10% decrease
}
}
# Compare IPv4 routes with 10% threshold
ipv4_result, ipv4_passed = compare_with_threshold(
pre_state,
post_state,
threshold=10, # 10% threshold
path="routes.ipv4"
)
# Compare IPv6 routes with 5% threshold
ipv6_result, ipv6_passed = compare_with_threshold(
pre_state,
post_state,
threshold=5, # 5% threshold
path="routes.ipv6"
)
# Print results
print("Route Count Comparison Results:")
print(f"Pre-state: {pre_state}")
print(f"Post-state: {post_state}")
print(f"\nIPv4 routes comparison (10% threshold):")
print(f"Passed: {ipv4_passed}")
print(f"Differences: {ipv4_result}")
print(f"\nIPv6 routes comparison (5% threshold):")
print(f"Passed: {ipv6_passed}")
print(f"Differences: {ipv6_result}")
if __name__ == "__main__":
main()
Here is the output from the script above:
# python 02_threshold_comparison.py
Route Count Comparison Results:
Pre-state: {'routes': {'ipv4': 1000, 'ipv6': 500}}
Post-state: {'routes': {'ipv4': 950, 'ipv6': 450}}
IPv4 routes comparison (10% threshold):
Passed: True
Differences: {}
IPv6 routes comparison (5% threshold):
Passed: False
Differences: {'index_element': {'new_value': 450, 'old_value': 500}}
3. Network State Comparison with exact_match and specific path
The third example shows how to compare network device states with specific comparison paths, such as interface state, IP addresses, and description.
In the example below, we use JMESPath expressions to target specific configuration elements and use the exact_match CheckType object, as we saw from the first example, for comparison. For the paths, we can use interface.*.state for interface comparison and interface.*.ip and interface.*.description for IP address and interface description comparison.
Since both GigabitEthernet 1 and GigabitEthernet 2 state were both “up” in the before state, but GigabitEthernet 2 state was “down” in the after state, the interface state comparison state would fail. However, the IP address and descriptions should pass since no changes were detected.
#03_network_stat_comparison
#!/usr/bin/env python3
"""
Network device state comparison example using jdiff.
This example demonstrates how to compare network device states before and after changes,
focusing on specific aspects like interface states and configurations.
"""
from typing import Dict, Any, Optional, List, Tuple
from jdiff import CheckType
def compare_network_states(
pre_state: Dict[str, Any],
post_state: Dict[str, Any],
paths: Optional[List[str]] = None
) -> Dict[str, Tuple[Dict, bool]]:
"""
Compare network device states across multiple paths.
Args:
pre_state: Dictionary containing the pre-change device state
post_state: Dictionary containing the post-change device state
paths: List of JMESPath expressions to compare
Returns:
Dict[str, Tuple[Dict, bool]]: Dictionary of comparison results for each path
"""
results = {}
if paths is None:
paths = ["interfaces.*.state", "interfaces.*.ip"]
try:
# Create an exact match check type
check = CheckType.create("exact_match")
for path in paths:
# For each path, evaluate the comparison
result, passed = check.evaluate(pre_state, post_state, path=path)
results[path] = (result, passed)
return results
except Exception as e:
print(f"Error during comparison: {e}")
return {path: ({}, False) for path in paths}
def main() -> None:
"""Main function demonstrating network state comparison usage."""
# Example network device states
pre_change = {
"interfaces": {
"GigabitEthernet1": {
"state": "up",
"ip": "192.168.1.1",
"description": "Primary Uplink"
},
"GigabitEthernet2": {
"state": "up",
"ip": "192.168.1.2",
"description": "Secondary Uplink"
}
}
}
post_change = {
"interfaces": {
"GigabitEthernet1": {
"state": "up",
"ip": "192.168.1.1",
"description": "Primary Uplink"
},
"GigabitEthernet2": {
"state": "down",
"ip": "192.168.1.2",
"description": "Secondary Uplink"
}
}
}
# Define paths to compare
comparison_paths = [
"interfaces.*.state",
"interfaces.*.ip",
"interfaces.*.description"
]
# Perform the comparison
results = compare_network_states(pre_change, post_change, comparison_paths)
# Print results
print("Network Device State Comparison Results:")
print(f"Pre-change state: {pre_change}")
print(f"Post-change state: {post_change}")
print("\nComparison Results:")
for path, (result, passed) in results.items():
print(f"\nPath: {path}")
print(f"Passed: {passed}")
print(f"Differences: {result}")
if __name__ == "__main__":
main()Here is the output from the script above:
# python 03_network_state_comparison.py
Network Device State Comparison Results:
Pre-change state: {'interfaces': {'GigabitEthernet1': {'state': 'up', 'ip': '192.168.1.1', 'description': 'Primary Uplink'}, 'GigabitEthernet2': {'state': 'up', 'ip': '192.168.1.2', 'description': 'Secondary Uplink'}}}
Post-change state: {'interfaces': {'GigabitEthernet1': {'state': 'up', 'ip': '192.168.1.1', 'description': 'Primary Uplink'}, 'GigabitEthernet2': {'state': 'down', 'ip': '192.168.1.2', 'description': 'Secondary Uplink'}}}
Comparison Results:
Path: interfaces.*.state
Passed: False
Differences: {'interfaces': {'GigabitEthernet2': {'state': {'new_value': 'down', 'old_value': 'up'}}}}
Path: interfaces.*.ip
Passed: True
Differences: {}
Path: interfaces.*.description
Passed: True
Differences: {}Best Practices
- The library only focuses on data analysis, not on data gathering—which should be handled externally.
- Use Specific Paths: Specify the exact path to the data you want to compare to improve performance and provide clarity.
- Document Your Comparisons: Keep track of what you are comparing as the data gets more complex and extensive.
Conclusion
Jdiff is an excellent tool for anyone who works with JSON-structured data, particularly within network operations. Its ability to compare data simplifies change validation, state monitoring, and system consistency. Jdiff empowers network administrators, DevOps practitioners, and data analysts to make more informed decisions about their data.
Additional Resources
- jdiff readthedoc: https://jdiff.readthedocs.io/en/latest/
- jdiff GitHub Repository: https://github.com/networktocode/jdiff
- JMESPath documentation: https://jmespath.org/
- Discover more NTC open source projects: https://networktocode.com/community/open-source/
– Eric
