Enhancing Scalability and Monitoring for Keysight Vision Orchestrator at HCLTech

Client: Keysight Technologies (via HCLTech)

Industry: Network Monitoring and Visibility Solutions

Project Duration: March 2022 – March 2025 (Individual Contribution)

Hilights: Backend Development (Go), Distributed Systems, Scalability Solutions, Caching Strategies, Monitoring and Observability (Grafana, Prometheus), API Development, Data Analysis and Visualization.

About HCLTech & Keysight Technologies:

HCLTech (https://www.hcltech.com/) is a global technology company with a vast workforce and a presence in numerous countries, providing industry-leading capabilities in digital, engineering, and cloud technologies. They partner with prominent clients like Keysight Technologies.

Keysight Technologies (https://www.keysight.com/) is a leading technology company that helps enterprises, service providers, and governments accelerate innovation to connect and secure the world. Their Keysight Vision Orchestrator (KVO) platform (https://www.keysight.com/it/en/product/SUB-KVO-10D/keysight-vision-orchestrator.html) is a key solution designed to centrally manage, configure, and monitor diverse network devices, including taps, virtual taps, data monitoring switches, and network packet brokers from both Keysight and third-party vendors. KVO empowers network administrators with comprehensive visibility and control over their network infrastructure.

The Challenge:

Keysight’s Vision Orchestrator (KVO), being a critical platform for managing complex network environments, faced ongoing challenges related to scalability, performance under increasing device loads, and the need for enhanced monitoring and observability to provide network administrators with actionable insights. Specific areas requiring improvement included:

• Scalability of Device Status Monitoring: Efficiently managing and retrieving the status of a growing number of network devices required optimization to prevent performance bottlenecks.
• Comprehensive Network Visibility: Providing detailed insights into policies, devices, and port activity was crucial for effective network management and troubleshooting.
• Accurate Network Statistics: Deriving precise statistics based on port bindings and device connections required improved labeling and data aggregation.
• Real-time Transceiver Monitoring: Network administrators needed the ability to monitor the light levels of transceivers for proactive identification of potential connectivity issues.

The Solution (Individual Contribution as Go Developer within HCLTech):

As a Go developer within the KVO Core backend team, my contributions focused on addressing these challenges through the design and implementation of key features and improvements:

• Scalability Improvements through Caching:
  • Analyzed the existing system for bottlenecks in retrieving device status information.
  • Designed and implemented caching strategies to store frequently accessed device status data in memory.
  • This significantly reduced the load on underlying systems and improved the responsiveness of the KVO platform when querying device status, enhancing overall scalability.
• Enhanced Monitoring with Grafana Dashboards:
  • Collaborated in team discussions to define key metrics for monitoring policies, devices, and ports within the network.
  • Developed and configured Grafana dashboards to visualize this data, utilizing both Prometheus (for standard metrics) and custom data sources (for KVO-specific information).
  • These dashboards provided network administrators with real-time insights into the health and performance of their network infrastructure managed by KVO.
• Improved Port Binding Labels for Accurate Statistics:
  • Identified limitations in the existing labeling system for port bindings (Attachment Points, Tools, etc.).
  • Proposed and implemented improvements to the labeling schema to provide more granular and accurate information about port connections.
  • This enabled the collection of more precise statistics using Prometheus, offering a clearer understanding of network utilization and traffic flow.
• API Development for Transceiver Light Level Monitoring:
  • Designed and implemented a new API endpoint to expose real-time monitoring data for transceiver light levels on network devices managed by KVO.
  • This API allowed network administrators to proactively monitor the health of their physical connections and identify potential issues before they impact network connectivity.
  • The API was designed for easy integration with other monitoring tools and systems.
• Agile Development Participation:
  • Actively participated in team design discussions, contributing technical expertise and insights.
  • Engaged in sprint retrospective meetings to identify areas for improvement in the development process.
  • Presented implemented features and progress during sprint demo sessions.

Technologies Utilized:

• Go (Golang)
• nodejs (Typescript)
• Kubernetes
• neo4j
• Cassandra
• Prometheus
• Grafana
• RESTful APIs

The Outcome & Impact:

My contributions to the KVO Core backend team resulted in several key improvements for the Keysight Vision Orchestrator platform:

• Increased Scalability: Implementing caching strategies significantly improved the scalability of the platform’s device status monitoring capabilities, allowing KVO to efficiently manage larger and more complex network environments.
• Enhanced Network Visibility: The development of comprehensive Grafana dashboards provided network administrators with richer and more actionable insights into their network infrastructure, facilitating better management and faster troubleshooting.
• More Accurate Network Analytics: Improved port binding labels enabled the collection of more precise statistics using Prometheus, leading to a better understanding of network utilization and resource allocation.
• Proactive Issue Detection: The new API for monitoring transceiver light levels empowered network administrators to proactively identify and address potential physical connectivity issues, reducing the risk of network outages.
• Improved Development Process: Active participation in agile development practices contributed to a more efficient and collaborative development environment.

Conclusion:

This experience highlights the expertise in backend development using Go, designing and implementing scalable solutions for distributed systems, enhancing monitoring and observability through tools like Grafana and Prometheus, and developing robust APIs for data access. The contributions to Keysight’s Vision Orchestrator directly addressed critical challenges related to scalability, visibility, and proactive network management, ultimately improving the value and effectiveness of the KVO platform for its users.

Do you need expertise in building scalable backend systems, implementing robust monitoring solutions, or developing efficient APIs? Contact me today to discuss how we can help you optimize your technology solutions.

I’m excited to learn about your challenges and help your team achieve smoother and more efficient software development. Let’s have a friendly chat about your specific needs and how my expertise can guide you towards building amazing things without the unnecessary stress.