The SD-WAN Setup Mistake That Left a San Jose Company Offline for a Day
When a mid sized financial services firm in San Jose decided to upgrade their legacy network infrastructure to SD-WAN, they expected faster performance, reduced costs, and better reliability. What they got instead was a full day of downtime, frustrated employees, and a six figure loss in productivity. The culprit wasn’t faulty hardware or a cyberattack. It was a fundamental SD-WAN setup mistake that could have been avoided with proper planning and expert guidance.
This cautionary tale highlights why businesses across California, from Silicon Valley to Los Angeles, need to approach SD-WAN deployment with care. While the technology offers tremendous benefits, the implementation process requires specialized knowledge that many internal IT teams simply don’t possess. For companies in Los Angeles considering SD-WAN or any network infrastructure upgrade, understanding what went wrong in San Jose could save you from a similar disaster.
Understanding SD-WAN and Why San Jose Businesses Are Adopting It
SD-WAN, or Software Defined Wide Area Networking, represents one of the most significant advancements in network technology over the past decade. Unlike traditional WAN architectures that rely on expensive MPLS circuits and manual configuration, SD-WAN uses software to intelligently route traffic across multiple connection types including broadband, LTE, and MPLS. This approach delivers better performance, lower costs, and greater flexibility for modern businesses.
The San Jose company, like many organizations in the Bay Area, was drawn to SD-WAN for several compelling reasons. They had offices spread across California with a headquarters in San Jose, a satellite office in San Francisco, and remote workers throughout the state. Their legacy network was struggling to handle the bandwidth demands of cloud applications, video conferencing, and large file transfers. The CFO was also unhappy with the escalating costs of their MPLS contracts.
According to a 2024 report from Statista, the global SD-WAN market is projected to reach $31.7 billion by 2027, growing at a compound annual growth rate of over 30 percent. This explosive growth reflects how businesses are recognizing the technology’s value. In California specifically, where technology adoption tends to lead national trends, SD-WAN implementation has accelerated rapidly among mid market and enterprise organizations.
The California Technology Association noted in their 2024 State of Technology report that network modernization, including SD-WAN adoption, ranks among the top three IT priorities for California businesses. With the state’s diverse economy spanning technology, finance, healthcare, and entertainment, the need for robust, flexible networking solutions has never been greater.
The San Jose SD-WAN Disaster: What Actually Happened
The San Jose company had been operating with a traditional hub and spoke network topology for over a decade. Their headquarters served as the central hub where all traffic flowed through before reaching its destination. This architecture worked well enough when most applications resided in their on premises data center, but it created significant bottlenecks as they migrated to cloud services.
Their IT director, under pressure from leadership to reduce costs and improve performance, made the decision to deploy SD-WAN. Rather than engaging a specialized network consulting firm, he assigned the project to two of his senior network engineers. Both were competent professionals with strong backgrounds in traditional networking, but neither had practical experience with SD-WAN deployment.
The team chose a well known SD-WAN vendor and began the implementation. The initial setup went smoothly, with the new SD-WAN appliances connecting to the existing infrastructure. However, the team made a critical error during the configuration phase. They inadvertently created a routing loop by failing to properly segment the network traffic between the legacy infrastructure and the new SD-WAN overlay.
This mistake went unnoticed during the weekend maintenance window when testing was limited. Come Monday morning, when hundreds of employees logged in simultaneously, the routing loop caused network traffic to bounce endlessly between the legacy routers and the SD-WAN appliances. The result was complete network paralysis.
The IT team spent the entire day troubleshooting the issue while the company’s operations ground to a halt. Customer service representatives couldn’t access client records. Traders couldn’t execute transactions. Executives couldn’t access critical reports. By the time an external consultant was brought in to diagnose and fix the problem, the company had lost an entire business day.
This scenario is more common than many business leaders realize. A 2025 survey from CompTIA found that 43 percent of organizations that deployed SD-WAN experienced significant issues during implementation, with 18 percent reporting at least one full day of downtime. The survey also revealed that companies working with experienced network service providers were 60 percent less likely to encounter deployment problems.
Why Network Infrastructure Requires Professional Expertise
The San Jose incident illustrates a fundamental truth about modern networking. The complexity of today’s infrastructure requires specialized expertise that goes beyond what generalist IT professionals can provide. Network infrastructure has evolved dramatically over the past decade, and the skills required to design, implement, and manage these systems have become increasingly specialized.
Network infrastructure encompasses far more than just connecting computers to the internet. It involves sophisticated routing protocols, security policies, quality of service configurations, and integration with cloud services. When you add SD-WAN to the mix, the complexity increases exponentially. SD-WAN introduces overlay networks, dynamic path selection, application aware routing, and integration with security services like secure web gateways and cloud access security brokers.
For businesses in Los Angeles and across California, the stakes are particularly high. The state’s economy is heavily dependent on digital infrastructure, and network outages can have devastating consequences. The California Public Utilities Commission has documented numerous instances where network failures disrupted critical services across the state.
This is where professional IT service providers like IT Training & Consulting, Inc. (ITTC) add tremendous value. With a team including Network Support Specialist Abner Navarro and IT Field Technicians Jerry Duque and Nestor Turcios, ITTC brings deep expertise in network infrastructure deployment and management. “The complexity of modern networks means that even experienced internal IT teams can miss critical configuration details,” says Abner Navarro. “A proper network assessment before any major upgrade is essential to identify potential issues before they become problems.”
The Hidden Costs of DIY SD-WAN Implementation
Many businesses are tempted to handle SD-WAN deployment internally to save money. The San Jose company certainly thought they were being cost conscious by using internal resources rather than hiring external experts. However, this decision ultimately cost them far more than they saved.
Let’s break down the actual costs of the San Jose incident. First, there was the direct productivity loss. With 200 employees unable to work for a full day, and average compensation of $50 per hour including benefits, the direct labor cost alone exceeded $80,000. Second, there was the lost revenue. As a financial services firm, their traders couldn’t execute transactions, resulting in an estimated $150,000 in lost commission revenue. Third, there were the consulting fees to bring in outside help to fix the problem, which added another $25,000. Finally, there was the damage to their reputation and the potential loss of client trust, which is difficult to quantify but extremely significant.
Total costs: over $250,000 for a single day of downtime.
This doesn’t even account for the hidden costs that continued to accrue after the immediate crisis was resolved. The rushed fix that got them back online created technical debt that would need to be addressed later. The IT team had to spend additional time documenting the issues and implementing permanent solutions. Employee confidence in the network was shaken, and the IT department’s credibility was damaged.
A 2024 study from the Uptime Institute found that the average cost of network downtime for mid sized businesses in California ranges from $5,000 to $15,000 per hour, depending on the industry. For financial services firms, that number can easily exceed $20,000 per hour. The San Jose company’s experience aligns with these figures.
How Proper Network Assessment Prevents SD-WAN Failures
One of the most critical steps the San Jose company skipped was a comprehensive network assessment before beginning the SD-WAN deployment. A proper assessment would have revealed several issues that contributed to their failure.
First, the assessment would have documented their current network topology in detail, including all existing routing protocols, firewall rules, and traffic patterns. This documentation would have made it obvious that their legacy hub and spoke architecture needed to be modified before introducing the SD-WAN overlay.
Second, the assessment would have identified the potential for routing loops. Experienced network engineers know to look for these issues when integrating new routing domains with existing ones. The San Jose team, lacking SD-WAN experience, simply didn’t know what to look for.
Third, the assessment would have included a migration strategy with clear rollback procedures. When the issue occurred, the San Jose team had no way to quickly revert to their previous configuration. They were stuck trying to fix a broken system rather than being able to fall back to a known good state.
A comprehensive network assessment typically includes inventory of all network devices, analysis of current traffic patterns, evaluation of security policies, review of network documentation, and development of a detailed migration plan. This assessment should be conducted by experienced professionals who understand both legacy and modern networking technologies.
ITTC offers comprehensive network assessment and planning services through their Network Management & Hardware Support division. Their approach begins with a thorough evaluation of existing infrastructure before recommending any upgrades or changes. This methodical approach has helped numerous Los Angeles businesses avoid the kind of disaster that befell the San Jose company.
Cloud Connectivity: A Critical Consideration for SD-WAN Success
The San Jose company’s SD-WAN failure was compounded by their complex cloud connectivity requirements. As a financial services firm, they relied heavily on several cloud based applications for trading, client management, and regulatory compliance. Their SD-WAN deployment needed to support these cloud services with low latency and high reliability.
When the routing loop occurred, it didn’t just affect internal connectivity. It also disrupted their connections to cloud service providers. The traffic destined for their cloud applications got caught in the same routing loop, effectively isolating them from critical external services.
This aspect of their failure highlights the importance of designing SD-WAN architectures with cloud connectivity as a primary consideration. Modern SD-WAN solutions offer sophisticated capabilities for cloud integration, including direct internet access, cloud on ramp services, and integration with cloud security platforms. However, these features must be properly configured and tested.
For Los Angeles businesses exploring corporate cloud computing solutions alongside SD-WAN, the integration between these technologies is crucial. The SD-WAN must be designed to optimize traffic to cloud services while maintaining security and compliance requirements. This often requires specialized knowledge that goes beyond basic networking skills.
Stanley Ung, Database Manager at ITTC, emphasizes the importance of this integration: “Your SD-WAN is only as effective as its integration with your cloud environment. We frequently see businesses that implement SD-WAN without considering their cloud architecture, and they end up with performance issues that negate the benefits of the technology.”
The Role of Structured Cabling in Network Performance
One often overlooked aspect of network infrastructure that contributed to the San Jose company’s challenges was their physical cabling infrastructure. When the IT team began troubleshooting the routing loop, they discovered that several of their network links were operating at suboptimal speeds due to degraded cabling.
This is a common issue in older facilities. Cabling that was installed years ago may not meet current performance standards. Additionally, physical damage to cables can occur over time, impacting network reliability and speed. In the San Jose case, the degraded cabling made the SD-WAN troubleshooting more difficult by introducing intermittent connectivity issues that were initially mistaken for SD-WAN configuration problems.
For businesses in Los Angeles, many office buildings are older structures with outdated cabling infrastructure. Before implementing a new network technology like SD-WAN, it’s essential to verify that the physical infrastructure can support it. This includes checking cable categories, termination quality, and pathway integrity.
ITTC provides professional phone cabling solutions and structured cabling services that ensure the physical foundation of your network is solid. Their team of experienced technicians can assess existing cabling infrastructure, recommend upgrades where needed, and install new cabling according to industry standards. Jerry Duque, IT Field Technician, often reminds clients that “no amount of software defined networking can compensate for a physical infrastructure that’s not up to the task.”
The Connected World: Virtualization and Network Agility
The SD-WAN technology that the San Jose company was deploying could have integrated effectively with their existing virtualization environment to provide greater agility and resilience. However, their implementation didn’t account for this integration, and the resulting failure had impacts that rippled through their virtual infrastructure.
Their virtualized servers, which hosted critical applications and databases, became unreachable when the network failed. This meant that even their virtual disaster recovery capabilities were ineffective because the network connectivity required to access failover systems was disrupted.
A properly designed SD-WAN solution should complement and enhance virtualization strategies. By intelligently routing traffic between virtual environments and physical infrastructure, SD-WAN can ensure that virtual workloads maintain connectivity even when some network links fail. It can also optimize traffic to virtual desktop infrastructure and other virtualized services.
ITTC’s virtualization services are designed with this integration in mind. Their team, including Full Stack Developer Abbas Arif and Software Engineer Juan Alvarez, understands how network architecture and virtualization interact. This comprehensive perspective allows them to design solutions that work across the entire technology stack.
Outsourcing Your IT: A Smarter Approach to Network Infrastructure
The San Jose company’s decision to handle their SD-WAN implementation internally was driven by a desire to maintain control and save money. However, as we’ve seen, this decision led to a costly failure. The experience demonstrates the value of outsourcing complex IT projects to specialized professionals.
The concept of outsourcing your IT has evolved significantly in recent years. It’s no longer about simply offloading routine tasks. Modern IT outsourcing involves partnering with experts who bring specialized knowledge and experience that internal teams may lack. This partnership approach allows businesses to access skills and capabilities that would be difficult or expensive to develop internally.
For a project like SD-WAN deployment, an outsourcing partner brings several advantages. They have experience with multiple vendors and deployment scenarios. They’ve likely encountered and solved similar issues in previous implementations. They have established methodologies and best practices that reduce risk. And they can provide ongoing support after the implementation is complete.
For Los Angeles businesses considering outsourcing IT functions, it’s important to select a partner with relevant experience. ITTC has extensive experience with network infrastructure projects for businesses across Southern California. Their team understands the unique requirements of California businesses and can provide guidance tailored to local conditions.
“The companies that succeed with network modernization are the ones that recognize their limitations and partner with experts,” says Juan Turcios, President and CEO of ITTC. “We consistently see better outcomes when businesses bring us in early in the planning process rather than after a problem has occurred.”
Monthly IT Support Services: Preventing Issues Before They Start
One of the most effective ways to avoid network failures is through proactive monitoring and maintenance. The San Jose company had basic IT support in place, but they didn’t have a comprehensive monitoring and maintenance program. As a result, they didn’t discover the configuration errors until they caused a full system failure.
Monthly IT support services can prevent this scenario by providing ongoing network monitoring, regular health checks, and proactive maintenance. These services identify potential issues before they cause outages, allowing for corrective action during planned maintenance windows rather than emergency situations.
For businesses in Los Angeles, monthly IT support offers particular advantages. The region’s frequent seismic activity and occasional severe weather events create unique challenges for network infrastructure. Regular maintenance and monitoring can help ensure that networks remain resilient in the face of these environmental factors.
ITTC’s monthly IT support services include comprehensive network monitoring, regular system health checks, patch management, and proactive maintenance. Their support desk, reachable through IT Support Services in LA, provides responsive assistance when issues do arise. This proactive approach has helped many Los Angeles businesses maintain high network availability and avoid costly outages.
What Los Angeles Businesses Can Learn from San Jose
The San Jose SD-WAN disaster offers valuable lessons for Los Angeles businesses planning network upgrades. While the specific mistake that caused the outage was technical, the underlying issues were organizational and procedural.
First, don’t underestimate the complexity of modern network technologies. SD-WAN may be marketed as simple to deploy, but the reality is that successful implementation requires deep expertise. Even experienced network engineers need specialized training and experience with SD-WAN to avoid pitfalls.
Second, invest in proper planning and assessment. The cost of a comprehensive network assessment is small compared to the cost of an outage. A thorough assessment can identify potential issues before they cause problems and provide a roadmap for successful implementation.
Third, consider engaging external experts for complex projects. There’s no shame in admitting that your internal team lacks specific expertise. Bringing in specialists can actually reduce costs by preventing mistakes and accelerating the project timeline.
Fourth, establish a clear migration strategy with rollback procedures. Every network upgrade should include a plan for reverting to the previous configuration if problems arise. This safety net can prevent extended outages and reduce pressure during the implementation.
Finally, don’t neglect ongoing monitoring and maintenance. Network failures often result from configuration drift or hardware degradation that could have been detected through proactive monitoring. Regular health checks and maintenance are essential investments in network reliability.
The Los Angeles business community is closely connected to the Silicon Valley technology ecosystem, and many of the same challenges that impact San Jose companies also affect those in Southern California. By learning from the mistakes of others, Los Angeles businesses can avoid similar failures.
Project Based IT Support for Network Modernization
For businesses that want to modernize their network infrastructure but don’t have the internal resources to do so, project based IT support offers an ideal solution. This approach provides access to specialized expertise for specific initiatives while maintaining flexibility.
Project based IT support is particularly well suited for SD-WAN deployments because these projects have clear scopes and timelines. A project based engagement allows businesses to bring in specialists for the planning, implementation, and transition phases while keeping ongoing operations with internal staff.
ITTC’s project-based IT support in LA provides businesses with access to experienced network engineers, project managers, and support specialists on a flexible basis. Whether you need help with planning, implementation, or troubleshooting, project based support offers the expertise you need without the overhead of additional full time staff.
The San Jose company would have benefited from project based support during their SD-WAN deployment. An experienced network consultant could have reviewed their configuration, identified potential issues, and provided guidance on best practices. The cost of this consultation would have been small compared to the cost of their eventual outage.
The Importance of IT Strategy and Planning
At its core, the San Jose SD-WAN failure was a planning failure. The company rushed into a major technology deployment without adequate planning, assessment, or testing. This is a common pattern that IT strategy and planning services are designed to prevent.
IT strategy and planning involves taking a comprehensive view of your technology needs and developing a roadmap for meeting them. This process considers business objectives, current technology capabilities, industry trends, and resource constraints. The result is a clear plan for technology investments that aligns with business goals and manages risk.
For businesses in California, IT strategy and planning is particularly important given the state’s regulatory environment and competitive business climate. California has stringent data privacy regulations, including the California Consumer Privacy Act (CCPA), that create specific requirements for network security and data handling. A well designed IT strategy ensures that these requirements are met while also supporting business objectives.
ITTC’s IT Strategy & Planning services help Los Angeles businesses develop comprehensive technology roadmaps that address both current needs and future growth. Their team works closely with business leaders to understand their objectives and design technology solutions that support them. This strategic approach has helped many businesses avoid the kind of costly mistakes that plagued the San Jose company.
“When we develop an IT strategy for a client, we’re not just thinking about technology,” explains Nestor Turcios, IT Field Technician. “We’re thinking about their business goals, their budget, their timeline, and their risk tolerance. This comprehensive perspective helps us design solutions that work for the whole organization, not just the IT department.”
Call to Action
Don’t let your Los Angeles business become another cautionary tale. The network infrastructure decisions you make today will impact your operations, security, and competitiveness for years to come. Whether you’re planning an SD-WAN deployment, upgrading your network infrastructure, or simply looking to improve your network reliability, expert guidance can make the difference between success and failure.
IT Training & Consulting, Inc. (ITTC) has been helping Los Angeles businesses navigate complex network challenges for years. Our team of experienced professionals brings deep expertise in network infrastructure, cloud computing, virtualization, and IT strategy. We understand the unique needs of California businesses and can provide the guidance you need to make informed technology decisions.
Take the first step toward a more reliable and efficient network. Our network assessment services can identify potential issues before they become problems and provide a clear roadmap for improvement. Whether you need project-based support for a specific initiative or ongoing management of your network infrastructure, we have the expertise to help.
Call us today at (844) 804-4882 to schedule a consultation. You can also reach out through our Contact Us page at https://www.it-tc.com/contact-us/. Our team is ready to help you avoid the mistakes that left the San Jose company offline for a day and ensure your network infrastructure supports your business goals.
