Software Provides Network Visibility

Aug. 1, 2006

Healthcare system solves compliance concerns with new centralized control of its network.

Doug Torre was a network manager with network management problems. His staff lacked visibility into its network. They could not determine whether compliance requirements were being met. They did not have the proper tools for pushing updates to the network.

Doug Torre was a network manager with network management problems. His staff lacked visibility into its network. They could not determine whether compliance requirements were being met. They did not have the proper tools for pushing updates to the network.

Catholic Health System Director of Networking and Technical Services Doug Torre found a network management solution that provided the controls he needed for regulatory compliance.

Torre is the director of networking and technical services at Catholic Health System (CHS), one of Buffalo’s largest healthcare providers, with 8,300 associates and 1,200 physicians, and a network of 40 locations, including four hospitals, 10 primary care centers, nine diagnostic and treatment centers, and a free-standing surgery center. With network security and Health Insurance Portability and Accountability Act (HIPAA) compliance in mind, CHS faced a number of management challenges with its constantly changing healthcare network environment.

One critical network management issue was lack of up-to-date visibility into its networks. As a result, CHS did not know how compliant it was at any point in time, nor what it needed to do to bring out-of-compliance systems back in line, explains Torre.

“Without proper monitoring, there was no easy way to verify a machine’s configuration, the patch levels, what it accessed, what machines communicated with it, even its security policies,” he explains. “Not knowing what’s out there or what’s happening with our systems did not allow us to manage our networks or compliance.”

Another management challenge Torre faced was a lack of automation for tasks such as checking, rolling out and enforcing policies. According to Torre, CHS undergoes frequent network changes and security policy updates. Torre needed a tool to centralize network-management updates, as well as handle compliance checking.

CHS security staff members, for example, conducted manual walk-throughs of their facilities to verify computer security compliance. During the walk-throughs, they checked on HIPAA requirements such as computer security policies, local account privileges and authorization timeouts.

“The manual network-management updates were labor intensive, and assessments on compliance were slow and inefficient,” says Torre. “We were looking for a more effective way to manage HIPAA security and internal policy compliance, something that could provide CHS with an automatic net­work status.”

Torre and his staff considered purchasing 2,500 host-based firewalls with configuration controls. They also looked into additional network-management and surveillance tools, such as security event management products. In addition, they considered port-level protection to examine systems and quarantine non-compliant machines.

Torre then learned about a security compliance management solution from San Mateo, Calif.-based Elemental Security that costs approximately the same as any one of those alternatives, but provides additional features with its combined security and compliance functionality. The system is a client-server software product that provides visibility into all machines connected to the network, or attempting to access it. It lets administrators control and contain users or user groups through automated security policies, and centralizes policy and host configuration management, along with network access, and discovery/inventory in one solution.

I liken [managing security] to a castle, protected by a moat and secured by a drawbridge. For those allowed in the castle, you still need locks on the doors to secure some domains, or locks on a treasure chest to protect valuables from those able to enter the room.

“The software captures and measures data creating a baseline view,” Torre offers. “From there, we can monitor and improve our compliance. The system lets us gauge performance against corporate policies. It handles our network surveillance and logs the activity in a real-world view.”

CHS purchased 2,500 Elemental client and 150 server agents, along with one central server. The system installed in its primary data center in about an hour, and CHS immediately began deploying agents to its various locations. CHS’ preferred server platform is Windows, but Solaris, AIX and Linux servers also provide an important role.
Its desktop platforms of choice are Windows XP and Windows 2000.

“The centralized control and view into our dynamic network, with its extensive computer policies, are valuable in tracking and improving our security and compliance posture,” says Aaron Shackelford, CHS network engineer. “Elemental takes our network security policies and helps implement them throughout the system to automate discovery and control. Network management and security are implemented and maintained without operator intervention.”

The solution’s unified view of what is currently happening with CHS computers helps staff address the HIPAA requirement to record and examine activity in systems that contain or use electronic-protected health information (PHI), according to Torre.

“Some organizations are willing to accept an annual or quarterly point-in-time vulnerability analysis, but what Elemental provides is orders of magnitude better,” he says. “It gives us new levels of information, and delivers a new perspective to managing our network and security—not with a quarterly snapshot, but with a current daily view. This is important because we need to know patch levels on a machine, as well as memory, usage scenarios, who the host communicates with and the policies deployed to it.”

The software’s packet filter, part of the system agent, also provides granular access controls based on a machine’s current compliance status. This control helps CHS efforts to comply with the HIPAA guideline to secure electronic PHI and grant access only to those users or software programs that have been granted access rights, says Shackelford.

According to Shackelford, one of the key systems-management features is the automatic dynamic grouping—if one computer or group fails to meet security policy settings, it can be automatically partitioned to protect sensitive systems and data. If a computer’s policy is altered, the software can correct the configuration or apply policies without intervention. If security vulnerabilities are detected, CHS can quickly find out which computers are at risk, and determine what to do to fix them.

“If security on a computer degrades, such as if its antivirus software stops running, it is no longer allowed to communicate with our database server,” Shackelford explains. “New computers coming on the network can either be automatically segregated, or added to the network, based on the configuration of the system.”

The centralized control and view into our dynamic network, with its extensive computer policies, are valuable in tracking and improving our security
and compliance posture.

Torre explains that CHS used to make private networks to support various requirements, but now can create groups logically within the CHS network. “These private networks add a lot of complexity to our network,” he says. “They are rigid and create isolated islands that are hard to manage.”

Like many organizations, CHS was interested in building multiple layers of security on its network. In today’s dynamic, partner-rich environment, it needed to manage the network differently, not with a wall around it.

“Managing security at multiple levels makes sense,” says Torre. “I liken it to a castle, protected by a moat and secured by a draw bridge. For those allowed in the castle, you still need locks on the doors to secure some domains, or locks on a treasure chest to protect valuables from those able to enter the room.”

Elemental complements its network perimeter security by adding security around every host. “Managing and defending the CHS network perimeters was an exercise in futility and decreasing returns,” Torre explains. “Now we can manage security policies where our information resides, as well, right down to the individual host or virtual host grouping.”

CHS is also better prepared to manage security audits, and can now report how the network and hosts are operating at any moment. The technology demonstrates compliance against CHS’ policy baselines, and creates a warehouse of detailed information so CHS can look back at compliance levels over days, weeks or months.

“As far as ROI, in addition to time saved, improved audit results, and better management of systems, security and the network, Elemental gave us cost avoidance because we didn’t need to purchase or manage other products and services,” Torre explains. “It delivers multiple features in one product, where we would typically need various products to try to accomplish the same results.”

Editor’s Note: This article was written for and originally appeared in HMT’s sister publication, Communications News, in the January 2006 issue.

Analyze This!

IT departments should be able to meticulously monitor their networks without incurring high operating expenses and absorbing precious staff time. This is the challenge that RGIS Inventory Specialists faced with its worldwide network of more than 400 offices–the IT department needed to find a way to analyze and manage this worldwide network.

RGIS provides inventory services in areas such as retail, warehousing, merchandising, distributing and pharmaceuticals. RGIS’s business depends on the performance of its network; therefore, one of the company’s priorities was to obtain an analysis tool that would allow the IT team to manage its current network, and handle expansion without much staff effort. The IT department also required a tool that could help minimize network downtime by being proactive with issues on the network.

An analysis tool was needed to monitor the network without adding additional staff to run it, and could be operated by everyone in the department, from junior staff to senior executives. RGIS researched network-analysis tools from a variety of leading vendors and determined that NetMRI, from Netcordia, best met its needs.

First, RGIS installed a NetMRI evaluation unit in its headquarters to test for one month. Within 48 hours, NetMRI discovered issues on the network that had never been revealed before. Before the week ended, Jimmy Willard, project manager in research and engineering for RGIS, recommended that RGIS buy the unit.

“After only two days of analyzing our network, NetMRI found routing and VLAN issues I didn’t even know existed,” Willard says. “This tool has paid for itself several times over.”

The tool evaluates the network as a whole system, including both physical and logical devices and links. Further, Willard says, it discovers issues in the network before they become problems.

Before NetMRI was installed, the biggest issue RGIS faced was verifying the correctness of every configuration file in its routers and switches against network configuration policies. This was a tedious task that required the network manager to search and correct each individual file. After NetMRI was implemented, this problem was quickly resolved.

The tool allows RGIS to automatically check the configuration files every day. It also autogenerates daily reports highlighting configuration issues, as well as any other issues in the network, and provides correctional guidance as well as compliance proof source. This provides the IT staff with a readily adaptable management tool that covers the entire worldwide network infrastructure.

“Since NetMRI has been running on the network, we don’t replace devices nearly as often as we did before,” offers Willard. “We can now easily differentiate between a hardware problem and a configuration problem, which is something we had issues with in the past.”

NetMRI also enhances RGIS’s network engineering effectiveness, Willard reports, as the information is now made available clearly and quickly. It allows the IT team to focus on the real issues affecting the infrastructure, instead of trying to troubleshoot dead-ends. This is accomplished by correlating the statistics and applying rules of logic for troubleshooting at the system-level functional areas, such as root bridge placement in VLANs, or security settings in wireless LANs. The tool complements RGIS’s real-time systems with in-depth root-cause analysis presented daily in an understandable, browser-based view or report.

“NetMRI enables our entire network and IT staff to be proactive and run more efficiently,” says Willard. “You don’t have to be a rocket scientist, or have 25 years experience to use this tool.”

This article was provided by Netcordia, Annapolis, Md.
For more information:

Probe Your Network

The distributed analyzer is a flexible, economical method of analyzing and monitoring switch-based networks. It consists of any number of probes reporting back to a central console, providing visibility of the different segments on the network. Probes observe and collect the data traversing links, similar to traffic cameras.

The most efficient probes do analysis on site and only send display updates to the console to minimize network overhead. Without probes, you would have to connect a dedicated analyzer to multiple switches, and even then you would have no way of seeing all of the data in a comprehensive view.

Deploying probes across every segment of the network for complete visibility, however, is not practical. Realistically, probes should be deployed on heavily utilized or business-critical links.

Ultimately, the architecture of a particular network and where visibility is required determines the best location to deploy probes. Placing probes on the full-duplex links that connect servers or server farms to core switches, for example, lets you see all traffic between servers and their clients.

Connecting additional probe appliances at the edge of the network will let you focus in on select segments or stations on the network for detailed problem resolution. Deploying a specialized probe on a WAN link makes WAN frames visible, in addition to showing all traffic flowing in and out through the link.

The following are examples of probe placement on a “common” network. Every network is different, so these examples may not look like your network but the concepts demonstrated should be applicable in most situations.

Ethernet probe. Connected to a switch SPAN or port mirror, an Ethernet probe can show you top network users connected to that switch, help enforce corporate usage policies and aid in troubleshooting station connections.

WAN probe.Deployed via a test access port (TAP) on a WAN link, a WAN probe can help to verify service-level agreements, monitor for intruders and aid in troubleshooting branch office connections.

Gigabit trunk probe. A trunk-aware probe deployed via a TAP on a trunk can show server, link and application performance, as well as aid in tweaking and troubleshooting trunk performance and troubleshooting station connections.

Wireless probe. A wireless probe helps to detect security threats, detect and shut down rogue access points and troubleshoot 802.11 connections.

Deploying probes at key areas on the network should give you sufficient visibility and the confidence that you are getting a comprehensive and accurate picture of the network. Failing to deploy probes in critical places on the network can result in blind spots, leading to inefficient troubleshooting and expensive mistakes.

Even if probes are deployed at the most effective places on a network, however, they only show your analyzer the data that is visible to those probes. An Ethernet probe, for example, is limited to what a particular switch’s SPAN can deliver. SPAN ports do not report errors and will drop information on highly utilized links. Using a TAP on designated links will provide access to all data—including errors—that traverse that link, even if bandwidth is running at maximum capacity. So TAPs are essential on critical links, while using a SPAN port may be sufficient on less-critical links.

Deploying probes that work with switches or TAPs across the network gives you the visibility to effectively monitor the network and boost troubleshooting power to ensure optimal traffic patterns across the network. Deploying probes the right way can arm you with the information needed to keep your network up and moving along.

This article was provided by Charles Thompson, senior systems engineer for Network Instruments, Minneapolis.
For more information: