A well-designed network has network management included in its basic requirements. At each stage of the design process, one of the key questions should be "how will this be managed"? The network designer should know from the outset where the network-management servers will be, both physically and logically, on the network. If special protocols are used for network management, then the design must ensure that this information can be delivered. If protocol analyzers or RMON probes are used to monitor network performance and assist in troubleshooting, then these devices should be placed in the design.
A probe is used to watch all of the traffic passively as it passes by. The issue of where to put probes is particularly difficult. In a switched or VLAN-based environment, probes are nearly useless if they are not deployed carefully.
Before switches, when Ethernet networks were made up of large bus configurations, it was possible to deploy a few probes and see everything. The probes would be placed near the Core of the network. From there, they could easily be switched to whatever LAN segment needed monitoring. However, in a switched Ethernet design, every end device is on its own LAN segment. This situation fundamentally changes how network monitoring has to be done.
One way to use a probe is to look at all traffic going to and coming from a particular end device by configuring the probe's switch port to mirror the port connecting to this device. This mirroring requires, of course, that a probe be available for use with this switch. In the ideal case where everything can be monitored centrally without having to leave the network operations center, this implies that there must be a separate probe on every switch. This prospect can be rather expensive. Thus, many organizations use either full or partial RMON probes built into their switches instead. The use of these probes allows good monitoring capabilities for every device in the network.
Another way to use a probe is on trunk links. In a hierarchical VLAN architecture, keeping a close watch on trunk utilization is important because this is where congestion problems usually first arise.
The discussion of hierarchical designs in Chapter 3 showed that trunks are used in four ways. They connect the Access Level to the Distribution Level and the Distribution to the Core. Internal trunks also exist within the Distribution and Core Levels. The important thing is that, while most of the switches wind up being at the Access Level, all trunks have at least one end in the Distribution or Core Levels. Thus, there is no need to deploy probes for monitoring trunk links to the Access Level. Not needing to deploy the probes at every switch should provide an important cost savings.
Chapter 3 also mentioned another important design issue for large-scale LAN environments in the discussion of hierarchical VLAN designs—the presence of a dedicated network-management VLAN. Obviously, the same VLAN cannot be present in different VLAN Distribution Areas, but every Distribution Area should have such a VLAN.
There are several reasons for having a separate network management VLAN that contains no user traffic:
If you monitor traffic on a user VLAN, you don't want to see the probe traffic mixed in with user traffic.
If you have to transfer configuration or software to or from the network devices, you don't want this traffic to interfere with production-application traffic.
Separating the management VLAN from user traffic can be useful for security reasons. A router can then completely block SNMP and other management-specific traffic from passing between user and management VLANs. Blocking the traffic greatly reduces the chances of a successful attack.
Perhaps most importantly, if there is a serious problem with a user VLAN, having a separate network-management VLAN allows the network engineer to get to the switch and hopefully fix the problem.
A network-management VLAN should always be part of the network design for every Distribution Area. This VLAN should contain the management addresses for all hubs and switches in the managed Distribution Area. It should also hold the management interfaces for all probes and protocol analyzers in the area. If any devices, such as Inverse Terminal Servers, are used for out-of-band management, then these devices should also be connected through this management VLAN.
The management VLAN can suffer failures without affecting production traffic. Thus, it is not necessary to provide the same level of redundancy for this VLAN as for the rest of the network. However, if a large number of devices are to be managed through this VLAN, then it is wise to make it fairly robust. Economy of design may mean just building this VLAN according to the same specifications as the rest of the production network.
A network designer can do several different things at the physical layer to ensure that a network is designed to be managed effectively. These steps generally involve ease of access, clear labeling, and logical layout.
By ease of access, I mean that equipment that needs to be touched frequently should be in a prominent position. It should be safe from being bumped accidentally; there should also be no obstructions such as walls or poles to prevent a technician from seeing or handling the equipment. A good example would be cabling patch panels. Patch panels are almost always the network elements that need the most frequent physical access. Fiber patch panels tend to be used less frequently than the patch panels for user LAN drops. It is common for fiber patch panels to be mounted too high or too low to access easily.
Usually, the best way to handle LAN-cabling patch panels is to mount them in logical groups in equipment cabinets with locking doors. When the technician needs to make changes, the door can be easily unlocked and opened. The changes can then be documented and the door locked again to prevent tampering.
Documenting Patch-Panel ChangesThere are two good methods for documenting patch-panel changes. One method is to have every change accompanied by a work order. The technician notes the changes on the work order. Then, when the work order is closed, the changes can be input into a database or spreadsheet of cabling information. The other method, which is probably more effective in most organizations, is to simply have a printed spreadsheet of the patching information taped to the inside of the cabinet door. When a technician makes a change, he immediately notes it on this sheet of paper. Then somebody needs to gather up the paper sheets periodically, input the changes, and print out new paper sheets to tape back inside the cabinets. The principle advantage to this method is that not all changes are accompanied by work orders. In particular, emergency changes usually have to be done quickly by whoever is on call. This person may not have time to access the work-order system and update databases with the small changes that they had to make to fix the problem. |
A clear, consistent, and simple labeling scheme is critical, particularly for patch panels and patch cords. Every patch-panel port should have a unique code number, and the formats of these codes should be consistent. These codes should clarify to what this port attaches. Suppose, for example, that you want to number the patch panels in a wiring closet that supports a large number of end users. In general, there should be a consistent floor plan in which every user work area is numbered.
Then, if each work area has three or four drops, you usually label the patch-panel ports with the work-area number followed by an alphabetic character to indicate to which cable drop the port connects. Each patch-panel port has a unique number that is easily associated with a particular cable drop in a particular work area. Thus, for example, desk number 99 may have 3 jacks beside it. If two of these jacks are for data and one is for voice, then you might number them 99-A, 99-B, and 99-V, respectively. This way, which ports are for what purposes is completely clear both at the desk and in the wiring closet.
If you later found that you had to run an additional data cable to this desk, you could number it 99-C. An additional voice line, perhaps for a fax machine, could be numbered 99-W.
These designations are merely intended as examples. Every network is different, so the network designer has to come up with a locally appropriate scheme.
Giving consistent labels to the patch cords that connect to these patch panels is also important. There are many different ways of doing this. Some organizations like to label the patch cord with a tag that indicates what is on the other end. For example, suppose that a cord connects panel 1, port 2 to panel 2, port 3. Then the first end plugs into panel 1, port 2, and it has a label saying "panel 2, port 3." This method is quite common, and it is generally not very good. The problem is that, at some point, somebody will need to move that patch cord. If they fail to change the label in the heat of the moment, then they will have a situation that is worse than having no labels at all because the labels cannot be trusted.
The simplest and most effective method for labeling patch cords that I have seen is simply to give every patch cord a unique number. These cables can be prenumbered and left in convenient locations in each wiring closet. Whenever a patch cable is connected between two ports, the technician writes the information on the sheet inside the cabinet. Patch panel 1, port 2 connects to patch cord number 445, which connects to panel 2, port 3. This system greatly facilitates the process of auditing patch panels. All that you need to do is go through the patch panels port by port and write down what patch-cord number connects to that port. Then you can put all of this information into a spreadsheet and sort it by patch-cord number to see all of the cross-connections immediately.
If the spreadsheets get badly outdated, and there is an emergency problem involving a particular port, then the technician will have to trace the cable manually regardless. An effective labeling scheme will be of no help if the spreadsheets are outdated.
Having universal rules for what constitutes a logical patch-panel layout is difficult. This is because what is logical depends on how the cabling is used. For example, suppose every user workstation has two LAN drops, labeled A and B. The first drop is for data and is connected to a computer. The second drop is for an IP telephone. In this case, it makes sense to separate the patch panels to put all drop As together in one group of panels and all drop Bs together in another group. Alternatively, if all drops are intended for user workstations and many users simply have two workstations, then grouping the A and B drops together may be simpler. In this case, the pattern might even alternate A and B on the same patch panel.
What is universal is that the patch-panel layout should make finding devices easy. Usually, workstations are numbered logically through the user work area. Consecutive numbers should indicate roughly adjacent workstations. Then the ports on the patch panel should be arranged in numerical order. In this way, it becomes easy for the technician who usually deals with cabling in this wiring closet to look at the patch panel and know at least approximately where the corresponding workstations are.
However, even with the best of initial intentions, the pattern can be badly broken over time. This is because you frequently have to deal with changes to the work area. Sometimes a cubicle pattern may change on the floor, and sometimes you need to run extra cabling to support extra devices. The network designer and manager have to tread a very careful line between forcing these changes into the logical flow of the entire area and wanting to minimize changes to unaffected areas. This situation usually means that any new drops are taken as exceptions and put at the end of the existing group of patch panels.
One of the most important considerations in designing a network to be manageable is deciding how and where to connect the network-management equipment. Is there a separate network-management center to accommodate? Do nonoperational staff members like the network designer sit in a different area? Do they require access to the network-management center's equipment through the network?
In general, the design should include a separate VLAN just for network-management equipment. This VLAN is not necessarily the same one mentioned earlier. That management VLAN was used to access management functions on remote network equipment. This network management-equipment VLAN houses servers and workstations used to manage the network.
This VLAN is usually as close to the Core of the network as possible. However, it is not always close to the Core. Many organizations are opting to outsource their network-management functions. This outsourcing permits highly trained staff to be available at all hours. It also means that network management must be done from offsite, usually from behind a firewall.