This page started out as a documentation project showing the condition of the 111 8th Avenue POP as we received it (at the beginning of December, 2000) and the work we did to bring it up to production standards. The first pictures were taken during the reconstruction of the POP. We had design input in the early stages, but weren't able to have errors corrected during construction when we noticed them, due to a change process with a very long lead time. Instead, we decided to take the work as-is, as well as not having additional work (for example, the fiber duct) done by the contractor.

Since then, the page has evolved to an ongoing documentation project showing the type of work we're doing. Each photograph has a date next to it showing when it was taken, and each subject area may have multiple pictures taken at different dates showing the ongoing work. I hope that these pages can be used as a reference for future construction projects.

Each image is clickable to show a larger version.

If you have any suggestions for improving the presentation of this material, please email me at terry@tmk.com.

---

12/03/2000

This is the provided wiring on the DSX-3 panels. Note that there is only one coax on each position, despite T3's needing two. Also note the style of lacing done, where the cables are not laced to the support brackets. Not shown in this picture is the front of the panels, where the designation strips and windows are missing.

---

04/15/2001

This is the corrected wiring on the DSX-3 panels. I expect the difference mostly speaks for itself.

---

12/03/2000

This picture shows the installation of the provided wiring on the ladders. Note that even with only half the necessary number of cables (since they only ran one coax per T3) this is starting to look messy. The cables need to be laid flat on the ladders and tied to the ladders, not just to each other. We removed all the existing coax cabling and are in the process of replacing it.

---

04/15/2001

This is 4 OC-12's worth of T3 cabling (48 T3's). The view is looking above rack 4.08 toward rack 3.08. At this point, the cabling in the picture has been connected at both ends, but an additional 4 bundles need to be run for an additional carrier patch panel.

---

04/15/2001

This detail shows the "flying junction" crossovers of the various groups of T3 cables as they are routed to different patch panels.

---

12/03/2000

This is a junction of the provided ladder rack. Despite my specifying Newton 12" ladder rack for this job (and, in fact, we purchased the material), it was changed to 24" Chatsworth by someone. With only 3 cross-row (front-to-back) ladder sections, the distance between them was over 9' 6", the factory length of the material. This caused short splice sections in the pieces that ran along the rows of racks, causing yet more sag in those sections.

You can also see the J-bolts drilled into the top angles very close to the edge of the angles. 24" ladder on 23" racks isn't normally a good idea. Further, you can see V-bolts on the center of each rack top angle. In order to keep the row straight, there need to be 2 V-bolts on each rack.

---

12/18/2000

Here you can see the new V-bolts installed on the racks, two per rack. You can also see the rack identification labels to the right of the left V-bolt on each rack.

---

12/18/2000

This picture shows a detail of the newly-installed ladder rack. We've placed 12" ladder in a front-to-back configuration (across rows) over every other rack, with side-to-side ladder rack between them, above the racks. This side-to-side ladder is positioned as far to the back of the row as it can be, resting up against the plates of the V-bolts in the front. This is necessary in order to be able to add the fiber duct support brackets later on.

---

12/03/2000

While it's somewhat hard to see because of the light fixture, this is a free-hanging ladder "tail" which is attached to the front-to-back ladder at only the left side - the right side isn't connected to anything and can bounce around.

---

12/03/2000

This picture shows the site after the ladder and coax cables have been removed. The cable ties visible on the left-hand row are supporting fiber optic cables inside innerduct, which used to sit on the ladders.

---

12/18/2000

This view shows the aisle between rows 5 and 4, looking toward 5.01/4.01. You can see the ladders that connect the aisles as well as the sidewall ladder on top of row 5 and normal ladder on top of row 4.

---

12/03/2000

This is a view of the plywood that the carrier innerducts/fiber are supposed to be attached to. The blue and black cables coiled on the floor are the Ethernet and coax cables we removed to relocate the NTT handoff, and will be re-attached later. The other loops are as the various carriers left them. In a later picture you'll see how we neatened this up.

---

12/18/2000

This is the carrier slack innerduct on the plywood after we neatened it up and attached the innerduct to the plywood. Also, the coax and Ethernet cables have been re-installed on the ladder.

---

12/18/2000

The contractor routed Ethernet cables through the 24" ladder rack, despite our telling them that that space was needed for jumper cables. Additionally, the 24" ladder with side rails (to hold jumpers), which we provided to the contractor, was not used.

This picture shows the re-routing of the cables out of the ladder rack (which has been removed) and into a cable management bracket on the rear of the rack. This involved cutting the ladder with a power hacksaw as the contractor had routed the cables through the rungs.

---

12/18/2000

At the same time this work was going on, we re-mounted the Catalyst 6509 switches using the proper mounting brackets. Previously, they had simply been placed on shelves. The power cables have been dressed and tie-wrapped and the plugs replaced with twist-lock ones to prevent inadvertent outages while work is being done under the floor.

Side note/diatribe: It seems that almost no vendors bother reading the published industry specs for rack mount ears. On the brackets in this picture, the ends of the brackets stick out 1/2" past the side of the rack. If we didn't have these cabling sections (instead, having racks bolted to each other) we wouldn't be able to mount switches in adjacent racks. As it is, we had to machine the brackets to be able to put the cabling section covers on. Cisco makes 2 additional mistakes - on much of their equipment (2900XL's, 75xx, etc.) there isn't enough clearance between the end of the mounting hole and the inside edge of the bracket. Juniper makes the same mistake on the M20 brackets, though oddly the M40 is Ok. Foundry does the same thing on the BI4000. Cisco has yet another foulup on the 2500 series - the upper and lower mounting holes are dimpled, don't line up with the rack holes, and are too small. This leaves only a single hole in the center of the bracket, which has the usual inadequate clearance problem. The 2600 gets it better - the upper and lower holes are the right size, having only the clearance problem.

---

12/18/2000

Here you can see a detail of the Catalyst mounting bracket, including the cuts needed to mount the bracket on the rack (look at the top and bottom of the bracket, where the web of the bracket angles in. The dangling cables are the out-of-band console and Gigabit Ethernet fiber, which will be re-routed later on.

---

04/15/2001

As promised, the dangling cables have been properly dressed. At the top right of the picture you can see the fiber protected by spiral wrap tubing. Additionally, the line cards have been swapped - a 16-port Gigabit Ethernet card is now in slot 3, and a pair of enhanced QOS RJ45 cards have replaced the telco card and un-enhanced RJ45 cards from the earlier picture.

---

12/18/2000

This is a view from the top of the completed ladder rack, looking toward rack 5.06. Note the ladder with sidewall over the front row of racks. This is needed to support the large number of CAT 5 jumpers that will be installed between the Catalyst switches and the patch panels.

---

12/18/2000

This is a slightly different top view, looking at rack 5.02. Again, note the sidewall ladder.

---

12/18/2000

A lot has happened since the previous pictures. The fiber duct has been installed and fibers run in it, and the routers and switches have been re-located and installed using the proper mounting brackets. Fiber optic slack jumper shelves have been added to hold excess fiber lengths.

---

04/15/2001

Here we see the Cisco GSR (12000 series), with some very neatly dressed fibers.

---

12/18/2000

The same location as the previous picture, from a different angle. This gives you a better view of the fiber optic ducting. Along the floor, we have moved the vented floor tiles around to balance the airflow in the room.

---

04/15/2001

This view looks the other way from the previous picture. Two additional Juniper M20's for international circuits have been installed in the foreground and a second M40 has been installed at the far end of the row for the forthcoming OC-48 upgrade.

---

04/15/2001

This Juniper M40 has been mounted in preparation for some OC-48 circuit deliveries. Right now it only has a Gigabit Ethernet, management Ethernet, and out-of-band serial console cables connected.

---

12/18/2000

This is a close-up view of a Foundry switch and a slack jumper shelf. Under the switch, you can see a lashing bar, used to provide additional support to the switch. The switch is mounted on 5" offsets, in order to center the switch in the rack and minimize protrusion on either side of the aisle.

---

12/18/2000

This is the relocated NTT handoff. On top are single-mode fiber terminations, with a slack jumper shelf below. Next are multimode fiber terminations, also with a slack jumper shelf. Below that are the Ethernet handoffs, not yet punched down. Not visible at the bottom of the rack are the T3 handoffs.

---

12/18/2000

Looking up from the NTT handoff, we see 3 fiber innerducts, 4 coax cables (for 24 T3's) and 24 Ethernet cables. At the top of the picture, single- mode fiber is visible through the mounting holes at the bottom of the fiber duct.

---

12/18/2000

Here we can see the innerducts from the previous position routed under the fiber duct, as well as the XO handoff.

---

12/18/2000

This is a view of the same area as the previous picture, but from above the ladder rack.

---

12/18/2000

We subjected Worldcom's handoff to an "involuntary neatening" - they were using some zip strips and rubber bands to manage their fiber slack. We replaced that with a pair of slack jumper shelves, one for the Worldcom cables between their two ADC panels and one for our fibers.

---

04/15/2001

While we'd already neatened up the Worldcom handoff, they added a second 48-strand patch panel, and we'd replaced our duplex fiber jumpers with space-saving DFX jumpers.

---

12/18/2000

This is a view of row 5, looking from 5.06 down to 5.01. Note the cable management brackets on the cabling sections, as well as the spiral wrap protecting the Gigabit Ethernet fiber uplinks. Eventually, there will be many hundreds of Ethernet jumpers between the Catalysts and the patch panels.

---

04/15/2001

Ethernet jumpers for customer circuits (green), management circuits (red), and special-purpose circuits (purple) are starting to appear in the ladder and on the patch panels.

---

04/15/2001

Looking at the patch panels in 5.01, we can see the Ethernet jumpers. Note that all these jumpers are made to length in order to avoid having random slack dangling on the panels, congesting them.

---

12/18/2000

We're now looking from 5.01 toward 5.14.

---

04/15/2001

This view, taken from further down the row, shows the Verizon muxes in greater detail. In the forground is a FLM150 (OC-3, 84 T1's) lobed from a FLM600 (OC-12). To the right is a FLM2400, wired for 48 T3's.

---

12/18/2000

This is a view of the area which will hold regional routers. Note the two fiber downspouts for future needs.

---

12/18/2000

Here we're looking at one of the downspouts in the regional area. You can see why it's necessary to carefully position the ladder that runs along the top of the racks - there are 2 45-degree offsets in the downspout to bring it flush with the cabling section. If the ladder was further forward, the 45-degree part would be longer, moving the usable part of the downspout too low on the cabling section.

---

04/15/2001

The regional routers have been installed. There are two rows of these arranged in a mirror image of each other. There are 2 Juniper M20's, a Foundry BI4000, and a Cisco 7513 in each row. You can also see the slack jumper shelf as well as 5 DSX-3 panels. The top 3 panels each go to a different row, for different carriers, while the bottom two panels connect to the various routers in the row. Cross-connect cables are placed between the panels as needed.

---

04/15/2001

This is a view of the back of the regional equipment. This photograph is actually of the equipment in row 3, the mirror image of row 4 from the previous photograph.

---

04/15/2001

Here we can see another advantage of made-to-length cabling. Unlike the usual rat's nest of cables at the back of the router, we have very neat cabling. 24 T3's are currently cabled, with room for another 12 in the future. The current cabling provides service for 672 T1 customers (24 T3's with 28 T1's in each) or even more if the circuits are used for fractional T1 or Frame Relay services.

---

04/15/2001

This is an AS2511-RJ router, used for OOB (out-of-band) access to the various systems. It is one of 3 OOB routers at the POP. There is another 2511 in the mirror-image regional row, as well as a 2600 in the backbone row. The black cable on the top of the router is for the dialup modem, temporarily disconnected to allow this router to be reached from the 2600 router instead.

---

12/18/2000

This detail shows the mounting of a fiber duct "X" piece above the ladder rack. Note the use of the duct brackets to hold the fiber duct several inches above the ladder. This is necessary to permit cables/innerduct/etc. to be run on the ladder. On this job, we used brackets normally designed to hold the duct below the ladder, since we had severe height constraints - in some spots, the top of the fiber duct is only an inch or so below the light fixtures. The normal brackets would have raised the duct to about 10" above the ladder. In most cases, the ladder would be suspended from the ceiling over the racks and not attached to the racks themselves, but this room has a very low ceiling and indirect lighting, which meant we needed to do things differently.

---

12/18/2000

Here you can see a fiber duct 90-degree corner.

---

12/18/2000

And here is a T junction, showing the installation of the mounting hardware.

---

12/18/2000

Lastly, a detail of the way the support brackets are attached to the ladder and fiber duct.