Field report: rooftop datacentre installation

Three-day onsite installation and commissioning of a new primary rooftop compute environment and a physically separate basement recovery platform for a five-storey office. All scheduled infrastructure tests completed successfully before a subsequent building event.

Assignment record

Day 1, 07:34 — Site arrival

The installation team arrived while the office was already in active use. Staff occupied parts of the lower floors and continued normal work throughout the attendance. A desktop deployment team from a separate supplier was completing the final desks, docking stations, displays and workstation connections as additional client personnel arrived.

The principal network cabling had been installed during the office fit-out. Cabinet destinations, floor outlets and wireless access-point positions were labelled, and the majority of the server, storage and network equipment had been configured before delivery. This reduced the intended onsite work to positioning, physical installation, final patching, power integration and commissioning.

Day 1, 08:16 — Rack delivery and positioning

The two new 42U cabinets were unloaded into the building’s service area and transferred to the rooftop plant level using the designated goods route. Both cabinets passed through the available access points without modification. They were moved into the marked positions in the rooftop service room and aligned with the preinstalled power, containment and data-entry points.

The dual online UPS units and standby generator interfaces were located on the same roof-level complex. Equipment clearances, cable routes and access to the front and rear rack faces were checked before the cabinets were populated. No obstruction, movement or visible problem with the room fabric was recorded.

Day 1, 10:42 — Rack population

Preconfigured compute nodes, core and access switches, firewalls, management appliances and storage controllers were installed in the agreed rack positions. Equipment was distributed between the cabinets according to the supplied elevation, with redundant power supplies divided across the two UPS-backed distribution paths.

Preinstalled fibre and copper cabling was presented at the overhead containment and patched into the designated interfaces. Cable bundles were dressed by destination and service, secured with releasable ties and checked against the fit-out contractor’s test schedule. End-to-end verification found two reversed labels and one incomplete floor outlet; all three were corrected without affecting the programme.

Day 2, 08:03 — Network and platform integration

Core switching, firewall transit links, server uplinks and management networks were brought online in a controlled sequence. Redundant paths were tested individually before aggregated links were enabled. VLAN, DHCP relay, DNS, directory, wireless and external connectivity checks completed from representative points on every occupied floor.

The other supplier continued connecting desktops while these checks were performed. Newly occupied desks received addresses, authenticated against Active Directory and reached the required application services as staff gradually filled the building. The concurrency produced several duplicated test records but no unresolved technical fault.

Day 2, 13:27 — Basement recovery environment

The separate 20U emergency backup rack in the basement was populated with recovery switching, management equipment and the high-capacity backup storage array. Its power, network and environmental connections were commissioned independently of the rooftop room. Initial replication from the primary platform completed overnight and sample restore points were opened successfully.

An offsite backup service was scheduled as a later phase. At the time of installation, the client company had not selected a backup site or approved a carrier, hosting room or connection method. No offsite target could therefore be commissioned, and the available data protection arrangement consisted of the rooftop production environment and its independent basement copy.

Day 3, 08:11 — Resilience testing

Commissioning resumed with individual loss-of-feed tests across both UPS paths. The platform remained operational while each path was isolated in turn. Battery state, bypass availability, load distribution and restoration were recorded at the UPS panels and through the management interfaces.

The roof-mounted standby generator completed its automatic start sequence and accepted the simulated building load. The UPS systems transferred without interruption and returned to mains operation after the prescribed stabilisation interval. Network path failure, firewall failover, server power-supply loss and recovery-platform reachability tests also passed within the acceptance thresholds.

Day 3, 15:48 — Commissioning complete

Final cabinet elevations, cable schedules, serial records and test results were reconciled against the delivered installation. All front and rear doors closed correctly, temperature readings were stable and both management systems showed a normal state. The remaining desktop team confirmed that its last group of workstations had connected successfully.

The client representative received the commissioning record, known-items schedule and access information. No outstanding defect prevented use of the platform. The only deferred item was the offsite backup phase, which remained dependent on a client decision regarding location.

18:26 — Informal close-out

After leaving the site, the installation team met for a self-congratulatory evening beer at a bar on the same office estate. The premises faced across a courtyard toward the converted warehouse, placing the client’s building and rooftop service-room structure within clear view from the table.

The building remained occupied and illuminated. Staff could be seen working at several windows, and people continued to enter and leave through the main reception. There was no visible indication that the completed installation required further attendance.

18:41 — Initial indication

A loud, isolated crack was heard from the direction of the client building. Several office workers then left through the main entrance at a faster than normal pace. Confused shouting could be heard in the courtyard, although no alarm was immediately distinguishable from the bar.

A second sound followed. This was a louder, slower sequence of cracking and crushing noises that appeared to travel from the roof into the upper structure. The team moved outside as occupants began leaving the building in greater numbers.

18:42 — Progressive vertical collapse

The rooftop service-room floor failed beneath the newly commissioned datacentre, its two 42U racks, both UPS systems and the standby generator assembly. The combined complex moved downward into the fifth floor and produced the first of five extremely loud impacts.

Four further crashes followed in sequence as the equipment and accumulating structural material passed through the fourth, third, second and first floors. Each floor impact expelled debris and shattered exterior windows at the corresponding level. Glass, ceiling material and office contents fell into the courtyard while the descending mass remained substantially concentrated on the same vertical line.

After the fifth crash, the rooftop plant, primary cabinets, power equipment and sections of all five floor slabs came to rest in the basement. Dust obscured the lower windows and reception area. Building occupants continued evacuating, and the installation team notified emergency services from the courtyard without re-entering the premises.

18:45 — Generator fuel fire

The generator day tank and fuel lines ruptured during the descent. Diesel released into the basement debris and across damaged electrical equipment, where it ignited before the building supply could be confirmed isolated. An orange flame became visible through the failed ground-floor glazing, followed by dense black smoke rising through the openings left in all five floors.

The fire spread along the fuel-contaminated material surrounding the collapsed generator and rack assemblies. Several smaller flare-ups were visible as trapped fuel drained from the distorted generator enclosure. The team remained in the courtyard with evacuated staff and provided the emergency operator with the generator specification, nominal tank capacity and last known test state.

18:52 — Fire brigade attendance

Three fire appliances entered the office estate and crews established an exclusion area around the building. Firefighters confirmed that all persons reported onsite had reached the assembly point, isolated the available utility supplies and approached the basement through a protected access route. The visible fire was treated as a fuel-fed incident within an unstable structure.

The installation team watched from outside the bar as crews applied firefighting foam and water into the basement and through the open floor path. Flames reduced, returned briefly around the generator assembly and were then extinguished. Water continued to be applied while thermal checks were performed across the debris, leaving the collapsed equipment exposed to sustained water, foam, soot and runoff from the floors above.

Firefighters remained at the premises into the evening to monitor the structure and isolate residual hot spots. No technical inspection was permitted that night. The team left the estate after providing contact details and agreeing to return when the fire brigade and structural engineer considered access safe.

Following day, 09:18 — Controlled access

The installation team returned the next morning. The fire brigade confirmed that the fire was extinguished and, after a further atmosphere and temperature check, granted limited access to the basement under the structural engineer’s controls. Standing water, firefighting foam, soot and wet building material covered the lower level. The equipment remained in the position in which it had landed and had not been energised or moved overnight.

Structural position and attribution

The equipment had been installed in the positions shown on the approved fit-out drawings and within the room identified by the client team as the datacentre location. Rack levelling, cabinet loading and generator commissioning produced no abnormal movement or reading during the three-day attendance. No structural capacity calculation formed part of the IT installation scope, and the room, floor and supporting building fabric had been treated as completed works made available by others.

The subsequent failure occurred after commissioning and handover. Pending examination of the converted warehouse structure, historic floor alterations, cumulative rooftop loading and the supporting columns, no installation activity can be isolated as the initiating event. The collapse is therefore recorded as a spontaneous structural departure coincident with, but not demonstrably caused by, the successful completion of the technology works.

Site assessment

The post-fire inspection found a continuous vertical failure path from the former rooftop service room through all five occupied office floors to the basement. Floor plates were locally absent around the path, adjacent beams and columns showed deformation, and exterior glazing had failed at every impacted level. Smoke staining and heat damage extended upward through the collapse opening from the basement.

Power, data, fire detection, sprinkler and drainage services crossing the collapse line had been severed. Firefighting water, foam, soot, dust and displaced office contents affected areas beyond the immediate openings. The roof-level room no longer existed as an independently recognisable space, and the office was assessed as unavailable for occupation pending extensive temporary works and a full structural investigation.

Equipment assessment

Both 42U cabinets were found folded into the basement debris with their frames, rails and installed equipment deformed beyond economic recovery. Server chassis, network appliances, primary storage controllers and power-distribution units had sustained combined impact, compression, fire, smoke, firefighting water and foam damage. The dual UPS systems had separated from their enclosures. The ruptured generator assembly was embedded within material from the lower floor slabs and showed extensive fire damage around its fuel system. No component of the rooftop production platform was considered suitable for energisation or data recovery in place.

The vertical services were severed at multiple levels, and the newly tested office cabling terminated at open floor edges. Workstations close to the collapse line were destroyed, buried, smoke affected or saturated by water applied through the floor openings; surviving desktop equipment could not be used because the building, server platform, directory services and core network were unavailable.

Removal of the wet upper debris exposed the separate 20U emergency backup rack directly beneath the primary wreckage. The rooftop datacentre complex, including the two production racks, dual UPS equipment and generator, had landed on the basement recovery system and its high-capacity backup storage array. The rack and all installed media were crushed, burned and flooded beneath the primary platform. With the offsite service still awaiting selection of a backup site, the completed installation had no remaining primary, onsite backup or offsite recovery copy.