One of our clients is a pivotal leader in electronic design, building upon more than 30 years of computational software expertise. Their ~9,000 employees serve the world’s most innovative companies in 23 countries, delivering extraordinary electronic products – from chips to boards to systems for dynamic market applications.
Flagship Facility Services has been providing janitorial services at for this organization’s corporate headquarters in San Jose, California for over 15 years.
A heat wave early in the summer of 2000 overloaded their Headquarters’ utility supply equipment resulting in a power outage that directly threatened the integrity of its core data centers – rightly deemed as mission critical for daily operations.
Because of the conﬁguration of the untested emergency backup system, when the generators took the full load from the UPS, a mechanical failure caused ﬁrst one, then three of the ﬁve generators to shut down. This triggered an attempted transfer of load back to the non-functioning UPS. Inevitably, the power failed to the associated Data Centers in turn.
Although not directly contracted to maintain the utility supply equipment, Flagship worked with our client to conduct a robust postmortem analysis of the event and the multi-vendor response to that event.
The Flagship team began to develop a services solution stack to scope the current system state, develop an MOP/SOW (Scope of Work) for a comprehensive maintenance program, create emergency event communications and protocols and detail the facilities’ data requirements to avoid a repeat of the system failure. Speciﬁc components of the ﬁnal solution include:
- Individual assessment of each component of the emergency back up system with the associated vendor to include deﬁning of single points of failure, history of equipment, and recommendations of best practices in industry
- Coordinate with controls vendor to develop and implement a robust controls package to monitor, alarm, and notify of real time status of incoming utility power, UPS, ATS, and Generator systems
- Identify and perform equipment overhauls, replacements where most needed, and revisions of preventive maintenance standards to include annual load bank testing
- Creation and implementation of an Emergency Response Plan agreed upon by all stakeholders
- Development of written communications protocols and procedures
- Training of Facility Management team to maintain and facilitate the various elements of the new Emergency Backup Systems program
Our client and the Flagship Facilities Maintenance Team formulated a dependable and scalable critical systems maintenance and event response program.
In the process this client’s internal customers gained a higher awareness and sense of accountability regarding the importance of ensuring the health of the physical infrastructure and the responsibility of the Facility Maintenance teams in maintaining the critical environment to a high standard of reliability and readiness.
Additionally, and just as valuable, is the esprit de corps and collaboration that arose as a byproduct of working together to create a solid process that functions as the foundation for future strategic planning, communication, and problem solving to protect physical assets as well as the health and wellness of the employees that work in those facilities.