Data centers are driving the growth of many hi-tech industries but a consequence of their rapid increase in capacity is greater consumption of energy and resources, and increased greenhouse gas emissions. Between 2000 and 2006, the amount of energy consumed by data centers nationwide has doubled and in 2011, the projected annual energy use can be as much as 100 billion kWh. Demand for more and larger data centers continues to increase, but their enormous energy consumption, along with rising energy costs, has expanded the focus of data center management from basic day-to-day maintenance to include more focused operations and energy efficiency plans. This article discusses the contributors to the high energy use and explores areas for improvement of energy use in data centers.
In 2007, the Environmental Protection Agency published a report on Server and Data Center Energy Efficiency. The EPA report details the rapidly growing energy costs of data centers from $4.5 billion in 2006 to $7.4 billion projected in 2011. During the same time the increase in data center energy consumption was projected to climb from 61 billion kWh in 2006 to 100 billion by 2011 (Figure 1). Pacific Gas & Electric Company (PG&E) estimates that data centers were responsible for 500 MW of peak electrical load in PG&E's service territory in 2007 (out of total system wide peak load of 20 GW - 2.5 percent of total). This is equivalent to the generating capacity of a large base load power plant.
|Figure 1: Energy Consumption Growth Trends - Past and Future (depending of operation scenarios implemented)|
Source: EPA Report to Congress on Server and data Center Energy Efficiency 2007 - 2011 (page 10)
These growth factors have led to increased scrutiny from regulators, customers, and other stake holders. Already, several regulatory agencies are studying energy consumption of data centers and working to identify energy efficiency opportunities.Some notable activities include:
- US Public Law 109-431 requiring the EPA to submit a report on energy consumption of data centers to US Congress
- The EPA has advocated use of separate energy meters for large data centers and development of procurement standards
- The EU Is developing a voluntary Code of Conduct for data centers prescribing energy efficiency best practices
- In February 2010 representatives from the U.S., European Union and Japan met to identify goals and desired outcomes in developing and applying energy efficiency metrics, and developed guiding principles for measuring energy efficiency in data centers.
These efforts are designed to help the industry to reach a common understanding of energy efficiency metrics to improve data center efficiency and reduce energy consumption.
Measuring Energy Consumption in Data Centers
The impact of data centers on the power grid first came to the public's attention during the late-1990s internet boom. Because data centers run entirely on electricity, their growing power demand from the grid began to cause concern among utility planners. Additionally, operating costs have become a matter of great concern to owners and facility operators and are beginning to overshadow other technology investments. As a result, measuring a data center's energy use has become common practice.
The chart below (Figure 2) demonstrates a typical breakdown of energy consumption by a data center. Greater than 80% of its energy is consumed by IT infrastructure and the cooling and environmental equipment used to maintain proper IT equipment temperatures and airflow.
|Figure 2: Energy Use Breakdown in high-performance data centers|
Source: EPA 2007 Report to Congress on Server and Data Center Energy Efficiency
Another method to measure a data center's energy use is the Power Usage Effectiveness (PUE), introduced by Green Grid in 2007. The PUE provides data center operators with a better measure of a facility's overall performance by providing feedback on how efficiently a data center uses its power. The PUE is the ratio of the total amount of power used in the facility to the power delivered to computing equipment. Many of the industry's leading data center operators have begun sharing tips, strategies, and best practices in design and operations to help achieve better PUE.
PUE = Total facility power / IT equipment power
In addition to measured data, the EPA has also explored implementing a new ENERGY STAR benchmark for data centers as described in a 2007 report to Congress. Since the report's publication, most major IT manufacturers have announced "Green" products and initiatives.
As a result of better measured data, and newer efficient technologies, many opportunities exist to improve the efficiency of data center operations.
CADE stands for Corporate Average Data center Efficiency. It is another metric used to rate the overall energy efficiency of data centers. It was introduced by the Uptime Institute and McKinsey to be a single key performance indicator to compare the energy consumption of data centers. CADE combines measurements of the energy efficiency and utilization of IT equipment and facilities into a single percentage. Higher CADE number indicates a more efficient data center.
CADE is represented by the following formula:
CADE = Facility Efficiency (FE) x Asset Efficiency (AE)
- Facility Efficiency (FE) is equal to (Facility Energy Efficiency) x (Facility Utilization)
- Asset Efficiency (AE) is equal to (IT Energy Efficiency) x (IT utilization)
CADE is used to:
- Monitor data center performance.
- Compare data centers with those of a competitor
- Gauge the return on investment (ROI) on a green computing initiative.
Although CADE doesn't accurately reflect utilitization gains made by removing downed servers nor does it address the utilitization or energy efficiency of outsourced servers at hosting providers, it was widely adopted by operators and regulators. It serves a similar purpose as the mandatory CAFE (Corporate Average Fuel Economy) standards enacted by US Congress in 1975 for the automotive industry.
Key Areas for Improvement of Data Center Efficiency
Energy saving opportunities may be grouped into three main categories:
- More efficient IT equipment
- Improved IT equipment operations
- More efficient HVAC and heat removal
#1 - More Efficient IT Equipment
Microprocessors: Major advances in the design and manufacture of integrated circuits have led to dramatic improvements in microprocessor speed and performance over the last two decades. Between 1986 and 2002 microprocessor performance improved roughly 50% per year. Due to continuous advancement, three key trends in server microprocessor technology show great promise for reducing server energy use in the near future: (1) the shift to multiple cores; (2) the development of dynamic frequency and voltage scaling capabilities; and, (3) the development of virtualization capabilities.
Servers: More energy-efficient servers have entered data center markets in order to meet demand for energy efficient equipment. The use of power management (i.e., dynamic frequency and voltage scaling) with multiple-core microprocessors and virtualization capabilities is resulting in more efficient operation. Also, the use of high-efficiency power supplies, and internal variable speed fans for on demand cooling are contributing to energy efficiency.
Improvements in underutilization of servers are also producing energy efficiency gains in data centers. Maximizing the utilization rate of existing servers is one of the most significant opportunities for energy savings in U.S. data centers.
Storage Devices: HDD storage devices are becoming more efficient due to a shift to smaller form factor disk drives and increasing use of serial advanced technology attachment (SATA) drives. The average power use per drive is expected to decrease by about 7% between by 2010 as a result of these trends.
Improved management of storage resources should result in significant data center energy savings. Management strategies include storage virtualization, data de-duplication, storage tiering, and movement of archival data to storage devices that can be powered down when not in use. Also, increased reliance on solid-state flash memory would create an emerging energy-efficient storage option in data centers.
#2 - Improved IT Equipment Operations
Start with the "low hanging fruit" of data center efficiency by making sure that all existing efficiency measures are being utilized, or 'active'.Be sure to turn 'ON' energy management features on all IT equipment with such features.
Utilize "best practices" and technologies found in the most energy-efficient facilities operating today. Facilities employing proven technologies and management practices represent the "best in class" of today's data centers. Many cost-effective technologies and practices are not widespread but may prove to be worth the effort of going the extra mile to implement them.
Application of improved operation and best practices above can be broken down to the equipment level to show specific examples of energy-efficiency measures to be pursued:
- Computing software
- Avoid excess code and inefficiencies in software design
- Shift computational load among systems to maximize energy efficiency
- Manage upgrades to minimize number of legacy servers
- Turn off or remove dead, obsolete, or excess equipment so it won't be used for extended periods of time
- Keep power management options 'ON' on all equipment.
- Use shared data storage, data compression and data de-duplication
- Use centralized servers - large systems to improve sharing of computer resources.
- Use high-efficiency UPS units
- Use equipment w/ built-in power monitoring
- Electrical Systems
- Use high-efficiency power distribution of higher-voltage AC & moderate-voltage DC - 50-600 VDC
- Use high-efficiency UPS units
- Controls and Management
- Use system management hardware/software to enable powering down parts of server clusters during times of low utilization
- Monitor power use, platform temperature, and processor utilizations to allow operators to better understand and manage energy use in their data centers.
- Optimize cooling controls to dynamically match the cooling supply to the IT heat load.
#3 - More Efficient HVAC and Heat Removal
- Use premium-efficiency motors in fans and pumps
- Use improved heat removal airflow management - Isolation of hot and cold isles can improve efficiency
- Optimize data center airflow design configuration by using CFD (computational fluid dynamics) modeling
- Use high-efficiency variable-speed air-handler fans and chilled water pumps
- Use variable-speed chillers and variable-speed, primary-only chilled water pumping
- Use high-efficiency chiller and chilled water supply motors
- Use high-efficiency Computer Room Air Conditioning (CRAC) units
- Use air-side economizers (outdoor air) for free cooling when outdoor conditions permit
- Provide commissioning of the infrastructure systems to ensure optimal functionality of all systems within design parameters
- Rebalance air-handlers after any significant IT reconfiguration
- Increase chilled water supply and return temperature difference to reduce chilled water flow.
- Optimize chillers and cooling towers
- Reuse waste heat for space heating (in not data center spaces of building)
- Use direct liquid cooling water or other dielectric liquid to take advantage of better heat transfer properties of liquid cooling compared to air cooling systems
As the demand for data storage and cloud computing increases for both businesses and consumers, reliance on data centers will continue to grow. Implementing energy-efficiency strategies for these facilities could considerably reduce peak loads and ongoing energy consumption attributable to data centers. The strategies outlined in this article can largely be applied to both new and existing data centers. The energy reductions will lead to less expensive utility bills, and would also help minimize the strain on power grids in large metropolitan areas with high concentrations of data centers. Because data center inefficiency is widespread it is a major concern worldwide. The recommendations outlined above could serve as guidelines for better data center management resulting in lower energy consumption, lower costs and reduction in carbon emissions.
California utilities offer outstanding educational opportunities that focus on the design, construction and operation of energy-efficient buildings. Listed here are a few of the many upcoming classes and events; for complete schedules, visit each utility's website.
Energy Efficiency's Role in Your Sustainability Agenda
Benchmarking Energy Use In Commercial Buildings
read more >
Data Center Analysis Tools
read more >
Don't miss future issues - to sign up for a free email subscription, please visit our
newsletter subscription page. Send letters to the editor, suggestions on topics for future
issues, or other comments to the e-News editor via our
Comments & Feedback form.
e-News is published by Energy Design
an online resource center for information on energy efficiency design practices in
Savings By Design (www.savingsbydesign.com)
offers design assistance and incentives to design teams and building owners in California
to encourage high-performance nonresidential building design and construction.
Energy Design Resources and Savings By Design are funded by California utility customers
and administered by Pacific Gas and Electric Company, Sacramento Municipal Utility
District, San Diego Gas and Electric, Southern California Edison and Southern California
Gas Company, under the auspices of the California Public Utilities Commission.