Characterization of a Rack-Level Thermosyphon-Based Cooling System

Cooling and other secondary equipment in a data center (DC) consume an average of about 43% of total DC power [1]. No major reduction in the power consumption of DC cooling systems has occurred since 2013 [1]. In addition, cooling and information technology (IT) equipment themselves generates a significant amount of low-grade waste heat, which is of little economic value, and is typically discharged to the ambient without any intermediate use.

An opportunity thus exists to both (1) reduce the cooling power requirement of legacy DCs, and (2) capture, transport and reuse this heat. This opportunity, along with the increasing rack density in data centers, represents an opportunity for liquid cooling—with its effective heat removal capability—to replace traditional air-cooling techniques as a data center cooling strategy.

Liquid cooling allows thermal engineers to operate with condensing temperatures in the 40–60 °C range, depending on the choice of a single or multiphase liquid cooling system [2,3], while maintaining chip case temperatures within reliability limits (e.g., the ASHRAE Recommended Range for air-cooled IT equipment [4]). The use of vapor-compression driven liquid cooling allows for condensing temperatures up to 90 °C [5]. A higher condensing temperature leads to effective waste heat harvesting.

Adoption of liquid cooling presents unique solutions to the challenges and technological barriers faced by air-cooling. These include successfully increasing IT equipment heat density without compromising reliability, improving the energy efficiency of the overall cooling system, reducing acoustic noise and total cost of ownership (TCO) of the DC, improving the potential for waste heat re-use and accommodating server densification.

The above reasoning makes it clear that liquid cooling presents several advantages for DC equipment cooling. However, the inhibition toward readily adopting liquid-cooled solutions in the DC industry stems in part from a lack of infrastructure available to legacy data centers to support new liquid cooling equipment. Furthermore, active (pump-based) liquid cooling systems require prime movers that, if failed, could cause a temperature increase in the working fluid large enough to generate boiling, raising the system pressure and causing leaks. Such an event can be catastrophic for the DC equipment, especially if water is used as the working fluid. Hence, while ASHRAE Technical Committee 9.9 adopted liquid cooling in DCs back in 2006 [6], standards such as NEBS and ETSI are yet to “readily” adopt liquid cooled systems.

Loop thermosyphon based cooling systems, however, present an opportunity for passive cooling of high-density ICT equipment in a compact form-factor that can be readily integrated at the telecom equipment/server level and conveniently scaled to an entire telecom cabinet/IT rack-level. The relative ease with which passive thermosyphon systems scale stems from their lack of moving parts, which occupy space and require weight-bearing support structures. Their closed-loop design is virtually maintenance free, without any fouling of the equipment (as in the case of water) or loss of the working fluid. Finally, the passive design eliminates the need to develop and deploy active controls to run the cooling system (e.g., active systems require pump speed modulation to avoid reaching the critical heat flux for transient IT loads [7]). In addition, if designed correctly, passive systems are self-sustaining, self-regulating, and virtually energy free (depending on application).

The use of thermosyphon-based cooling is growing in interest, and several studies containing theoretical system models exist (e.g., [8,9]). Zhang et al. [10] describe the increased exploration of thermosyphons specific to passive cooling in air-cooled data centers, with few studies at the time focusing on combined cooling and heat recovery. The review by Ding et al. [11] indicates a tradeoff between increasing passive cooling effectiveness and increasing infrastructure modification as one moves away from room-level thermosyphon use toward chip-level use, indicating a balance of effectiveness and expense at the rack-level. Tong et al. [12] designed a system that combined a thermosyphon with a natural cold source, demonstrating energy savings of up to 59%. Zou et al. [13] demonstrated that the retrofitting of a legacy air-cooled data center with a thermosyphon-based cooling with water-side economization can lead to significant energy and carbon savings, with a payback period of 1.6–3.6 years. Amalfi et al. [14] designed a rack containing dual (server and rack-level) thermosyphons for passive cooling, demonstrating heat rejection exceeding 5000 kW/m³/K.

Studies have investigated alternative refrigerants (e.g., R32 [15], CO₂ [16,17], and R744 [18], with additional information in Ding et al.'s review [11]) for thermosyphon loops specific for passive data center thermal management. The study by Albertsen and Schmitz [19] explores the use of fourth-generation, environmentally friendly refrigerants R1233zd(E) and R1224 yd(Z), the former being the working fluid for the present study. The authors find both fluids yield similar thermal performance and system stability with a maximum tested heat load of 1 kW at the evaporator.

Several studies (e.g., Han et al. [20], Zhang et al. [21], and Zhang et al. [22]) have explored the use of vapor compression cycles within thermosyphon-based cooling systems, but their focus was on improving cooling effectiveness and energy efficiency. Shao et al. [23] demonstrated that the use of an evaporative condenser, while increasing on-site water consumption, expands the annual free cooling time by up to 14% as compared to rejecting that heat to a single-phase fluid. Studies such as that by Wang et al. [24] explore issues surrounding the placement of the condenser portion of the thermosyphon outdoors. Thermosyphons have been proposed in conjunction with latent thermal energy storage systems for emergency cooling in case of power failure [25].

At the bench-scale, Sbaity et al. [26] showed a cooling capacity of nearly 3 kW for a thermosyphon-based cooling system when pumped water is used for heat removal at the condenser. Further, Wang et al. [27] studied cooling with heat recovery and developed a bench-scale system containing a thermosyphon-based microchannel cold plate cooling system, demonstrating an ability to recover over 85% of waste heat for a 500 W load and 82% for a rack containing two servers [28]. Their work demonstrates the potential for IT cooling and waste heat recovery.

Studies by Cataldo et al. [29], Szczukiewicz et al. [30], Amalfi et al. [31], Li [32], Meng [33], and Koito [34] extend server-level loop thermosyphon cooling to the rack-level for cooling entire DCs using loop thermosyphons. The study by Cataldo et al. [29], in particular, utilizes a scheme where the CPUs on individual servers are cooled by individual mini-thermosyphons housed within the server, while the condensers' of the server-level thermosyphons are cooled by the rack-level thermosyphon using water at 20 °C. The studies by Li and Meng [33] couple the thermosyphon loop with a mechanical refrigeration loop, similar to the system proposed in the current study. The work by Koito [34] adds a jet explosion stream (JES) to the evaporator of the loop thermosyphon. By doing so, the authors demonstrate that the overall height of the thermosyphon can be lowered from 1200 mm to 480 mm with a 158 mm initial liquid level, making it suitable for rack-level thermal management and high heat flux electronics. Finally, it is noteworthy that other rack-level cooling technologies have also been explored in the literature, such as the use of pulsating heat pipes [35]. Ding et al. [11] provide a comprehensive review of the application of loop thermosyphons to DC cooling.

However, no known studies address the issue of how to adapt liquid cooling systems into data centers containing heterogeneous (i.e., combination of low and high-density) IT equipment, which is seen in colocation facilities. It is important, therefore, to examine how air-cooled IT equipment can be seamlessly integrated into a liquid cooling framework and to address the limitations of thermosyphon-based cooling in this manner. We believe that the most feasible way to integrate air-cooled equipment into thermosyphon-based liquid cooling is at the rack level since it brings liquid cooling as close as possible to the air-cooled servers without needing to reconfigure the IT equipment. The novelty of our study, therefore, lies in (1) experimental characterization of a rack-scale system that utilizes thermosyphon-based cooling using R1233zd(E) and (2) uncovering the behavior of the system for racks containing air-cooled IT equipment.

Figure 1 depicts a novel system that not only efficiently cools and captures heat from hardware components via liquid cooling in DCs and telecom central offices, but also allows for heat recovery via a vapor recompression (i.e., heat pump) system. The equipment to be cooled (computer servers in this case) is contained in an air-cooled rack. The IT equipment is cooled by air that is continuously circulated within the cabinet, while the air itself is cooled via an air-to-refrigerant heat exchanger (HX) that is the evaporator of a thermosyphon loop. In particular, the evaporator is a finned tube HX that is widely used in refrigeration and air-conditioning applications. The refrigerant flows in a thermosyphon loop between a source and sink (evaporator and condenser, respectively). A reservoir tank (LV-separator) connects the two HXs and physically separates the refrigerant liquid and vapor phases.

The vapor refrigerant either can reject heat directly through a water-cooled condenser (called “low-pressure condenser” here) or, through the opening and closing of ball valves, be driven through vapor recompression, thereby boosting its pressure (and temperature) for enhanced heat recovery. Refrigerant does not flow simultaneously to the LP condenser and HP condenser. The cooling water for the high temperature condenser (called “high-pressure condenser” here) can then be used for various applications such as district heating, as service fluid within process industries, or as potable water in the same or colocated buildings. Alternatively, the high-pressure condenser of Fig. 1 can be replaced or augmented with an absorption chiller to produce cold water, which can be supplied to an HVAC system or used to supplement IT equipment cooling. The reader is referred to Ref. [36] for case studies detailing the applications and efficacy of the heat recovery system. Note that Fig. 1 does not display the direct connection between the liquid–vapor separator and the low-pressure condenser.

The cooling scheme indicates the evaporator (as a finned-tube HX) and condenser as key components in the proposed cooling and waste heat recovery system. System performance in the form of thermal management of the rack-mounted IT equipment and waste heat recovery ability is strongly dependent upon the thermal performance of these components, while the heat recovery process is strongly dependent on the efficiency of the compressor in use. The proposed system serves two purposes: (i) provide energy-smart thermal management of IT equipment by minimizing secondary (cooling system) power consumption and (ii) capture and upgrade waste heat to enhance re-usability, which leads to higher revenue generating activity [36].

Figure 2 shows the pilot scale experimental setup with key components labeled. The evaporator is located inside the cabinet directly behind the fan array (upstream of the cabinet fans) and cannot be seen without opening the cabinet. The pilot-scale setup uses refrigerant HCFO-1233zd(E), a low GWP alternative refrigerant to HFC-245fa, as the working fluid.

Table 1 presents a comparison of the commercially used refrigerant R134a along with other environmentally friendly counterparts. From Table 1, refrigerants R245fa and R1233zd(E) establish themselves as the most suitable candidates for the proposed rack-level cooling and enhanced heat recovery (CEHR) system since they both have low ozone depletion potential (ODP) and relatively high boiling points, which leads to lower evaporating and condensing pressures. Between the two refrigerants, R245fa is more economical while R1233zd(E) is more environmentally friendly (low GWP). Given the increasing trend toward adoption of “green” refrigerants and phasing-out of environmentally harmful ones, R1233zd(E) is chosen to be the primary refrigerant of choice for use in the proposed system. Refrigerant R245fa is, therefore, deemed to be the secondary choice. Choosing R1233zd(E) as the working fluid has several additional advantages:

• Based on a brief literature survey, no study on implementing R1233zd(E) in an actual data center/rack-level environment has been performed to-date.

• R1233zd(E) contains a high nominal boiling point (NBP), which leads to low (near ambient) system pressures. This feature allows for safe operating pressures, minimizes the chance of leaks, and does not require special code stamping for the separator tank and heat exchangers.

• R1233zd(E) is the green version of R245fa, so the two refrigerants exhibit similar performance characteristics in a given cooling system.

• R1233zd(E)is non-flammable.

• R1233zd(E) is readily available at an affordable price point ($47/lb in the U.S. [38] versus $27/lb for the commercially used R134a [39]).

• R1233zd(E) allows for exploration of refrigerants other than R134a. R134a has been the refrigerant of choice for most vapor compression driven or hybrid cooling loops (e.g., [3,40–42]) but will be phased-out in the near future due to its high global warming potential.

The electrical power supplied to the load banks was measured via the cabinet PDU. This data was acquired remotely via a web interface provided by the PDU manufacturer and sampled at a frequency of 0.2 Hz (one sample every five minutes), since the power stayed nearly constant at steady-state.

Air temperature and velocity measurements were made at four distinct locations (state points 7a, 8a, 9a, and 10a per Fig. 3) across the evaporator using air temperature and velocity sensors. These sensors consist of a hot-wire anemometer and a thermocouple in a low-profile shape and were used to measure the air velocity and temperature at the inlet and exit of the load banks and evaporator. Air temperature and velocity measurements were collected simultaneously for all sensors, sampled at a frequency of 0.2 Hz. Airside anemometry is detailed in a separate study featuring the evaporator of the proposed system [37].

Refrigerant temperature and pressure measurements were taken at the inlet of the evaporator, while only temperature was measured at the evaporator outlet in the vertical section of the riser. Refrigerant flowrate in the evaporator was measured on the downcomer using an ultrasonic flowmeter. The relative locations of the instrumentation used on the refrigerant and waterside are depicted in Fig. 4, while instrumentation specifications are detailed in Table 2.

The voltage and current signals generated by the instrumentation employed on the refrigerant and waterside were acquired using a National Instrument data acquisition system. Based on test conditions, it took the overall system about 40-45 min to reach statistically steady-state conditions, as defined by further changes in temperature falling below data noise levels, following which all data channels were scanned simultaneously at a frequency of 1 Hz for a duration of 15 min. A total of 15 data points across five test runs were collected over a 45-day period, spanning a cabinet heat load range between 2 and 7.5 kW in increments of 2 kW. An arithmetic average of the acquired data samples for each heat load tested was used to process the data.

It should be noted that temperature measurements were obtained indirectly as the difference in voltage (generated by a thermocouple) between the state point of interest and an ice bath, taken as the reference voltage, and converted to a temperature reading via a ninth-order polynomial [44]. This is the reason for specifying the thermocouple uncertainty as a percentage rather than a value in Table 2.

where $ρ$⁠, $Q$⁠, and $cp$ represent mass density, volumetric flow, and specific heat capacity, respectively. Figure 4 identifies the water and refrigerant-side state points used in the above equation.

where $PLB$ is the load bank power output.

Results show that the difference is within 10% in general, except for the 8 kW cases, where the maximum difference is within 15%. Finally, since all the waterside energy balances close within 15%, it can be inferred with confidence that all energy balances on the air and refrigerant sides also agree within 15%. Hence, the results presented in this study hold merit.

A summary of the key parameters of the system is presented in Table 3. The summary includes parameters that carry the least uncertainty (electrical power into the load banks and the rate of heat gain by the waterside) and important parameters that characterize system performance. Note that the first data point for test run 1 (3.96 kW) was not directly obtained from a cold start; rather, an intermediate heat load value of 1.9 kW was first obtained, but no data recorded. The initial heat load for the remaining test runs was however obtained from a cold start. This variation helps to identify any variation in the results that could occur due to an intermediate heat load.

The key parameters presented in Table 3 are depicted graphically in Figs. 5–13. Figures 5 and 6 confirm previously known thermosyphon-based system trends and characteristics, such as an increase in the thermosyphon flowrate with increasing system heat load. Further, the trend depicted by Fig. 5 reveals that the thermosyphon is in the gravity-dominated regime [19,46,47] as confirmed by the increasing flowrate with increasing heat load. However, beyond a heat load of 8 kW, the thermosyphon appears to be nearing a transition point and will eventually enter the friction-dominant regime [19,46,47]³ as evident by the near flattening flowrate trend. Figure 5 also reveals the hysteresis in the thermosyphon flowrate values as the heat load is increased and then correspondingly decreased along the same path, indicating a path dependence that may be related to the presence of different flow boiling regimes.

The riser quality data provided in Table 3 (based on Eq. (3b)) reveals that the refrigerant vapor quality is low. In fact, the riser quality is near zero (well below 20%), which shows that the system is practically running in single-phase and not able to take complete advantage of the high heat transfer coefficients afforded by the phase-change process. However, note that the void fraction (area ratio) varies between 20% and 47% between the lowest and highest heat loads, indicating that there is enough vapor to generate significant buoyancy forces to drive the flow. The void fraction is determined based on the experimental data through use of the Rouhani-Axelsson correlation for vertical flow [48], which is based on the drift-flux model and validated through empirical data. From a system design perspective, the evaporator outlet quality reveals that the chosen line sizes of 1-inch and 1.5-inch nominal diameter for the downcomer and riser, respectively, may be too large. A future prototype, therefore, is recommended to contain line sizes of 0.5 inch and 1 inch for the downcomer and riser, respectively.

In addition, the evaporator and condenser are connected via the separator tank, leading to two flowrate values instead of one, coupled via the vapor quality, as per Eq. (3a). Figures 5 and 6 graphically depict the two flowrates versus rack-level heat loads. For the chosen fill ratio of 44%, the flowrates through the downcomer (evaporator side) are within the same range as reported for other rack-level studies (e.g., [49] and [50] report values between 450 and 900 g/s per rack). The flowrates achieved in the present system can be lowered by reducing the size of the separator tank and optimizing the filling ratio. Note that the separator tank was oversized to dampen vapor pulsations.

The flowrate obtained at a given heat load depends on the previous system state. In the case of Fig. 5, the highest heat load for the increasing load case (blue curve) was obtained from a previous, one-step lower heat load, while that of the decreasing load case (red curve) was obtained from a cold start. The hysteresis at the highest heat load may be due to a different boiling regime, the time elapsed since the system is first powered on, or the influence of previous flowrate, but the reason cannot be stated for certain. In essence, the system history in achieving a specific state influences the flow characteristics.

A thermosyphon-based system's inherent simplicity results in a single operating point. Thereby, the flowrate in the system is directly linked to the system pressure. As the heat load increases, so does the system flowrate as seen in Figs. 5 and 6, as well as the system pressure, as measured in the separator tank and at the inlet to the evaporator (Table 3). Figure 7 depicts the trend in the system pressure, as gauged by the pressure drop across the evaporator, which on average, is about 21.4 kPa (3.1 psi) and of the same order as water or refrigerant-cooled rear door HXs [51,52]. The evaporator dimensions and geometry are provided in Ref. [53].

Figure 8 shows a plot of the thermosyphon's resistance as a function of heat load. The trend reveals a decrease in the thermosyphon's thermal resistance with increasing heat load, with a minimum value around 5.7 kW and subsequent increase in resistance as the heat load is further increased. Thus, the 5 to 6 kW range represents the thermally optimum load range for operation of this thermosyphon. This is because the air temperature at the evaporator inlet rises and drops slower than the waterside heat gain as the load bank power approaches the 5.5 kW range. The water inlet temperature stays between 6 °C and 7 °C range. The better heat transfer characteristics stem from the initially rapid increase in the refrigerant mass flowrate as the heat load approaches the optimum mark and slowing down beyond that, as frictional effects begin to catch up to gravity effects. The mass flowrate plots of Figs. 5 and 6 depict this trend.

Figures 9 and 10 plot the temperature of the air as it circulates through the cabinet, moving through the load banks and gaining heat while losing it as it flows through the evaporator. Specifically, Fig. 9 shows that the air inlet temperature to the load banks is well beyond the ASHRAE recommended range, even for lower heat loads such as 4 kW. However, this is due to the design of the evaporator, which was sized for an air inlet temperature of 40 °C to maximize the potential for heat recovery, discussed in detail in Ref. [36]. The high air inlet temperatures lead to even higher air outlet temperatures for the load banks, which in the 8 kW case, exceed the recommended limit for air cooling set by OEMs (e.g., 50 °C). From a design perspective, suitable air temperatures can be achieved by proper sizing of the evaporator and condenser. Alternatively, while in operation, the air inlet temperature to the IT can be controlled by having enough cooling water to the condenser for a given heat load and cabinet airflow rate.

Finally, it is noteworthy that the cabinet fans are located upstream of the evaporator (Fig. 2), so airflow inside the cabinet is across the evaporator and through the load banks. The default fan speed is 0.80 PWM. An increase in the air temperature at the evaporator outlet, as measured by the pre-installed temperature sensors, beyond 26 °C causes the fans to ramp up to their maximum speed. This results in the dip in air temperature at the evaporator outlet (Fig. 10) and load banks inlet (Fig. 9) near the 4 kW heat load, as the air temperature exceeds the manufacturer specified threshold.

Heat loss from the enclosed cabinet to the room ambient is determined as the difference between the applied power to the load banks and the heat gained on the condenser waterside, determined using Eq. (2). Values for these two quantities are provided in Table 3.

Figure 11 shows a plot of the cabinet heat losses as a function of applied power. The figure reveals that heat losses from the enclosed cabinet to the room ambient because of natural convection and radiation are of the order of tens of Watts for power input up to 4 kW, which can be considered negligible. However, the losses increase sharply (by two orders of magnitude) as the heat load is increased to about 7.5 kW (associated U-values are 0.3 W/m² K and 9.0 W/m² K for the lowest and highest heat loads, respectively). This trend shows that the hydraulic characteristics of the cabinet airflow can be improved, especially for higher heat loads.

Figure 12 plots the impedence (R–Q) curve for the condenser, which shows the same profile as the overall thermosyphon. Since the flowrate in the condenser (and evaporator) is proportional to the heat load, then the heat load values can be directly substituted for the mass flowrate without affecting the trend. The curve in Fig. 12 reveals a minimum at the flowrate corresponding to the 5.7 kW heat load, with a nearly asymptotic trend moving forward. This trend is typical of heat exchangers.

Further, the pressure at the condenser outlet is actually higher than at the inlet, as evident by the negative pressure drop values. This is due to the momentum component of pressure drop, which causes an increase in pressure as the condensing fluid decelarates as it flows through the condenser (decreasing shear stress at the liquid–vapor interface). The momentum component of pressure drop is aided by the static component, which also leads to a gain in pressure as the condensing fluid flows downwards, aided by gravity. The overall pressure gain across the condenser increases with increasing condenser flowrate, which itself increases with system heat load.

In a thermosyphon-based system, the condenser secondary side controls the system behavior and determines the refrigerant flowrates in the thermosyphon loop. The water temperature entering the condenser determines the condenser subcooling and sets the system pressure, which can be gauged from the pressure in the separator tank. Since the fluid in the tank is a two-phase mixture, the pressure determines the refrigerant temperature in the tank and subsequently in the evaporator.

Further, the water flowrate and temperature indirectly affect the air temperature inside the cabinet (through the refrigerant side), whereby higher water flowrates will lead to lower air temperatures for the same IT heat load and vice versa. Although the condenser design heat load is about 9 kW, the HX is likely oversized for the maximum heat load tested (7.5 kW), i.e., there is excess heat transfer surface area. This result stems from the extremely large subcooling (∼26 °C) at the condenser outlet for the maximum heat load (Table 3) and points to the fact that the condenser is flooded.

Finally, Fig. 13 shows the transient ramp-up behavior of the system. The figure indicates that the system time constant is 25.2 min, or that steady-state behavior is reached after approximately 40 min. This time constant is consistent in the refrigerant at states 1, 2, and 3 (i.e., the downcomer, riser, and liquid–vapor separation tank) per Fig. 3.

In addition to the insights gained from the plots and trends of Figs. 5–13, the following is observed based on the experimental data collected:

• A maximum rack heat load of 7.5 kW can be removed before exceeding the air temperature threshold of 50 °C.

• A maximum inlet air temperature of around 40 °C can be supplied for a 7.5 kW heat load, where the exit temperature would be at or slightly above the 50 °C threshold set by OEMs for air-cooled IT equipment.

• The maximum junction temperatures are seen for the case of fully throttling the servers and are in the range 75°–79 °C for an IT heat load of about 7.5 kW and air temperatures at the server exhaust just exceeding 50 °C. The minimum junction temperatures are between 39° and 43 °C when all servers were idling and at an IT heat load of about 3.8 kW and server air inlet temperatures near 30 °C.

• A refrigerant temperature of 32 °C at the evaporator inlet is observed at the highest tested heat load of about 7.5 kW. Since the fluid experiences a change of phase as it flows through the evaporator, the outlet temperature is also near 32 °C. This temperature will increase as the heat load increases, reducing the work required by the compressor to boost the discharge vapor temperature to a target value (e.g., 75 °C) for waste heat recovery.

• The refrigerant quality at the evaporator outlet is low (around 5%), even at the maximum tested rack heat load of 7.5 kW.

Further, the following logical deductions can be made from the data and plots presented:

• (a)

  The evaporator outlet quality is low enough to dissipate significantly higher heat loads before reaching dry-out conditions.

• (b)

  By maximizing the evaporator geometry while still fitting in the cabinet footprint and avoiding refrigerant dry-out, the maximum rack-level heat load that can be dissipated is about 28 kW (indicated in Fig. 15).

  • This requires increasing the finned tube length from 49 in. to 72 in. and the number of tubes per row from 7 to 8.

  • This load yields air temperatures entering and exiting the load banks of 22 °C and 50 °C, respectively, while using the same cabinet fans.

  • This load limits the refrigerant quality at the evaporator outlet to 70-75%.

• (c)

  The evaporator design heat load of 10 kW provides a safety factor of 1.33 relative to the maximum tested heat load of 7.5 kW.

• (d)

  Heat transfer calculations indicate energy conservation at the evaporator and condenser within 15% for all cases (within 10% for all cases below 7.5 kW input heat load), which is acceptable from a scientific perspective.

  Finally, based on repeated trial and data collection runs of the proposed pilot-scale system, the following positive aspects about the system were observed:

• (e)

  The system performance is stable.

• (f)

  The thermosyphon is self-sustaining and self-regulating.

• (g)

  It is observed that the thermosyphon works at heat loads as low as 300 W.

• (h)

  The low-GWP, low-ODP R1233zd(E) is a suitable choice for the system since it exhibits near ambient system pressure (and therefore has a low chance of leakage), stable operation, and a low enthalpy of vaporization, leading to easy boiling.

A first-order uncertainty analysis is applied to characterize the propagation of errors from the measured values to the final results, as per the approach outlined by ASME [54] (based primarily on the work of Coleman and Steele [55]). The expanded uncertainty in the measured variables is based on instrument uncertainties specified in Table 2. Further, the uncertainty values for various thermodynamic and transport properties used for data reduction are stated in Table 4. The expanded uncertainty range was determined for each value for the high-impact variables based on a 95% confidence interval, and are provided in Table 5.

The uncertainty in airside pressure readings was not determined since there is little to no variation in this parameter. Similarly, uncertainty in the airside relative humidity was not determined since it is only used to fix the air state (determine humidity ratio) and since no condensation occurs inside the evaporator, the humidity ratio remains constant throughout the cabinet for all data points. Hence, these variables are de-emed to be low-impact and specific uncertainty values not required which may impact further results.

Finally, the reason for the relatively high uncertainty in the quality values is due to the propagation of error resulting from the outlet quality being determined from the inlet enthalpy, refrigerant mass flowrate through the evaporator and evaporator duty, as opposed to the outlet enthalpy being determined directly based on measured temperature and pressure at the evaporator outlet.

The plot on the left in Fig. 14 shows that the evaporator effectiveness displays a linearly increasing trend and lies within a 35%–60% range. The trend is opposite to what is expected due to a decreasing trend in subcooling at the evaporator inlet with increasing heat load. This is due to the heat gained by the refrigerant from the hot cabinet air as it flows through the downcomer and enters the evaporator, despite the tubing being insulated.

The plot on the right in Fig. 14 show that the condenser effectiveness lies within the range 30%–99% and displays a decreasing trend with increasing heat load, effectively captured with a second-degree polynomial (R² = 0.92) for heat loads ranging from 2 kW to 7.5 kW. The high effectiveness value of close to unity is displayed at the lowest heat load, where the low degree of subcooling at the condenser outlet does not adversely affect the refrigerant flow. This result is largely due to the low water flowrate at this lowest head load, compared to a seven times higher yet steady flowrate for the remaining, higher heat loads. Barring the lowest heat load, the majority of condenser effectiveness values lie within the 30%–60% range.

Conversely, the lowest effectiveness and highest degree of subcooling occur at the highest heat load, with values of 0.28 and 26 °C, respectively. This shows that the condenser effectiveness decreases with increasing degree of subcooling at its outlet, linked to the growing pool of subcooled liquid at the base of the condenser, which leaves less volume for desuperheating and condensing, in addition to creating a backpressure that has an adverse impact on refrigerant flow. The condenser design can thus be improved to maintain the degree of subcooling within a specified value at the highest heat load.

The two curves of Fig. 14 suggest an optimal operating point of 4.25 kW for the proposed system, which maximizes both HXs effectiveness, yielding a common value close to 50%.

At the highest heat load tested, the heat exchange effectiveness for the load banks, evaporator, and condenser are, on average, 0.32, 0.60, and 0.30, respectively. The load bank effectiveness is calculated by treating the hot bed of electronics as an infinite heat capacity fluid stream at 79 °C (junction temperature at maximum heat load) and the air stream as the fluid with the lower heat capacity rate. These effectiveness values can be used for comparing the proposed system's performance with similar systems.

Assume a data center with N racks and an average rack IT power density of P kW/rack. For such a configuration, a comparison of the proposed CEHR system versus a traditional air-cooled CRAH based system is presented in Table 6, where the CEHR system contains a maximum cooling load of 900 W per rack due to the cabinet fans.

Table 6 indicates that an average rack power density exceeding 1.2 kW ensures a PUE below 1.74, thereby making the proposed CEHR system more energy efficient than the average air-cooled data center. This, minimal IT power density makes the CEHR adoption feasible as rack densities continue to increase (e.g., 15–20 kW/rack by 2025 with the increasing adoption of AI workloads [62]).

Further, the major difference between the proposed system and a traditional air-cooled data center utilizing a CRAH is that the blower is replaced with cabinet fans, which require significantly lower power. For an IT rack density of 10 kW, this would result in a PUE of 1.09, representing a greater than 50% improvement over the average DC PUE of 1.74.

Benefits in addition to the proposed system's energy saving potential include:

• Enhanced heat recovery by coupling the cooling loop with a vapor compression cycle

• Working fluid is low-pressure and environment friendly

• Easy retrofitting in legacy data centers via a rear door heat exchanger (RDHX)

• Enhanced reliability than direct water cooling (DWC) in event of a leak or pump failure

• Maintenance free by virtue of passive design; ideal for remote installations

• Does not require a CDU; saves valuable rack space and lowers OPEX

• Enables multi-scale installation (server, rack and row level [36])

• Rack-top water cooled condenser can be replaced by single, rooftop air-cooled condenser or single row-level water cooled condenser

Finally, a comparison with other cooling schemes is presented in Fig. 15 below (sources [63–70]). The comparison reveals that the proposed close-coupled cooling solution performs better than traditional air-cooling but poorer than direct liquid and immersion cooling but with the added benefit of enhanced waste heat recovery.

A data center rack-level cooling and heat recovery system has been designed, assembled and tested. The results reveal stable system performance using a low-pressure, environmentally friendly working fluid, R1233zd(E). The proposed system is a pump free design that is self-sustaining and self-regulating. The design reveals a maximum heat load removal capability of 7.5 kW using air-cooling, which can be extended to higher heat loads by either redesigning the air-cooled evaporator for lower air inlet temperatures or by switching to direct-to-chip liquid cooling using microchannel heat sinks mounted on top of the high heat flux devices.

However, bottlenecks in system performance such as large hysteresis in the mass flowrate values, excess subcooling at the condenser outlet and near zero quality at the evaporator outlet al.so emerge. These issues point to flaws in the thermosyphon design, in particular, downcomer and riser oversizing and the limiting heat removal capacity of the condenser. In effect, design optimization by reducing the line size and separator tank capacity, as well as appropriately sizing the two HXs can reduce upfront cost, save on refrigerant inventory and optimize thermal performance.

The modeling efforts reveal that the evaporator performance, as measured by its effectiveness, increases with heat load while the condenser effectiveness decreases due to condenser flooding as subcooled refrigerant pools at the base of the heat exchanger, as evident by the extremely large subcooling of 26 °C at the highest heat load tested. The heat transport capability of the system performs well for low heat loads till 4 kW, beyond which stray heat losses to the ambient cause degradation in the system performance. This situation can be circumvented by better system design in future iterations.

In addition to better system design, an enhanced level of telemetry and data collection can improve system performance. Key to this are improved instrumentation on the airside to accurately capture temperature and velocity measurements at multiple points in a 2D gridlike manner, e.g., using equipment as given in Refs. [71] and [72], and an accurate measurement of the evaporator refrigerant-side pressure drop using a differential pressure sensor, such as in Ref. [73].

This material is based upon work supported by the NSF IUCRC Award No. IIP-1738782. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

• National Science Foundation (Award No. 1738782; Funder ID: 10.13039/100000001).

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

Roman

c =

cabinet (enclosure to mount Information Technology equipment)

$cp$ =

specific feat capacity (⁠$J/(kg K)$⁠)

cond =

condenser

dec =

decreasing

Diff =

difference

evap =

evaporator

Ext =

external

inc =

increasing

$m$ =

mass (kg)

$m˙$ =

mass flowrate (kg/s, unless stated otherwise)

$p$ =

pressure (Pa)

$P$ =

power (electrical) (W)

R =

resistance (W/K)

r =

refrigerant (used to denote state point on refrigerant side)

Q =

volumetric flowrate (m³/s)

$Q˙$ =

heat rate (W)

Sat =

saturated

$T$ =

temperature

tn =

tank (liquid-vapor separator tank)

w =

water (used to denote state point on waterside)

$x$ =

quality

Greek Symbols

$Δhfg$ =

enthalpy of vaporization (J/kg)

$ε$ =

effectiveness

$ρ$ =

density (kg/m³)

Subscripts

air =

air-side

cond =

condenser

evap =

evaporator

in =

inlet

l =

liquid

LV,sep =

liquid-vapor separator tank

out =

outlet

r, ref =

refrigerant

sc =

sub-cooling

v =

vapor

wtr =

water

Abbreviations

ASHRAE =

American Society for Heating, Refrigerating, and Air Conditioning Engineers

CAPEX =

capital expenditure

CDU =

coolant distribution unit

CPU =

central processing unit

CEHR =

cooling and enhanced heat recovery system

DAQ =

data acquisition

DC =

data center

D2C =

direct-to-chip

EEV =

electronic expansion valve

ETSI =

European Telecommunications Standards Institute

EV =

end value

HP =

high pressure

HX =

heat exchanger

ICT =

Information and Communications Technology

IT =

Information Technology

LB =

load bank

LP =

low pressure

LV =

liquid-vapor

MFM =

Magnetic Flowmeter

MV =

measured value

NEBS =

network equipment-building system

NI =

national Instruments (manufacturer)

OEM =

original equipment manufacturer

OPEX =

operating expense

PDU =

power distribution unit

PG =

pressure gauge

PT =

pressure transmitter

PUE =

power usage effectiveness

PWM =

pulse width modulation

RDHX =

rear door heat exchanger

TC =

thermocouple

TCO =

total cost of ownership

UFM =

ultrasonic flowmeter

VFD =

variable frequency drive