The Institute for Digital Research and Education (IDRE) is a cooperative of faculty and technologists working to advance the existing body of computing knowledge and expertise at UCLA. An interdisciplinary research institute with an emphasis on the integration of high performance computing, applied algorithmic research and development, computer science, and informatics, IDRE will be central to UCLA’s Smart Manufacturing and Smart Power Management initiatives.
IDRE supports research and innovative scholarship that takes advantage of new technologies and encourages collaboration between faculty from different departments and disciplines at UCLA, the opening of new research questions, and the enrichment of the learning environment. The goal of this campus-wide collective is to ensure UCLA’s reputation as a world leader in high-performance computing and visualization research and education.
IDRE’s unique computational capability includes High Performance Computing resources and expertise, Grid and Cloud Access Services, Cluster Services, the Grid Portal, and UCLA’s data center system.
IDRE has a research staff that includes scientists, scholars, and post-doctoral researchers. The areas of specialization within the IDRE-High Performance Computing group include:
IDRE’s cloud and virtual computing research and development work is mainly focused on Infrastructure as a Service (IaaS) and Storage as a Service (SaaS). These services provide users with the ability to provision processing, storage, networks, and other fundamental computing resources to deploy and run arbitrary software, which can include operating systems and applications and the ability to create storage space on demand.
UCLA’s Shared Hoffman2 Cluster has 64-bit nodes, currently 774 and 7,508 cores, with an Ethernet network and Infiniband interconnect, that includes a scheduler, GCC and the best performing compiler for C, C++, Fortran 77, 90 and 95 on the current Shared Cluster architecture, applications and software libraries that offer languages, compilers and software specific to Chemistry, Chemical Engineering, Engineering, Mathematics, Visualization, Programming and an array of miscellaneous software. The current peak performance of the cluster is in order of 75 Trillion Floating Point, double precision, operations per second (TFLOPS). Hoffman2 is currently the largest and most power cluster in the University of California system.
Hoffman2 additional resources for researchers include complete system administration for contributed cores, cluster access through a 10Gb network interconnect to the campus backbone, high performance home and scratch storage space, capability to run large parallel jobs that can take advantage of the cluster's InfiniBand interconnect, and web access to the Hoffman2 Cluster through the UCLA Grid Portal, and access to the BlueArc and Panasas storage systems.
The cluster is also an end point on Globus online service using 10Gb network interconnect backbone, thus providing researchers a facility for fast and reliable data movement between Hoffman2 and most leadership class facilities across USA.
UCLA’s Dawson2 GPU CLuster, ranked 148 in the Top500, comprises 96 HP ProLiant SL390 G7 systems, each having dual socket Intel Xeon X5650 processors, 3 Nvidia M2070 Graphics processors, and 48 GB of main memory giving peak performance of 1.66 double precision Trillion Floating Point operations per second (TFLOPS). The cluster uses QDR Infiniband networking and 160 Terabytes of high performance common disk space from Panasas for communication and storage respectively.
UCLA’s Grid Portal provides a single web interface to eight UCLA computational clusters that represent 844 nodes. Aggregate peak performance is 48,000 GFlops. The Grid Portal provides a real-time status of all accessible clusters, the ability to submit a job and review results through the web, file transfer from local machines and among clusters, file management and editing and the ability to visualize chemistry data and 2D and 3D gridded data.
IDRE is in the process of creating a comprehensive software management system that will be accessible through web and will have capabilities such as software management, software distribution, and software monitoring and tracking.
IDRE has been instrumental in creating and implementing a strategic campus data center plan to most effectively support research computing. The data center system includes shared space in the following three facilities:
The UCLA Math Sciences data center, which is shared with the campus administrative systems unit, houses research computing clusters, including part of the main Hoffman2 shared cluster. The Math Sciences data center also provides backend services to Visualization Portal and Modeling Lab. Approximately 2,700 square feet of the Math Sciences data center’s 5,000 square feet are dedicated to supporting IDRE research computing. The compute resources in this facility and the IDRE data center are networked through 10gigabit Ethernet as well as wide-area Infiniband fabric. The Math Sciences facility is a Tier 3 data center space with greater than 600kW of power with full UPS and motor generator backup and 170 tons of redundant air conditioning capacity. The facility has a strong physical security, which includes 24 x 7 staff monitoring and physical presence in the center. In addition to the cluster nodes our central Blue Arc-based storage appliance, currently three-quarters of a petabyte, and online Tivoli tape backup system are housed here.
The Math Sciences Data Center has been extended through the innovative use of the HP Performance Optimized Data Center (POD) making available for researchers an additional 1,535 nodes and 18,000+ cores. The POD is a retrofitted 40' by 8' shipping container that has been extensively modified to support more than 1,500 compute nodes with associated network and interconnect equipment. The POD provides 600KW of power and 170 tons of air conditioning in a highly efficient manner that is roughly 40% better than a conventional data center at a fraction of the cost.
Located in the California Nanosystems Institute (CNSI) building, the IDRE data center consists of 2,500 square feet of Tier 2 data center space equipped with 200KW of power and 60 tons of air conditioning. The IDRE facility has space for 400 compute nodes. The data center is connected via a redundant, 10Gb fiber connection to the campus backbone through two Cisco 6509 switches in addition to four fiber-linked Infiniband connections to UCLA’s Math Sciences data center where there is space for 500 more nodes.