STORAGE SYSTEM CONFIGURATION
| Parallel File System | Lustre |
| Lnet Nodes | 39 |
| Storage Array | Sonexion 3000 |
| Storage Racks | 7 Racks + 6 Tape Library Frames |
Sonexion 3000
The storage for the Cray system consists of 5 Sonexion 3000 Arrays which provides high performance Lustre file systems for User Home and Scratch areas respectively.
Sonexion file systems are made of two basic building blocks: scalable storage units (SSUs) and a metadata management unit (MMU). Each SSU houses disk drives and two OSSs that are connected to an even or odd top of-rack IB switch and then to the system storage network
The Cray system consists of 39 Lnet Router Nodes which routes traffic between Sonexion Lustre Servers on the IB Network and the Lustre Clients on the Aries HSN (High Speed Network).
Below is the Configuration Details for each Storage Subsystem:
Storage Subsystem for Home
- One MMU and 26 Scalable Storage Units (SSU).
- 2 MDS (Active/Passive) and 52 Object Storage Servers (OSS) nodes in Active/Active Configuration
- The filesystem data is stored on 6TB ,7.2K RPM HDDs on GridRAID (RAID 6)
- The filesystem metadata is stored on 900GB,10K RPM drive (RAID 10).
Storage Subsystem for Scratch
- One MMU and 4 Scalable Storage Units (SSU).
- 2 MDS and 8 OSS housed in a single Sonexion Base rack.
- The Scratch filesystem data is stored on 4TB ,10K RPM HDDs on GridRAID (RAID 6)
- Filesystem metadata is stored on 900GB,10K RPM drive (RAID 10).
Parallel Filesystems
- For Home - Lustre with a Useable capacity 7,211 TB and Sustained throughput of 217 GB/s.
- For Scratch - Lustre with a useable capacity 801 TB and sustained throughput of 49 GB/s.
Analyse :
The 8 utility nodes are apt for pre and post data processing needs of the HPC users.
Archive:
The backup and archiving is enabled with two Cray tiered Adaptive Storage (TAS) systems .
The TAS Solution is paired with the TAS Connector for Lustre to provide a unique and tightly
Integrated solution for tiered data management directly from the Lustre filesystem.
The details of the HSM/Archive Tier is as below :
- Total usable archival space of 24.06PB.
- Storage Array Disk Cache of 233TB for HSM provided by a Netapp E5600 series device.
- 6 SpectraLogic Tfinity Tape Libray Frames
- 48 LTO-7 Drives
- 5190 LTO-7 Media/Slot
- Versity VSM 1.0 for HSM
Cray TAS racks details :
- 2 Metadata Controllers
- 2 Connection Managers (CMM)
- 4 Connector Agents (CMA)
- 2 External Gateways
Tape Library
It also has Spectra Logic TFinity tape library for the data backup with 48 number of drives LTO-7, 5190 media slots LTO7, with HSM Solution which internally uses Versity Software and gives uncompressed capacity of 24.06 PiB
HSM Archive system
Cray TAS HSM Solution ensures data fluidity across storage tiers i.e data movement from the User Home areas to Tape Libraries and vice versa without affecting users or running applications. It has NetApp E5600 Storage Array providing a Disk cache with the useable space of 233 TiB and sustained throughput of 20 GB/s.
