HPC & Machine learning Experts

Japanese Government Agency 3PB Storage System

Japanese Government Agency Background

Pacific Teck has implemented a 3PB project in Japan for a Japanese government agency. The storage unit for this project is mission critical and having a tolerance for zero down time was a key part of the requirement for this project.

Project Background

Usable storage of more than 3PB was a key requirement for this project.;
RAID set of the disks for the storage must have a data protection function equivalent to 16D3P or more;

The HDDs being used are NLSAS 7200 RPM with a MTBF of 2 million hours or more. In order to reduce the failure rate in many HDD environments, a rotary vibration sensor is built in and helium is filled so as to reduce the flutter received by the internal magnetic head. The manufacturer announces that it is designed with the assumption of operation of storage device 24 hours 365 days a year.

In case of a disk failure, a hot swap replacement can be performed while the storage is operating.

The storage controller/file server has redundancy in the sense that reads and writes will continue in the unlikely event that one of the storage controller fails.

The storage controller has a total of 12 CPU core cores and each core must operate with an operation clock of 2.0 GHz or more. However, multiple socket configurations are also allowed.

The storage controller can access the file system from the client via NFS. Both the file service and NFS service can continue even if one storage controller fails.
We are also supporting snapshots and can hold more than 256 generations of snapshots per volume.

The capacity used can be limited for users, groups, and directories.

storagesystemtopology

System Configuration

2x Supermicro servers is for BeeGFS IO server (metadata and storage service)
1x Supermicro server is for client and management
4x 90 bay Supermicro enclosure is for storage
90 bay Supermicro enclosure doesn’t have HW RAID support. So we used 16+3 ZFS RAID 6 configuration for the storage target.
4x SSD/NVMe for ZFS for metadata.

Pacific Teck’s Role

Pacific Teck assisted the SI in analyzing the system architecture that was required in the customer specification. We helped determine how we can be compliant with the customer requirements which resulted in a successful tender bid. We aided the SI in the installation of the software and communication with the developers of BeeGFS as necessary during the bring up phase. The system is extremely stable today and Pacific Tcck continues to assist with post-sales support.


BeeGFS

Learn More about BeeGFS