HPC & Machine learning Experts

Spookfish

Spookfish

Spookfish Background

Spookfish is an aerial survey system based in Western Australia​. They were using NFS storage to provide high resolution images to customers who need up to date information on terrain they plan to utilize​. This information can be fed into GIS and CAD applications.​ The NFS filesystem, however, was a performance bottleneck and they need a faster solution.

Spookfish

 

Project Background

Spookfish installed BeeGFS in converged storage with application, metadata, and storage all combined in a single server. 40 of these converged storage servers ingest map data from cameras in airplanes​, processing large and small file sizes rapidly.

 

System Architecture​

Spookfish

Metadata server x 6​

  • Supermicro chassis with 4 x Intel Xeon X7560 and 256GB RAM​
  • Only performs MDS Services​
  • Metadata target x6 with buddy mirroring​

Converged storage server x 40​

  • DELL R730 with 2 x Intel Xeon E5-2650v4 CPU’s and 128GB of RAM​
  • Storage servers also perform processing for applications​
  • Uses Linux cgroups to avoid out-of-memory events​
  • cgroups not used for CPU usage and so far no issues of CPU shortage​
  • Storage target x 160 with buddy mirroring​

10GB/s Ethernet

 

Result

“The result [of switching to BeeGFS] is that we’re now able to process about 3 times faster with BeeGFS than with our old NFS server. We’re seeing speeds of up to 10GB/s read and 5-6GB/s write.” -Spookfish​

 

Pacific Teck’s role

Pacific Teck partnered closely with ThinkParQ developers to provide advanced tuning for this converged storage solution. While other parallel file systems struggle to maintain performance and flexibility, we were able to showcase BeeGFS’s unique ability to provide the best of both worlds. As Spookfish continues to push the boundries of storage technology with BeeGFS, Pacific Teck is the first line of support to help make the product perform even better.

 
 

BeeGFS

Learn More about BeeGFS