Storage slowdowns due to failing disks

CLUSTERS INVOLVED: emory/tardis, ase1

Hey folks,

We’ve gone ahead and replaced some disks in your storage as the type of failures they are generating right now cause dramatic slowdowns in I/O performance for the disk arrays.

As a result of the replacements, the array will remain slow for a period of ~5 or so hours as the arrays rebuild themselves to have the appropriate redundancy.

We’ll be keeping an eye on this problem as we have recently noticed a spike in the number of these events as of late.