Gear6
Gear6 is a Silicon Valley startup that just came up with an incredible product — a machine “CACHEfx” that uses standard computer RAM as storage cache. They come in blocks of 250 or 500 GB, and the operating system will virtualize it into a single pool of upto 5 TB.
The box plugs into a standard Gigabit Ethernet link and caches frequently used files. CACHEfx has a latency of less than 500 microseconds. They also have a handy tool to assess the speedup that you’ll gain by using their product.
If my lab had some more money and the need, we could get one of these. Currently, runs of fluidmatch can generate outputs of upto half a gigabyte. Multiply this by a hundred, and simple things like moving or copying files needs to be scheduled for overnight.