Comments on: The MalStone Benchmark, TeraSort and Clouds For Data Intensive Computing

By: DataRush posts 2 TB per hour on MalStone B | insideHPC.com

DataRush posts 2 TB per hour on MalStone B | insideHPC.com — Wed, 25 Aug 2010 14:16:49 +0000

[...] on MalStone B (a stylized benchmark for data intensive computing, Robert Grossman describes it here) using a 32-core Intel Xeon 7550 server. MalStone B10 has 10 billion records, which equates to just [...]

By: HIMSS 2010 - DataRush in Health IT making a difference in patient care - Pervasive DataRush

Thu, 11 Mar 2010 13:41:50 +0000

[...] cyber-security, Pervasive DataRush presented results from a benchmark study based on Malstone algorithm for web exploits. The Malstone DataRush implementation is capable of processing 1Tb of [...]

By: Open Source Cloud Computing Software at SC 09 « From Data to Decisions

Open Source Cloud Computing Software at SC 09 « From Data to Decisions — Thu, 12 Nov 2009 17:51:57 +0000

[...] measured by the MalStone Benchmark, Sector was over 2x fast as Hadoop. Sector was one of six technologies selected by SC 09 as a [...]

By: Large Data Clouds FAQ « From Data to Decisions

Large Data Clouds FAQ « From Data to Decisions — Fri, 17 Jul 2009 01:38:21 +0000

[...] How do I get started? The easiest way to get started is to download one of the applications and to work through some basic examples. The example that most people work through is word count. Another common example is the terasort example (soring 10 billion 100 byte records where the first 10 bytes is the key that is sorted and the remaining 90 bytes is the payload). A simple analytic to try is MalStone, which I have described in another post. [...]

By: Test Drive the Sector Public Cloud « From Data to Decisions

Test Drive the Sector Public Cloud « From Data to Decisions — Tue, 23 Jun 2009 17:28:05 +0000

[...] is some information on the performance of Sector/Sphere in my post on the MalStone Benchmark, a benchmark for clouds that support data intensive computing. Possibly related posts: [...]

By: rgrossman

rgrossman — Thu, 04 Jun 2009 13:58:19 +0000

Sample implementations of the benchmark code can now be found on the MalGen web site: code.google.com/p/malgen.

Both a technical report about MalStone and sample implementations are available in the “Feature Downloads” section along the right hand edge.

We view both the code implementations and the timings as preliminary and look forward to working with the community on flushing out and improving the benchmark and optimizing the particular implementations.

A more detailed technical report about MalStone is in preparation.

By: rgrossman

rgrossman — Wed, 27 May 2009 01:52:41 +0000

Doug,

Thanks for your note. Please consider the posted results preliminary. We’ll be making the code available shortly along with a technical document that provides more details about MalStone. I’ll update the post when we do. We also found the results suprising and it could well be that with more work optimising the code these numbers would change.

–Bob

By: Doug Cutting

Doug Cutting — Tue, 26 May 2009 21:51:46 +0000

Have you posted your benchmark code publicly? It doesn’t appear to be with the MalGen code.

I can’t think why “Hadoop MapReduce” is slower than “Hadoop Streams”. I assume the former is a Java implementation, which is nearly always faster than Hadoop Streaming, yet, in your case it is 5x slower. That seems strange.