Map-Reduce for GuineaPig: see [GuineaPig Wiki]
Options:
--input : gpfs:sharded
--mapper : python streamNaiveBayesLearner.py --streamTrain 100
--numReduceTasks : 3
--output : gpfs:events
--reducer : python sum-events.py
Statistics:
mapper summary: 3/3 finished/started progress [###]
reducer summary: 3/3 finished/started progress [###]
shuffle summary: 3/3 finished/started progress [###]
__top level task : finished in 3.264 sec
_init mappers and shufflers : finished in 0.078 sec
_init reduce buffer queues : finished in 0.001 sec
_init reducers : finished in 0.114 sec
_join mappers : finished in 1.843 sec
_join reducer queues : finished in 0.000 sec
_join reducers : finished in 1.226 sec
mapper-from-gpfs:sharded/part00000 : finished in 1.605 sec chars: input 1990944(1.9M) output 6784645(6.5M) log 206
mapper-from-gpfs:sharded/part00001 : finished in 1.424 sec chars: input 1927812(1.8M) output 6587275(6.3M) log 206
mapper-from-gpfs:sharded/part00002 : finished in 1.586 sec chars: input 1935917(1.8M) output 6714692(6.4M) log 206
reducer-to-gpfs:events/part00000 : finished in 1.160 sec chars: input 6651662(6.3M) output 1299450(1.2M) log 0
reducer-to-gpfs:events/part00001 : finished in 1.263 sec chars: input 6927496(6.6M) output 1316031(1.3M) log 0
reducer-to-gpfs:events/part00002 : finished in 1.151 sec chars: input 6507454(6.2M) output 1311985(1.3M) log 0
shuffle from gpfs:sharded/part00000 : finished in 0.273 sec
shuffle from gpfs:sharded/part00001 : finished in 0.146 sec
shuffle from gpfs:sharded/part00002 : finished in 0.160 sec
Subprocess Logs:
chars: 206(0.0M) _logs/mapper-from-gpfs:sharded/part00000 [cat|download|head|tail]
chars: 206(0.0M) _logs/mapper-from-gpfs:sharded/part00001 [cat|download|head|tail]
chars: 206(0.0M) _logs/mapper-from-gpfs:sharded/part00002 [cat|download|head|tail]
chars: 0(0.0M) _logs/reducer-to-gpfs:events/part00000 [cat|download|head|tail]
chars: 0(0.0M) _logs/reducer-to-gpfs:events/part00001 [cat|download|head|tail]
chars: 0(0.0M) _logs/reducer-to-gpfs:events/part00002 [cat|download|head|tail]
[List directories | Task History | See Report on last task] File system size: 16663645(15.9M)