Hadoop 測試

Hadoop 測試

MRUnit單元測試Mapper和Reducer類在內(nèi)存上獨(dú)立運(yùn)行, PipelineMapReduceDriver單線程運(yùn)行.

LocalJobRunner單線程運(yùn)行, 且僅有一個(gè) Reducer能夠啟動conf.set("mapred.job.tracker", "local"); conf.set("fs.default.name", "file:////"); FileSystem fs = FileSystem.getLocal(conf);

MiniMRCluster, MiniYarnCluster, MiniDFSCluster 多線程