Posted by: Wildan Maulana | January 22, 2009

Kick Starting Hadoop

Cheat sheet starting & stopping multinode Hadoop cluster :

Starting :

On master node :

./bin/start-dfs.sh
./bin/start-mapred.sh

Stopping (the opposite of starting process):
On master node :

./bin/stop-mapred.sh
./bin/stop-dfs.sh

When running wordcount example on my little sweet multi node cluster, i have found errors msg on the task tracker log :

————cut———————

hadoop@tobeThink:/opt/hadoop$ tail -f  logs/hadoop-hadoop-tasktracker-tobeThink.log
………………………………………….

2009-01-28 14:19:02,901 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200901281323_0001_r_000000_0 0.15277779% reduce > copy (11 of 24 at 0.00 MB/s) >
2009-01-28 14:19:03,658 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200901281323_0001/attempt_200901281323_0001_r_000000_0/output/file.out in any of the configured local directories
2009-01-28 14:19:03,658 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200901281323_0001/attempt_200901281323_0001_r_000002_0/output/file.out in any of the configured local directories
…………………………………..

————-cut———————

and on the console :

———-cut —————-

hadoop@tobeThink:/opt/hadoop$ ./bin/hadoop jar hadoop-0.19.0-examples.jar wordcount gutenberg gutenberg-output
09/01/28 14:11:02 INFO mapred.FileInputFormat: Total input paths to process : 12
09/01/28 14:11:03 INFO mapred.JobClient: Running job: job_200901281323_0001
09/01/28 14:11:04 INFO mapred.JobClient:  map 0% reduce 0%
……………………………………………………………………………………………

09/01/28 14:11:10 INFO mapred.JobClient:  map 4% reduce 0%
09/01/28 14:12:11 INFO mapred.JobClient:  map 100% reduce 16%
09/01/28 14:14:37 INFO mapred.JobClient: Task Id : attempt_200901281323_0001_m_000001_0, Status : FAILED
Too many fetch-failures
09/01/28 14:14:41 INFO mapred.JobClient: Task Id : attempt_200901281323_0001_m_000000_0, Status : FAILED
Too many fetch-failures
09/01/28 14:14:41 WARN mapred.JobClient: Error reading task outputhadoop2
09/01/28 14:14:41 WARN mapred.JobClient: Error reading task outputhadoop2
09/01/28 14:14:53 INFO mapred.JobClient:  map 100% reduce 17%
09/01/28 14:14:55 INFO mapred.JobClient:  map 100% reduce 18%
09/01/28 14:20:04 INFO mapred.JobClient: Task Id : attempt_200901281323_0001_m_000002_0, Status : FAILED
Too many fetch-failures
09/01/28 14:20:04 WARN mapred.JobClient: Error reading task outputhadoop2
09/01/28 14:20:04 WARN mapred.JobClient: Error reading task outputhadoop2
09/01/28 14:20:05 INFO mapred.JobClient: Task Id : attempt_200901281323_0001_m_000004_0, Status : FAILED
Too many fetch-failures
09/01/28 14:20:22 INFO mapred.JobClient:  map 100% reduce 19%
09/01/28 14:25:27 INFO mapred.JobClient: Task Id : attempt_200901281323_0001_m_000003_0, Status : FAILED
Too many fetch-failures
09/01/28 14:25:27 WARN mapred.JobClient: Error reading task outputhadoop2
09/01/28 14:25:27 WARN mapred.JobClient: Error reading task outputhadoop2
09/01/28 14:25:32 INFO mapred.JobClient: Task Id : attempt_200901281323_0001_m_000006_0, Status : FAILED
Too many fetch-failures
09/01/28 14:25:40 INFO mapred.JobClient:  map 100% reduce 20%
09/01/28 14:30:55 INFO mapred.JobClient: Task Id : attempt_200901281323_0001_m_000005_0, Status : FAILED
Too many fetch-failures
09/01/28 14:30:55 WARN mapred.JobClient: Error reading task outputhadoop2
09/01/28 14:30:55 WARN mapred.JobClient: Error reading task outputhadoop2
09/01/28 14:30:56 INFO mapred.JobClient: Task Id : attempt_200901281323_0001_m_000008_0, Status : FAILED
Too many fetch-failures
09/01/28 14:31:08 INFO mapred.JobClient:  map 100% reduce 21%
09/01/28 14:31:13 INFO mapred.JobClient:  map 100% reduce 22%

———–cut—————–

I’m using Hadoop Release 0.19.0 – 2008-11-18, after googling, i saw Amareshwari Sriramadasu on JIRA , already fix the problem (https://issues.apache.org/jira/browse/HADOOP-4963) ..

no i’m trying to rebuild hadoop from source ..(svn checkout) ..

10 minutes later …

Ok done  building … :

——–cut————–

wildan@tobeThink:/opt/hadoop-trunk$  ./bin/hadoop version
Hadoop 0.21.0-dev
Subversion  -r
Compiled by wildan on Wed Jan 28 15:19:53 WIT 2009

hadoop@hadoop2:/opt/hadoop$ ./bin/hadoop version
Hadoop 0.21.0-dev
Subversion  -r
Compiled by wildan on Wed Jan 28 15:19:53 WIT 2009

———–cut————-

re-run the wordcount examples one more time … :

hadoop@tobeThink:/opt/hadoop$ ./bin/hadoop dfs -copyFromLocal /home/hadoop/gutenberg/ gutenberg
hadoop@tobeThink:/opt/hadoop$ ./bin/hadoop jar hadoop-0.21.0-dev-examples.jar wordcount gutenberg gutenberg-output

Ups …, there was an error in 0.21.0 examples :

——————cut——————————-
09/01/28 16:16:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
09/01/28 16:16:40 INFO input.FileInputFormat: Total input paths to process : 12
09/01/28 16:16:40 INFO mapred.FileInputFormat: Total input paths to process : 12
09/01/28 16:16:40 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:16:41 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:16:41 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:16:41 WARN mapred.LocalJobRunner: job_local_0001
java.lang.ClassCastException: org.apache.hadoop.mapred.FileSplit cannot be cast to org.apache.hadoop.mapreduce.lib.input.FileSplit
at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:55)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:402)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:500)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:293)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:140)

——————cut——————————-

trying the 0.19 examples … :

$ ./bin/hadoop jar hadoop-0.19.0-examples.jar wordcount     gutenberg   gutenberg-output
……

Done .., without any error …, but the console output is more verbose comparing to 0.19 output .., :

———————–cut——————————————-

hadoop@tobeThink:/opt/hadoop$ ./bin/hadoop jar hadoop-0.19.0-examples.jar wordcount gutenberg gutenberg-output
09/01/28 16:23:06 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
09/01/28 16:23:06 INFO mapred.FileInputFormat: Total input paths to process : 12
09/01/28 16:23:07 INFO mapred.JobClient: Running job: job_local_0001
09/01/28 16:23:07 INFO mapred.FileInputFormat: Total input paths to process : 12
09/01/28 16:23:07 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:07 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:07 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:07 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:08 INFO mapred.JobClient:  map 0% reduce 0%
09/01/28 16:23:08 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:09 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:09 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting
09/01/28 16:23:09 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/10940-8.txt:0+987864
09/01/28 16:23:09 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000000_0’ done.
09/01/28 16:23:09 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:09 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:09 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:09 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:09 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:10 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:10 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000001_0 is done. And is in the process of commiting
09/01/28 16:23:10 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/12254-8.txt:0+502062
09/01/28 16:23:10 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000001_0’ done.
09/01/28 16:23:10 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:10 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:10 INFO mapred.JobClient:  map 100% reduce 0%
09/01/28 16:23:10 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:10 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:10 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:10 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:10 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000002_0 is done. And is in the process of commiting
09/01/28 16:23:10 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/132.txt:0+343694
09/01/28 16:23:10 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000002_0’ done.
09/01/28 16:23:10 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:10 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:10 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:10 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:11 INFO mapred.MapTask: Spilling map output: record full = true
09/01/28 16:23:11 INFO mapred.MapTask: bufstart = 0; bufend = 2617729; bufvoid = 99614720
09/01/28 16:23:11 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
09/01/28 16:23:11 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:11 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:11 INFO mapred.MapTask: Finished spill 1
09/01/28 16:23:11 INFO mapred.Merger: Merging 2 sorted segments
09/01/28 16:23:11 INFO mapred.Merger: Down to the last merge-pass, with 2 segments left of total size: 807802 bytes
09/01/28 16:23:12 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000003_0 is done. And is in the process of commiting
09/01/28 16:23:12 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/19699-8.txt:0+1944509
09/01/28 16:23:12 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000003_0’ done.
09/01/28 16:23:12 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:12 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:12 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:12 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:12 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:13 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:13 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000004_0 is done. And is in the process of commiting
09/01/28 16:23:13 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/20417-8.txt:0+674425
09/01/28 16:23:13 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000004_0’ done.
09/01/28 16:23:13 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:13 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:13 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:13 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:13 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:13 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:13 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000005_0 is done. And is in the process of commiting
09/01/28 16:23:13 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/20417.txt:0+674762
09/01/28 16:23:13 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000005_0’ done.
09/01/28 16:23:13 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:13 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:13 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:13 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:14 INFO mapred.MapTask: Spilling map output: record full = true
09/01/28 16:23:14 INFO mapred.MapTask: bufstart = 0; bufend = 2551697; bufvoid = 99614720
09/01/28 16:23:14 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
09/01/28 16:23:14 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:14 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:14 INFO mapred.MapTask: Finished spill 1
09/01/28 16:23:14 INFO mapred.Merger: Merging 2 sorted segments
09/01/28 16:23:14 INFO mapred.Merger: Down to the last merge-pass, with 2 segments left of total size: 537246 bytes
09/01/28 16:23:14 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000006_0 is done. And is in the process of commiting
09/01/28 16:23:14 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/22094-0.txt:0+1610961
09/01/28 16:23:14 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000006_0’ done.
09/01/28 16:23:15 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:15 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:15 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:15 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:15 INFO mapred.MapTask: Spilling map output: record full = true
09/01/28 16:23:15 INFO mapred.MapTask: bufstart = 0; bufend = 2546156; bufvoid = 99614720
09/01/28 16:23:15 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
09/01/28 16:23:15 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:16 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:16 INFO mapred.MapTask: Finished spill 1
09/01/28 16:23:16 INFO mapred.Merger: Merging 2 sorted segments
09/01/28 16:23:16 INFO mapred.Merger: Down to the last merge-pass, with 2 segments left of total size: 740543 bytes
09/01/28 16:23:16 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000007_0 is done. And is in the process of commiting
09/01/28 16:23:16 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/4300-8.txt:0+1573048
09/01/28 16:23:16 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000007_0’ done.
09/01/28 16:23:16 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:16 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:16 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:16 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:16 INFO mapred.MapTask: Spilling map output: record full = true
09/01/28 16:23:16 INFO mapred.MapTask: bufstart = 0; bufend = 2545957; bufvoid = 99614720
09/01/28 16:23:16 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
09/01/28 16:23:16 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:17 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:17 INFO mapred.MapTask: Finished spill 1
09/01/28 16:23:17 INFO mapred.Merger: Merging 2 sorted segments
09/01/28 16:23:17 INFO mapred.Merger: Down to the last merge-pass, with 2 segments left of total size: 740234 bytes
09/01/28 16:23:17 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000008_0 is done. And is in the process of commiting
09/01/28 16:23:17 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/4300.txt:0+1573044
09/01/28 16:23:17 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000008_0’ done.
09/01/28 16:23:17 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:17 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:17 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:17 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:18 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:18 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:18 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000009_0 is done. And is in the process of commiting
09/01/28 16:23:18 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/8ldvc10.txt:0+1427769
09/01/28 16:23:18 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000009_0’ done.
09/01/28 16:23:18 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:18 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:18 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:18 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:19 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:19 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:19 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000010_0 is done. And is in the process of commiting
09/01/28 16:23:19 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/972.txt:0+393994
09/01/28 16:23:19 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000010_0’ done.
09/01/28 16:23:19 INFO mapred.MapTask: numReduceTasks: 1
09/01/28 16:23:19 INFO mapred.MapTask: io.sort.mb = 100
09/01/28 16:23:19 INFO mapred.MapTask: data buffer = 79691776/99614720
09/01/28 16:23:19 INFO mapred.MapTask: record buffer = 262144/327680
09/01/28 16:23:19 INFO mapred.MapTask: Starting flush of map output
09/01/28 16:23:19 INFO mapred.MapTask: Finished spill 0
09/01/28 16:23:19 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000011_0 is done. And is in the process of commiting
09/01/28 16:23:19 INFO mapred.LocalJobRunner: hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg/advsh12.txt:0+590093
09/01/28 16:23:19 INFO mapred.TaskRunner: Task ‘attempt_local_0001_m_000011_0’ done.
09/01/28 16:23:20 INFO mapred.Merger: Merging 12 sorted segments
09/01/28 16:23:20 INFO mapred.Merger: Merging 3 intermediate segments out of a total of 12
09/01/28 16:23:20 INFO mapred.Merger: Down to the last merge-pass, with 10 segments left of total size: 5046270 bytes
09/01/28 16:23:20 INFO mapred.TaskRunner: Task:attempt_local_0001_r_000000_0 is done. And is in the process of commiting
09/01/28 16:23:20 INFO mapred.LocalJobRunner:
09/01/28 16:23:20 INFO mapred.TaskRunner: Task attempt_local_0001_r_000000_0 is allowed to commit now
09/01/28 16:23:20 INFO mapred.FileOutputCommitter: Saved output of task ‘attempt_local_0001_r_000000_0’ to hdfs://tobethink.pappiptek.lipi.go.id:54310/user/hadoop/gutenberg-output
09/01/28 16:23:20 INFO mapred.LocalJobRunner: reduce > reduce
09/01/28 16:23:20 INFO mapred.TaskRunner: Task ‘attempt_local_0001_r_000000_0’ done.
09/01/28 16:23:21 INFO mapred.JobClient: Job complete: job_local_0001
09/01/28 16:23:21 INFO mapred.JobClient: Counters: 15
09/01/28 16:23:21 INFO mapred.JobClient:   FileSystemCounters
09/01/28 16:23:21 INFO mapred.JobClient:     FILE_BYTES_READ=27372898
09/01/28 16:23:21 INFO mapred.JobClient:     HDFS_BYTES_READ=90436465
09/01/28 16:23:21 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=57594146
09/01/28 16:23:21 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=3770703
09/01/28 16:23:21 INFO mapred.JobClient:   Map-Reduce Framework
09/01/28 16:23:21 INFO mapred.JobClient:     Reduce input groups=157716
09/01/28 16:23:21 INFO mapred.JobClient:     Combine output records=345496
09/01/28 16:23:21 INFO mapred.JobClient:     Map input records=234498
09/01/28 16:23:21 INFO mapred.JobClient:     Reduce shuffle bytes=0
09/01/28 16:23:21 INFO mapred.JobClient:     Reduce output records=157716
09/01/28 16:23:21 INFO mapred.JobClient:     Spilled Records=925570
09/01/28 16:23:21 INFO mapred.JobClient:     Map output bytes=20212650
09/01/28 16:23:21 INFO mapred.JobClient:     Map input bytes=12296225
09/01/28 16:23:21 INFO mapred.JobClient:     Combine input records=2067113
09/01/28 16:23:21 INFO mapred.JobClient:     Map output records=2067113
09/01/28 16:23:21 INFO mapred.JobClient:     Reduce input records=345496
————————————————————cut—————————————————-

Kalau mendapatkan error : Connection refused pada slave, gunakan FQDN untuk nama host, lihat [3]

Referensi :

[1]Running Hadoop On Ubuntu Linux (Single-Node Cluster),  http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Single-Node_Cluster)

[2]Running Hadoop On Ubuntu Linux (Multi-Node Cluster),  http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Multi-Node_Cluster)

[3] http://wiki.apache.org/hadoop/HowToSetupYourDevelopmentEnvironment


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: