hadoop mapreduce - Container killed on request. Exit code is 143

Job fail일 발생했다. AM이 죽었다는건가?

안되는 영어독해를 해보니 물리메모리 제한을 넘어서 실행을 했다고?

reduce시작하고 저러는 걸 보니 그쪽문제인가..

container physical memory check를 비활성화하고 해봤다.

그래도 죽는다..

연습도 할겸 며칠 후 2.7.6으로 재설치를 했다.

테스트 당시는 2.7.5였다.

뭥미 container physical memory check나 기타 한게 없다....정상이다..

jdk1.8때문이라는 내용도 있다.상세한 설명이 없지만..

 

18/04/14 17:41:07 INFO mapreduce.Job: Task Id : attempt_1523694716869_0001_r_000000_0, Status : FAILED
Container [pid=13601,containerID=container_1523694716869_0001_01_000038] is running beyond physical memory limits. Current usage: 1.0 GB of 1 GB physical memory used; 3.7 GB of 4 GB virtual memory used. Killing container.
Dump of the process-tree for container_1523694716869_0001_01_000038 :
        |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
        |- 13607 13601 13601 13601 (java) 2828 480 3910541312 263833 /jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Djava.security.egd=file:/dev/../urandom -Djava.io.tmpdir=/hadoop/data/yarn/nm-local-dir/usercache/hadoop/appcache/application_1523694716869_0001/container_1523694716869_0001_01_000038/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/hadoop/logs/userlogs/application_1523694716869_0001/container_1523694716869_0001_01_000038 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dyarn.app.mapreduce.shuffle.logger=INFO,shuffleCLA -Dyarn.app.mapreduce.shuffle.logfile=syslog.shuffle -Dyarn.app.mapreduce.shuffle.log.filesize=0 -Dyarn.app.mapreduce.shuffle.log.backups=0 org.apache.hadoop.mapred.YarnChild 192.168.100.184 40587 attempt_1523694716869_0001_r_000000_0 38
        |- 13601 13597 13601 13601 (bash) 0 0 115851264 97 /bin/bash -c /jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN  -Djava.security.egd=file:/dev/../urandom -Djava.io.tmpdir=/hadoop/data/yarn/nm-local-dir/usercache/hadoop/appcache/application_1523694716869_0001/container_1523694716869_0001_01_000038/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/hadoop/logs/userlogs/application_1523694716869_0001/container_1523694716869_0001_01_000038 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dyarn.app.mapreduce.shuffle.logger=INFO,shuffleCLA -Dyarn.app.mapreduce.shuffle.logfile=syslog.shuffle -Dyarn.app.mapreduce.shuffle.log.filesize=0 -Dyarn.app.mapreduce.shuffle.log.backups=0 org.apache.hadoop.mapred.YarnChild 192.168.100.184 40587 attempt_1523694716869_0001_r_000000_0 38 1>/hadoop/logs/userlogs/application_1523694716869_0001/container_1523694716869_0001_01_000038/stdout 2>/hadoop/logs/userlogs/application_1523694716869_0001/container_1523694716869_0001_01_000038/stderr

Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

[hadoop@big-master hadoop]$ vi yarn-site.xml
<property>
   <name>yarn.nodemanager.pmem-check-enabled</name>
   <value>false</value>
</property>
<property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
</property>