Notice
Recent Posts
Recent Comments
Link
일 | 월 | 화 | 수 | 목 | 금 | 토 |
---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | 6 | 7 |
8 | 9 | 10 | 11 | 12 | 13 | 14 |
15 | 16 | 17 | 18 | 19 | 20 | 21 |
22 | 23 | 24 | 25 | 26 | 27 | 28 |
29 | 30 |
Tags
- Kotlin
- Java
- GIT
- SPC
- JavaScript
- Spring
- hadoop
- Express
- Android
- Eclipse
- tomcat
- SSL
- 공정능력
- table
- 보조정렬
- NPM
- SQL
- vaadin
- Sqoop
- mybatis
- plugin
- R
- Python
- MSSQL
- xPlatform
- IntelliJ
- mapreduce
- es6
- window
- react
Archives
- Today
- Total
DBILITY
hadoop mapreduce - Container killed on request. Exit code is 143 본문
bigdata/hadoop
hadoop mapreduce - Container killed on request. Exit code is 143
DBILITY 2018. 4. 14. 17:48반응형
Job fail일 발생했다. AM이 죽었다는건가?
안되는 영어독해를 해보니 물리메모리 제한을 넘어서 실행을 했다고?
reduce시작하고 저러는 걸 보니 그쪽문제인가..
container physical memory check를 비활성화하고 해봤다.
그래도 죽는다..
연습도 할겸 며칠 후 2.7.6으로 재설치를 했다.
테스트 당시는 2.7.5였다.
뭥미 container physical memory check나 기타 한게 없다....정상이다..
jdk1.8때문이라는 내용도 있다.상세한 설명이 없지만..
18/04/14 17:41:07 INFO mapreduce.Job: Task Id : attempt_1523694716869_0001_r_000000_0, Status : FAILED
Container [pid=13601,containerID=container_1523694716869_0001_01_000038] is running beyond physical memory limits. Current usage: 1.0 GB of 1 GB physical memory used; 3.7 GB of 4 GB virtual memory used. Killing container.
Dump of the process-tree for container_1523694716869_0001_01_000038 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 13607 13601 13601 13601 (java) 2828 480 3910541312 263833 /jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Djava.security.egd=file:/dev/../urandom -Djava.io.tmpdir=/hadoop/data/yarn/nm-local-dir/usercache/hadoop/appcache/application_1523694716869_0001/container_1523694716869_0001_01_000038/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/hadoop/logs/userlogs/application_1523694716869_0001/container_1523694716869_0001_01_000038 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dyarn.app.mapreduce.shuffle.logger=INFO,shuffleCLA -Dyarn.app.mapreduce.shuffle.logfile=syslog.shuffle -Dyarn.app.mapreduce.shuffle.log.filesize=0 -Dyarn.app.mapreduce.shuffle.log.backups=0 org.apache.hadoop.mapred.YarnChild 192.168.100.184 40587 attempt_1523694716869_0001_r_000000_0 38
|- 13601 13597 13601 13601 (bash) 0 0 115851264 97 /bin/bash -c /jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Djava.security.egd=file:/dev/../urandom -Djava.io.tmpdir=/hadoop/data/yarn/nm-local-dir/usercache/hadoop/appcache/application_1523694716869_0001/container_1523694716869_0001_01_000038/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/hadoop/logs/userlogs/application_1523694716869_0001/container_1523694716869_0001_01_000038 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dyarn.app.mapreduce.shuffle.logger=INFO,shuffleCLA -Dyarn.app.mapreduce.shuffle.logfile=syslog.shuffle -Dyarn.app.mapreduce.shuffle.log.filesize=0 -Dyarn.app.mapreduce.shuffle.log.backups=0 org.apache.hadoop.mapred.YarnChild 192.168.100.184 40587 attempt_1523694716869_0001_r_000000_0 38 1>/hadoop/logs/userlogs/application_1523694716869_0001/container_1523694716869_0001_01_000038/stdout 2>/hadoop/logs/userlogs/application_1523694716869_0001/container_1523694716869_0001_01_000038/stderr
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
[hadoop@big-master hadoop]$ vi yarn-site.xml
<property>
<name>yarn.nodemanager.pmem-check-enabled</name>
<value>false</value>
</property>
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>
반응형
'bigdata > hadoop' 카테고리의 다른 글
apache hadoop HA 설치 (0) | 2018.04.17 |
---|---|
hadoop 2.7.5 compile (0) | 2018.04.11 |
hadoop job control (0) | 2017.04.16 |
hadoop 2.6.4 window 10 pseudo distribution mode 설치 (0) | 2017.04.01 |
hadoop 2.x winutils (0) | 2017.03.27 |
Comments