Commands to check Spark YARN Logs, Kill Spark appliation

Views: 140   

1.) To Check spark YARN logs :-
Below Command print will print out the contents of all log files from all containers of the given application.

[email protected]:~# yarn logs -applicationId <app ID>

is application ID provided by YARN to Spark Job, We can find out the by looking into spark job logs or in YARN console. By Default YARN console runs on 8088 port number.

2.) To Kill Spark Job running in YARN
To kill through command prompt

[email protected]:~# yarn application -kill <app ID>

(or)
i) opening Spark application UI.
ii) gotojobs tab.
Iii) find job among running jobs.
iv) click on kill link and confirm.

3.) To Kill Spark Job running in standalone

[email protected]:~# spark-submit --kill [submission ID] --master [spark://...]
[email protected]:~# spark-submit --master spark://xxxx --kill application_xxx
On By