- Executing a python 3 script which is intended to execute on the Hadoop Cluster as if it were a spark-scala job with the following command:
/opt/spark/bin/spark-submit --master yarn --deploy-mode cluster --num-executors 23 --executor-cores 5 --executor-memory 35GB profiling.py
- The job is submitted but fails with the following error message:
ERROR org.apache.spark.deploy.yarn.ApplicationMaster (Logging.scala:logError(91)) - User class threw exception: java.io.IOException: Cannot run /usr/miniconda3/bin/python - execve failed (ENOENT - No such file or directory)
- SAP Cloud Platform Big Data Services (BDS) 4.3
- SAP Cloud Platform Big Data Services (BDS) 5.0
Altiscale, Portal, HDFS
Hive, Spark, Pig, Tez, Oozie, Alation, MapReduce, SparkContext, YARN , KBA , BC-NEO-BDS , HCP Big Data Service , Problem
About this pageThis is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP ONE Support launchpad (Login required).
Search for additional results
Visit SAP Support Portal's SAP Notes and KBA Search.