−
== Use Apache Spark on Grid5000 ==
+
== How to use Apache Spark on Grid5000 ==
          
     oarsub -I -t allow_classic_ssh -l nodes=6,walltime=2
 
     oarsub -I -t allow_classic_ssh -l nodes=6,walltime=2
 +
 +
'''Known issue'''
 +
 +
It is possible that trying to run the hg5k and spark_g5k scripts while in interactive reservation mode raises a "DistributionNotFound" python error.
 +
In this case, you should try adding the keywork "deploy" to your reservation command.
 +
for example:
 +
    oarsub -I -t allow_classic_ssh -l nodes=2,walltime=0:30 deploy
    
'''Cluster initialization'''
 
'''Cluster initialization'''
 
'''Get result file named res'''
 
'''Get result file named res'''
   Exception encountered, of type "Error"
[8cf9f41e] /bigdata/index.php?diff=cur&oldid=31&title=How_to_use_Spark_on_Grid5000 Error from line 434 of /home/bigdata/includes/diff/DairikiDiff.php: Call to undefined function each()
Backtrace:
#0 /home/bigdata/includes/diff/DairikiDiff.php(544): DiffEngine->diag()
#1 /home/bigdata/includes/diff/DairikiDiff.php(344): DiffEngine->compareSeq()
#2 /home/bigdata/includes/diff/DairikiDiff.php(227): DiffEngine->diffLocal()
#3 /home/bigdata/includes/diff/DairikiDiff.php(721): DiffEngine->diff()
#4 /home/bigdata/includes/diff/DairikiDiff.php(859): Diff->__construct()
#5 /home/bigdata/includes/diff/DairikiDiff.php(980): MappedDiff->__construct()
#6 /home/bigdata/includes/diff/TableDiffFormatter.php(194): WordLevelDiff->__construct()
#7 /home/bigdata/includes/diff/DiffFormatter.php(140): TableDiffFormatter->changed()
#8 /home/bigdata/includes/diff/DiffFormatter.php(82): DiffFormatter->block()
#9 /home/bigdata/includes/diff/DifferenceEngine.php(881): DiffFormatter->format()
#10 /home/bigdata/includes/diff/DifferenceEngine.php(797): DifferenceEngine->generateTextDiffBody()
#11 /home/bigdata/includes/diff/DifferenceEngine.php(728): DifferenceEngine->generateContentDiffBody()
#12 /home/bigdata/includes/diff/DifferenceEngine.php(662): DifferenceEngine->getDiffBody()
#13 /home/bigdata/includes/diff/DifferenceEngine.php(632): DifferenceEngine->getDiff()
#14 /home/bigdata/includes/diff/DifferenceEngine.php(453): DifferenceEngine->showDiff()
#15 /home/bigdata/includes/page/Article.php(797): DifferenceEngine->showDiffPage()
#16 /home/bigdata/includes/page/Article.php(508): Article->showDiffPage()
#17 /home/bigdata/includes/actions/ViewAction.php(44): Article->view()
#18 /home/bigdata/includes/MediaWiki.php(490): ViewAction->show()
#19 /home/bigdata/includes/MediaWiki.php(287): MediaWiki->performAction()
#20 /home/bigdata/includes/MediaWiki.php(714): MediaWiki->performRequest()
#21 /home/bigdata/includes/MediaWiki.php(508): MediaWiki->main()
#22 /home/bigdata/index.php(41): MediaWiki->run()
#23 {main}