scala - SparkPi running slow with more than 1 slice -


Tried to run sparky examples on a new spark and a standalone 12 core three machine cluster. I have failed to understand that running this example with a slice gives better performance than using 12 slices. One such case was when I was using parallel function. Along with adding each slice, the time is almost nearly linear. Please tell me if I am doing something wrong code snippet is given below: 

  val spark = new spark contact ("spark: // telecom: 7077", "sparky", system .getenv ("SPARK_HOME"), list ("target /scala-2.10/sparkpii_2.10-1.0.jar")) Val Slice = 1 Well N = 10000000 * Slice Val count = Spark. Parallel (1 to n, slice) .Map {i = & gt; Val x = random * 2 - 1 val y = random * 2 - 1 if (x * x + y * y & lt; 1) 1 and 0} .readuce (_ + _) println ("Pi roughly +" 4.0 * count / n) spark.stop ()   
 UPDATE: The problem was random, because it was a synchronized method, it could not scale to many cores.   
 
 The random function used in sparky examples is a synchronized method and can not do multiple core scale. This is an easy enough example to deploy on your cluster, but do not use it to check spark performance and scalability.

Comments

Post a Comment

Popular posts from this blog

How can I add a seperate action listener to all 14 of my buttons at once that can determine player turn?(java) -

I have a big code, for a game I'm calling Tommy's Kumamak. I know that this is not beautiful, but I am very naive and for some reasons I know from using arrays, anyway, my problem is that I can not add a variable outside the listener and within the action listener I can modify the variable (pTurn ++) from How can I go about adding a count to that variable from inside the action listener? 1 error was found: file: c: \ user \ public \ togiz_kumalak.java [line: 130] error: local variables pTurn is accessed from within internal class; Need to be declared final int pTurn = 0; Button1.addActionListener {new zero-action action (action event e) {pTurn ++; if (pTurn% 2 == 0) {display.append ("\ n Player 2 cup 1");} If ( PTurn% 2! = 0) {display.append ("\ nPlayer 1Cup 1")}}); In that case, move the contents of the functionalities into orbit and then call the action For example, display the method from: Private Zero Button Press {pTurn ++; If (pTurn% 2...
Read more

php - Sorting an multidimension array using usort fails -

I tried to use custom-made YouTube fun cats when I ran 5 cell arrays in 30 seconds and then a Throwing a serious error. I tried to know that I have been made an infinite loop, but he did not install it. iPhone 5S 32GB [Price] => 0 [Description] => A new apple product [Image_URL] = & Gt; [Business_ID] = & gt; 6 [keyword] = & gt; ["iPhone", "iPhone 5s", "Apple", "32 GB"]) [1] = & gt; array ([name] = & Gt; iPhone 4S 32g [Price] => 300 [Description] => Apple iPhone 4S = [Image_URL] => [Business_ID] => 6 [Keyword] = & gt; [ "IPhone 4S", "" iPhone "," Apple "," 32 GB "])) search: $ searchTerm =" iphone 5s "; Code: if (! SearchTerm below!) {$ SearchTermWords = Explosion (" ", $ searchTerm); $$ $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ ['Keyword']; $ bKeywords = $...
Read more

Simple file compression in C -

I am new to programming with C, and I need elsewhere that I can read on file compression in C I am I only need a simple accredited method, for example it will help but mostly I can read it to understand. Probably a good starting point RLE is very simple but it is not so insignificant to apply see it :
Read more