I've just read an article about a breakthrough on matrix multiplication; an algorithm that is O(n^2.373). But I guess matrix multiplication is something that can be parallelized. So, if we ever start producing thousandth-cores processors, will this become irrelevant? How would things change?

Parallel execution doesn't change the basics of the complexity for a particular algorithm -- at best, you're just taking the time for some given size, and dividing by the number of cores. This may reduce time for a given size by a constant factor, but has no effect on the algorithm's complexity.

At the same time, parallel execution does sometimes change *which* algorithm(s) you want to use for particular tasks. Some algorithms that work well in serial code just don't split up into parallel tasks very well. Others that have higher *complexity* might be faster for practical-sized problems because they run better in parallel.

For an extremely large number of cores, the complexity of the calculation itself may become secondary to simply getting the necessary data to/from all the cores to do the calculation. most computations of big-O don't take these effects into account for a serial calculation, but it can become quite important for parallel calculations, especially for some models of parallel machines that don't give uniform access to all nodes.

By Amdahl's law, **for the same size of problem**, parallelization will come to a point of diminishing return with the increase in the number of cores (theoretically). In reality, from a certain degree of parallelization, the overhead of parallelization will actually decrease the performance of the program.

However, by Gustafson's law, the increase of number of cores actually helps as the size of the problem increases. That is the motivation behind cluster computing. As we have more computing power, we can tackle problem at a larger scale or better precision with the help of parallelization.

Algorithms that we learn as is may or may not be paralellizable. Sometimes, a separate algorithm must be used to efficiently execute the same task in parallel. Either way, the Big-O notation must be re-analyze for the parallel case to take into consideration the effect of parallelization on the time complexity of the algorithm.

If quantum computing comes to something practical some day, then yes, complexity of algorithms will change.

In the meantime, parallelizing an algorithm, with a fixed number of processors, justs divides its runtime proportional (and that, in the best case, when there are no dependencies between the tasks performed at every processor). That means, dividing the runtime by a constant, and so the complexity remains the same.

Similar Questions

I want to ask you about the notation in probability. I know that P(A | B) = the conditional probability that event A occurs given that event B has occurred already But I cannot find what A,B or in

I'm studying a degree in computer science and at class we're using big-theta notation much more often than big-O notation. Although while reading articles about algorithms and its running times, I har

Is it possible to retrieve all the primary keys affected by an update query? Ex: Table: +-------------+---------+--------+ |idx (PK, int)|FirstName|LastName| +-------------+---------+--------+ |1 |Jan

In my library I need to support devices of compute capability 2.0 and higher. For CC 3.5+ devices I’ve implemented optimized kernels which utilize Dynamic Parallelism. It seems that nvcc compiler does

First, here is my code for the O(n): import java.util.*; public class BigO{ public static void main(String[] args) { Scanner in = new Scanner(System.in); System.out.print(Enter a number: ); String u

I'm trying to use the parfor loop in the matlab parallelism package. I'm having a similar problem to this guy : MATLAB parfor slicing issue? . The output matrix doesn't seem to be recognized as a slic

I am trying to understand the difference between .Notation and [] notation. In my problem below when I use if (object[key] === true) I get the correct answer. When I use if (object.key === true) it do

Snape’s “Unfriendly Algorithms for Wizards” textbook claims the running time of merge sort is O(n^4). Is this claim correct? Solution: Yes. This claim is technically correct, because O(n^4) only gives

Im trying to understand downscaling. I can see how interpolation algorithms such as bicubic and nearest neighbour can be used when when upscaling, to fill in the blanks between the old, known points

From the PHP documentation: only four types of code are affected by namespaces: classes, interfaces, functions and constants. But, it seems to me that TRAITS are also affected: namespace FOO; trait

I have studied in the past the classical DP problems and algorithms (coins, longest increasing subsequence, longest common subsequence, etc). I know that these algorithms have practical applications (

I have seen research papers about using genetic algorithms and neural networks(neurocrypto) for encryption purposes. Can't we use ant colony optimization algorithms for encryption? Is there any recent

I am doing a research in my class on algorithms complexities , I need to know if there is any other complexities of algorithms , what I know and studied is 2 types 1- is the in the BIG O complexity th

This question already has an answer here: Plain English explanation of Big O 21 answers How is Big Oh calculated for sorting algorithms? I have written a program to sort a deck of cards ( n = 5

What's the difference between using oracle's plus notation (+) over the ansi standard join notation? Is there a difference in performance? Is the plus notation deprecated?

I'm studying multicore parallelism in F#. I have to admit that immutability really helps to write correct parallel implementation. However, it's hard to achieve good speedup and good scalability when

Coq - IP Notation I want to create a notation for ip addresses. The following is my notation definition that works fine: Inductive IP := ip : nat -> nat -> nat -> nat -> IP. Notation a .

I am using zend framework and I see save() function used in the code, would like to know if there is a way to get the rows affected after this to see number of rows affected .. I checked this document

What areas of math are prerequisite for learning algorithms?

In their arXiv paper, the original authors of Julia mention the following: 2.14 Parallelism. Parallel execution is provided by a message-based multi-processing system implemented in Julia in the sta

I'm trying to understand the concept of Algorithms and basically how they improve performance of a computer program. So assume, I have to write a program which generates a list of numbers by, Starts

How can I convert this query into dot notation var x = from Prods n in Cat.Prod.GetAllProds() orderby n.Name select new { Name = n.Name, Cost = n.Cost };

I'm faced with a 3 dimensional bin packing problem and am currently conducting some preliminary research as to which algorithms/heuristics are currently yielding the best results. Since the problem is

Do you know any .NET library for text algorithms?? Especially I'm interested in strings match, and full-text-search algorithms like Bitap algorithm Levenshtein distance Damerau–Levenshtein distance

I would like to use parallel execution on LET_TESTATE_LETTURE without forcing the full table scan, I want to use force the parallelism on index. How can I solve? alter session enable parallel dml; CRE

Good morning, Does anyone know about efficient algorithms for partial string matching? For example, given the two strings woods and woodes, the algorithm could/should possibly return wood+s, or

Possible Duplicate: What is the purpose of wrapping whole Javascript files in anonymous functions like “(function(){ … })()”? Hello all, I have seen several JavaScript files using this notation: Sta

Velocity has two forms of references: $foo (the default notation) and $!foo (the quiet notation). If 'foo' can't be resolved in the context, in the first case you'll just get $foo rendered in the ou

I would like to create a task that executes a member function of a struct or class: import std.parallelism, std.stdio; struct S { void foo() { writeln(S.foo()); } } void main() { S s; auto pool = ne

I like STL algorithms, and prefer use algorithms rather than usual loops. Almost all STL algorithms are usually used as: std::algorithm_name( container.begin(), container.end(), ..... ) container.be

I already managed to create function which checks if user is using a trial/demo or full/purchased app on Windows Phone 7. Is it possible somehow to know if user has reviewed or rated a current app?

I need som help understanding how this works! how do I go about calculating the complexity of 'Computing the first half of an array of n items' or 'displaying the third element in a linked list' ? I n

How do I get the number of rows affected after a SaveChanges()?

This article talks about the optimization of code and discusses Instruction level parallelism. They give an example of GPU vector math where the float4 vector math can be performed on the vector rathe

I recently bumped in the problem illustrated at Uncorrectable ECC error. Shortly speaking, from time to time I receive an Uncorrectable ECC error and my dynamic parallelism code generates uncorrect re

I am trying to determine why the bigO of this algorithm is m^2*n, and why the innermost loop is executing in m^2*n steps. int m=10, n=15; int inLoop = 0, midLoop = 0, outLoop = 0; for(int i=1;i<=

I am interested in ways to improve or come up with algorithms that are able to solve the Travelling salesman problem for about n = 100 to 200 cities. The wikipedia link I gave lists various optimizati

I am working on Context Free Grammars and I am stuck into the first step: understanding how Top-Down parsing algorithms are structured. My problem revolves around top-down parsers. And I have three al

for i=1 to n | n+1 for j=1 to i^3 | ??? x=x+1 | Problem: Find O notation for n number of times the statement x=x+1 executes. I am confused with the second for loop here. I have the first statement th

I'm appending values into a div through jQuery, but I've realized what gets appended isn't affected by my javascript functions. $(.div).append(<input type='text' class='textForm' placement='Valu

Can anyone provide some pseudo code for a roulette selection function? How would I implement this: I don't really understand how to read this math notation. I never took any probability or statistics

I'm trying to understand how q defines dot notation for dates. Case 1: Temporal Variable Issue For ex. if I do following to get day from a date: q) d:2014.06.14 q) d.dd 14 i But if I try doing this wi

I found this example here Data Parallelism GParsPool.withPool() { Closure longLastingCalculation = {calculate()} Closure fastCalculation = longLastingCalculation.async() Future result=fastCalculation(

Quick question, could anyone help me out with regards to writing this without dot notation? (i.e. [self setView? ...) self.view.transform = CGAffineTransformIdentity; many thanks, gary

I came upon a notation I'm not too familiar with and would like some direction to where to look. the notation is selector event : function() {} so for example .elementClass change : function() {}

I was wondering what big numbers are, and what are some common algorithms used to handle them. I heard this term mentioned in Coders at Work, where one was asked in an interview to create a library to

I'm trying to learn twitter storm by following the great article Understanding the parallelism of a Storm topology However I'm a bit confused by the concept of task. Is a task an running instance

I've been looking around scipy and sklearn for clustering algorithms for a particular problem I have. I need some way of characterizing a population of N particles into k groups, where k is not necess

We are embarking on some R&D for a staff rostering system, and I know that there are some suggested algorithms such as the memetic algorithm etc., but I cannot find any additional information on t

I am using the AJAX rating control to rate the articles. It is working fine. But I wish to add a functionality to it. I want to give the user a message when the user has already rated the article. How