parallel programming in c#?????????
-
Parallel programming in .NET is threading; essentially the same thing.
Tech, life, family, faith: Give me a visit. I'm currently blogging about: Roman Catholic Relevance? The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
Parallel programming is not the same as multi-threading. In a parallel processing system you have multiple processors each with their own memory. These processors can run the same computational process, different processes, or parts of a single process independently and simulataneously without requiring context switching. Multi-threading allows multiple threads of execution to occur on a single processor and requires context switching to move between the threads. These are really two completely different concepts.
Scott.
—In just two days, tomorrow will be yesterday. [Forum Guidelines] [Articles] [Blog]
-
Parallel programming is not the same as multi-threading. In a parallel processing system you have multiple processors each with their own memory. These processors can run the same computational process, different processes, or parts of a single process independently and simulataneously without requiring context switching. Multi-threading allows multiple threads of execution to occur on a single processor and requires context switching to move between the threads. These are really two completely different concepts.
Scott.
—In just two days, tomorrow will be yesterday. [Forum Guidelines] [Articles] [Blog]
Hmm, well, I bow to your ultimate coding supremacy, Scott. :) But I'm not sure about this part:
Scott Dorman wrote:
Multi-threading allows multiple threads of execution to occur on a single processor
That's not true, right? I mean, the OS can schedule threads to run on any CPU or CPU core it chooses. Multi-threading enables one to program code that's run in parallel. On single-CPU machines, the OS simply switches between the different executing threads (context switching), thus simulating real parallel programs. Wikipedia says, "Some people consider parallel programming to be synonymous with concurrent programming. Others draw a distinction between parallel programming, which uses well-defined and structured patterns of communications between processes and focuses on parallel execution of processes to enhance throughput, and concurrent programming, which typically involves defining new patterns of communication between processes that may have been made concurrent for reasons other than performance. In either case, communication between processes is performed either via shared memory or with message passing, either of which may be implemented in terms of the other." I guess I've always considered parallel programming to mean writing software that executes in parallel. You do this using threads, which I think was the question of the post, after all, he said, "is parallel programming similar to threading or not?" All that said, I admit I'm not an expert at threading. I'm probably wrong and you're probably right. :)
Tech, life, family, faith: Give me a visit. I'm currently blogging about: I Plead (audio) The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
-
Hmm, well, I bow to your ultimate coding supremacy, Scott. :) But I'm not sure about this part:
Scott Dorman wrote:
Multi-threading allows multiple threads of execution to occur on a single processor
That's not true, right? I mean, the OS can schedule threads to run on any CPU or CPU core it chooses. Multi-threading enables one to program code that's run in parallel. On single-CPU machines, the OS simply switches between the different executing threads (context switching), thus simulating real parallel programs. Wikipedia says, "Some people consider parallel programming to be synonymous with concurrent programming. Others draw a distinction between parallel programming, which uses well-defined and structured patterns of communications between processes and focuses on parallel execution of processes to enhance throughput, and concurrent programming, which typically involves defining new patterns of communication between processes that may have been made concurrent for reasons other than performance. In either case, communication between processes is performed either via shared memory or with message passing, either of which may be implemented in terms of the other." I guess I've always considered parallel programming to mean writing software that executes in parallel. You do this using threads, which I think was the question of the post, after all, he said, "is parallel programming similar to threading or not?" All that said, I admit I'm not an expert at threading. I'm probably wrong and you're probably right. :)
Tech, life, family, faith: Give me a visit. I'm currently blogging about: I Plead (audio) The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
Judah Himango wrote:
I guess I've always considered parallel programming to mean writing software that executes in parallel. You do this using threads, which I think was the question of the post, after all, he said, "is parallel programming similar to threading or not?"
The distinction becomes clear if you think of each parallel process as an entirely separate machine (this is not really accurate, but serves to demonstrate what I mean) - for example, a Beowulf cluster. Threads are light processes which share memory with the main process; a truly parallel process has its own memory space. You can run them at the same time on 1 CPU, sure, but that rather defeats the purpose, since the CPU will still have to do twice as much work.
-
Parallel programming is not the same as multi-threading. In a parallel processing system you have multiple processors each with their own memory. These processors can run the same computational process, different processes, or parts of a single process independently and simulataneously without requiring context switching. Multi-threading allows multiple threads of execution to occur on a single processor and requires context switching to move between the threads. A single (physical) processor system can perform multi-threaded activities but not parallel processing, which a parallel processing system can be both multi-threaded and parallel. As to being able to do parallel processing in C#, it currently isn't really possible. You need specific compilers and language instructions that allow you to take advantage of a parallel processor system. These are not available in C# or any of the .NET languages.
Scott.
—In just two days, tomorrow will be yesterday. [Forum Guidelines] [Articles] [Blog]
Scott Dorman wrote:
These are not available in C# or any of the .NET languages.
I wonder how hard it would be to wrap MPI in C# wrappers? We only use it at a very high level at my job, so I don't know the vagaries and minuscule details of how it works, but I imagine it would be quite the undertaking, if it's even possible.
-
Parallel programming is not the same as multi-threading. In a parallel processing system you have multiple processors each with their own memory. These processors can run the same computational process, different processes, or parts of a single process independently and simulataneously without requiring context switching. Multi-threading allows multiple threads of execution to occur on a single processor and requires context switching to move between the threads. A single (physical) processor system can perform multi-threaded activities but not parallel processing, which a parallel processing system can be both multi-threaded and parallel. As to being able to do parallel processing in C#, it currently isn't really possible. You need specific compilers and language instructions that allow you to take advantage of a parallel processor system. These are not available in C# or any of the .NET languages.
Scott.
—In just two days, tomorrow will be yesterday. [Forum Guidelines] [Articles] [Blog]
Scott Dorman wrote:
As to being able to do parallel processing in C#, it currently isn't really possible.
Parallel processing to a certain degree is possible in C# see my post above for the link. That is the entire purpose of the Accelerator framework to bring parallel processing (of a certain degree) into the .net platform based upon the current GPU hardware. Its work inprogress but a step in the right direction. There is also a video on channel9 on which two developers talk about Accelerator in great detail.
-
Judah Himango wrote:
I guess I've always considered parallel programming to mean writing software that executes in parallel. You do this using threads, which I think was the question of the post, after all, he said, "is parallel programming similar to threading or not?"
The distinction becomes clear if you think of each parallel process as an entirely separate machine (this is not really accurate, but serves to demonstrate what I mean) - for example, a Beowulf cluster. Threads are light processes which share memory with the main process; a truly parallel process has its own memory space. You can run them at the same time on 1 CPU, sure, but that rather defeats the purpose, since the CPU will still have to do twice as much work.
Interesting, thanks. So where do multi-CPU or multi-core CPUs fit into the picture?
Tech, life, family, faith: Give me a visit. I'm currently blogging about: Roman Catholic Relevance? The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
-
in my opinion i wouldnt consider parallel programming to be the same as threading, purely because with parallel programming you need dedicated memory per processor. If you really want to experiment with parallel programming then I would recommend investigating a new framework for .NET from microsoft which utilises GPGPU concepts. The framework is called Microsoft Research Accelerator. In a nutshell GPGPU concepts utilises the graphic processor units parallel processing ability. http://channel9.msdn.com/wiki/default.aspx/Accelerator.HomePage[^] http://research.microsoft.com/act/[^] Accelerator is very exciting because it gives you an idea the future path the .net framework itself may take, add to that the advent of multicore processors. This is as close as you will get to true parallel programming in c#
Very interesting, thanks.
Tech, life, family, faith: Give me a visit. I'm currently blogging about: Roman Catholic Relevance? The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
-
Interesting, thanks. So where do multi-CPU or multi-core CPUs fit into the picture?
Tech, life, family, faith: Give me a visit. I'm currently blogging about: Roman Catholic Relevance? The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
Judah Himango wrote:
So where do multi-CPU or multi-core CPUs fit into the picture?
Until very, VERY recently, they didn't. Until just a few months ago, parallel programming libraries would only be aware of CPUs and could only fork jobs to CPU's - for example, the most popular parallel processing library, MPI. Even that wasn't a guarantee that the CPU you sent the job to would be the one doing it. I've seen Linux decide to move the jobs back and forth between CPUs to prevent them from overheating. btw, when I say 'fork jobs to CPUs' I mean the very heavy process of serializing the entire process to memory, moving it to another CPU, and having that CPU continue the job where it was left off. This is the distinction between a thread and a truly parallel process. People have been doing this for years on single CPU's using fork(), but it takes on a whole new level of meaning when you move it between CPUs. Recently though, several libraries have come out designed explicitly to give developers access to the multiple cores on each CPU, so I expect in several years we'll see parallel processing making use of every possible core on every possible machine. My office works almost exclusively in cluster computing so I expect I'll have to deal with this eventually....
-
Scott Dorman wrote:
As to being able to do parallel processing in C#, it currently isn't really possible.
Parallel processing to a certain degree is possible in C# see my post above for the link. That is the entire purpose of the Accelerator framework to bring parallel processing (of a certain degree) into the .net platform based upon the current GPU hardware. Its work inprogress but a step in the right direction. There is also a video on channel9 on which two developers talk about Accelerator in great detail.
Interesting link and some pretty cool research. My point still stands, however, in that it isn't possible to do parallel processing in C#...unless you are willing to install the research software, which most companies wouldn't allow.
Scott.
—In just two days, tomorrow will be yesterday. [Forum Guidelines] [Articles] [Blog]
-
Scott Dorman wrote:
These are not available in C# or any of the .NET languages.
I wonder how hard it would be to wrap MPI in C# wrappers? We only use it at a very high level at my job, so I don't know the vagaries and minuscule details of how it works, but I imagine it would be quite the undertaking, if it's even possible.
I'm sure it's possible but probably not worth the effort it would take to get everything right. There are a lot of concepts that are present in single processor systems that take on entirely new meanings in the world of parallel processors.
Scott.
—In just two days, tomorrow will be yesterday. [Forum Guidelines] [Articles] [Blog]
-
Parallel programming in .NET is threading; essentially the same thing.
Tech, life, family, faith: Give me a visit. I'm currently blogging about: Roman Catholic Relevance? The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
As previous comments have stated, threading is not equivalent to paralellel processing. Currently MS operating systems do not choose a processor for a thread or a process to run on. That has to be done in the application by the application. If you would like to further investigate this, MSDN has some examples and I am not certain that CP does or does not.
Phil
-
Hmm, well, I bow to your ultimate coding supremacy, Scott. :) But I'm not sure about this part:
Scott Dorman wrote:
Multi-threading allows multiple threads of execution to occur on a single processor
That's not true, right? I mean, the OS can schedule threads to run on any CPU or CPU core it chooses. Multi-threading enables one to program code that's run in parallel. On single-CPU machines, the OS simply switches between the different executing threads (context switching), thus simulating real parallel programs. Wikipedia says, "Some people consider parallel programming to be synonymous with concurrent programming. Others draw a distinction between parallel programming, which uses well-defined and structured patterns of communications between processes and focuses on parallel execution of processes to enhance throughput, and concurrent programming, which typically involves defining new patterns of communication between processes that may have been made concurrent for reasons other than performance. In either case, communication between processes is performed either via shared memory or with message passing, either of which may be implemented in terms of the other." I guess I've always considered parallel programming to mean writing software that executes in parallel. You do this using threads, which I think was the question of the post, after all, he said, "is parallel programming similar to threading or not?" All that said, I admit I'm not an expert at threading. I'm probably wrong and you're probably right. :)
Tech, life, family, faith: Give me a visit. I'm currently blogging about: I Plead (audio) The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
It looks like Patric gave you a pretty good explanation on this one. As to your other question regarding multi-CPU and multi-core CPUs, they really didn't become a factor until fairly recently. I'm not exactly sure what you mean by "multi-CPU" other than to say 2 or more physical CPU chips on one board. In that scenario, you are "technically" running a parallel processing system, although I'm not sure the OS really treats it as such. It can schedule processes to run on one CPU or the other but I don't believe you (as the programmer) have control through the OS as to what gets scheduled where, etc. Multi-core CPUs muddy the waters even further, as they are physically one chip and each core generally shares the same memory. Theoretically you could treat it as 2 CPUs, but I think you still end up with the same issues of context switches, etc.
Scott.
—In just two days, tomorrow will be yesterday. [Forum Guidelines] [Articles] [Blog]
-
As previous comments have stated, threading is not equivalent to paralellel processing. Currently MS operating systems do not choose a processor for a thread or a process to run on. That has to be done in the application by the application. If you would like to further investigate this, MSDN has some examples and I am not certain that CP does or does not.
Phil
pbraun wrote:
Currently MS operating systems do not choose a processor for a thread or a process to run on. That has to be done in the application by the application.
That seems to conflict with what Patrick and Scott are saying. Are you sure about this?
Tech, life, family, faith: Give me a visit. I'm currently blogging about: Roman Catholic Relevance? The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
-
pbraun wrote:
Currently MS operating systems do not choose a processor for a thread or a process to run on. That has to be done in the application by the application.
That seems to conflict with what Patrick and Scott are saying. Are you sure about this?
Tech, life, family, faith: Give me a visit. I'm currently blogging about: Roman Catholic Relevance? The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
Unless they've changed the scheduler in XP SP? and Vista for user processes, user processes are only scheduled on CPU 0. You can check this by writing a simple process that displays the CPU identifier and makes the CPU busy. Then run a few instances of the process to see which CPU they end up on. I don't have the source code for the test program I used to check that with anymore. It will take some time to re-create it.
Phil
-
Unless they've changed the scheduler in XP SP? and Vista for user processes, user processes are only scheduled on CPU 0. You can check this by writing a simple process that displays the CPU identifier and makes the CPU busy. Then run a few instances of the process to see which CPU they end up on. I don't have the source code for the test program I used to check that with anymore. It will take some time to re-create it.
Phil
I just built a test app that uses spawns a new thread to do lots of work. I've got a dual core CPU here and I see that both cores are kept partially busy while the thread runs. In other words, the OS is scheduling both cores to bear the load of a single thread.
Tech, life, family, faith: Give me a visit. I'm currently blogging about: I Plead (audio) The apostle Paul, modernly speaking: Epistles of Paul Judah Himango