What's the benefits of multi-processing when we already have mult-threading?

I'm confused whether using multiple processes for a web application will improve the performance. Apache's mod_wsgi provides an option to set the number of processes to be started for the daemon process group. I used fastcgi with lighttpd before and it also had an option to configure the max number of processes for each fastcgi application.

While I don't know how multi-processing is better, I do know something bad about it compared to single-process multi-threading model. For example, logging will be harder to implement in multi-processing scenario (link), especially when you also want log rotating. And since memory can't be shared, if you cache something in memory (the most straightforward way), you have multiple duplicate copies.

Do multiple processes better utilize multi-core computing power, or do they yield higher throughput? Or are they just there for some single threaded applications?

-------------Problems Reply------------

In the case of Python, or more specifically CPython as used by mod_wsgi, the issue is the Python GIL. Although you may have multiple threads in Python, the global interpreter lock effectively means that only one thread can be running Python code at a time. This means it cannot make use of multiple processors/cores properly on a system. Using multiple processes however does allow you to use all those processors/cores.

That said, for mod_wsgi it isn't all Python code but has a lot of C code and with Apache also being C code. During execution of the C code, the GIL is unlocked by that thread meaning that a thread running in C code can run in parallel to a thread running in Python code. Still not the best one can achieve, but can still make partial use of all the processors/core on your system.

For me details of this in relation to mod_wsgi read:

http://blog.dscpl.com.au/2007/09/parallel-python-discussion-and-modwsgi.html http://blog.dscpl.com.au/2007/07/web-hosting-landscape-and-modwsgi.html

Multi-processing is less efficient than multi-threading, but it's more resilient to failure. Each process gets its own independent memory space and may be terminated and restarted (recycled) independent of other processes.

Category:multithreading Views:1 Time:2011-08-09

Related post

  • Performance difference for multi-thread and multi-process 2010-07-31

    A few years ago, in the Windows environment, I did some testing, by letting multiple instances of CPU computation intensive + memory access intensive + I/O access intensive application run. I developed 2 versions: One is running under multi-processin

  • Multi-process builds in Visual Studio 2010: Worth it? 2010-05-26

    I've started testing our C++ software with VS2010 and the build times are really bad (30-45 minutes, about double the VS2005 times). I've been reading about the /MP switch for multi-process compilation. Unfortunately, it is incompatible with some fea

  • Chrome / IE8 multi-process design, is it possible in .NET? 2009-03-10

    Google Chrome and IE8 (among others) aim to provide greater reliability/stability by isolating each tab (web page) in a separate process (over-simplified, I know). This would seem to be much more heavyweight then multiple threads, but has the major b

  • Multi-process support within the same app? 2009-05-11

    Google Chrome and IE8 implement multi-process support for each tab. How would I implement something like that using c# and vs2008? --------------Solutions------------- Umm... look up Inter-process communication. Then make your processes communcate wi

  • Designing a multi-process spider in Python 2009-11-23

    I'm working on a multi-process spider in Python. It should start scraping one page for links and work from there. Specifically, the top-level page contains a list of categories, the second-level pages events in those categories, and the final, third-

  • Multi-process webserver vs multi threaded web server? 2009-12-31

    I would like to know why we prefer to make web servers multi-threaded instead of make it multi-process web servers .... Is it because of legacy issues..... I would like to hear practical reasons as well as theoretical reasons --------------Solutions-

  • python multi-processing queue: is putting independent from getting? 2010-02-16

    Is putting an object in a multi-processing queue independent from getting an object from it? In other words, will putting an object block the process P1 if another process P2 is getting from it? Update: I am assuming an infinite queue. --------------

  • SQLite multi process access 2010-05-05

    We are using SQLite in a multi processes and multi threaded application. The SQLite database files are encrypted using the embedded SQLite encryption. The FAQ states that SQLite should be able to manage multi process accesses using locks mechanism. W

  • How to create a multi-process webkitgtk application (Vala)? 2010-07-12

    I want to make a web browser in Vala using webkit. But, I don't know how to make it multi-process. I want each tab to have its own process. How can I do that using Vala and Gtk+. Thanks for your answer. --------------Solutions------------- Check out

  • Running PHP exec() asynchronously/multi-processes 2010-07-26

    I'm currently running these lines in a PHP script: foreach ($list as $route) { exec('php-cgi ./nextbus-route_stop_predictions.php route=' . $route['route_id']); } nextbus-route_stop_predictions.php takes about 15 seconds to run and exec() is ran abou

  • Is there a multi-process unit testing framework / junit addon? 2010-08-18

    Imagine two servers (each within an own jvm process) which communicate using some form of messages (e.g. simple producer/consumer) I'd like to write unit tests that will test the behaviour of these two servers. My questions are: Are there some framew

  • Multi processing subprocess 2010-08-20

    I'm new to subprocess module of python, currently my implementation is not multi processed. import subprocess,shlex def forcedParsing(fname): cmd = 'strings "%s"' % (fname) #print cmd args= shlex.split(cmd) try: sp = subprocess.Popen( args, shell = F

  • How to have a multi process architecture for my Flex AIR based application? 2010-11-11

    I developing an AIR application in Flex that would be dealing with playing different SWFs. I am finding some issues as these SWFs that I am playing are heavy. I find that the animation is not smooth and is causing the graph of CPU usage to go high (m

  • Which is better choice for browser's tab: multi-thread or multi-process? 2010-11-24

    I wonder which is a better choice for implementing tabs in a web browser? (Ex: Firefox use multi-threading for their tabs, while Google Chrome use multi-processes ... ) --------------Solutions------------- It depends on the quality of your implementa

  • boost::asio server multi-process 2011-03-01

    I would to make a simple multi process (not thread) server. I've seen the iterative example in which it handles one request at a time. Instead I need to handle more requests(more on less 10) at the same time. In the classic c and c++ examples, I've s

  • Sample for Multi-process C# app like Google Chrome 2011-04-14

    Is Any body knows good sample for Multi-process C# app like Google Chrome. Thanks for help. --------------Solutions------------- Useful blog post (with actual C# source code): http://wyday.com/blog/2010/multi-process-c-sharp-application-like-google-c

  • Apache multi process model vs Tomact multi thread model 2011-08-02

    I'm a newbie to the J2EE world and I was wondering about the performance of Tomcat Multi Threaded model for Servlets as opposed to Apache Multi Process model. In a traditional round robin process scheduling OS, apache has an advantage since it has mu

  • What is the expected overhead of a multi-process system compared to multi-threaded application? 2011-11-26

    For some operating systems (OpenBSD & ...) threads are userland-only and mostly exist for compatibility. Disregarding that type of implementation, how does the use of kernel-level threads compare to a multi-process design? What are the security i

  • How to get an id that is unique under multi-thread, multi-process, web gardens and web farms situations? 2011-12-01

    I am trying to go around the problem of log4net not being multi-process safe. Because sometimes the log file gets locked and another thread of same application cannot write onto it. What I am trying to do is to create the log file with a unique id to

Copyright (C) dskims.com, All Rights Reserved.

processed in 0.078 (s). 11 q(s)