This is an article from the Python world, but it is still applicable to the entire programming field, although multithreading allows us to process requests faster, but there is also a ceiling, green (micro-thread) threads are the solution.
Multithreaded software development solves a large number of problems, especially for network-centric applications that require demanding performance to respond quickly to users. Unfortunately, multithreading is not enough to solve large-scaleConcurrencysexual problems.
Addressing these issues requires changing programming models, using asynchronous events and callback-based mechanisms. At Druva, we created a python library-based called Dhaga to solve large-scaleConcurrency, while the programming model does not require significant changes.
Software developers live in oneConcurrencyworld. Threads are first-class citizens today, especially during development, especially when your application performs intensive network operations, like the inSync system (network security synchronization product) like Druva. Multithreading helps the flow of programming code for network operations simple and orderly. When our application needs performance enhancements or improvements, it can be improvedElasticity, we can increase the number of threads.
But when it comes to thousands of scalesConcurrencyrequests, threads are not enough.
We found that multithreading has the following drawbacks:
1. The inSync system client needs to back up a large number of files to the server via network RPC calls. A typical way for developers to speed things up is to use threads. However, the performance brought by multi-threading increases the cost of memory and CPU; Developers need to maintain a balance between speed and thread count.
2. Our servers need to handle between the inSync system and thousands of customersConcurrencyConnections and notifications. To handle connections efficiently, we use threads to handle requests. But the growing number of inSync system customers also means that we have to continue to increase the number of threads, which consumes a lot of server memory and CPU.
3. Our web server needs to handle thousands of parallel HTTP requests. Most of the work is on the network sockets that receive and send data and pass them to the backend of the inSync system. Causes most threads to wait for network operations. Causing the C10K problem, when there are thousands of synchronous requests to the web server, generating a thread for each request is quite non-scalable (Scale).
Limitations of asynchronous frameworks
Many async frameworks, including Twisted, Tornado Tornado, and asyncore, can help developers move away from the popular ways of using threads. These frameworks rely on non-blocking sockets and callback mechanisms (similar to Node.js). If we use these frameworks as is, the main parts of our Druva code will have to be refactored. That's not what we want to do. Refactoring code increases development and testing cycles, preventing us from meeting our scale requirements. Given that multiple parts of the product need to be massive, each of us will have to refactor them – hence the effort of doubling or triple.
To avoid changing so much code, we had to move away from using the existing framework directly. Fortunately, we found some useful tools.
Because we want to control the execution of code on the network I/O, we need a way to divide a thread into micro-threads. We findgreenlets。 It provides a non-implicit microthread scheduling called co-routine coroutine. In other words. It's useful when you want to control your code running. You can build microthreads for custom schedules because you can control when greenlets yield pauses. This is perfect for us because it gives us full control over the scheduling of our code.
Tornado is a simple, non-blocking web server framework written in Python designed to handle thousands of asynchronous requests. We use its core component, IOLoop IOStream. IOLoop is a non-blocking socket I/O event loop; It uses epoll (on Linux) or queues (BSD and Mac OS X), otherwise select (on Windows) if they are available. IOStream provides non-blocking sockets such as convenient packaging for reading and writing. We delegate all socket operations to Tornado and then use callbacks to trigger code operations to complete (banq note: very similar to Node.js mechanism).
It's a good start, but we need more. If we use the above module directly in our code, a lot of our RPC code will have to change, scheduling RPC through greenlets, making sure greenlets don't block (if greenlets get clogged, it will clog the entire thread and all the others), handle callback functions from tornado.
We need an abstraction to manage and arrange greenlets to avoid clogging them with external calls, and this abstraction can be massively scalable beyond threads. This abstraction is Dhaga, which allows the application code flow to be programmed like a traditional synchronous sequence, but the execution is asynchronous.
Dhaga (from Hindi, which means thread) is an execution framework for a lightweight thread that we abstract. The Dhaga class is derived from greenlets and uses stack switching to execute multiple code flows in a single operating system thread. Threads of one operating system execute multiple dhagases using collaborative scheduling. Whenever a dhaga waits (mainly waiting for an RPC call to return), it yields control to the parent level (i.e., the execution context of the OS-level thread that created it). The parent level then schedules another dhaga to be ready to run. The RPC call will be passed to the tornado web server to write the Socket asynchronously, and then register a callback when it returns, and when this RPC returns, the waiting dhaga will be added to the executable queue and then picked up by the parent thread. (Banq note: similar to node.js principle)
We can use Dhaga instead of threads for high latency operations, and we use 512 dhagas in a single thread when the number of threads increases beyond a reasonable limit for throughput.
|
Posted on 12/6/2014 10:21:58 PM
|
|
|
Posted on 12/7/2014 5:22:55 PM
