Benchmark asyncio vs gevent vs native epoll

Everyone knows the asyncio module in python schedules all coroutines in a single thread. That means it helps you code easier, and you can’t gain any performance from it.

But what is the performance of python’s asyncio module? How fast can it run compared to traditional gevent and native epoll ?

Requirements

pip3 install hiredis gevent

Enter fullscreen mode Exit fullscreen mode

An optional package uvloop can also be install if working on Linux:

pip3 install uvloop

Enter fullscreen mode Exit fullscreen mode

Source

The following code uses both asyncio and gevent to simulate a redis server on port 5000, 5001, and 5002.

They can be tested with redis-benchmark.

content of echo_bench_gevent.py:

import sys
import gevent
import gevent.monkey
import hiredis

from gevent.server import StreamServer
gevent.monkey.patch_all()

d = {}

def process(req):
    # only support get/set     cmd = req[0].lower()
    if cmd == b'set':
        d[req[1]] = req[2]
        return b"+OK\r\n"
    elif cmd == b'get':
        v = d.get(req[1])
        if v is None:
            return b'$-1\r\n'
        else:
            return b'$1\r\n1\r\n'
    else:
        print(cmd)
        raise NotImplementedError()
    return b''

def handle(sock, addr):
    reader = hiredis.Reader()
    while True:
        buf = sock.recv(4096)
        if not buf:
            return
        reader.feed(buf)
        while True:
            req = reader.gets()
            if not req:
                break
            sock.sendall(process(req))
    return 0

print('serving on 0.0.0.0:5000')
server = StreamServer(('0.0.0.0', 5000), handle)
server.serve_forever()

Enter fullscreen mode Exit fullscreen mode

content of echo_bench_asyncio.py:

import asyncio
import hiredis

d = {}

def process(req):
    cmd = req[0].lower()
    if cmd == b'set':
        d[req[1]] = req[2]
        return b"+OK\r\n"
    elif cmd == b'get':
        v = d.get(req[1])
        if v is None:
            return b'$-1\r\n'
        else:
            return b'$1\r\n1\r\n'
    elif cmd == b'config':
        return b'-ERROR\r\n'
    else:
        return b'-ERROR\r\n'
    return b''

async def echo_server(reader, writer):
    hireader = hiredis.Reader()
    while True:
        s = await reader.read(4096)
        if not s:
            break
        hireader.feed(s)
        while True:
            req = hireader.gets()
            if not req:
                break
            res = process(req)
            writer.write(res)
            await writer.drain()
    return 0

async def main():
    server = await asyncio.start_server(echo_server, '0.0.0.0', 5001)
    print('serving on {}'.format(server.sockets[0].getsockname()))
    await server.serve_forever()
    return 0

asyncio.run(main())

Enter fullscreen mode Exit fullscreen mode

content of echo_bench_asyncio_uvloop.py:

import asyncio
import hiredis

d = {}

def process(req):
    cmd = req[0].lower()
    if cmd == b'set':
        d[req[1]] = req[2]
        return b"+OK\r\n"
    elif cmd == b'get':
        v = d.get(req[1])
        if v is None:
            return b'$-1\r\n'
        else:
            return b'$1\r\n1\r\n'
    elif cmd == b'config':
        return b'-ERROR\r\n'
    else:
        return b'-ERROR\r\n'
    return b''

async def echo_server(reader, writer):
    hireader = hiredis.Reader()
    while True:
        s = await reader.read(4096)
        if not s:
            break
        hireader.feed(s)
        while True:
            req = hireader.gets()
            if not req:
                break
            res = process(req)
            writer.write(res)
            await writer.drain()
    return 0

async def main():
    server = await asyncio.start_server(echo_server, '0.0.0.0', 5002)
    print('serving on {}'.format(server.sockets[0].getsockname()))
    await server.serve_forever()
    return 0

try:
    import uvloop
    uvloop.install()
    print('uvloop is enabled')
except ImportError:
    print('uvloop is not available')

asyncio.run(main())

Enter fullscreen mode Exit fullscreen mode

Start servers

python3 echo_bench_gevent.py          # will listen on port 5000
python3 echo_bench_asyncio.py         # will listen on port 5001
python3 echo_bench_asyncio_uvloop.py  # will listen on port 5002

Enter fullscreen mode Exit fullscreen mode

Test

redis-benchmark -p 5000 -t get -n 100000 -r 100000000
redis-benchmark -p 5001 -t get -n 100000 -r 100000000
redis-benchmark -p 5002 -t get -n 100000 -r 100000000

Enter fullscreen mode Exit fullscreen mode

Result

Native epoll

Redis is implemented with epoll in C, and we can test redis directly:

redis-benchmark -p 6379 -t get -n 100000 -r 100000000

Enter fullscreen mode Exit fullscreen mode

Output:

75244.55 requests per second

Enter fullscreen mode Exit fullscreen mode

Conclusion

• asyncio is 50% faster than gevent in Python 3.11
• asyncio can run twice as fast as gevent with uvloop.
• asyncio can go up to 68% of a native epoll program.
• asyncio can go up to 88% of a native epoll program with uvloop.

原文链接：Benchmark asyncio vs gevent vs native epoll