Skip to content

an attempt to bypass the `sendto()` bottleneck

Peter V. Saveliev edited this page May 15, 2016 · 1 revision

An attempt to use an unsafe C sendto() implementation.

The reason: sendto() became a bottlenck. The solution: replace with a custom sendto() implementation. The conclusion: no help.

parser2 branch

https://github.com/svinota/pyroute2/tree/parser2

{parser2} $ sudo time -v python -m cProfile -s tottime t.py 
         616136 function calls (610298 primitive calls) in 4.488 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      230    1.483    0.006    1.483    0.006 {method 'sendto' of '_socket.socket' objects}
      265    1.180    0.004    1.180    0.004 {time.sleep}
     9022    0.152    0.000    0.152    0.000 {method 'acquire' of 'thread.lock' objects}
      275    0.125    0.000    0.125    0.000 {method 'recv' of '_socket.socket' objects}
      160    0.124    0.001    0.446    0.003 transactional.py:154(pick)
112159/109766    0.101    0.000    0.124    0.000 common.py:181(__getattribute__)
     5419    0.090    0.000    0.107    0.000 __init__.py:607(__init__)
      850    0.079    0.000    0.166    0.000 __init__.py:1274(decode_nlas)
3198/3172    0.060    0.000    0.276    0.000 __init__.py:826(decode)
6976/6896    0.059    0.000    0.117    0.000 transactional.py:414(__setitem__)
    19388    0.058    0.000    0.201    0.000 threading.py:147(acquire)
       80    0.058    0.001    0.118    0.001 transactional.py:229(__floordiv__)
     1845    0.049    0.000    0.050    0.000 threading.py:260(__init__)
    19388    0.045    0.000    0.054    0.000 threading.py:187(release)
      227    0.040    0.000    0.515    0.002 nlsocket.py:560(get)
7696/5605    0.039    0.000    0.321    0.000 transactional.py:42(decorated)
      277    0.029    0.000    0.050    0.000 transactional.py:89(__init__)
      705    0.026    0.000    0.177    0.000 main.py:902(_addr_add)
    39190    0.026    0.000    0.026    0.000 threading.py:64(_note)
    48043    0.026    0.000    0.026    0.000 {isinstance}
    16387    0.025    0.000    0.025    0.000 {_struct.unpack_from}
4203/4177    0.023    0.000    0.113    0.000 __init__.py:1038(get_attrs)
     2025    0.023    0.000    0.196    0.000 __init__.py:1138(_ft_decode_generic)
      685    0.022    0.000    0.025    0.000 nlsocket.py:225(release)
  431/230    0.020    0.000    0.050    0.000 __init__.py:909(encode)
     1651    0.019    0.000    0.095    0.000 linkedset.py:24(__init__)
      791    0.019    0.000    0.019    0.000 {method '__enter__' of 'thread.lock' objects}
     6178    0.018    0.000    0.018    0.000 {method 'items' of 'dict' objects}
21847/21793    0.018    0.000    0.095    0.000 __init__.py:1335(__getitem__)
…

send_from branch

https://github.com/svinota/pyroute2/tree/send_from

{send_from} $ sudo time -v python -m cProfile -s tottime t.py 
         618687 function calls (612809 primitive calls) in 4.631 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      230    1.667    0.007    1.667    0.007 {send.send_from}
      245    1.198    0.005    1.198    0.005 {time.sleep}
     8981    0.182    0.000    0.182    0.000 {method 'acquire' of 'thread.lock' objects}
      160    0.136    0.001    0.445    0.003 transactional.py:154(pick)
112169/109776    0.108    0.000    0.146    0.000 common.py:181(__getattribute__)
     5475    0.088    0.000    0.094    0.000 __init__.py:607(__init__)
       80    0.068    0.001    0.126    0.002 transactional.py:229(__floordiv__)
6976/6896    0.060    0.000    0.122    0.000 transactional.py:414(__setitem__)
      860    0.057    0.000    0.140    0.000 __init__.py:1274(decode_nlas)
    19404    0.050    0.000    0.218    0.000 threading.py:147(acquire)
      275    0.050    0.000    0.050    0.000 {method 'recv' of '_socket.socket' objects}
    19404    0.044    0.000    0.054    0.000 threading.py:187(release)
7696/5605    0.040    0.000    0.331    0.000 transactional.py:42(decorated)
3236/3210    0.034    0.000    0.221    0.000 __init__.py:826(decode)
      227    0.034    0.000    0.405    0.002 nlsocket.py:560(get)
      277    0.031    0.000    0.054    0.000 transactional.py:89(__init__)
      685    0.031    0.000    0.033    0.000 nlsocket.py:220(acquire)
  431/230    0.028    0.000    0.060    0.000 __init__.py:909(encode)
4253/4227    0.028    0.000    0.099    0.000 __init__.py:1038(get_attrs)
    16571    0.028    0.000    0.028    0.000 {_struct.unpack_from}
    48125    0.027    0.000    0.027    0.000 {isinstance}
  229/227    0.026    0.000    0.044    0.000 common.py:390(alloc)
     1845    0.025    0.000    0.026    0.000 threading.py:260(__init__)
    39217    0.021    0.000    0.021    0.000 threading.py:64(_note)
      685    0.021    0.000    0.024    0.000 nlsocket.py:225(release)
       80    0.020    0.000    4.119    0.051 interface.py:460(commit)
     2045    0.019    0.000    0.167    0.000 __init__.py:1138(_ft_decode_generic)
      787    0.019    0.000    0.019    0.000 {method '__enter__' of 'thread.lock' objects}
      275    0.016    0.000    0.223    0.001 nlsocket.py:139(parse)
1363/1323    0.016    0.000    0.019    0.000 req.py:263(__setitem__)
    18091    0.016    0.000    0.065    0.000 threading.py:215(__exit__)
     7345    0.014    0.000    0.016    0.000 common.py:197(__setattr__)
4249/4223    0.014    0.000    0.113    0.000 __init__.py:1025(get_attr)
     1651    0.014    0.000    0.063    0.000 linkedset.py:24(__init__)
22115/22061    0.013    0.000    0.077    0.000 __init__.py:1335(__getitem__)
…
Clone this wiki locally