So I had a little python script that scraps a certain website for product data. A couple of functions that fetches data, parses for one type of content, another type of data.
Now I had this issue where the css selectors returns None which breaks all my sneaky list indices.
Currently my program looks like this.
<span>def</span> <span>get_x_info</span><span>(</span><span>url</span><span>):</span><span># piece of code that works all the time </span><span># piece of code that crashes </span><span>return</span> <span>result</span><span>def</span> <span>get_x_info</span><span>(</span><span>url</span><span>):</span> <span># piece of code that works all the time </span> <span># piece of code that crashes </span> <span>return</span> <span>result</span>def get_x_info(url): # piece of code that works all the time # piece of code that crashes return result
Applying try and except
to the piece of code that crashes wasn’t quite convincing because of the tedious task of doing this to all of the probable volatile code. And I wasn’t quite sure of what part might crash.
Hence, Python Decorators.
A syntactic sugar for wrapping functions with a reusable code.
Learn about it here
Here is how I used decorators error handle the functions that throws errors most of the time.
<span>def</span> <span>safe_run</span><span>(</span><span>func</span><span>):</span><span>def</span> <span>func_wrapper</span><span>(</span><span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>):</span><span>try</span><span>:</span><span>return</span> <span>func</span><span>(</span><span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>)</span><span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span><span>print</span><span>(</span><span>e</span><span>)</span><span>return</span> <span>None</span><span>return</span> <span>func_wrapper</span><span>def</span> <span>safe_run</span><span>(</span><span>func</span><span>):</span> <span>def</span> <span>func_wrapper</span><span>(</span><span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>):</span> <span>try</span><span>:</span> <span>return</span> <span>func</span><span>(</span><span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>)</span> <span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span> <span>print</span><span>(</span><span>e</span><span>)</span> <span>return</span> <span>None</span> <span>return</span> <span>func_wrapper</span>def safe_run(func): def func_wrapper(*args, **kwargs): try: return func(*args, **kwargs) except Exception as e: print(e) return None return func_wrapper
Now apply safe_run
as a decorator to the function declarations in the main code
<span>@</span><span>safe_run</span><span>def</span> <span>get_x_info</span><span>(</span><span>url</span><span>):</span><span>....</span><span>@</span><span>safe_run</span> <span>def</span> <span>get_x_info</span><span>(</span><span>url</span><span>):</span> <span>....</span>@safe_run def get_x_info(url): ....
Quick terrible fix that works 🙂
暂无评论内容