Enfold the odyssey to Python brilliance, a journey illuminated with the allure of elegance and efficiency. This article, inspired by Saa, my Python package that translates time into human-friendly spoken expressions, takes on a voyage to explore four advanced Python concepts, unearthing the true prowess of Python language. With every stride, witness your code transform, mirroring the elegance of poetry and the finesse of an art masterpiece.
️ code heavy: set out in not-too-distant galax, this article explores advance concepts
1. Single Dispatcher: The Chameleon Functions
Observe the allure of singledispatch from functools, a decorator that bestows your functions the gift to morph based on the type of the first argument, a beacon for writing seamless generic code.
<span>from</span> <span>__future__</span> <span>import</span> <span>annotations</span><span>from</span> <span>functools</span> <span>import</span> <span>singledispatch</span><span>from</span> <span>datetime</span> <span>import</span> <span>time</span><span>,</span> <span>datetime</span><span>TimeType</span> <span>=</span> <span>str</span> <span>|</span> <span>time</span> <span>|</span> <span>datetime</span><span>@singledispatch</span><span>def</span> <span>clock</span><span>(</span><span>_</span><span>:</span> <span>TimeType</span><span>)</span> <span>-></span> <span>time</span><span>:</span><span>"""</span><span>Clock Parser Accepts string, time or datetime and return time object Args: _ (TimeType): string, time or datetime object Raises: NotImplementedError: shell for dispatching Returns: time: python time object </span><span>"""</span><span>raise</span> <span>NotImplementedError</span><span>@clock.register</span><span>(</span><span>str</span><span>)</span><span>def</span> <span>_</span><span>(</span><span>t</span><span>:</span> <span>str</span><span>)</span> <span>-></span> <span>time</span><span>:</span><span>return</span> <span>datetime</span><span>.</span><span>strptime</span><span>(</span><span>t</span><span>,</span> <span>"</span><span>%H:%M</span><span>"</span><span>).</span><span>time</span><span>()</span><span>@clock.register</span><span>(</span><span>datetime</span><span>)</span><span>def</span> <span>_</span><span>(</span><span>t</span><span>:</span> <span>datetime</span><span>)</span> <span>-></span> <span>time</span><span>:</span><span>return</span> <span>t</span><span>.</span><span>time</span><span>()</span><span>@clock.register</span><span>(</span><span>time</span><span>)</span><span>def</span> <span>_</span><span>(</span><span>t</span><span>:</span> <span>time</span><span>)</span> <span>-></span> <span>time</span><span>:</span><span>return</span> <span>t</span><span>from</span> <span>__future__</span> <span>import</span> <span>annotations</span> <span>from</span> <span>functools</span> <span>import</span> <span>singledispatch</span> <span>from</span> <span>datetime</span> <span>import</span> <span>time</span><span>,</span> <span>datetime</span> <span>TimeType</span> <span>=</span> <span>str</span> <span>|</span> <span>time</span> <span>|</span> <span>datetime</span> <span>@singledispatch</span> <span>def</span> <span>clock</span><span>(</span><span>_</span><span>:</span> <span>TimeType</span><span>)</span> <span>-></span> <span>time</span><span>:</span> <span>"""</span><span>Clock Parser Accepts string, time or datetime and return time object Args: _ (TimeType): string, time or datetime object Raises: NotImplementedError: shell for dispatching Returns: time: python time object </span><span>"""</span> <span>raise</span> <span>NotImplementedError</span> <span>@clock.register</span><span>(</span><span>str</span><span>)</span> <span>def</span> <span>_</span><span>(</span><span>t</span><span>:</span> <span>str</span><span>)</span> <span>-></span> <span>time</span><span>:</span> <span>return</span> <span>datetime</span><span>.</span><span>strptime</span><span>(</span><span>t</span><span>,</span> <span>"</span><span>%H:%M</span><span>"</span><span>).</span><span>time</span><span>()</span> <span>@clock.register</span><span>(</span><span>datetime</span><span>)</span> <span>def</span> <span>_</span><span>(</span><span>t</span><span>:</span> <span>datetime</span><span>)</span> <span>-></span> <span>time</span><span>:</span> <span>return</span> <span>t</span><span>.</span><span>time</span><span>()</span> <span>@clock.register</span><span>(</span><span>time</span><span>)</span> <span>def</span> <span>_</span><span>(</span><span>t</span><span>:</span> <span>time</span><span>)</span> <span>-></span> <span>time</span><span>:</span> <span>return</span> <span>t</span>from __future__ import annotations from functools import singledispatch from datetime import time, datetime TimeType = str | time | datetime @singledispatch def clock(_: TimeType) -> time: """Clock Parser Accepts string, time or datetime and return time object Args: _ (TimeType): string, time or datetime object Raises: NotImplementedError: shell for dispatching Returns: time: python time object """ raise NotImplementedError @clock.register(str) def _(t: str) -> time: return datetime.strptime(t, "%H:%M").time() @clock.register(datetime) def _(t: datetime) -> time: return t.time() @clock.register(time) def _(t: time) -> time: return t
Enter fullscreen mode Exit fullscreen mode
Test we must, young Padawan. Reflect upon the use cases, we shall. In the vast galaxy where stars twinkle, mirror the universe’s myriad possibilities, our trials will.
<span>from</span> <span>datetime</span> <span>import</span> <span>datetime</span><span>,</span> <span>time</span><span>import</span> <span>pytest</span><span>import</span> <span>clock</span><span>@pytest.fixture</span><span>(</span><span>params</span><span>=</span><span>[</span><span>"</span><span>12:34</span><span>"</span><span>,</span> <span>datetime</span><span>.</span><span>now</span><span>().</span><span>replace</span><span>(</span><span>hour</span><span>=</span><span>12</span><span>,</span> <span>minute</span><span>=</span><span>34</span><span>),</span> <span>time</span><span>(</span><span>12</span><span>,</span> <span>34</span><span>)])</span><span>def</span> <span>valid_time_input</span><span>(</span><span>request</span><span>):</span><span>yield</span> <span>request</span><span>.</span><span>param</span><span>@pytest.fixture</span><span>(</span><span>params</span><span>=</span><span>[</span><span>1234</span><span>,</span> <span>12.34</span><span>,</span> <span>[</span><span>12</span><span>,</span> <span>34</span><span>],</span> <span>{</span><span>"</span><span>hour</span><span>"</span><span>:</span> <span>12</span><span>,</span> <span>"</span><span>minute</span><span>"</span><span>:</span> <span>34</span><span>}])</span><span>def</span> <span>invalid_time_input</span><span>(</span><span>request</span><span>):</span><span>yield</span> <span>request</span><span>.</span><span>param</span><span>def</span> <span>test_clock_with_valid_input</span><span>(</span><span>valid_time_input</span><span>):</span><span>"</span><span>Test with valid inputs</span><span>"</span><span>result</span> <span>=</span> <span>clock</span><span>(</span><span>valid_time_input</span><span>)</span><span>assert</span> <span>isinstance</span><span>(</span><span>result</span><span>,</span> <span>time</span><span>)</span><span>assert</span> <span>result</span><span>.</span><span>hour</span> <span>==</span> <span>12</span><span>assert</span> <span>result</span><span>.</span><span>minute</span> <span>==</span> <span>34</span><span>def</span> <span>test_clock_with_invalid_input</span><span>(</span><span>invalid_time_input</span><span>):</span><span>"</span><span>Test with invalid input, expecting NotImplementedError</span><span>"</span><span>with</span> <span>pytest</span><span>.</span><span>raises</span><span>(</span><span>NotImplementedError</span><span>):</span><span>clock</span><span>(</span><span>invalid_time_input</span><span>)</span><span>from</span> <span>datetime</span> <span>import</span> <span>datetime</span><span>,</span> <span>time</span> <span>import</span> <span>pytest</span> <span>import</span> <span>clock</span> <span>@pytest.fixture</span><span>(</span><span>params</span><span>=</span><span>[</span><span>"</span><span>12:34</span><span>"</span><span>,</span> <span>datetime</span><span>.</span><span>now</span><span>().</span><span>replace</span><span>(</span><span>hour</span><span>=</span><span>12</span><span>,</span> <span>minute</span><span>=</span><span>34</span><span>),</span> <span>time</span><span>(</span><span>12</span><span>,</span> <span>34</span><span>)])</span> <span>def</span> <span>valid_time_input</span><span>(</span><span>request</span><span>):</span> <span>yield</span> <span>request</span><span>.</span><span>param</span> <span>@pytest.fixture</span><span>(</span><span>params</span><span>=</span><span>[</span><span>1234</span><span>,</span> <span>12.34</span><span>,</span> <span>[</span><span>12</span><span>,</span> <span>34</span><span>],</span> <span>{</span><span>"</span><span>hour</span><span>"</span><span>:</span> <span>12</span><span>,</span> <span>"</span><span>minute</span><span>"</span><span>:</span> <span>34</span><span>}])</span> <span>def</span> <span>invalid_time_input</span><span>(</span><span>request</span><span>):</span> <span>yield</span> <span>request</span><span>.</span><span>param</span> <span>def</span> <span>test_clock_with_valid_input</span><span>(</span><span>valid_time_input</span><span>):</span> <span>"</span><span>Test with valid inputs</span><span>"</span> <span>result</span> <span>=</span> <span>clock</span><span>(</span><span>valid_time_input</span><span>)</span> <span>assert</span> <span>isinstance</span><span>(</span><span>result</span><span>,</span> <span>time</span><span>)</span> <span>assert</span> <span>result</span><span>.</span><span>hour</span> <span>==</span> <span>12</span> <span>assert</span> <span>result</span><span>.</span><span>minute</span> <span>==</span> <span>34</span> <span>def</span> <span>test_clock_with_invalid_input</span><span>(</span><span>invalid_time_input</span><span>):</span> <span>"</span><span>Test with invalid input, expecting NotImplementedError</span><span>"</span> <span>with</span> <span>pytest</span><span>.</span><span>raises</span><span>(</span><span>NotImplementedError</span><span>):</span> <span>clock</span><span>(</span><span>invalid_time_input</span><span>)</span>from datetime import datetime, time import pytest import clock @pytest.fixture(params=["12:34", datetime.now().replace(hour=12, minute=34), time(12, 34)]) def valid_time_input(request): yield request.param @pytest.fixture(params=[1234, 12.34, [12, 34], {"hour": 12, "minute": 34}]) def invalid_time_input(request): yield request.param def test_clock_with_valid_input(valid_time_input): "Test with valid inputs" result = clock(valid_time_input) assert isinstance(result, time) assert result.hour == 12 assert result.minute == 34 def test_clock_with_invalid_input(invalid_time_input): "Test with invalid input, expecting NotImplementedError" with pytest.raises(NotImplementedError): clock(invalid_time_input)
Enter fullscreen mode Exit fullscreen mode
Reflect and Contemplate:
Pave this path cautiously. Ensure the base function is etched to raise a NotImplementedError and adorn each additional implementation with @function_name.register(type).
2. Setters and Getters: The Shield and Sword
Conjure the @property decorator, your shield and sword, ensuring your objects stand resilient, cloaked in encapsulation.
Beginning with elemental steps, let’s unveil the potential to not only encapsulate but also validate inputs at the setting.
<span>from</span> <span>__future__</span> <span>import</span> <span>annotations</span><span>π</span> <span>=</span> <span>22</span><span>/</span><span>7</span> <span>#3.14... </span><span>def</span> <span>radius_validator</span><span>(</span><span>value</span><span>:</span> <span>int</span><span>|</span><span>float</span><span>)</span> <span>-></span> <span>int</span> <span>|</span> <span>float</span><span>:</span><span>if</span> <span>not</span> <span>isinstance</span><span>(</span><span>value</span><span>,</span> <span>(</span><span>int</span><span>,</span> <span>float</span><span>)):</span><span>raise</span> <span>TypeError</span><span>(</span><span>f</span><span>"</span><span>Radius has to be a positive int or float </span><span>"</span><span>)</span><span>elif</span> <span>value</span> <span><</span> <span>0</span><span>:</span><span>raise</span> <span>ValueError</span><span>(</span><span>f</span><span>"</span><span>Radius cannot be negative </span><span>"</span><span>)</span><span>return</span> <span>value</span><span>class</span> <span>Circle</span><span>:</span><span>def</span> <span>__init__</span><span>(</span><span>self</span><span>,</span> <span>radius</span><span>:</span> <span>int</span><span>|</span><span>float</span><span>):</span><span>self</span><span>.</span><span>radius</span> <span>=</span> <span>radius</span><span>@property</span><span>def</span> <span>radius</span><span>(</span><span>self</span><span>)</span> <span>-></span> <span>int</span><span>|</span><span>float</span><span>:</span><span>return</span> <span>self</span><span>.</span><span>_radius</span><span>@radius.setter</span><span>def</span> <span>radius</span><span>(</span><span>self</span><span>,</span> <span>value</span><span>:</span> <span>int</span><span>|</span><span>float</span><span>):</span><span>self</span><span>.</span><span>_radius</span> <span>=</span> <span>radius_validator</span><span>(</span><span>value</span><span>)</span><span>def</span> <span>area</span><span>(</span><span>self</span><span>)</span> <span>-></span> <span>float</span><span>:</span><span>return</span> <span>π</span> <span>*</span> <span>self</span><span>.</span><span>_radius</span><span>**</span><span>2</span><span>def</span> <span>circumference</span><span>(</span><span>self</span><span>)</span> <span>-></span> <span>float</span><span>:</span><span>return</span> <span>2</span> <span>*</span> <span>π</span> <span>*</span> <span>self</span><span>.</span><span>_radius</span><span>def</span> <span>__repr__</span><span>(</span><span>self</span><span>):</span><span>return</span> <span>f</span><span>"</span><span>area=</span><span>{</span><span>self</span><span>.</span><span>area</span><span>()</span><span>:</span> <span>.</span><span>2</span><span>f</span><span>}</span><span>;circumference=</span><span>{</span><span>self</span><span>.</span><span>circumference</span><span>()</span><span>:</span><span>.</span><span>2</span><span>f</span><span>}</span><span>"</span><span># c = Circle(radius=42) # c.radius = -1 # => throws ValueError(f"Radius cannot be negative ") # c.radius = "42" # => throws TypeError(f"Radius has to be a positive int or float ") </span><span>from</span> <span>__future__</span> <span>import</span> <span>annotations</span> <span>π</span> <span>=</span> <span>22</span><span>/</span><span>7</span> <span>#3.14... </span> <span>def</span> <span>radius_validator</span><span>(</span><span>value</span><span>:</span> <span>int</span><span>|</span><span>float</span><span>)</span> <span>-></span> <span>int</span> <span>|</span> <span>float</span><span>:</span> <span>if</span> <span>not</span> <span>isinstance</span><span>(</span><span>value</span><span>,</span> <span>(</span><span>int</span><span>,</span> <span>float</span><span>)):</span> <span>raise</span> <span>TypeError</span><span>(</span><span>f</span><span>"</span><span>Radius has to be a positive int or float </span><span>"</span><span>)</span> <span>elif</span> <span>value</span> <span><</span> <span>0</span><span>:</span> <span>raise</span> <span>ValueError</span><span>(</span><span>f</span><span>"</span><span>Radius cannot be negative </span><span>"</span><span>)</span> <span>return</span> <span>value</span> <span>class</span> <span>Circle</span><span>:</span> <span>def</span> <span>__init__</span><span>(</span><span>self</span><span>,</span> <span>radius</span><span>:</span> <span>int</span><span>|</span><span>float</span><span>):</span> <span>self</span><span>.</span><span>radius</span> <span>=</span> <span>radius</span> <span>@property</span> <span>def</span> <span>radius</span><span>(</span><span>self</span><span>)</span> <span>-></span> <span>int</span><span>|</span><span>float</span><span>:</span> <span>return</span> <span>self</span><span>.</span><span>_radius</span> <span>@radius.setter</span> <span>def</span> <span>radius</span><span>(</span><span>self</span><span>,</span> <span>value</span><span>:</span> <span>int</span><span>|</span><span>float</span><span>):</span> <span>self</span><span>.</span><span>_radius</span> <span>=</span> <span>radius_validator</span><span>(</span><span>value</span><span>)</span> <span>def</span> <span>area</span><span>(</span><span>self</span><span>)</span> <span>-></span> <span>float</span><span>:</span> <span>return</span> <span>π</span> <span>*</span> <span>self</span><span>.</span><span>_radius</span><span>**</span><span>2</span> <span>def</span> <span>circumference</span><span>(</span><span>self</span><span>)</span> <span>-></span> <span>float</span><span>:</span> <span>return</span> <span>2</span> <span>*</span> <span>π</span> <span>*</span> <span>self</span><span>.</span><span>_radius</span> <span>def</span> <span>__repr__</span><span>(</span><span>self</span><span>):</span> <span>return</span> <span>f</span><span>"</span><span>area=</span><span>{</span><span>self</span><span>.</span><span>area</span><span>()</span><span>:</span> <span>.</span><span>2</span><span>f</span><span>}</span><span>;circumference=</span><span>{</span><span>self</span><span>.</span><span>circumference</span><span>()</span><span>:</span><span>.</span><span>2</span><span>f</span><span>}</span><span>"</span> <span># c = Circle(radius=42) # c.radius = -1 # => throws ValueError(f"Radius cannot be negative ") # c.radius = "42" # => throws TypeError(f"Radius has to be a positive int or float ") </span>from __future__ import annotations π = 22/7 #3.14... def radius_validator(value: int|float) -> int | float: if not isinstance(value, (int, float)): raise TypeError(f"Radius has to be a positive int or float ") elif value < 0: raise ValueError(f"Radius cannot be negative ") return value class Circle: def __init__(self, radius: int|float): self.radius = radius @property def radius(self) -> int|float: return self._radius @radius.setter def radius(self, value: int|float): self._radius = radius_validator(value) def area(self) -> float: return π * self._radius**2 def circumference(self) -> float: return 2 * π * self._radius def __repr__(self): return f"area={self.area(): .2f};circumference={self.circumference():.2f}" # c = Circle(radius=42) # c.radius = -1 # => throws ValueError(f"Radius cannot be negative ") # c.radius = "42" # => throws TypeError(f"Radius has to be a positive int or float ")
Enter fullscreen mode Exit fullscreen mode
Up to this point, things are looking promising. The power of setters and getters extends far beyond mere value validation at assignments; they unlock Jedi-like capabilities. Consider this: in the realm of classical Machine Learning, what if we employed a setter to save a fitted transformer and later, during prediction, utilized a getter to retrieve it? Intriguing, isn’t it?
<span>import</span> <span>pickle</span><span>from</span> <span>pathlib</span> <span>import</span> <span>Path</span><span>from</span> <span>typing</span> <span>import</span> <span>Literal</span><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><span>from</span> <span>sklearn.compose</span> <span>import</span> <span>ColumnTransformer</span><span>class</span> <span>TransformerTask</span><span>:</span><span>def</span> <span>__init__</span><span>(</span><span>self</span><span>,</span> <span>transformer</span><span>:</span><span>ColumnTransformer</span><span>,</span><span>stage</span><span>:</span> <span>Literal</span><span>[</span><span>"</span><span>train</span><span>"</span><span>,</span> <span>"</span><span>predict</span><span>"</span><span>]</span> <span>=</span> <span>"</span><span>train</span><span>"</span><span>,</span><span>file_path</span><span>:</span><span>str</span><span>=</span><span>"</span><span>models/transformer.pkl</span><span>"</span><span>,):</span><span>self</span><span>.</span><span>stage</span> <span>=</span> <span>stage</span><span>self</span><span>.</span><span>file_path</span> <span>=</span> <span>Path</span><span>(</span><span>file_path</span><span>)</span><span>self</span><span>.</span><span>_transformer</span> <span>=</span> <span>transformer</span><span>@property</span><span>def</span> <span>transformer</span><span>(</span><span>self</span><span>)</span> <span>-></span> <span>ColumnTransformer</span><span>:</span><span>if</span> <span>self</span><span>.</span><span>stage</span> <span>==</span> <span>"</span><span>predict</span><span>"</span> <span>and</span> <span>not</span> <span>self</span><span>.</span><span>file_path</span><span>.</span><span>exists</span><span>():</span><span>raise</span> <span>FileNotFoundError</span><span>(</span><span>f</span><span>"</span><span>{</span><span>self</span><span>.</span><span>file_path</span><span>}</span><span> was not found.</span><span>"</span><span>)</span><span>if</span> <span>self</span><span>.</span><span>stage</span> <span>==</span> <span>"</span><span>predict</span><span>"</span> <span>and</span> <span>self</span><span>.</span><span>file_path</span><span>.</span><span>exists</span><span>():</span><span>self</span><span>.</span><span>_transformer</span> <span>=</span> <span>pickle</span><span>.</span><span>loads</span><span>(</span><span>self</span><span>.</span><span>file_path</span><span>.</span><span>read_bytes</span><span>())</span><span>return</span> <span>self</span><span>.</span><span>_transformer</span><span>@transformer.setter</span><span>def</span> <span>transformer</span><span>(</span><span>self</span><span>,</span> <span>transformer</span><span>:</span><span>ColumnTransformer</span><span>)</span> <span>-></span> <span>None</span><span>:</span><span>self</span><span>.</span><span>file_path</span><span>.</span><span>write_bytes</span><span>(</span><span>pickle</span><span>.</span><span>dumps</span><span>(</span><span>transformer</span><span>))</span><span>def</span> <span>run</span><span>(</span><span>self</span><span>,</span> <span>data</span><span>:</span> <span>pd</span><span>.</span><span>DataFrame</span><span>)</span> <span>-></span> <span>pd</span><span>.</span><span>DataFrame</span><span>:</span><span>operations</span> <span>=</span> <span>{</span><span>"</span><span>train</span><span>"</span><span>:</span> <span>self</span><span>.</span><span>_train</span><span>,</span><span>"</span><span>predict</span><span>"</span><span>:</span> <span>self</span><span>.</span><span>_predict</span><span>}</span><span>return</span> <span>operations</span><span>.</span><span>get</span><span>(</span><span>self</span><span>.</span><span>stage</span><span>,</span> <span>lambda</span> <span>d</span><span>:</span> <span>d</span><span>)(</span><span>data</span><span>)</span><span>def</span> <span>_train</span><span>(</span><span>self</span><span>,</span> <span>data</span><span>:</span> <span>pd</span><span>.</span><span>DataFrame</span><span>):</span><span>cleaned_data</span> <span>=</span> <span>self</span><span>.</span><span>transformer</span><span>.</span><span>fit_transform</span><span>(</span><span>data</span><span>)</span><span>self</span><span>.</span><span>transformer</span> <span>=</span> <span>self</span><span>.</span><span>_transformer</span><span>return</span> <span>cleaned_data</span><span>def</span> <span>_predict</span><span>(</span><span>self</span><span>,</span> <span>data</span><span>:</span> <span>pd</span><span>.</span><span>DataFrame</span><span>):</span><span>return</span> <span>self</span><span>.</span><span>transformer</span><span>.</span><span>transform</span><span>(</span><span>data</span><span>)</span><span>import</span> <span>pickle</span> <span>from</span> <span>pathlib</span> <span>import</span> <span>Path</span> <span>from</span> <span>typing</span> <span>import</span> <span>Literal</span> <span>import</span> <span>pandas</span> <span>as</span> <span>pd</span> <span>from</span> <span>sklearn.compose</span> <span>import</span> <span>ColumnTransformer</span> <span>class</span> <span>TransformerTask</span><span>:</span> <span>def</span> <span>__init__</span><span>(</span><span>self</span><span>,</span> <span>transformer</span><span>:</span><span>ColumnTransformer</span><span>,</span> <span>stage</span><span>:</span> <span>Literal</span><span>[</span><span>"</span><span>train</span><span>"</span><span>,</span> <span>"</span><span>predict</span><span>"</span><span>]</span> <span>=</span> <span>"</span><span>train</span><span>"</span><span>,</span> <span>file_path</span><span>:</span><span>str</span><span>=</span><span>"</span><span>models/transformer.pkl</span><span>"</span><span>,):</span> <span>self</span><span>.</span><span>stage</span> <span>=</span> <span>stage</span> <span>self</span><span>.</span><span>file_path</span> <span>=</span> <span>Path</span><span>(</span><span>file_path</span><span>)</span> <span>self</span><span>.</span><span>_transformer</span> <span>=</span> <span>transformer</span> <span>@property</span> <span>def</span> <span>transformer</span><span>(</span><span>self</span><span>)</span> <span>-></span> <span>ColumnTransformer</span><span>:</span> <span>if</span> <span>self</span><span>.</span><span>stage</span> <span>==</span> <span>"</span><span>predict</span><span>"</span> <span>and</span> <span>not</span> <span>self</span><span>.</span><span>file_path</span><span>.</span><span>exists</span><span>():</span> <span>raise</span> <span>FileNotFoundError</span><span>(</span><span>f</span><span>"</span><span>{</span><span>self</span><span>.</span><span>file_path</span><span>}</span><span> was not found.</span><span>"</span><span>)</span> <span>if</span> <span>self</span><span>.</span><span>stage</span> <span>==</span> <span>"</span><span>predict</span><span>"</span> <span>and</span> <span>self</span><span>.</span><span>file_path</span><span>.</span><span>exists</span><span>():</span> <span>self</span><span>.</span><span>_transformer</span> <span>=</span> <span>pickle</span><span>.</span><span>loads</span><span>(</span><span>self</span><span>.</span><span>file_path</span><span>.</span><span>read_bytes</span><span>())</span> <span>return</span> <span>self</span><span>.</span><span>_transformer</span> <span>@transformer.setter</span> <span>def</span> <span>transformer</span><span>(</span><span>self</span><span>,</span> <span>transformer</span><span>:</span><span>ColumnTransformer</span><span>)</span> <span>-></span> <span>None</span><span>:</span> <span>self</span><span>.</span><span>file_path</span><span>.</span><span>write_bytes</span><span>(</span><span>pickle</span><span>.</span><span>dumps</span><span>(</span><span>transformer</span><span>))</span> <span>def</span> <span>run</span><span>(</span><span>self</span><span>,</span> <span>data</span><span>:</span> <span>pd</span><span>.</span><span>DataFrame</span><span>)</span> <span>-></span> <span>pd</span><span>.</span><span>DataFrame</span><span>:</span> <span>operations</span> <span>=</span> <span>{</span> <span>"</span><span>train</span><span>"</span><span>:</span> <span>self</span><span>.</span><span>_train</span><span>,</span> <span>"</span><span>predict</span><span>"</span><span>:</span> <span>self</span><span>.</span><span>_predict</span> <span>}</span> <span>return</span> <span>operations</span><span>.</span><span>get</span><span>(</span><span>self</span><span>.</span><span>stage</span><span>,</span> <span>lambda</span> <span>d</span><span>:</span> <span>d</span><span>)(</span><span>data</span><span>)</span> <span>def</span> <span>_train</span><span>(</span><span>self</span><span>,</span> <span>data</span><span>:</span> <span>pd</span><span>.</span><span>DataFrame</span><span>):</span> <span>cleaned_data</span> <span>=</span> <span>self</span><span>.</span><span>transformer</span><span>.</span><span>fit_transform</span><span>(</span><span>data</span><span>)</span> <span>self</span><span>.</span><span>transformer</span> <span>=</span> <span>self</span><span>.</span><span>_transformer</span> <span>return</span> <span>cleaned_data</span> <span>def</span> <span>_predict</span><span>(</span><span>self</span><span>,</span> <span>data</span><span>:</span> <span>pd</span><span>.</span><span>DataFrame</span><span>):</span> <span>return</span> <span>self</span><span>.</span><span>transformer</span><span>.</span><span>transform</span><span>(</span><span>data</span><span>)</span>import pickle from pathlib import Path from typing import Literal import pandas as pd from sklearn.compose import ColumnTransformer class TransformerTask: def __init__(self, transformer:ColumnTransformer, stage: Literal["train", "predict"] = "train", file_path:str="models/transformer.pkl",): self.stage = stage self.file_path = Path(file_path) self._transformer = transformer @property def transformer(self) -> ColumnTransformer: if self.stage == "predict" and not self.file_path.exists(): raise FileNotFoundError(f"{self.file_path} was not found.") if self.stage == "predict" and self.file_path.exists(): self._transformer = pickle.loads(self.file_path.read_bytes()) return self._transformer @transformer.setter def transformer(self, transformer:ColumnTransformer) -> None: self.file_path.write_bytes(pickle.dumps(transformer)) def run(self, data: pd.DataFrame) -> pd.DataFrame: operations = { "train": self._train, "predict": self._predict } return operations.get(self.stage, lambda d: d)(data) def _train(self, data: pd.DataFrame): cleaned_data = self.transformer.fit_transform(data) self.transformer = self._transformer return cleaned_data def _predict(self, data: pd.DataFrame): return self.transformer.transform(data)
Enter fullscreen mode Exit fullscreen mode
Test we must, again young Padawan …
<span>from</span> <span>sklearn.compose</span> <span>import</span> <span>make_column_transformer</span><span>from</span> <span>sklearn.preprocessing</span> <span>import</span> <span>OneHotEncoder</span><span>from</span> <span>sklearn</span> <span>import</span> <span>set_config</span><span>from</span> <span>srp</span> <span>import</span> <span>config</span><span>set_config</span><span>(</span><span>transform_output</span><span>=</span><span>"</span><span>pandas</span><span>"</span><span>)</span><span>transformer</span> <span>=</span> <span>make_column_transformer</span><span>(</span><span>(</span><span>OneHotEncoder</span><span>(</span><span>sparse_output</span><span>=</span><span>False</span><span>),[</span><span>config</span><span>.</span><span>COLUMNS_TO_ONEHOTENCODE</span><span>,</span><span>],</span><span>),</span><span>verbose_feature_names_out</span><span>=</span><span>False</span><span>,</span><span>remainder</span><span>=</span><span>"</span><span>passthrough</span><span>"</span><span>,</span><span>)</span><span>from</span> <span>sklearn.compose</span> <span>import</span> <span>make_column_transformer</span> <span>from</span> <span>sklearn.preprocessing</span> <span>import</span> <span>OneHotEncoder</span> <span>from</span> <span>sklearn</span> <span>import</span> <span>set_config</span> <span>from</span> <span>srp</span> <span>import</span> <span>config</span> <span>set_config</span><span>(</span><span>transform_output</span><span>=</span><span>"</span><span>pandas</span><span>"</span><span>)</span> <span>transformer</span> <span>=</span> <span>make_column_transformer</span><span>(</span> <span>(</span> <span>OneHotEncoder</span><span>(</span><span>sparse_output</span><span>=</span><span>False</span><span>),[</span> <span>config</span><span>.</span><span>COLUMNS_TO_ONEHOTENCODE</span><span>,</span> <span>],</span> <span>),</span> <span>verbose_feature_names_out</span><span>=</span><span>False</span><span>,</span> <span>remainder</span><span>=</span><span>"</span><span>passthrough</span><span>"</span><span>,</span> <span>)</span>from sklearn.compose import make_column_transformer from sklearn.preprocessing import OneHotEncoder from sklearn import set_config from srp import config set_config(transform_output="pandas") transformer = make_column_transformer( ( OneHotEncoder(sparse_output=False),[ config.COLUMNS_TO_ONEHOTENCODE, ], ), verbose_feature_names_out=False, remainder="passthrough", )
Enter fullscreen mode Exit fullscreen mode
Oops, now test, we can 🥹
<span>from</span> <span>pathlib</span> <span>import</span> <span>Path</span><span>import</span> <span>pytest</span><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><span>from</span> <span>tasks</span> <span>import</span> <span>TransformerTask</span><span>from</span> <span>preprocessors</span> <span>import</span> <span>transformer</span><span># URI for the dataset </span><span>URI</span> <span>=</span> <span>"</span><span>https://raw.githubusercontent.com/mwaskom/seaborn-data/master/penguins.csv</span><span>"</span><span># Model File path </span><span>MODEL_FILE</span> <span>=</span> <span>Path</span><span>(</span><span>"</span><span>models/transformer.pkl</span><span>"</span><span>)</span><span>def</span> <span>cleanup</span><span>(</span><span>file</span><span>=</span><span>MODEL_FILE</span><span>):</span><span>if</span> <span>MODEL_FILE</span><span>.</span><span>exists</span><span>():</span><span>MODEL_FILE</span><span>.</span><span>unlink</span><span>()</span><span>@pytest.fixture</span><span>(</span><span>autouse</span><span>=</span><span>True</span><span>)</span><span>def</span> <span>startup_and_teardown</span><span>():</span><span>"</span><span>Execute before and after a test run</span><span>"</span><span># Startup: remove model file </span> <span>cleanup</span><span>()</span><span>yield</span><span># Teardown : remove model file </span> <span>cleanup</span><span>()</span><span>@pytest.fixture</span><span>def</span> <span>load_data</span><span>():</span><span>yield</span> <span>pd</span><span>.</span><span>read_csv</span><span>(</span><span>URI</span><span>)</span><span>@pytest.fixture</span><span>def</span> <span>transformer_task</span><span>():</span><span>yield</span> <span>TransformerTask</span><span>(</span><span>transformer</span><span>,</span> <span>stage</span><span>=</span><span>"</span><span>train</span><span>"</span><span>)</span><span>def</span> <span>test_transformer_train</span><span>(</span><span>load_data</span><span>,</span> <span>transformer_task</span><span>):</span><span>"</span><span>Test the training stage</span><span>"</span><span>result</span> <span>=</span> <span>transformer_task</span><span>.</span><span>run</span><span>(</span><span>load_data</span><span>)</span><span>assert</span> <span>not</span> <span>result</span><span>.</span><span>empty</span><span>assert</span> <span>MODEL_FILE</span><span>.</span><span>exists</span><span>()</span><span>def</span> <span>test_transformer_predict</span><span>(</span><span>transformer_task</span><span>,</span> <span>load_data</span><span>):</span><span>"</span><span>Test the prediction stage</span><span>"</span><span># First, run the training stage to ensure the transformer is fitted and saved </span> <span>transformer_task</span><span>.</span><span>run</span><span>(</span><span>load_data</span><span>)</span><span>transformer_task</span><span>.</span><span>stage</span> <span>=</span> <span>"</span><span>predict</span><span>"</span><span># Testing prediction stage </span> <span>assert</span> <span>MODEL_FILE</span><span>.</span><span>exists</span><span>()</span><span>assert</span> <span>transformer_task</span><span>.</span><span>stage</span> <span>==</span> <span>"</span><span>predict</span><span>"</span><span>result</span> <span>=</span> <span>transformer_task</span><span>.</span><span>run</span><span>(</span><span>load_data</span><span>)</span><span>assert</span> <span>not</span> <span>result</span><span>.</span><span>empty</span><span>from</span> <span>pathlib</span> <span>import</span> <span>Path</span> <span>import</span> <span>pytest</span> <span>import</span> <span>pandas</span> <span>as</span> <span>pd</span> <span>from</span> <span>tasks</span> <span>import</span> <span>TransformerTask</span> <span>from</span> <span>preprocessors</span> <span>import</span> <span>transformer</span> <span># URI for the dataset </span><span>URI</span> <span>=</span> <span>"</span><span>https://raw.githubusercontent.com/mwaskom/seaborn-data/master/penguins.csv</span><span>"</span> <span># Model File path </span><span>MODEL_FILE</span> <span>=</span> <span>Path</span><span>(</span><span>"</span><span>models/transformer.pkl</span><span>"</span><span>)</span> <span>def</span> <span>cleanup</span><span>(</span><span>file</span><span>=</span><span>MODEL_FILE</span><span>):</span> <span>if</span> <span>MODEL_FILE</span><span>.</span><span>exists</span><span>():</span> <span>MODEL_FILE</span><span>.</span><span>unlink</span><span>()</span> <span>@pytest.fixture</span><span>(</span><span>autouse</span><span>=</span><span>True</span><span>)</span> <span>def</span> <span>startup_and_teardown</span><span>():</span> <span>"</span><span>Execute before and after a test run</span><span>"</span> <span># Startup: remove model file </span> <span>cleanup</span><span>()</span> <span>yield</span> <span># Teardown : remove model file </span> <span>cleanup</span><span>()</span> <span>@pytest.fixture</span> <span>def</span> <span>load_data</span><span>():</span> <span>yield</span> <span>pd</span><span>.</span><span>read_csv</span><span>(</span><span>URI</span><span>)</span> <span>@pytest.fixture</span> <span>def</span> <span>transformer_task</span><span>():</span> <span>yield</span> <span>TransformerTask</span><span>(</span><span>transformer</span><span>,</span> <span>stage</span><span>=</span><span>"</span><span>train</span><span>"</span><span>)</span> <span>def</span> <span>test_transformer_train</span><span>(</span><span>load_data</span><span>,</span> <span>transformer_task</span><span>):</span> <span>"</span><span>Test the training stage</span><span>"</span> <span>result</span> <span>=</span> <span>transformer_task</span><span>.</span><span>run</span><span>(</span><span>load_data</span><span>)</span> <span>assert</span> <span>not</span> <span>result</span><span>.</span><span>empty</span> <span>assert</span> <span>MODEL_FILE</span><span>.</span><span>exists</span><span>()</span> <span>def</span> <span>test_transformer_predict</span><span>(</span><span>transformer_task</span><span>,</span> <span>load_data</span><span>):</span> <span>"</span><span>Test the prediction stage</span><span>"</span> <span># First, run the training stage to ensure the transformer is fitted and saved </span> <span>transformer_task</span><span>.</span><span>run</span><span>(</span><span>load_data</span><span>)</span> <span>transformer_task</span><span>.</span><span>stage</span> <span>=</span> <span>"</span><span>predict</span><span>"</span> <span># Testing prediction stage </span> <span>assert</span> <span>MODEL_FILE</span><span>.</span><span>exists</span><span>()</span> <span>assert</span> <span>transformer_task</span><span>.</span><span>stage</span> <span>==</span> <span>"</span><span>predict</span><span>"</span> <span>result</span> <span>=</span> <span>transformer_task</span><span>.</span><span>run</span><span>(</span><span>load_data</span><span>)</span> <span>assert</span> <span>not</span> <span>result</span><span>.</span><span>empty</span>from pathlib import Path import pytest import pandas as pd from tasks import TransformerTask from preprocessors import transformer # URI for the dataset URI = "https://raw.githubusercontent.com/mwaskom/seaborn-data/master/penguins.csv" # Model File path MODEL_FILE = Path("models/transformer.pkl") def cleanup(file=MODEL_FILE): if MODEL_FILE.exists(): MODEL_FILE.unlink() @pytest.fixture(autouse=True) def startup_and_teardown(): "Execute before and after a test run" # Startup: remove model file cleanup() yield # Teardown : remove model file cleanup() @pytest.fixture def load_data(): yield pd.read_csv(URI) @pytest.fixture def transformer_task(): yield TransformerTask(transformer, stage="train") def test_transformer_train(load_data, transformer_task): "Test the training stage" result = transformer_task.run(load_data) assert not result.empty assert MODEL_FILE.exists() def test_transformer_predict(transformer_task, load_data): "Test the prediction stage" # First, run the training stage to ensure the transformer is fitted and saved transformer_task.run(load_data) transformer_task.stage = "predict" # Testing prediction stage assert MODEL_FILE.exists() assert transformer_task.stage == "predict" result = transformer_task.run(load_data) assert not result.empty
Enter fullscreen mode Exit fullscreen mode
Reflect and Contemplate:
Wield this tool with a balance to avert the shadows of accidental recursion or the echoes of unwanted side effects.
3. Decorators: The Enchanters
Summon the might of Decorators, enchanters that weave their spells to transform the behaviour of your functions or methods, bestowing upon them new realms of possibilities.
Create a decorator, we must. Allow it will, to profile bottlenecks within our functions/methods, hmmm.
Start simple, we shall. Expand later, we will, young Padawan.
<span>from</span> <span>typing</span> <span>import</span> <span>Callable</span><span>import</span> <span>numpy</span> <span>as</span> <span>np</span><span>def</span> <span>regression</span><span>(</span><span>func</span><span>:</span> <span>Callable</span><span>)</span> <span>-></span> <span>Callable</span><span>:</span><span>func</span><span>.</span><span>kind</span> <span>=</span> <span>"</span><span>regression</span><span>"</span><span>return</span> <span>func</span><span>@regression</span><span>def</span> <span>mse</span><span>(</span><span>y_true</span><span>,</span> <span>y_pred</span><span>):</span><span>return </span><span>((</span><span>y_true</span> <span>-</span> <span>y_pred</span><span>)</span><span>**</span><span>2</span><span>).</span><span>mean</span><span>()</span><span>print</span><span>((</span><span>mse</span><span>.</span><span>__name__</span><span>,</span> <span>mse</span><span>.</span><span>kind</span><span>))</span><span># ('mse', 'regression') </span><span>from</span> <span>typing</span> <span>import</span> <span>Callable</span> <span>import</span> <span>numpy</span> <span>as</span> <span>np</span> <span>def</span> <span>regression</span><span>(</span><span>func</span><span>:</span> <span>Callable</span><span>)</span> <span>-></span> <span>Callable</span><span>:</span> <span>func</span><span>.</span><span>kind</span> <span>=</span> <span>"</span><span>regression</span><span>"</span> <span>return</span> <span>func</span> <span>@regression</span> <span>def</span> <span>mse</span><span>(</span><span>y_true</span><span>,</span> <span>y_pred</span><span>):</span> <span>return </span><span>((</span><span>y_true</span> <span>-</span> <span>y_pred</span><span>)</span><span>**</span><span>2</span><span>).</span><span>mean</span><span>()</span> <span>print</span><span>((</span><span>mse</span><span>.</span><span>__name__</span><span>,</span> <span>mse</span><span>.</span><span>kind</span><span>))</span> <span># ('mse', 'regression') </span>from typing import Callable import numpy as np def regression(func: Callable) -> Callable: func.kind = "regression" return func @regression def mse(y_true, y_pred): return ((y_true - y_pred)**2).mean() print((mse.__name__, mse.kind)) # ('mse', 'regression')
Enter fullscreen mode Exit fullscreen mode
Let’s level up, doing a hyperspace jump
<span>from</span> <span>cProfile</span> <span>import</span> <span>Profile</span><span>from</span> <span>functools</span> <span>import</span> <span>wraps</span><span>import</span> <span>pstats</span><span>def</span> <span>profiler</span><span>(</span><span>func</span><span>):</span><span>@wraps</span><span>(</span><span>func</span><span>)</span><span>def</span> <span>wrapper</span><span>(</span><span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>):</span><span>with</span> <span>Profile</span><span>()</span> <span>as</span> <span>p</span><span>:</span><span># run the function and collect profile data </span> <span>results</span> <span>=</span> <span>p</span><span>.</span><span>runcall</span><span>(</span><span>func</span><span>,</span> <span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>)</span><span># sort => name, time, file https://docs.python.org/3/library/profile.html </span> <span>ps</span> <span>=</span> <span>pstats</span><span>.</span><span>Stats</span><span>(</span><span>p</span><span>,</span> <span>stream</span><span>=</span><span>None</span><span>).</span><span>sort_stats</span><span>(</span><span>'</span><span>cumulative</span><span>'</span><span>)</span><span>ps</span><span>.</span><span>print_stats</span><span>(</span><span>5</span><span>)</span> <span># restriction of print </span> <span>return</span> <span>results</span><span>return</span> <span>wrapper</span><span>@profiler</span><span>@regression</span><span>def</span> <span>mse</span><span>(</span><span>y_true</span><span>,</span> <span>y_pred</span><span>):</span><span>return </span><span>((</span><span>y_true</span> <span>-</span> <span>y_pred</span><span>)</span><span>**</span><span>2</span><span>).</span><span>mean</span><span>()</span><span>if</span> <span>__name__</span> <span>==</span> <span>"</span><span>__main__</span><span>"</span><span>:</span><span>y_true</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>([</span><span>1</span><span>,</span> <span>2</span><span>,</span> <span>3</span><span>,</span> <span>4</span><span>,</span> <span>5</span><span>])</span><span>y_pred</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>([</span><span>1</span><span>,</span> <span>3</span><span>,</span> <span>2</span><span>,</span> <span>3</span><span>,</span> <span>5</span><span>])</span><span>print</span><span>((</span><span>mse</span><span>.</span><span>__name__</span><span>,</span> <span>mse</span><span>.</span><span>kind</span><span>,</span> <span>results</span><span>))</span><span># some profile logs </span> <span># ('mse', 'regression', 0.6) </span><span>from</span> <span>cProfile</span> <span>import</span> <span>Profile</span> <span>from</span> <span>functools</span> <span>import</span> <span>wraps</span> <span>import</span> <span>pstats</span> <span>def</span> <span>profiler</span><span>(</span><span>func</span><span>):</span> <span>@wraps</span><span>(</span><span>func</span><span>)</span> <span>def</span> <span>wrapper</span><span>(</span><span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>):</span> <span>with</span> <span>Profile</span><span>()</span> <span>as</span> <span>p</span><span>:</span> <span># run the function and collect profile data </span> <span>results</span> <span>=</span> <span>p</span><span>.</span><span>runcall</span><span>(</span><span>func</span><span>,</span> <span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>)</span> <span># sort => name, time, file https://docs.python.org/3/library/profile.html </span> <span>ps</span> <span>=</span> <span>pstats</span><span>.</span><span>Stats</span><span>(</span><span>p</span><span>,</span> <span>stream</span><span>=</span><span>None</span><span>).</span><span>sort_stats</span><span>(</span><span>'</span><span>cumulative</span><span>'</span><span>)</span> <span>ps</span><span>.</span><span>print_stats</span><span>(</span><span>5</span><span>)</span> <span># restriction of print </span> <span>return</span> <span>results</span> <span>return</span> <span>wrapper</span> <span>@profiler</span> <span>@regression</span> <span>def</span> <span>mse</span><span>(</span><span>y_true</span><span>,</span> <span>y_pred</span><span>):</span> <span>return </span><span>((</span><span>y_true</span> <span>-</span> <span>y_pred</span><span>)</span><span>**</span><span>2</span><span>).</span><span>mean</span><span>()</span> <span>if</span> <span>__name__</span> <span>==</span> <span>"</span><span>__main__</span><span>"</span><span>:</span> <span>y_true</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>([</span><span>1</span><span>,</span> <span>2</span><span>,</span> <span>3</span><span>,</span> <span>4</span><span>,</span> <span>5</span><span>])</span> <span>y_pred</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>([</span><span>1</span><span>,</span> <span>3</span><span>,</span> <span>2</span><span>,</span> <span>3</span><span>,</span> <span>5</span><span>])</span> <span>print</span><span>((</span><span>mse</span><span>.</span><span>__name__</span><span>,</span> <span>mse</span><span>.</span><span>kind</span><span>,</span> <span>results</span><span>))</span> <span># some profile logs </span> <span># ('mse', 'regression', 0.6) </span>from cProfile import Profile from functools import wraps import pstats def profiler(func): @wraps(func) def wrapper(*args, **kwargs): with Profile() as p: # run the function and collect profile data results = p.runcall(func, *args, **kwargs) # sort => name, time, file https://docs.python.org/3/library/profile.html ps = pstats.Stats(p, stream=None).sort_stats('cumulative') ps.print_stats(5) # restriction of print return results return wrapper @profiler @regression def mse(y_true, y_pred): return ((y_true - y_pred)**2).mean() if __name__ == "__main__": y_true = np.array([1, 2, 3, 4, 5]) y_pred = np.array([1, 3, 2, 3, 5]) print((mse.__name__, mse.kind, results)) # some profile logs # ('mse', 'regression', 0.6)
Enter fullscreen mode Exit fullscreen mode
Another way of writing the same decorator, using class, would look like this:
<span>class</span> <span>profiler</span><span>:</span><span>def</span> <span>__init__</span><span>(</span><span>self</span><span>,</span> <span>func</span><span>):</span><span>self</span><span>.</span><span>func</span> <span>=</span> <span>func</span><span>self</span><span>.</span><span>__name__</span> <span>=</span> <span>func</span><span>.</span><span>__name__</span><span>def</span> <span>__call__</span><span>(</span><span>self</span><span>,</span> <span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>):</span><span>with</span> <span>Profile</span><span>()</span> <span>as</span> <span>p</span><span>:</span><span>results</span> <span>=</span> <span>p</span><span>.</span><span>runcall</span><span>(</span><span>self</span><span>.</span><span>func</span><span>,</span> <span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>)</span><span>ps</span> <span>=</span> <span>pstats</span><span>.</span><span>Stats</span><span>(</span><span>p</span><span>,</span> <span>stream</span><span>=</span><span>None</span><span>).</span><span>sort_stats</span><span>(</span><span>'</span><span>cumulative</span><span>'</span><span>)</span><span>ps</span><span>.</span><span>print_stats</span><span>(</span><span>5</span><span>)</span><span>return</span> <span>results</span><span>class</span> <span>profiler</span><span>:</span> <span>def</span> <span>__init__</span><span>(</span><span>self</span><span>,</span> <span>func</span><span>):</span> <span>self</span><span>.</span><span>func</span> <span>=</span> <span>func</span> <span>self</span><span>.</span><span>__name__</span> <span>=</span> <span>func</span><span>.</span><span>__name__</span> <span>def</span> <span>__call__</span><span>(</span><span>self</span><span>,</span> <span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>):</span> <span>with</span> <span>Profile</span><span>()</span> <span>as</span> <span>p</span><span>:</span> <span>results</span> <span>=</span> <span>p</span><span>.</span><span>runcall</span><span>(</span><span>self</span><span>.</span><span>func</span><span>,</span> <span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>)</span> <span>ps</span> <span>=</span> <span>pstats</span><span>.</span><span>Stats</span><span>(</span><span>p</span><span>,</span> <span>stream</span><span>=</span><span>None</span><span>).</span><span>sort_stats</span><span>(</span><span>'</span><span>cumulative</span><span>'</span><span>)</span> <span>ps</span><span>.</span><span>print_stats</span><span>(</span><span>5</span><span>)</span> <span>return</span> <span>results</span>class profiler: def __init__(self, func): self.func = func self.__name__ = func.__name__ def __call__(self, *args, **kwargs): with Profile() as p: results = p.runcall(self.func, *args, **kwargs) ps = pstats.Stats(p, stream=None).sort_stats('cumulative') ps.print_stats(5) return results
Enter fullscreen mode Exit fullscreen mode
Hark! A disturbance in the Force. Hardcoded sorting and stats printing lines. A desire to inject them, I sense. Embark on this quest, we must. Bring balance to the code, we shall. Doing that, let us.
<span># functional way </span><span>def</span> <span>profiler</span><span>(</span><span>sort_by</span><span>=</span><span>'</span><span>cumulative</span><span>'</span><span>,</span> <span>restriction</span><span>=</span><span>5</span><span>,</span> <span>streams</span><span>=</span><span>None</span><span>):</span><span>def</span> <span>inner_function</span><span>(</span><span>func</span><span>):</span><span>@wraps</span><span>(</span><span>func</span><span>)</span><span>def</span> <span>wrapper</span><span>(</span><span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>):</span><span>with</span> <span>Profile</span><span>()</span> <span>as</span> <span>p</span><span>:</span><span>result</span> <span>=</span> <span>p</span><span>.</span><span>runcall</span><span>(</span><span>func</span><span>,</span> <span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>)</span><span>ps</span> <span>=</span> <span>pstats</span><span>.</span><span>Stats</span><span>(</span><span>p</span><span>,</span> <span>stream</span><span>=</span><span>streams</span><span>).</span><span>sort_stats</span><span>(</span><span>sort_by</span><span>)</span><span>ps</span><span>.</span><span>print_stats</span><span>(</span><span>restriction</span><span>)</span><span>return</span> <span>result</span><span>return</span> <span>wrapper</span><span>return</span> <span>inner_function</span><span># class way </span><span>class</span> <span>profiler</span><span>:</span><span>def</span> <span>__init__</span><span>(</span><span>self</span><span>,</span> <span>sort_by</span><span>=</span><span>'</span><span>cumulative</span><span>'</span><span>,</span> <span>restriction</span><span>=</span><span>5</span><span>,</span> <span>streams</span><span>=</span><span>None</span><span>):</span><span>self</span><span>.</span><span>sort_by</span> <span>=</span> <span>sort_by</span><span>self</span><span>.</span><span>restriction</span> <span>=</span> <span>restriction</span><span>self</span><span>.</span><span>streams</span> <span>=</span> <span>streams</span><span>def</span> <span>__call__</span><span>(</span><span>self</span><span>,</span> <span>func</span><span>):</span><span>def</span> <span>wrapper</span><span>(</span><span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>):</span><span>with</span> <span>Profile</span><span>()</span> <span>as</span> <span>p</span><span>:</span><span>result</span> <span>=</span> <span>p</span><span>.</span><span>runcall</span><span>(</span><span>func</span><span>,</span> <span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>)</span><span>ps</span> <span>=</span> <span>pstats</span><span>.</span><span>Stats</span><span>(</span><span>p</span><span>,</span> <span>stream</span><span>=</span><span>self</span><span>.</span><span>streams</span><span>).</span><span>sort_stats</span><span>(</span><span>self</span><span>.</span><span>sort_by</span><span>)</span><span>ps</span><span>.</span><span>print_stats</span><span>(</span><span>self</span><span>.</span><span>restriction</span><span>)</span><span>return</span> <span>result</span><span>wrapper</span><span>.</span><span>__name__</span> <span>=</span> <span>func</span><span>.</span><span>__name__</span><span>return</span> <span>wrapper</span><span>@regression</span><span>@profiler</span><span>(</span><span>sort_by</span><span>=</span><span>"</span><span>time</span><span>"</span><span>,</span> <span>restriction</span><span>=</span><span>3</span><span>)</span><span>def</span> <span>mse</span><span>(</span><span>y_true</span><span>,</span> <span>y_pred</span><span>):</span><span>return </span><span>((</span><span>y_true</span> <span>-</span> <span>y_pred</span><span>)</span><span>**</span><span>2</span><span>).</span><span>mean</span><span>()</span><span>if</span> <span>__name__</span> <span>==</span> <span>"</span><span>__main__</span><span>"</span><span>:</span><span>y_true</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>([</span><span>1</span><span>,</span> <span>2</span><span>,</span> <span>3</span><span>,</span> <span>4</span><span>,</span> <span>5</span><span>])</span><span>y_pred</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>([</span><span>1</span><span>,</span> <span>3</span><span>,</span> <span>2</span><span>,</span> <span>3</span><span>,</span> <span>5</span><span>])</span><span>results</span> <span>=</span> <span>mse</span><span>(</span><span>y_true</span><span>,</span> <span>y_pred</span><span>)</span><span>print</span><span>((</span><span>mse</span><span>.</span><span>__name__</span><span>,</span> <span>mse</span><span>.</span><span>kind</span><span>,</span> <span>results</span><span>))</span><span># some profile logs are sorted by time and show 3 lines </span> <span># ('mse', 'regression', 0.6) </span><span># functional way </span><span>def</span> <span>profiler</span><span>(</span><span>sort_by</span><span>=</span><span>'</span><span>cumulative</span><span>'</span><span>,</span> <span>restriction</span><span>=</span><span>5</span><span>,</span> <span>streams</span><span>=</span><span>None</span><span>):</span> <span>def</span> <span>inner_function</span><span>(</span><span>func</span><span>):</span> <span>@wraps</span><span>(</span><span>func</span><span>)</span> <span>def</span> <span>wrapper</span><span>(</span><span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>):</span> <span>with</span> <span>Profile</span><span>()</span> <span>as</span> <span>p</span><span>:</span> <span>result</span> <span>=</span> <span>p</span><span>.</span><span>runcall</span><span>(</span><span>func</span><span>,</span> <span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>)</span> <span>ps</span> <span>=</span> <span>pstats</span><span>.</span><span>Stats</span><span>(</span><span>p</span><span>,</span> <span>stream</span><span>=</span><span>streams</span><span>).</span><span>sort_stats</span><span>(</span><span>sort_by</span><span>)</span> <span>ps</span><span>.</span><span>print_stats</span><span>(</span><span>restriction</span><span>)</span> <span>return</span> <span>result</span> <span>return</span> <span>wrapper</span> <span>return</span> <span>inner_function</span> <span># class way </span><span>class</span> <span>profiler</span><span>:</span> <span>def</span> <span>__init__</span><span>(</span><span>self</span><span>,</span> <span>sort_by</span><span>=</span><span>'</span><span>cumulative</span><span>'</span><span>,</span> <span>restriction</span><span>=</span><span>5</span><span>,</span> <span>streams</span><span>=</span><span>None</span><span>):</span> <span>self</span><span>.</span><span>sort_by</span> <span>=</span> <span>sort_by</span> <span>self</span><span>.</span><span>restriction</span> <span>=</span> <span>restriction</span> <span>self</span><span>.</span><span>streams</span> <span>=</span> <span>streams</span> <span>def</span> <span>__call__</span><span>(</span><span>self</span><span>,</span> <span>func</span><span>):</span> <span>def</span> <span>wrapper</span><span>(</span><span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>):</span> <span>with</span> <span>Profile</span><span>()</span> <span>as</span> <span>p</span><span>:</span> <span>result</span> <span>=</span> <span>p</span><span>.</span><span>runcall</span><span>(</span><span>func</span><span>,</span> <span>*</span><span>args</span><span>,</span> <span>**</span><span>kwargs</span><span>)</span> <span>ps</span> <span>=</span> <span>pstats</span><span>.</span><span>Stats</span><span>(</span><span>p</span><span>,</span> <span>stream</span><span>=</span><span>self</span><span>.</span><span>streams</span><span>).</span><span>sort_stats</span><span>(</span><span>self</span><span>.</span><span>sort_by</span><span>)</span> <span>ps</span><span>.</span><span>print_stats</span><span>(</span><span>self</span><span>.</span><span>restriction</span><span>)</span> <span>return</span> <span>result</span> <span>wrapper</span><span>.</span><span>__name__</span> <span>=</span> <span>func</span><span>.</span><span>__name__</span> <span>return</span> <span>wrapper</span> <span>@regression</span> <span>@profiler</span><span>(</span><span>sort_by</span><span>=</span><span>"</span><span>time</span><span>"</span><span>,</span> <span>restriction</span><span>=</span><span>3</span><span>)</span> <span>def</span> <span>mse</span><span>(</span><span>y_true</span><span>,</span> <span>y_pred</span><span>):</span> <span>return </span><span>((</span><span>y_true</span> <span>-</span> <span>y_pred</span><span>)</span><span>**</span><span>2</span><span>).</span><span>mean</span><span>()</span> <span>if</span> <span>__name__</span> <span>==</span> <span>"</span><span>__main__</span><span>"</span><span>:</span> <span>y_true</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>([</span><span>1</span><span>,</span> <span>2</span><span>,</span> <span>3</span><span>,</span> <span>4</span><span>,</span> <span>5</span><span>])</span> <span>y_pred</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>([</span><span>1</span><span>,</span> <span>3</span><span>,</span> <span>2</span><span>,</span> <span>3</span><span>,</span> <span>5</span><span>])</span> <span>results</span> <span>=</span> <span>mse</span><span>(</span><span>y_true</span><span>,</span> <span>y_pred</span><span>)</span> <span>print</span><span>((</span><span>mse</span><span>.</span><span>__name__</span><span>,</span> <span>mse</span><span>.</span><span>kind</span><span>,</span> <span>results</span><span>))</span> <span># some profile logs are sorted by time and show 3 lines </span> <span># ('mse', 'regression', 0.6) </span># functional way def profiler(sort_by='cumulative', restriction=5, streams=None): def inner_function(func): @wraps(func) def wrapper(*args, **kwargs): with Profile() as p: result = p.runcall(func, *args, **kwargs) ps = pstats.Stats(p, stream=streams).sort_stats(sort_by) ps.print_stats(restriction) return result return wrapper return inner_function # class way class profiler: def __init__(self, sort_by='cumulative', restriction=5, streams=None): self.sort_by = sort_by self.restriction = restriction self.streams = streams def __call__(self, func): def wrapper(*args, **kwargs): with Profile() as p: result = p.runcall(func, *args, **kwargs) ps = pstats.Stats(p, stream=self.streams).sort_stats(self.sort_by) ps.print_stats(self.restriction) return result wrapper.__name__ = func.__name__ return wrapper @regression @profiler(sort_by="time", restriction=3) def mse(y_true, y_pred): return ((y_true - y_pred)**2).mean() if __name__ == "__main__": y_true = np.array([1, 2, 3, 4, 5]) y_pred = np.array([1, 3, 2, 3, 5]) results = mse(y_true, y_pred) print((mse.__name__, mse.kind, results)) # some profile logs are sorted by time and show 3 lines # ('mse', 'regression', 0.6)
Enter fullscreen mode Exit fullscreen mode
Ah, a wise query emerges amidst the stars! Speak of adorning methods within a cosmic class, do you? Fear not, for a class decorator we shall craft. Bestow our decorator upon the class methods, it shall. In unity, traverse the celestial pathways of Python, we will!
<span>def</span> <span>profilerx</span><span>(</span><span>decorate</span><span>=</span><span>None</span><span>):</span><span>if</span> <span>decorate</span> <span>is</span> <span>None</span><span>:</span><span>decorate</span> <span>=</span> <span>lambda</span> <span>d</span><span>:</span> <span>d</span><span>def</span> <span>wrapper</span><span>(</span><span>cls</span><span>):</span><span>name_functions</span> <span>=</span> <span>{</span><span>name</span><span>:</span><span>func</span> <span>for</span> <span>name</span><span>,</span> <span>func</span> <span>in</span> <span>vars</span><span>(</span><span>cls</span><span>).</span><span>items</span><span>()</span> <span>if</span> <span>not</span> <span>name</span><span>.</span><span>startswith</span><span>(</span><span>"</span><span>__</span><span>"</span><span>)}</span><span>for</span> <span>name</span><span>,</span> <span>func</span> <span>in</span> <span>vars</span><span>(</span><span>cls</span><span>).</span><span>items</span><span>():</span><span>if</span> <span>callable</span><span>(</span><span>func</span><span>):</span><span>setattr</span><span>(</span><span>cls</span><span>,</span> <span>name</span><span>,</span> <span>decorate</span><span>(</span><span>func</span><span>))</span><span>return</span> <span>cls</span><span>wrapper</span><span>.</span><span>__name__</span> <span>=</span> <span>cls</span><span>.</span><span>__name__</span><span>return</span> <span>wrapper</span><span># Decorate our class, we can ️ </span><span>@profilerx</span><span>(</span><span>decorate</span><span>=</span><span>profiler</span><span>(</span><span>sort_by</span><span>=</span><span>"</span><span>cumulative</span><span>"</span><span>,</span> <span>restriction</span><span>=</span><span>5</span><span>))</span><span>class</span> <span>Circle</span><span>:</span><span>def</span> <span>__init__</span><span>(</span><span>self</span><span>,</span> <span>radius</span><span>:</span> <span>int</span><span>|</span><span>float</span><span>):</span><span>self</span><span>.</span><span>_radius</span> <span>=</span> <span>radius_validator</span><span>(</span><span>radius</span><span>)</span><span>...</span><span>def</span> <span>profilerx</span><span>(</span><span>decorate</span><span>=</span><span>None</span><span>):</span> <span>if</span> <span>decorate</span> <span>is</span> <span>None</span><span>:</span> <span>decorate</span> <span>=</span> <span>lambda</span> <span>d</span><span>:</span> <span>d</span> <span>def</span> <span>wrapper</span><span>(</span><span>cls</span><span>):</span> <span>name_functions</span> <span>=</span> <span>{</span><span>name</span><span>:</span><span>func</span> <span>for</span> <span>name</span><span>,</span> <span>func</span> <span>in</span> <span>vars</span><span>(</span><span>cls</span><span>).</span><span>items</span><span>()</span> <span>if</span> <span>not</span> <span>name</span><span>.</span><span>startswith</span><span>(</span><span>"</span><span>__</span><span>"</span><span>)}</span> <span>for</span> <span>name</span><span>,</span> <span>func</span> <span>in</span> <span>vars</span><span>(</span><span>cls</span><span>).</span><span>items</span><span>():</span> <span>if</span> <span>callable</span><span>(</span><span>func</span><span>):</span> <span>setattr</span><span>(</span><span>cls</span><span>,</span> <span>name</span><span>,</span> <span>decorate</span><span>(</span><span>func</span><span>))</span> <span>return</span> <span>cls</span> <span>wrapper</span><span>.</span><span>__name__</span> <span>=</span> <span>cls</span><span>.</span><span>__name__</span> <span>return</span> <span>wrapper</span> <span># Decorate our class, we can ️ </span> <span>@profilerx</span><span>(</span><span>decorate</span><span>=</span><span>profiler</span><span>(</span><span>sort_by</span><span>=</span><span>"</span><span>cumulative</span><span>"</span><span>,</span> <span>restriction</span><span>=</span><span>5</span><span>))</span> <span>class</span> <span>Circle</span><span>:</span> <span>def</span> <span>__init__</span><span>(</span><span>self</span><span>,</span> <span>radius</span><span>:</span> <span>int</span><span>|</span><span>float</span><span>):</span> <span>self</span><span>.</span><span>_radius</span> <span>=</span> <span>radius_validator</span><span>(</span><span>radius</span><span>)</span> <span>...</span>def profilerx(decorate=None): if decorate is None: decorate = lambda d: d def wrapper(cls): name_functions = {name:func for name, func in vars(cls).items() if not name.startswith("__")} for name, func in vars(cls).items(): if callable(func): setattr(cls, name, decorate(func)) return cls wrapper.__name__ = cls.__name__ return wrapper # Decorate our class, we can ️ @profilerx(decorate=profiler(sort_by="cumulative", restriction=5)) class Circle: def __init__(self, radius: int|float): self._radius = radius_validator(radius) ...
Enter fullscreen mode Exit fullscreen mode
Reflect and Contemplate:
Embrace this magic with mindfulness, for within its allure, lies the labyrinth of complexity, whispering tales of code entangled in its own enchantment.
4. Metaprogramming: The Arcane Arts
As we traverse the realms of decorators, we find ourselves at the gateway to Metaprogramming, a universe where code breathes life into more code. Imagine a scenario where the reins of our crafted code slip out of our grasp, handed over to realms and teams beyond our dominion. In this abyss, where control eludes our touch, how do we ensure our shields are not tampered with? Behold the luminescence of metaprogramming, a beacon in the shadows of constraint and order.
Envision a creation of our own, a code entity named RejectPrint, destined to embark upon a voyage to distant teams and uses. As it journeys beyond our realm, we yearn to engrave a solemn vow upon its essence – the exile of the print command, guarding the sanctity of silence amidst its ventures in the unknown.
<span># our modulex.py </span><span>import</span> <span>logging</span><span>import</span> <span>inspect</span><span>from</span> <span>typing</span> <span>import</span> <span>Callable</span><span>def</span> <span>check_bad_usage</span><span>(</span><span>name</span><span>:</span> <span>str</span><span>,</span> <span>obj</span><span>:</span> <span>Callable</span><span>)</span><span>-></span> <span>bool</span><span>:</span><span>"""</span><span> checks if an object has a function with certain name </span><span>"""</span><span>func_names</span> <span>=</span> <span>obj</span><span>.</span><span>__code__</span><span>.</span><span>co_names</span><span>is_bad_usage</span> <span>=</span> <span>False</span><span>if</span> <span>name</span> <span>not</span> <span>in</span> <span>func_names</span><span>:</span><span>return</span> <span>is_bad_usage</span><span>is_bad_usage</span> <span>=</span> <span>True</span><span>for</span> <span>func_name</span> <span>in</span> <span>func_names</span><span>:</span><span>if</span> <span>name</span> <span>==</span> <span>func_name</span><span>:</span><span># echo the code with bad usage </span> <span>logging</span><span>.</span><span>error</span><span>(</span><span>inspect</span><span>.</span><span>getsource</span><span>(</span><span>obj</span><span>.</span><span>__code__</span><span>))</span><span>return</span> <span>is_bad_usage</span><span>class</span> <span>RejectPrintMeta</span><span>(</span><span>type</span><span>):</span><span>def</span> <span>__new__</span><span>(</span><span>cls</span><span>,</span> <span>name</span><span>,</span> <span>bases</span><span>,</span> <span>body</span><span>):</span><span>callable_obj</span> <span>=</span> <span>(</span><span>v</span> <span>for</span> <span>v</span> <span>in</span> <span>body</span><span>.</span><span>values</span><span>()</span> <span>if</span> <span>callable</span><span>(</span><span>v</span><span>))</span><span>for</span> <span>obj</span> <span>in</span> <span>callable_obj</span><span>:</span><span>is_bad_usage</span> <span>=</span> <span>check_bad_usage</span><span>(</span><span>"</span><span>print</span><span>"</span><span>,</span> <span>obj</span><span>)</span><span>if</span> <span>is_bad_usage</span><span>:</span><span>raise</span> <span>UserWarning</span><span>(</span><span>f</span><span>"</span><span>No way, `</span><span>{</span><span>obj</span><span>.</span><span>__name__</span><span>}</span><span>` contains `print` function!</span><span>"</span><span>)</span><span>return</span> <span>super</span><span>().</span><span>__new__</span><span>(</span><span>cls</span><span>,</span> <span>name</span><span>,</span> <span>bases</span><span>,</span> <span>body</span><span>)</span><span>class</span> <span>RejectPrint</span><span>(</span><span>metaclass</span><span>=</span><span>RejectPrintMeta</span><span>):</span><span>pass</span><span># our modulex.py </span><span>import</span> <span>logging</span> <span>import</span> <span>inspect</span> <span>from</span> <span>typing</span> <span>import</span> <span>Callable</span> <span>def</span> <span>check_bad_usage</span><span>(</span><span>name</span><span>:</span> <span>str</span><span>,</span> <span>obj</span><span>:</span> <span>Callable</span><span>)</span><span>-></span> <span>bool</span><span>:</span> <span>"""</span><span> checks if an object has a function with certain name </span><span>"""</span> <span>func_names</span> <span>=</span> <span>obj</span><span>.</span><span>__code__</span><span>.</span><span>co_names</span> <span>is_bad_usage</span> <span>=</span> <span>False</span> <span>if</span> <span>name</span> <span>not</span> <span>in</span> <span>func_names</span><span>:</span> <span>return</span> <span>is_bad_usage</span> <span>is_bad_usage</span> <span>=</span> <span>True</span> <span>for</span> <span>func_name</span> <span>in</span> <span>func_names</span><span>:</span> <span>if</span> <span>name</span> <span>==</span> <span>func_name</span><span>:</span> <span># echo the code with bad usage </span> <span>logging</span><span>.</span><span>error</span><span>(</span><span>inspect</span><span>.</span><span>getsource</span><span>(</span><span>obj</span><span>.</span><span>__code__</span><span>))</span> <span>return</span> <span>is_bad_usage</span> <span>class</span> <span>RejectPrintMeta</span><span>(</span><span>type</span><span>):</span> <span>def</span> <span>__new__</span><span>(</span><span>cls</span><span>,</span> <span>name</span><span>,</span> <span>bases</span><span>,</span> <span>body</span><span>):</span> <span>callable_obj</span> <span>=</span> <span>(</span><span>v</span> <span>for</span> <span>v</span> <span>in</span> <span>body</span><span>.</span><span>values</span><span>()</span> <span>if</span> <span>callable</span><span>(</span><span>v</span><span>))</span> <span>for</span> <span>obj</span> <span>in</span> <span>callable_obj</span><span>:</span> <span>is_bad_usage</span> <span>=</span> <span>check_bad_usage</span><span>(</span><span>"</span><span>print</span><span>"</span><span>,</span> <span>obj</span><span>)</span> <span>if</span> <span>is_bad_usage</span><span>:</span> <span>raise</span> <span>UserWarning</span><span>(</span><span>f</span><span>"</span><span>No way, `</span><span>{</span><span>obj</span><span>.</span><span>__name__</span><span>}</span><span>` contains `print` function!</span><span>"</span><span>)</span> <span>return</span> <span>super</span><span>().</span><span>__new__</span><span>(</span><span>cls</span><span>,</span> <span>name</span><span>,</span> <span>bases</span><span>,</span> <span>body</span><span>)</span> <span>class</span> <span>RejectPrint</span><span>(</span><span>metaclass</span><span>=</span><span>RejectPrintMeta</span><span>):</span> <span>pass</span># our modulex.py import logging import inspect from typing import Callable def check_bad_usage(name: str, obj: Callable)-> bool: """ checks if an object has a function with certain name """ func_names = obj.__code__.co_names is_bad_usage = False if name not in func_names: return is_bad_usage is_bad_usage = True for func_name in func_names: if name == func_name: # echo the code with bad usage logging.error(inspect.getsource(obj.__code__)) return is_bad_usage class RejectPrintMeta(type): def __new__(cls, name, bases, body): callable_obj = (v for v in body.values() if callable(v)) for obj in callable_obj: is_bad_usage = check_bad_usage("print", obj) if is_bad_usage: raise UserWarning(f"No way, `{obj.__name__}` contains `print` function!") return super().__new__(cls, name, bases, body) class RejectPrint(metaclass=RejectPrintMeta): pass
Enter fullscreen mode Exit fullscreen mode
As users employ our package, a silent guardian emerges at the birth of their classes. The use of print is gently yet firmly barred, ensuring unbroken harmony in the world of our tool’s operation.
<span># example.py # from modulex import RejectPrint </span><span>class</span> <span>UserPrint</span><span>(</span><span>RejectPrint</span><span>):</span><span>def</span> <span>no_print_used</span><span>(</span><span>self</span><span>):</span><span>pass</span><span>def</span> <span>print_used</span><span>(</span><span>self</span><span>):</span><span>print</span><span>(</span><span>'</span><span>using print :-o</span><span>'</span><span>)</span><span># example.py # from modulex import RejectPrint </span> <span>class</span> <span>UserPrint</span><span>(</span><span>RejectPrint</span><span>):</span> <span>def</span> <span>no_print_used</span><span>(</span><span>self</span><span>):</span> <span>pass</span> <span>def</span> <span>print_used</span><span>(</span><span>self</span><span>):</span> <span>print</span><span>(</span><span>'</span><span>using print :-o</span><span>'</span><span>)</span># example.py # from modulex import RejectPrint class UserPrint(RejectPrint): def no_print_used(self): pass def print_used(self): print('using print :-o')
Enter fullscreen mode Exit fullscreen mode
Reflect and Contemplate:
Tread with care in the realm of metaprogramming, for the powers it unleashes, while potent, whisper the tales of complexity and enigma.
Conclusion: Beginning of the Start
I incorporated these techniques in my repertoire, as a young Padawan. I watched the doors to Python mastery unfold before me. It is my hope that you continue to explore, learn, and adapt, and let the Force guide your own enlightening journey.
And so, the odyssey through the realms of advanced Python concludes yet begins. May your code flourish in elegance and efficiency, nurtured by the seeds of wisdom sown today.
Until then, may the force keep you coding.
Note: Generators, context managers, and plugins are additional Jedi tools that I’ve had to set aside for this post to avoid making it overly lengthy. If your curiosity is piqued and you desire to explore these realms further, do let me know. A sequel awaits the beckoning of your interest, ready to guide you further on this path to Python mastery.
© 版权声明
THE END
暂无评论内容