Mastering SQLAlchemy (4 Part Series)
1 Databases in Python Made Easy with SQLAlchemy
2 Implement ORM Data Models with SQLAlchemy
3 Queries as Python Code with SQLAlchemy’s Expression Language
4 Define Relationships Between SQLAlchemy Data Models
Our SQLAlchemy journey thus far has covered managing database connections and model creation. Yet, how do we extract the data we want from our database?
SQLAlchemy’s ORM query API simplifies the way we write database queries. Instead of writing raw SQL queries, we can construct queries on our SQLAlchemy session by chaining together methods to retrieve data. We’re going to dive into SQLAlchemy’s extensive query API to get an idea of all the ways we can query our data.
Create a Session
We covered SQLAlchemy session creation in the previous post and explained the concept of engines in the post before that. If you skipped those posts, don’t. The below is copy+pasta courtesy:
<span>"""Database engine & session creation."""</span><span>from</span> <span>sqlalchemy</span> <span>import</span> <span>create_engine</span><span>from</span> <span>sqlalchemy.orm</span> <span>import</span> <span>sessionmaker</span><span>engine</span> <span>=</span> <span>create_engine</span><span>(</span><span>'mysql+pymysql://user:password@host:3600/database'</span><span>,</span><span>echo</span><span>=</span><span>True</span><span>)</span><span>Session</span> <span>=</span> <span>sessionmaker</span><span>(</span><span>bind</span><span>=</span><span>engine</span><span>)</span><span>session</span> <span>=</span> <span>Session</span><span>()</span><span>"""Database engine & session creation."""</span> <span>from</span> <span>sqlalchemy</span> <span>import</span> <span>create_engine</span> <span>from</span> <span>sqlalchemy.orm</span> <span>import</span> <span>sessionmaker</span> <span>engine</span> <span>=</span> <span>create_engine</span><span>(</span> <span>'mysql+pymysql://user:password@host:3600/database'</span><span>,</span> <span>echo</span><span>=</span><span>True</span> <span>)</span> <span>Session</span> <span>=</span> <span>sessionmaker</span><span>(</span><span>bind</span><span>=</span><span>engine</span><span>)</span> <span>session</span> <span>=</span> <span>Session</span><span>()</span>"""Database engine & session creation.""" from sqlalchemy import create_engine from sqlalchemy.orm import sessionmaker engine = create_engine( 'mysql+pymysql://user:password@host:3600/database', echo=True ) Session = sessionmaker(bind=engine) session = Session()
Enter fullscreen mode Exit fullscreen mode database.py
Basic Query Syntax
Let’s quickly become familiar with the basic structure of SQLAlchemy’s query API. SQLAlchemy session objects have a query()
method which accepts the raw class of a data model we’ve previously defined. Below are the humble beginnings of a query to run on Customer
model; or in other words, a query on the customers SQL table:
<span>"""Construct database queries from SQLAlchemy sessions."""</span><span>from</span> <span>.database</span> <span>import</span> <span>session</span><span>from</span> <span>.models</span> <span>import</span> <span>Customer</span><span># Example structure of an ORM query </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span><span>.</span><span>FUNCTION</span><span>()</span><span>"""Construct database queries from SQLAlchemy sessions."""</span> <span>from</span> <span>.database</span> <span>import</span> <span>session</span> <span>from</span> <span>.models</span> <span>import</span> <span>Customer</span> <span># Example structure of an ORM query </span><span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span> <span>.</span><span>FUNCTION</span><span>()</span>"""Construct database queries from SQLAlchemy sessions.""" from .database import session from .models import Customer # Example structure of an ORM query records = session .query(Customer) .FUNCTION()
Enter fullscreen mode Exit fullscreen mode Select records from a SQLAlchemy session
Calling .query(Customer)
on our session isn’t a valid query until we add one more method to the chain. All session queries end with a final method to shape/anticipate the result(s) of our query:
-
all()
will return all records which match our query as a list of objects. If we were to use all on the query above, we would receive all customer records with the Python data typeList[Customer]
. -
first()
returns the first record matching our query, despite how many records match the query (what constitutes “first” depends on how your table is sorted). This is the equivalent of addingLIMIT 1
to a SQL query. As a result, the Python type to be returned would beCustomer
. -
one()
is extremely useful for cases where a maximum of one record should exist for the query we’re executing (think of querying by primary key). This syntax is notably useful when verifying whether or not a record exists prior to creating one. -
scalar()
returns a single value if one exists, None if no values exist, or raises an exception if multiple records are returned. -
get([VALUE(S)])
searches against a model’s primary key to return rows where the primary key is equal to the value provided.get()
also accepts tuples in the event that multiple foreign keys should be searched. Lastly,get()
can also accept a dictionary and return rows where the columns (dictionary keys) match the values provided.
To create more complex queries, we’d add to our query by chaining methods on our original query:
<span>"""Construct database queries from SQLAlchemy sessions."""</span><span>from</span> <span>.database</span> <span>import</span> <span>session</span><span>from</span> <span>.models</span> <span>import</span> <span>Customer</span><span># Example structure of an ORM query </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span><span>.</span><span>METHOD_1</span><span>()</span><span>.</span><span>METHOD_2</span><span>()</span><span>.</span><span>FUNCTION</span><span>()</span><span>"""Construct database queries from SQLAlchemy sessions."""</span> <span>from</span> <span>.database</span> <span>import</span> <span>session</span> <span>from</span> <span>.models</span> <span>import</span> <span>Customer</span> <span># Example structure of an ORM query </span><span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span> <span>.</span><span>METHOD_1</span><span>()</span> <span>.</span><span>METHOD_2</span><span>()</span> <span>.</span><span>FUNCTION</span><span>()</span>"""Construct database queries from SQLAlchemy sessions.""" from .database import session from .models import Customer # Example structure of an ORM query records = session .query(Customer) .METHOD_1() .METHOD_2() .FUNCTION()
Enter fullscreen mode Exit fullscreen mode Complex SELECT query
Query Results
If we execute a query that returns multiple records, we’ll need to loop through them to see the results:
<span>"""Construct database queries from SQLAlchemy sessions."""</span><span>from</span> <span>.database</span> <span>import</span> <span>session</span><span>from</span> <span>.models</span> <span>import</span> <span>Customer</span><span># Fetch all customer records </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span><span>.</span><span>all</span><span>()</span><span># Loop over records </span><span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span><span>print</span><span>(</span><span>record</span><span>)</span><span>"""Construct database queries from SQLAlchemy sessions."""</span> <span>from</span> <span>.database</span> <span>import</span> <span>session</span> <span>from</span> <span>.models</span> <span>import</span> <span>Customer</span> <span># Fetch all customer records </span><span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span> <span>.</span><span>all</span><span>()</span> <span># Loop over records </span><span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span> <span>print</span><span>(</span><span>record</span><span>)</span>"""Construct database queries from SQLAlchemy sessions.""" from .database import session from .models import Customer # Fetch all customer records records = session .query(Customer) .all() # Loop over records for record in records: print(record)
Enter fullscreen mode Exit fullscreen mode Fetch all customer records and print the output
The SQLAlchemy ORM will return an instance of a class by default, which means the above will result in the following output:
<Customer model 1><Customer model 2><Customer model 3><Customer model 4><Customer model 5><Customer model 6><Customer model 7><Customer model 8><Customer model 9><Customer model 10><Customer model 1> <Customer model 2> <Customer model 3> <Customer model 4> <Customer model 5> <Customer model 6> <Customer model 7> <Customer model 8> <Customer model 9> <Customer model 10><Customer model 1> <Customer model 2> <Customer model 3> <Customer model 4> <Customer model 5> <Customer model 6> <Customer model 7> <Customer model 8> <Customer model 9> <Customer model 10>
Enter fullscreen mode Exit fullscreen mode Output of a session query
If you’re looking to get dictionaries instead, use the built-in __dict__
method:
<span>...</span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span><span>.</span><span>all</span><span>()</span><span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span><span>pp</span><span>.</span><span>pprint</span><span>(</span><span>record</span><span>.</span> <span>__dict__</span> <span>)</span><span>...</span> <span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span> <span>.</span><span>all</span><span>()</span> <span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span> <span>pp</span><span>.</span><span>pprint</span><span>(</span><span>record</span><span>.</span> <span>__dict__</span> <span>)</span>... records = session .query(Customer) .all() for record in records: pp.pprint(record. __dict__ )
Enter fullscreen mode Exit fullscreen mode View query results as dictionaries
This instead returns dictionary objects for each row:
<span>{</span> <span>'_sa_instance_state'</span>: <sqlalchemy.orm.state.InstanceState object at 0x7f98c8221748>,<span>'email'</span>: <span>'kpaladini5i@senate.gov'</span>,<span>'first_name'</span>: <span>'Kenna'</span>,<span>'id'</span>: 199,<span>'join_date'</span>: datetime.datetime<span>(</span>2019, 4, 19, 0, 0<span>)</span>,<span>'last_name'</span>: <span>'Paladini'</span>,<span>'preferred_language'</span>: <span>'Bulgarian'</span><span>}</span><span>{</span> <span>'_sa_instance_state'</span>: <sqlalchemy.orm.state.InstanceState object at 0x7f9918192d68>,<span>'email'</span>: <span>'rlebrun5j@narod.ru'</span>,<span>'first_name'</span>: <span>'Rriocard'</span>,<span>'id'</span>: 200,<span>'join_date'</span>: datetime.datetime<span>(</span>2015, 6, 8, 0, 0<span>)</span>,<span>'last_name'</span>: <span>'Le Brun'</span>,<span>'preferred_language'</span>: <span>'Khmer'</span><span>}</span>,...<span>{</span> <span>'_sa_instance_state'</span>: <sqlalchemy.orm.state.InstanceState object at 0x7f98c8221748>, <span>'email'</span>: <span>'kpaladini5i@senate.gov'</span>, <span>'first_name'</span>: <span>'Kenna'</span>, <span>'id'</span>: 199, <span>'join_date'</span>: datetime.datetime<span>(</span>2019, 4, 19, 0, 0<span>)</span>, <span>'last_name'</span>: <span>'Paladini'</span>, <span>'preferred_language'</span>: <span>'Bulgarian'</span><span>}</span> <span>{</span> <span>'_sa_instance_state'</span>: <sqlalchemy.orm.state.InstanceState object at 0x7f9918192d68>, <span>'email'</span>: <span>'rlebrun5j@narod.ru'</span>, <span>'first_name'</span>: <span>'Rriocard'</span>, <span>'id'</span>: 200, <span>'join_date'</span>: datetime.datetime<span>(</span>2015, 6, 8, 0, 0<span>)</span>, <span>'last_name'</span>: <span>'Le Brun'</span>, <span>'preferred_language'</span>: <span>'Khmer'</span><span>}</span>, ...{ '_sa_instance_state': <sqlalchemy.orm.state.InstanceState object at 0x7f98c8221748>, 'email': 'kpaladini5i@senate.gov', 'first_name': 'Kenna', 'id': 199, 'join_date': datetime.datetime(2019, 4, 19, 0, 0), 'last_name': 'Paladini', 'preferred_language': 'Bulgarian'} { '_sa_instance_state': <sqlalchemy.orm.state.InstanceState object at 0x7f9918192d68>, 'email': 'rlebrun5j@narod.ru', 'first_name': 'Rriocard', 'id': 200, 'join_date': datetime.datetime(2015, 6, 8, 0, 0), 'last_name': 'Le Brun', 'preferred_language': 'Khmer'}, ...
Enter fullscreen mode Exit fullscreen mode Output of a session query as dictionaries
Of course, you could also create your own object instead to receive only the columns you want/need:
<span>...</span><span># Fetch all customers </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>Customer</span><span>).</span><span>all</span><span>()</span><span># Loop through records </span><span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span><span>recordObject</span> <span>=</span> <span>{</span><span>'name'</span><span>:</span> <span>record</span><span>.</span><span>name</span><span>,</span><span>'position'</span><span>:</span> <span>record</span><span>.</span><span>position</span><span>,</span><span>'team_name'</span><span>:</span> <span>record</span><span>.</span><span>team</span><span>.</span><span>name</span><span>,</span><span>'team_city'</span><span>:</span> <span>record</span><span>.</span><span>team</span><span>.</span><span>city</span><span>}</span><span>print</span><span>(</span><span>recordObject</span><span>)</span><span>...</span> <span># Fetch all customers </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>Customer</span><span>).</span><span>all</span><span>()</span> <span># Loop through records </span><span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span> <span>recordObject</span> <span>=</span> <span>{</span> <span>'name'</span><span>:</span> <span>record</span><span>.</span><span>name</span><span>,</span> <span>'position'</span><span>:</span> <span>record</span><span>.</span><span>position</span><span>,</span> <span>'team_name'</span><span>:</span> <span>record</span><span>.</span><span>team</span><span>.</span><span>name</span><span>,</span> <span>'team_city'</span><span>:</span> <span>record</span><span>.</span><span>team</span><span>.</span><span>city</span> <span>}</span> <span>print</span><span>(</span><span>recordObject</span><span>)</span>... # Fetch all customers records = session.query(Customer).all() # Loop through records for record in records: recordObject = { 'name': record.name, 'position': record.position, 'team_name': record.team.name, 'team_city': record.team.city } print(recordObject)
Enter fullscreen mode Exit fullscreen mode Deserialize results of a query
This outputs something a bit cleaner:
<span>{</span> <span>'email'</span>: <span>'kpaladini5i@senate.gov'</span>,<span>'first_name'</span>: <span>'Kenna'</span>,<span>'join_date'</span>: datetime.datetime<span>(</span>2019, 4, 19, 0, 0<span>)</span>,<span>'last_name'</span>: <span>'Paladini'</span>,<span>'preferred_language'</span>: <span>'Bulgarian'</span><span>}</span><span>{</span> <span>'email'</span>: <span>'rlebrun5j@narod.ru'</span>,<span>'first_name'</span>: <span>'Rriocard'</span>,<span>'join_date'</span>: datetime.datetime<span>(</span>2015, 6, 8, 0, 0<span>)</span>,<span>'last_name'</span>: <span>'Le Brun'</span>,<span>'preferred_language'</span>: <span>'Khmer'</span><span>}</span>,...<span>{</span> <span>'email'</span>: <span>'kpaladini5i@senate.gov'</span>, <span>'first_name'</span>: <span>'Kenna'</span>, <span>'join_date'</span>: datetime.datetime<span>(</span>2019, 4, 19, 0, 0<span>)</span>, <span>'last_name'</span>: <span>'Paladini'</span>, <span>'preferred_language'</span>: <span>'Bulgarian'</span><span>}</span> <span>{</span> <span>'email'</span>: <span>'rlebrun5j@narod.ru'</span>, <span>'first_name'</span>: <span>'Rriocard'</span>, <span>'join_date'</span>: datetime.datetime<span>(</span>2015, 6, 8, 0, 0<span>)</span>, <span>'last_name'</span>: <span>'Le Brun'</span>, <span>'preferred_language'</span>: <span>'Khmer'</span><span>}</span>, ...{ 'email': 'kpaladini5i@senate.gov', 'first_name': 'Kenna', 'join_date': datetime.datetime(2019, 4, 19, 0, 0), 'last_name': 'Paladini', 'preferred_language': 'Bulgarian'} { 'email': 'rlebrun5j@narod.ru', 'first_name': 'Rriocard', 'join_date': datetime.datetime(2015, 6, 8, 0, 0), 'last_name': 'Le Brun', 'preferred_language': 'Khmer'}, ...
Enter fullscreen mode Exit fullscreen mode Output of query deserialization
Filtering Results
Probably the most common method you’ll use on a query is the filter()
method. filter()
is the equivalent of a SQL WHERE clause to return only rows that match the criteria we want:
<span>...</span><span># Fetch records where `first_name` is `Carl` </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span><span>.</span><span>filter</span><span>(</span><span>Customer</span><span>)</span><span>.</span><span>first_name</span> <span>==</span> <span>'Carl'</span><span>)</span><span>.</span><span>all</span><span>()</span><span>...</span> <span># Fetch records where `first_name` is `Carl` </span><span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span> <span>.</span><span>filter</span><span>(</span><span>Customer</span><span>)</span> <span>.</span><span>first_name</span> <span>==</span> <span>'Carl'</span><span>)</span> <span>.</span><span>all</span><span>()</span>... # Fetch records where `first_name` is `Carl` records = session .query(Customer) .filter(Customer) .first_name == 'Carl') .all()
Enter fullscreen mode Exit fullscreen mode Select all customers named Carl
filter_by()
We could write the above query using the filter_by()
method instead like so:
<span>...</span><span># Fetch records where `first_name` is `Carl` </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span><span>.</span><span>filter_by</span><span>(</span><span>first_name</span><span>=</span><span>"Carl"</span><span>)</span><span>.</span><span>all</span><span>()</span><span>...</span> <span># Fetch records where `first_name` is `Carl` </span><span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span> <span>.</span><span>filter_by</span><span>(</span><span>first_name</span><span>=</span><span>"Carl"</span><span>)</span> <span>.</span><span>all</span><span>()</span>... # Fetch records where `first_name` is `Carl` records = session .query(Customer) .filter_by(first_name="Carl") .all()
Enter fullscreen mode Exit fullscreen mode Filter with
filter_by
Unlike filter()
, filter_by()
accepts keyword arguments (note the difference in syntax here: filter()
checks a conditional against a column object whereas filter_by()
finds columns that match the arguments we pass). filter_by()
can only search for exact values and serves as a kind of shorthand for simple filtering queries.
like()
We can do more than filter on simple conditionals. SQLAlchemy has a like()
method which works in an equivalent manner to SQL’s LIKE
:
<span>...</span><span># Fetch records where `first_name` begins with the letter `J` </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span><span>.</span><span>filter</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span><span>.</span><span>like</span><span>(</span><span>'J%'</span><span>))</span><span>.</span><span>all</span><span>()</span><span>...</span> <span># Fetch records where `first_name` begins with the letter `J` </span><span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span> <span>.</span><span>filter</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span><span>.</span><span>like</span><span>(</span><span>'J%'</span><span>))</span> <span>.</span><span>all</span><span>()</span>... # Fetch records where `first_name` begins with the letter `J` records = session .query(Customer) .filter(Customer.first_name.like('J%')) .all()
Enter fullscreen mode Exit fullscreen mode Select customer records where first names begin with “J”
As expected, this will give us all rows where the customer’s first name starts with a J :
<span>{</span> <span>'email'</span>: <span>'jpugsley9@netvibes.com'</span>,<span>'first_name'</span>: <span>'Jarid'</span>,<span>'join_date'</span>: datetime.datetime<span>(</span>2017, 10, 11, 0, 0<span>)</span>,<span>'last_name'</span>: <span>'Pugsley'</span>,<span>'preferred_language'</span>: <span>'Burmese'</span><span>}</span><span>{</span> <span>'email'</span>: <span>'jdymockek@is.gd'</span>,<span>'first_name'</span>: <span>'Jeanna'</span>,<span>'join_date'</span>: datetime.datetime<span>(</span>2017, 11, 13, 0, 0<span>)</span>,<span>'last_name'</span>: <span>'Dymocke'</span>,<span>'preferred_language'</span>: <span>'Malayalam'</span><span>}</span>...<span>{</span> <span>'email'</span>: <span>'jpugsley9@netvibes.com'</span>, <span>'first_name'</span>: <span>'Jarid'</span>, <span>'join_date'</span>: datetime.datetime<span>(</span>2017, 10, 11, 0, 0<span>)</span>, <span>'last_name'</span>: <span>'Pugsley'</span>, <span>'preferred_language'</span>: <span>'Burmese'</span><span>}</span> <span>{</span> <span>'email'</span>: <span>'jdymockek@is.gd'</span>, <span>'first_name'</span>: <span>'Jeanna'</span>, <span>'join_date'</span>: datetime.datetime<span>(</span>2017, 11, 13, 0, 0<span>)</span>, <span>'last_name'</span>: <span>'Dymocke'</span>, <span>'preferred_language'</span>: <span>'Malayalam'</span><span>}</span> ...{ 'email': 'jpugsley9@netvibes.com', 'first_name': 'Jarid', 'join_date': datetime.datetime(2017, 10, 11, 0, 0), 'last_name': 'Pugsley', 'preferred_language': 'Burmese'} { 'email': 'jdymockek@is.gd', 'first_name': 'Jeanna', 'join_date': datetime.datetime(2017, 11, 13, 0, 0), 'last_name': 'Dymocke', 'preferred_language': 'Malayalam'} ...
Enter fullscreen mode Exit fullscreen mode Output
High-level Query Methods
In addition to filter()
, there are a few basic methods we should be familiar with. Each of these corresponds to SQL keywords you’re probably familiar with:
-
limit([INTEGER])
: Limits the number of rows to a maximum of the number provided. -
order_by([COLUMN])
: Sorts results by the provided column. -
offset([INTEGER])
: Begins the query at row n.
This next part involves executing JOIN queries between models, which requires us to define relationships on our models first. Things are a bit out of order at the moment, as I actually don’t cover this until the next post. Sorry for the mess, I’m working on it!
Performing Joins & Unions
We’ve touched on JOINs a bit previously, but we’re about to kick it up a notch. We have two data models we’re working with: one for customers, and one for orders. Each customer
<span>...</span><span>import</span> <span>pprint</span><span>from</span> <span>.models</span> <span>import</span> <span>Order</span><span>,</span> <span>Customer</span><span>pp</span> <span>=</span> <span>pprint</span><span>.</span><span>PrettyPrinter</span><span>(</span><span>indent</span><span>=</span><span>4</span><span>)</span><span># Execute a SELECT query on JOINed tables </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span><span>.</span><span>join</span><span>(</span><span>Order</span><span>,</span> <span>Order</span><span>.</span><span>customer_id</span> <span>==</span> <span>Customer</span><span>.</span><span>id</span><span>)</span><span>.</span><span>all</span><span>()</span><span># Loop through results </span><span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span><span>record_object</span> <span>=</span> <span>{</span><span>'first_name'</span><span>:</span> <span>record</span><span>.</span><span>first_name</span><span>,</span><span>'last_name'</span><span>:</span> <span>record</span><span>.</span><span>last_name</span><span>,</span><span>'email'</span><span>:</span> <span>record</span><span>.</span><span>email</span><span>,</span><span>'preferred_language'</span><span>:</span> <span>record</span><span>.</span><span>preferred_language</span><span>,</span><span>'join_date'</span><span>:</span> <span>record</span><span>.</span><span>join_date</span><span>,</span><span>'orders'</span><span>:</span> <span>[]</span><span>}</span><span>for</span> <span>order</span> <span>in</span> <span>record</span><span>.</span><span>order</span><span>:</span><span>order</span> <span>=</span> <span>{</span><span>'order_price'</span><span>:</span> <span>order</span><span>.</span><span>price</span><span>,</span><span>'currency'</span><span>:</span> <span>order</span><span>.</span><span>currency</span><span>,</span><span>'purchase_date'</span><span>:</span> <span>order</span><span>.</span><span>purchase_date</span><span>,</span><span>'product'</span><span>:</span> <span>order</span><span>.</span><span>product</span><span>}</span><span>record_object</span><span>[</span><span>'orders'</span><span>].</span><span>append</span><span>(</span><span>order</span><span>)</span><span>pp</span><span>.</span><span>pprint</span><span>(</span><span>record_object</span><span>)</span><span>...</span> <span>import</span> <span>pprint</span> <span>from</span> <span>.models</span> <span>import</span> <span>Order</span><span>,</span> <span>Customer</span> <span>pp</span> <span>=</span> <span>pprint</span><span>.</span><span>PrettyPrinter</span><span>(</span><span>indent</span><span>=</span><span>4</span><span>)</span> <span># Execute a SELECT query on JOINed tables </span><span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span> <span>.</span><span>join</span><span>(</span><span>Order</span><span>,</span> <span>Order</span><span>.</span><span>customer_id</span> <span>==</span> <span>Customer</span><span>.</span><span>id</span><span>)</span> <span>.</span><span>all</span><span>()</span> <span># Loop through results </span><span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span> <span>record_object</span> <span>=</span> <span>{</span> <span>'first_name'</span><span>:</span> <span>record</span><span>.</span><span>first_name</span><span>,</span> <span>'last_name'</span><span>:</span> <span>record</span><span>.</span><span>last_name</span><span>,</span> <span>'email'</span><span>:</span> <span>record</span><span>.</span><span>email</span><span>,</span> <span>'preferred_language'</span><span>:</span> <span>record</span><span>.</span><span>preferred_language</span><span>,</span> <span>'join_date'</span><span>:</span> <span>record</span><span>.</span><span>join_date</span><span>,</span> <span>'orders'</span><span>:</span> <span>[]</span> <span>}</span> <span>for</span> <span>order</span> <span>in</span> <span>record</span><span>.</span><span>order</span><span>:</span> <span>order</span> <span>=</span> <span>{</span> <span>'order_price'</span><span>:</span> <span>order</span><span>.</span><span>price</span><span>,</span> <span>'currency'</span><span>:</span> <span>order</span><span>.</span><span>currency</span><span>,</span> <span>'purchase_date'</span><span>:</span> <span>order</span><span>.</span><span>purchase_date</span><span>,</span> <span>'product'</span><span>:</span> <span>order</span><span>.</span><span>product</span> <span>}</span> <span>record_object</span><span>[</span><span>'orders'</span><span>].</span><span>append</span><span>(</span><span>order</span><span>)</span> <span>pp</span><span>.</span><span>pprint</span><span>(</span><span>record_object</span><span>)</span>... import pprint from .models import Order, Customer pp = pprint.PrettyPrinter(indent=4) # Execute a SELECT query on JOINed tables records = session .query(Customer) .join(Order, Order.customer_id == Customer.id) .all() # Loop through results for record in records: record_object = { 'first_name': record.first_name, 'last_name': record.last_name, 'email': record.email, 'preferred_language': record.preferred_language, 'join_date': record.join_date, 'orders': [] } for order in record.order: order = { 'order_price': order.price, 'currency': order.currency, 'purchase_date': order.purchase_date, 'product': order.product } record_object['orders'].append(order) pp.pprint(record_object)
Enter fullscreen mode Exit fullscreen mode Join records from different tables and deserialize records
We perform our JOIN using the join()
method. The first parameter we pass is the data model we’ll be joining with on the “right.” We then specify what we’ll be joining “on”: the customer_id column of our order model, and the id column of our customer model.
Our outer loop gives us each customer, and our inner loop adds each order to the appropriate customer. Check out an example record:
<span>{</span> <span>'email'</span>: <span>'jtinline16@arizona.edu'</span>,<span>'first_name'</span>: <span>'Jerry'</span>,<span>'join_date'</span>: datetime.datetime<span>(</span>2016, 10, 27, 0, 0<span>)</span>,<span>'last_name'</span>: <span>'Tinline'</span>,<span>'preferred_language'</span>: <span>'Icelandic'</span>,<span>'orders'</span>: <span>[{</span><span>'currency'</span>: <span>'IDR'</span>,<span>'order_price'</span>: 34.24,<span>'product'</span>: <span>'Beer - Corona'</span>,<span>'purchase_date'</span>: datetime.datetime<span>(</span>2019, 5, 5, 0, 0<span>)}</span>,<span>{</span><span>'currency'</span>: <span>'GEL'</span>,<span>'order_price'</span>: 25.75,<span>'product'</span>: <span>'Creamers - 10%'</span>,<span>'purchase_date'</span>: datetime.datetime<span>(</span>2019, 1, 27, 0, 0<span>)}]}</span><span>{</span> <span>'email'</span>: <span>'jtinline16@arizona.edu'</span>, <span>'first_name'</span>: <span>'Jerry'</span>, <span>'join_date'</span>: datetime.datetime<span>(</span>2016, 10, 27, 0, 0<span>)</span>, <span>'last_name'</span>: <span>'Tinline'</span>, <span>'preferred_language'</span>: <span>'Icelandic'</span>, <span>'orders'</span>: <span>[{</span><span>'currency'</span>: <span>'IDR'</span>, <span>'order_price'</span>: 34.24, <span>'product'</span>: <span>'Beer - Corona'</span>, <span>'purchase_date'</span>: datetime.datetime<span>(</span>2019, 5, 5, 0, 0<span>)}</span>, <span>{</span><span>'currency'</span>: <span>'GEL'</span>, <span>'order_price'</span>: 25.75, <span>'product'</span>: <span>'Creamers - 10%'</span>, <span>'purchase_date'</span>: datetime.datetime<span>(</span>2019, 1, 27, 0, 0<span>)}]}</span>{ 'email': 'jtinline16@arizona.edu', 'first_name': 'Jerry', 'join_date': datetime.datetime(2016, 10, 27, 0, 0), 'last_name': 'Tinline', 'preferred_language': 'Icelandic', 'orders': [{'currency': 'IDR', 'order_price': 34.24, 'product': 'Beer - Corona', 'purchase_date': datetime.datetime(2019, 5, 5, 0, 0)}, {'currency': 'GEL', 'order_price': 25.75, 'product': 'Creamers - 10%', 'purchase_date': datetime.datetime(2019, 1, 27, 0, 0)}]}
Enter fullscreen mode Exit fullscreen mode Output
Our friend Jerry here has two orders: one for some Coronas, and another for creamers. Get at it, Jerry.
Outer JOINs
In addition to simple JOINs, we can perform outer JOINs using the same syntax:
<span>...</span><span>from</span> <span>.models</span> <span>import</span> <span>ExampleModel1</span><span>,</span> <span>ExampleModel2</span><span># Execute an outer JOIN </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>ExampleModel1</span><span>)</span><span>.</span><span>outerjoin</span><span>(</span><span>ExampleModel2</span><span>)</span><span>.</span><span>all</span><span>()</span><span>...</span> <span>from</span> <span>.models</span> <span>import</span> <span>ExampleModel1</span><span>,</span> <span>ExampleModel2</span> <span># Execute an outer JOIN </span><span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>ExampleModel1</span><span>)</span> <span>.</span><span>outerjoin</span><span>(</span><span>ExampleModel2</span><span>)</span> <span>.</span><span>all</span><span>()</span>... from .models import ExampleModel1, ExampleModel2 # Execute an outer JOIN records = session .query(ExampleModel1) .outerjoin(ExampleModel2) .all()
Enter fullscreen mode Exit fullscreen mode Perform an outer join
Unions
We can perform UNIONs and UNION ALLs as well:
<span>...</span><span>from</span> <span>.models</span> <span>import</span> <span>ExampleModel1</span><span>,</span> <span>ExampleModel2</span><span># Execute a UNION </span><span>records</span> <span>=</span> <span>ExampleModel1</span><span>.</span><span>union</span><span>(</span><span>ExampleModel2</span><span>)</span><span>...</span> <span>from</span> <span>.models</span> <span>import</span> <span>ExampleModel1</span><span>,</span> <span>ExampleModel2</span> <span># Execute a UNION </span><span>records</span> <span>=</span> <span>ExampleModel1</span><span>.</span><span>union</span><span>(</span><span>ExampleModel2</span><span>)</span>... from .models import ExampleModel1, ExampleModel2 # Execute a UNION records = ExampleModel1.union(ExampleModel2)
Enter fullscreen mode Exit fullscreen mode Perform a union
To perform a union all, simply replace union()
with union_all()
!
Aggregate Functions and Stats
As with all SQL-like query languages, we can perform some aggregate stats as well. The following are available to us:
-
count([COLUMN])
: Counts the number of records in a column. -
count(distinct([COLUMN]))
: Counts the distinct number of records in a column. -
sum([COLUMN])
: Adds the numerical values in a column.
Here’s how we’d perform a query that counts the values in a column:
<span>...</span><span>from</span> <span>sqlalchemy</span> <span>import</span> <span>func</span><span># Count number of records with a `first_name` value </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>func</span><span>.</span><span>count</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span><span>))</span><span>.</span><span>all</span><span>()</span><span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span><span>print</span><span>(</span><span>record</span><span>)</span><span>...</span> <span>from</span> <span>sqlalchemy</span> <span>import</span> <span>func</span> <span># Count number of records with a `first_name` value </span><span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>func</span><span>.</span><span>count</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span><span>))</span> <span>.</span><span>all</span><span>()</span> <span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span> <span>print</span><span>(</span><span>record</span><span>)</span>... from sqlalchemy import func # Count number of records with a `first_name` value records = session .query(func.count(Customer.first_name)) .all() for record in records: print(record)
Enter fullscreen mode Exit fullscreen mode Aggregate queries
Which outputs:
<span>(</span>200,<span>)</span><span>(</span>200,<span>)</span>(200,)
Enter fullscreen mode Exit fullscreen mode Output
This query can easily be modified to only count distinct values:
<span>...</span><span>from</span> <span>sqlalchemy</span> <span>import</span> <span>func</span><span>from</span> <span>sqlalchemy</span> <span>import</span> <span>distinct</span><span># Count number of DISTINCT `first_name` values </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>func</span><span>.</span><span>count</span><span>(</span><span>distinct</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span><span>)))</span><span>.</span><span>all</span><span>()</span><span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span><span>print</span><span>(</span><span>record</span><span>)</span><span>...</span> <span>from</span> <span>sqlalchemy</span> <span>import</span> <span>func</span> <span>from</span> <span>sqlalchemy</span> <span>import</span> <span>distinct</span> <span># Count number of DISTINCT `first_name` values </span><span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>func</span><span>.</span><span>count</span><span>(</span><span>distinct</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span><span>)))</span> <span>.</span><span>all</span><span>()</span> <span>for</span> <span>record</span> <span>in</span> <span>records</span><span>:</span> <span>print</span><span>(</span><span>record</span><span>)</span>... from sqlalchemy import func from sqlalchemy import distinct # Count number of DISTINCT `first_name` values records = session .query(func.count(distinct(Customer.first_name))) .all() for record in records: print(record)
Enter fullscreen mode Exit fullscreen mode Query to aggregate distinct results
Using Group_by()
Of course, we can use the group_by()
method on queries based around aggregates as well. group_by()
works similarly to what we’d expect from SQL and Pandas:
<span>...</span><span># Execute a `GROUP BY` aggregation query </span><span>records</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>func</span><span>.</span><span>count</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span><span>))</span><span>.</span><span>group_by</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span><span>)</span><span>.</span><span>all</span><span>()</span><span>...</span> <span># Execute a `GROUP BY` aggregation query </span><span>records</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>func</span><span>.</span><span>count</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span><span>))</span> <span>.</span><span>group_by</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span><span>)</span> <span>.</span><span>all</span><span>()</span>... # Execute a `GROUP BY` aggregation query records = session .query(func.count(Customer.first_name)) .group_by(Customer.first_name) .all()
Enter fullscreen mode Exit fullscreen mode “Group by” aggregation
Mutations
We’ve spent an awful lot of time going over how to extract data from our database, but haven’t talked about modifying our data yet! The last item on our agenda today is looking at how to add, remove, and change records using the SQLAlchemy ORM.
Inserting Rows
The first way we can add data is by using the add()
method. add()
expects an instance of a class (data model specifically) to be passed, and will create a new database row as a result:
<span>from</span> <span>.database</span> <span>import</span> <span>session</span><span>from</span> <span>.models</span> <span>import</span> <span>Customer</span><span># Inserting records via data models </span><span>customer</span> <span>=</span> <span>Customer</span><span>(</span><span>first_name</span><span>=</span><span>'Todd'</span><span>,</span><span>last_name</span><span>=</span><span>'Birchard'</span><span>,</span><span>email</span><span>=</span><span>'fake@example.com'</span><span>,</span><span>preferred_language</span><span>=</span><span>'English'</span><span>,</span><span>join_date</span><span>=</span><span>datetime</span><span>.</span><span>now</span><span>()</span><span>)</span><span>session</span><span>.</span><span>add</span><span>(</span><span>customer</span><span>)</span><span>session</span><span>.</span><span>commit</span><span>()</span><span>from</span> <span>.database</span> <span>import</span> <span>session</span> <span>from</span> <span>.models</span> <span>import</span> <span>Customer</span> <span># Inserting records via data models </span><span>customer</span> <span>=</span> <span>Customer</span><span>(</span> <span>first_name</span><span>=</span><span>'Todd'</span><span>,</span> <span>last_name</span><span>=</span><span>'Birchard'</span><span>,</span> <span>email</span><span>=</span><span>'fake@example.com'</span><span>,</span> <span>preferred_language</span><span>=</span><span>'English'</span><span>,</span> <span>join_date</span><span>=</span><span>datetime</span><span>.</span><span>now</span><span>()</span> <span>)</span> <span>session</span><span>.</span><span>add</span><span>(</span><span>customer</span><span>)</span> <span>session</span><span>.</span><span>commit</span><span>()</span>from .database import session from .models import Customer # Inserting records via data models customer = Customer( first_name='Todd', last_name='Birchard', email='fake@example.com', preferred_language='English', join_date=datetime.now() ) session.add(customer) session.commit()
Enter fullscreen mode Exit fullscreen mode Insert records via ORM
An alternative way to add data is by using the insert()
method. Unlike add()
, insert()
is called on an SQLAlchemy Table object and doesn’t rely on receiving a data model. insert()
is not part of the ORM:
<span>...</span><span># Inserting records via SQLAlchemy `Table` objects </span><span>insert</span> <span>=</span> <span>[</span><span>TABLE</span><span>]</span><span>.</span><span>insert</span><span>()</span><span>.</span><span>values</span><span>(</span><span>first_name</span><span>=</span><span>'Todd'</span><span>,</span><span>last_name</span><span>=</span><span>'Jack Jones'</span><span>,</span><span>email</span><span>=</span><span>'fake@example.com'</span><span>,</span><span>preferred_language</span><span>=</span><span>'English'</span><span>,</span><span>join_date</span><span>=</span><span>datetime</span><span>.</span><span>now</span><span>()</span><span>)</span><span>...</span> <span># Inserting records via SQLAlchemy `Table` objects </span><span>insert</span> <span>=</span> <span>[</span><span>TABLE</span><span>]</span> <span>.</span><span>insert</span><span>()</span> <span>.</span><span>values</span><span>(</span> <span>first_name</span><span>=</span><span>'Todd'</span><span>,</span> <span>last_name</span><span>=</span><span>'Jack Jones'</span><span>,</span> <span>email</span><span>=</span><span>'fake@example.com'</span><span>,</span> <span>preferred_language</span><span>=</span><span>'English'</span><span>,</span> <span>join_date</span><span>=</span><span>datetime</span><span>.</span><span>now</span><span>()</span> <span>)</span>... # Inserting records via SQLAlchemy `Table` objects insert = [TABLE] .insert() .values( first_name='Todd', last_name='Jack Jones', email='fake@example.com', preferred_language='English', join_date=datetime.now() )
Enter fullscreen mode Exit fullscreen mode Insert records
Updating
Building on the syntax of insert()
, we can drop in the update()
method to change an existing record’s values. We chain in the where()
method to specify which rows should be updated:
<span>...</span><span># Updating records via SQLAlchemy `Table` objects </span><span>result</span> <span>=</span> <span>[</span><span>TABLE</span><span>]</span><span>.</span><span>update</span><span>()</span><span>.</span><span>where</span><span>([</span><span>TABLE</span><span>].</span><span>c</span><span>.</span><span>name</span> <span>==</span> <span>'Todd'</span><span>)</span><span>.</span><span>values</span><span>(</span><span>email</span><span>=</span><span>'newemail@example.com'</span><span>)</span><span>...</span> <span># Updating records via SQLAlchemy `Table` objects </span><span>result</span> <span>=</span> <span>[</span><span>TABLE</span><span>]</span> <span>.</span><span>update</span><span>()</span> <span>.</span><span>where</span><span>([</span><span>TABLE</span><span>].</span><span>c</span><span>.</span><span>name</span> <span>==</span> <span>'Todd'</span><span>)</span> <span>.</span><span>values</span><span>(</span><span>email</span><span>=</span><span>'newemail@example.com'</span><span>)</span>... # Updating records via SQLAlchemy `Table` objects result = [TABLE] .update() .where([TABLE].c.name == 'Todd') .values(email='newemail@example.com')
Enter fullscreen mode Exit fullscreen mode Update records
Deleting
On any query we execute, we can append the delete()
method to delete all rows which are contained in that query (be careful!). The below deletes all records where the first_name column contains a value of “Carl”:
<span>...</span><span># Delete records where `first_name` is `Carl` </span><span>result</span> <span>=</span> <span>session</span><span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span><span>.</span><span>filter</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span> <span>==</span> <span>'Carl'</span><span>)</span><span>.</span><span>delete</span><span>()</span><span>...</span> <span># Delete records where `first_name` is `Carl` </span><span>result</span> <span>=</span> <span>session</span> <span>.</span><span>query</span><span>(</span><span>Customer</span><span>)</span> <span>.</span><span>filter</span><span>(</span><span>Customer</span><span>.</span><span>first_name</span> <span>==</span> <span>'Carl'</span><span>)</span> <span>.</span><span>delete</span><span>()</span>... # Delete records where `first_name` is `Carl` result = session .query(Customer) .filter(Customer.first_name == 'Carl') .delete()
Enter fullscreen mode Exit fullscreen mode Delete records
delete()
accepts the synchronize_session parameter, which determines how deletions should be handled:
-
False
won’t perform the delete until the session is committed. -
'fetch'
selects all rows to be deleted and removes matched rows. -
'evaluate'
will evaluate the objects in the current session to determine which rows should be removed.
Never Stop Exploring™
There’s a lot we’ve left out for the sake of simplicity. There are plenty of cool methods left to explore, like the correlate()
method, for instance. You’re armed with enough to be dangerous in SQLAlchemy now, but I encourage anybody to look over the query documentation and find the cool things we didn’t speak to in detail here.
I’m still working on throwing together the source code for this post in the Github repo below. Source for the previous chapters can be found there as well. In the meantime, I apologize for being a bit of a shit show:
hackersandslackers / sqlalchemy-tutorial
🧪 Use SQLAlchemy to connect, query, and interact with relational databases.
SQLAlchemy Tutorial
This repository contains the source code for a four-part tutorial series on SQLAlchemy:
- Databases in Python Made Easy with SQLAlchemy
- Implement an ORM with SQLAlchemy
- Relationships in SQLAlchemy Data Models
- Constructing Database Queries with SQLAlchemy
Getting Started
Get set up locally in two steps:
Environment Variables
Replace the values in .env.example with your values and rename this file to .env:
-
SQLALCHEMY_DATABASE_URI
: Connection URI of a SQL database. -
SQLALCHEMY_DATABASE_PEM
(Optional): PEM key for databases requiring an SSL connection.
Remember never to commit secrets saved in .env files to Github.
Installation
Get up and running with make deploy
:
$ git clone https://github.com/hackersandslackers/sqlalchemy-tutorial.git
$ cd sqlalchemy-tutorial
$ make deploy
Enter fullscreen mode Exit fullscreen mode
Hackers and Slackers tutorials are free of charge. If you found this tutorial helpful, a small donation would be greatly appreciated to keep us in business. All proceeds go towards coffee, and all coffee goes towards more content.
Mastering SQLAlchemy (4 Part Series)
1 Databases in Python Made Easy with SQLAlchemy
2 Implement ORM Data Models with SQLAlchemy
3 Queries as Python Code with SQLAlchemy’s Expression Language
4 Define Relationships Between SQLAlchemy Data Models
原文链接:Queries as Python Code with SQLAlchemy’s Expression Language
暂无评论内容