
Year birds: +34 (**165**),
State birds: +3 (**332**),
Life birds: +2 (**507**)

---

March was a great month of birding. I'm ending the first quarter of the year with 165 species, exactly 25 species ahead of Paul's 2013 pace. A portion of the birds I'm "ahead" with are more common ones that will come easier later in the year (and he had 16 I've yet to find), but I did pick up a couple really good ones this month!

### Out West

The month started out west. Keegan and I were camping at Lake Ogallala looking for gulls the last few days of January, and not finding much on the third day, we decided to go to Sutherland Reservoir to see if we could find one of the legendary massive flocks of Snow Geese. On the way in, by the cooling ponds, we saw a loon. Naturally, I snapped some photos, and we both wrote it off as a Common Loon, but something was still odd about it. It took until 2 hours later when we got back to camp for me to study the photos and realize that it was actually a **Pacific Loon** (132)! A really great bird to pick up early in the year, and a state bird for both of us. Later Boni E, Dave C, and others were able to relocate the bird and add it to their state lists!

[![Pacific Loon](https://cdn.download.ams.birds.cornell.edu/api/v2/asset/631530734/640)](https://macaulaylibrary.org/asset/631530734)

Also at Sutherland we did manage to find a massive flock of geese. Atempting to estimate the size was difficult, but with a little studying of the maps we were able to estimate a [flock size of 560,000](https://ebird.org/checklist/S216074079)!

### Gull-a-palooza!

As new year birds were coming in a bit slowly in mid March, I decided to go bird at Marsh Wren, just to see if any early shorebirds decided to stop there. Instead of shorebirds, I ran into a [MASSIVE flock of gulls](https://ebird.org/checklist/S218281421)!

Among the flock, I found a Glaucous Gull, multiple Lesser-black-backed Gulls, "Thayer's" and "Kumlien's" Iceland Gulls, 2 California Gulls (very rare for Lancaster county), and a [hybrid Herring x Lesser Black-backed Gull](https://ebird.org/checklist/S218690750)! The flock seemed to be resting there after foraging at the dump and showed up regularly for over a week! It was very cool to see many people come out and get life, state, and county birds from such a diverse and cooperative flock! Just goes to show that birding some spots in the "wrong" time of year can pay off!

[![California Gull](https://cdn.download.ams.birds.cornell.edu/api/v2/asset/632158431/640)](https://macaulaylibrary.org/asset/632158431)

### Cranes, Cranes, Cranes

March is Crane season in Nebraska, and I made 4 trips out to see them! The first trip was a blind tour at Crane Trust. The tour was amazing as always (even though Snow Geese tried to steal the show), but didn't produce any Whooping Cranes, so Keegan and I came out the next week to find a **Whooping Crane** (142) to add to our year lists.

The month of March also resulted in 2 other trips specifically to search for the Common Cranes. The first search was fruitless, but an extended second trip resulted in success! You can read more about the **Common Crane** (159) in it's own [blog post](/blog/2025/03/nebraska-big-year-common-crane-quest).

[![Common Crane](https://cdn.download.ams.birds.cornell.edu/api/v2/asset/632650420/640)](https://macaulaylibrary.org/asset/632650420)

### Prairie Birds

If I had a state nemesis bird going into the month, it would probably be Smith's Longspur. I went out to Spring Creek Prairie 5 times last fall searching and listening for the elusive prairie bird with no luck. So when Paul D texted me and gave me a few spots to checkout in the southeast portion of the state, I couldn't help but give it a shot.

I started at Pawnee Prairie WMA where I racked up over 20 Pawnee county birds. I _did_ have some Longspurs there, but I wasn't confident enough in the call notes to turn them into Smith's. I later visited Burchard WMA, but the strong winds made listening for the rattle calls nearly impossible. So finally I headed to Berg Tallgrass Prairie near Talmage, and not 5 minutes after stepping out of the car, a dozen **Smith's Longspurs** (157) flew over my head, rattling all the way and landed in the shortgrass. I was even able to snag a few photos before they flew off again and disappeared to the west.

[![Smith's Longspur](https://cdn.download.ams.birds.cornell.edu/api/v2/asset/632498819/640)](https://macaulaylibrary.org/asset/632498819)

### Shorebirds

Shorebird season is starting up again, and I'm off to a pretty good start! A couple of last year's go-to spots like Pawnee Lake and Branched Oak Lake seem to have much higher water this year, leaving little to no mudflats to work with, so I'm needing to branch out a bit more. I've scouted out a few places in the southwest side of Lancaster county that should be good for this year, especially Yankee Hill WMA, where I snagged my last 3 year birds of the month: **Lesser Yellowlegs** (163), **Marbled Godwit** (164), and **American Avocet** (165). Migration will begin in earnest soon and I'm ready for the challenge of finding everything that comes through!

Thank you so much to Caroline, Keegan, Paul, Dan, Colleen, and everyone else I birded with this month for all their help and encouragement!

---

[eBird Trip Report](https://ebird.org/tripreport/337360)

132. **Pacific Loon**, Lincoln
133. **American Pipit**, Lincoln
134. **Western Grebe**, Lincoln
135. **Common Grackle**, Lancaster
136. **Killdeer**, Lancaster
137. **Eastern Phoebe**, Hall
138. **Yellow-headed Blackbird**, Hall
139. **Eastern Meadowlark**, Seward
140. **American Woodcock**, Lancaster
141. **Spotted Towhee**, Lancaster
142. **Whooping Crane**, Hall
143. **Wood Duck**, Hall
144. **Savannah Sparrow**, Lancaster
145. **Blue-winged Teal**, Lancaster
146. **Horned Grebe**, Lancaster
147. **Loggerhead Shrike**, Lancaster
148. **Turkey Vulture**, Lancaster
149. **Franklin's Gull**, Lancaster
150. **Common Loon**, Lancaster
151. **Red-necked Grebe**, Lancaster
152. **Baird's Sandpiper**, Lancaster
153. **Field Sparrow**, Pawnee
154. **Brown Thrasher**, Pawnee
155. **Tree Swallow**, Pawnee
156. **Greater Yellowlegs**, Pawnee
157. **Smith's Longspur**, Nemaha
158. **Long-billed Dowitcher**, Lancaster
159. **Common Crane**, Keith
160. **Pectoral Sandpiper**, Lancaster
161. **Barn Swallow**, Lancaster
162. **Purple Martin**, Lancaster
163. **Lesser Yellowlegs**, Lancaster
164. **Marbled Godwit**, Lancaster
165. **American Avocet**, Lancaster


Nebraska Big Year: March Recap


March is crane season in Nebraska, the Platte River is covered with nearly a million Sandhill Cranes, and we often get handfuls of Whooping Cranes. But the most sought after crane in the state (besides the ultra-mega Hooded) might just be the Common Crane. The Common Crane is a eurasian species that breeds in northern Euraisa, all the way from Northern Europe to Siberia. Sandhill Cranes also breed in far eastern Russia, and sometimes post-breeding dispersal can result in a student exchange program of sorts, leading Sandhill cranes to end up in Europe and Common Cranes to end up here in Nebraska.

When setting out for a Big Year, the rarities are what matters. You can't get a very large number of birds with just the breeding birds, you need to go for the crazy rarities. So at the beginning of the year, Common Crane was in the back of my mind. Studying ebird data, I noticed that nearly every year there was a Common Crane spotted, sometimes near Grand Island, and sometimes near Lewellen way out by Lake Mac. I told myself if there was a Common Crane report, I would go for it.

### March 17, 2025

On March 17th, one of those reports came in. I was out enjoying the unseasonably warm weather, washing my mountain bike in preparation for a ride when I got a call from Keegan. "Did you see what I sent you?!?! Common Crane. Merrick County. Now." Sure enough, there it was. I couldn't bail on the bike ride with my friends, but Keegan and I quickly made plans and took the time off from work to head out the next day in search of the crane.

### March 18, 2025

We started near Phillips, searching through endless flocks of cranes. We branched out to nearby areas, searching the roads south where it had been in 2022. Paul Dunbar and others were out looking as well, but after 4 hours of searching we had nothing. We regrouped with lunch at Runza and decided to put in one last effort. With 2 more hours of fruitless searching, we decided it was time to give up. 100 disappointed miles back home. The haystack won today.

### March 23, 2025

CRANE ALERT, CRANE ALERT! Late Sunday afternoon, another report came in, this time closer to Kearney. Once again, Keegan and I hatched plans to set out the next day, this time bringing along long-time Nebraska birder and mentor, Dan.

### March 24, 2025

Once again, we started searching the area where it was previously reported. The heat haze was awful at midday, and I estimate we were only really searching 1/3 of all the cranes in the area. After 2 hours of searching we were beginning to lose hope, until I recieve a text from Boni with a photo taken just an hour earlier: "Common Crane in Clear Creek". Clear Creek? 3 hours away on the west end of Lake McConaughy? It was a decision point. We had spent 2 fruitless hours searching here, and someone was _currently looking at a Common Crane_ under 3 hours away. We had to take advantage of it, so we immediately set off.

2 hours and 34 minutes later when we arrived, Julie from game and parks was standing on a picnic table photographing the bird. We quickly joined, got onto it and let out a sigh of relief. It so nice, after dedicating many hours to searching and driving for one bird to _finally_ get it. It was a life bird for both Keegan and I, and it gave us great looks, showing off it's contrasting neck pattern against the nearby Sandhill Cranes!

Clear Creek WMA is a wonderful spot, with a few newly built ponds and wetland areas, so we spent some time birding in the area while also enjoying the crane. The cranes were roosting SO CLOSE, I don't think I'd ever been so close to a crane not in a car or blind, but the tall grasses between the path and the roost provided some cover and the cranes seemed not to mind. After about an hour (and with a 4 hour drive ahead of us), we decided to head back, having completed our quest. 650 miles. 1 life and year bird.

159. **Common Crane**, Keith

![Common Crane](https://cdn.download.ams.birds.cornell.edu/api/v2/asset/632650417/640)

Special thanks to Matt (for finding the bird), Paul and Colleen (for searching with us), Boni (for texting me about the bird), Julie (for being on it when we got there), and Dan (for coming along for the ride).


Nebraska Big Year: Common Crane Quest


Year birds: +13 (**131**),
State birds: +1 (**329**),
Life birds: +1 (**499**)

---

Overall February was a bit of a slower month, but I still ran into some great birds!

In early February I headed to Lake Ogallala specifically targeting Gulls. I spent the first night of the trip camping in North Platte, where Boni gave me some tips that led to a few years birds including **Short-eared Owl**. At Lake O, the trip continued to produce, netting me 11 year birds, including 2 very good gull targets: **Glaucous Gull** and **Short-billed Gull**. The biggest target I had that I _missed_ was Great Black-backed Gull, but I'll have another shot at that one this fall.

![Glaucous Gull](https://cdn.download.ams.birds.cornell.edu/api/v2/asset/630537712/640)

Back home in Lancaster County, I tried a few different places for **Long-eared Owl** and finally tracked down my first (and only) lifer of February.

Perhaps more importantly for the overall effort, February was a HUGE planning month. I refined my spreadsheet and spent hours and hours studying [Bird of Nebraska Online](https://birds.outdoornebraska.gov/) and ebird data, noting the rarity, locations, and estimated arrival dates of all the spring migrants in my big year spreadsheet.

---

[eBird Trip Report](https://ebird.org/tripreport/330554)

119. **Mountain Bluebird**, Lincoln
120. **Brown-headed Cowbird**, Lincoln
121. **Greater Prairie-Chicken**, Lincoln
122. **Short-eared Owl**, Lincoln
123. **Glaucous Gull**, Keith
124. **Red-breasted Merganser**, Keith
125. **American White Pelican**, Lincoln
126. **Double-crested Cormorant**, Lincoln
127. **Wilson's Snipe**, Lincoln
128. **Virginia Rail**, Lincoln
129. **Short-billed Gull**, Keith
130. **Long-eared Owl**, Lancaster
131. **Rusty Blackbird**, Lancaster


Nebraska Big Year: February Recap


As some of you may know, I'm giving a Nebraska state Big Year a crack this year. What does this mean you ask? It means I'm committing to driving thousands of miles, spending hundreds of hours, doing my absolute best to see as many species of birds in the state of Nebraska in one calendar year. It will be a long and tiring year, punctuated by exciting chases and bitter disappointements, but I am committed and ready to give it all I have. The current Big Year record is 347, set in 2013 by Paul Dunbar. This record is incredible, absolutely blowing the previous record of 331 out of the water. Nebraska on average gets about 370 species of birds either breeding or passing through each year, and in 2025 I'm setting out with a goal to see 350 of them. I'll be blogging as best I can on here, and at least doing monthly updates.

---

Year birds: +118 (**118**),
State birds: +3 (**328**),
Life birds: +3 (**497**)

---

### January 1

It was a cold winter morning in North Platte, and I woke up early at 5am to get my first bird of the year. I hopped in the car, and drove from my hotel to some canyons 30 miles to the southeast. Shortly after arriving at the spot (which I had scoped out on Dec 31st), I heard my target bird, a **Northern Saw-whet Owl**! Right after hearing it, it buzzed right over my head, and I got just a glimpse of it as it passed through my car headlights and deep into the woods. A pretty good bird to start off the year!

From there, I hit the road, aiming to be at Lake Ogallala by sunrise where a buffet of waterfowl was awaiting me. Shortly after arriving I found 4 of my key targets, all continuing birds: **Long-tailed Duck**, **White-winged Scoter**, **Black Scoter**, and **Surf Scoter**. All 4 of those birds are huge to get so early in the year, especially all at the same location, and will save me hours and hours of searching in the fall. After that, I settled in to find the big target. Boni and Colleen arrived shortly after and we began scoping. The rafts of waterfowl were huge, at least a few thousand, but after 15 minutes or so, we got onto it, my first lifer of the year: **Eurasian Wigeon**!

Crazy enough as it seems, from there I headed BACK to North Platte for 2 more rarities because Big Years are all about getting the rarest birds first. The bird I had just gotten as a lifer a few weeks prior showed off it's brilliant yellow to me again in the new year: **Couch's Kingbird**, a continuing first state record! HUGE thanks to the Nelsons for being such gracious hosts. One last big target was waiting for me at the North Platte WTP, and moments after spotting the **Barrow's Goldeneye**, I headed west.

I finished January 1st with 78 species, and 8 rare targets, and 1 lifer! I was off to the races.

![Couch's Kingbird](https://cdn.download.ams.birds.cornell.edu/api/v2/asset/628571041/640)

### January 2

I woke up again before dawn to drive from Gering to Harrison to participate in a Christmas Bird Count. I was in charge of Pants Butte and Sowbelly Roads, 2 excellent birding areas. Early in the morning, at the intersection of Pants Butte and Sowbelly, I had my 2nd life bird of the year: **Gray-crowned Rosy-Finch**! GCRF was one of my main targets, and I got pretty good views of a relatively large flock before they disappeared into the hills.

A few more excellent pickups for the day were **Common Raven**, **Sharp-tailed Grouse**, **Black-billed Magpie**, and **Golden Eagle**, all western specialties that I was very pleased to get so early. After meeting up around lunch to tally our CBC results, I took the scenic route down Henry Road, checking again for rosy-finches, but not coming up with much except **Rough-legged Hawks**.

### January 3

This was the day I planned to see Pinyon Jays, but it didn't start out as planned. A thin blanket of snow covered the ground in Gering as I set out for the day, but after a couple hundred yards of driving, I realized I had a flat tire (probably from driving 40 miles of gravelt the day before). I attempted to put the spare on, but my tiny wimpy car jack wouldn't do. Thanks to the Gering police department I was finally able to get the spare on and hit the road again. (Yes, I did buy a new jack immediately after getting home)

60 miles later on a spare tire, I pulled up to the traditional Pinyon Jay feeders, and for nearly 90 minutes all I saw were the usual species. I was beginning to lose hope, when finally I noticed the undulating flight pattern of a group of birds coming in from the west. After inspecting the feeders for a minute or so, my year's first **Pinyon Jays** overwhelmed the feeders! 3 days in and 3 life birds.

I spent the next 2 hours killing time in a coffee shop while my tire was repaired in Bridgeport. Not the best way to spend prime birding time during a time-crunched Big Year, but I had already driven 30 miles more than I should have on the spare.

I was able to salvage the rest of the day at Lake Minitaire, Buffalo Creek WMA and Wildcat Hills, notching another 3 western specialty year birds: **Rock Wren** (early or overwintering), **Townsend's Solitaire**, and **Pygmy Nuthatch**.

![Buffalo Creek WMA](https://live.staticflickr.com/65535/54249191699_21a566b628_z.jpg)

### January 4

Like the evening before, I camped out at the feeders at Wildcat Hills, hoping to glimpse a Cassin's Finch. After 90 minutes, and with the weather projected to get worse, I decided to call it and begin the long drive home.

I stopped off in Kearney to look for the Brant that was seen a few days before, but had no luck. I continued home in the freezing rain with a repsectable 93 species in 4 days.

![Wildcat Hills](https://live.staticflickr.com/65535/54249372910_5680027928_z.jpg)

### January 5-31

The rest of the month I spent in the east, knocking out a few more key targets. **Snow Bunting** in Platte, **Tundra Swan** in Douglas, and **Purple Finch** and **Hermit Thrush** in Lancaster were the most notable birds.

---

[eBird Trip Report](https://ebird.org/tripreport/325880)

1. **Northern Saw-whet Owl**, Lincoln
2. **Northern Cardinal**, Keith
3. **Song Sparrow**, Keith
4. **Dark-eyed Junco**, Keith
5. **Cedar Waxwing**, Keith
6. **American Robin**, Keith
7. **European Starling**, Keith
8. **Winter Wren**, Keith
9. **White-breasted Nuthatch**, Keith
10. **American Crow**, Keith
11. **Blue Jay**, Keith
12. **Northern Flicker**, Keith
13. **Hairy Woodpecker**, Keith
14. **Red-bellied Woodpecker**, Keith
15. **Eastern Screech-Owl**, Keith
16. **Red-tailed Hawk**, Keith
17. **Bald Eagle**, Keith
18. **Great Blue Heron**, Keith
19. **Eared Grebe**, Keith
20. **Pied-billed Grebe**, Keith
21. **Iceland Gull**, Keith
22. **Lesser Black-backed Gull**, Keith
23. **American Herring Gull**, Keith
24. **Ring-billed Gull**, Keith
25. **Bonaparte's Gull**, Keith
26. **American Coot**, Keith
27. **Eurasian Collared-Dove**, Keith
28. **Ruddy Duck**, Keith
29. **Common Merganser**, Keith
30. **Common Goldeneye**, Keith
31. **Bufflehead**, Keith
32. **Long-tailed Duck**, Keith
33. **Black Scoter**, Keith
34. **White-winged Scoter**, Keith
35. **Surf Scoter**, Keith
36. **Lesser Scaup**, Keith
37. **Greater Scaup**, Keith
38. **Redhead**, Keith
39. **Canvasback**, Keith
40. **Green-winged Teal**, Keith
41. **Mallard**, Keith
42. **American Wigeon**, Keith
43. **Eurasian Wigeon**, Keith
44. **Gadwall**, Keith
45. **Canada Goose**, Keith
46. **Cackling Goose**, Keith
47. **American Goldfinch**, Keith
48. **American Kestrel**, Keith
49. **California Gull**, Keith
50. **House Sparrow**, Keith
51. **Ross's Goose**, Keith
52. **Snow Goose**, Keith
53. **Rock Pigeon**, Keith
54. **Western Meadowlark**, Keith
55. **Northern Harrier**, Keith
56. **American Tree Sparrow**, Lincoln
57. **House Finch**, Lincoln
58. **Red-breasted Nuthatch**, Lincoln
59. **Couch's Kingbird**, Lincoln
60. **Downy Woodpecker**, Lincoln
61. **Great-tailed Grackle**, Lincoln
62. **Barrow's Goldeneye**, Lincoln
63. **Ring-necked Duck**, Lincoln
64. **Northern Pintail**, Lincoln
65. **Northern Shoveler**, Lincoln
66. **Greater White-fronted Goose**, Lincoln
67. **Horned Lark**, Keith
68. **Wild Turkey**, Garden
69. **Red-winged Blackbird**, Garden
70. **Ferruginous Hawk**, Garden
71. **Sharp-shinned Hawk**, Garden
72. **Northern Shrike**, Morrill
73. **Prairie Falcon**, Scotts Bluff
74. **Ring-necked Pheasant**, Scotts Bluff
75. **Cooper's Hawk**, Scotts Bluff
76. **White-crowned Sparrow**, Scotts Bluff
77. **Black-capped Chickadee**, Scotts Bluff
78. **Great Horned Owl**, Scotts Bluff
79. **Red Crossbill**, Sioux
80. **Black-billed Magpie**, Sioux
81. **Belted Kingfisher**, Sioux
82. **Golden Eagle**, Sioux
83. **Rough-legged Hawk**, Sioux
84. **Merlin**, Sioux
85. **Common Raven**, Sioux
86. **Sharp-tailed Grouse**, Sioux
87. **Gray-crowned Rosy-Finch**, Sioux
88. **Lapland Longspur**, Sioux
89. **Pine Siskin**, Morrill
90. **Pinyon Jay**, Morrill
91. **Rock Wren**, Scotts Bluff
92. **Townsend's Solitaire**, Scotts Bluff
93. **Pygmy Nuthatch**, Scotts Bluff
94. **Snow Bunting**, Platte
95. **White-throated Sparrow**, Lancaster
96. **Carolina Wren**, Lancaster
97. **Brown Creeper**, Lancaster
98. **Golden-crowned Kinglet**, Lancaster
99. **Barred Owl**, Lancaster
100. **Mourning Dove**, Lancaster
101. **Yellow-rumped Warbler**, Lancaster
102. **Eastern Bluebird**, Lancaster
103. **Peregrine Falcon**, Lancaster
104. **Trumpeter Swan**, Sarpy
105. **Harris's Sparrow**, Cass
106. **Tufted Titmouse**, Cass
107. **Fox Sparrow**, Sarpy
108. **Tundra Swan**, Douglas
109. **Purple Finch**, Lancaster
110. **Hermit Thrush**, Lancaster
111. **Red-shouldered Hawk**, Seward
112. **Red-headed Woodpecker**, Sarpy
113. **Pileated Woodpecker**, Sarpy
114. **Yellow-bellied Sapsucker**, Lancaster
115. **Northern Bobwhite**, Lancaster
116. **Hooded Merganser**, Cass
117. **Sandhill Crane**, Lancaster
118. **Brewer's Blackbird**, Lancaster


Nebraska Big Year: January Recap


In Django, fields with `auto_now=True` are not always updated correctly. A common practice to work around this is to override the `save()` method on the model, or to use a pre-save hook. However, this doesn't work when calling `Model.objects.update()` since `save()` is not called on all instances in the queryset.

A common way to fix this would be to alter the table, for example, in MySQL it may look like this:

```sql
ALTER TABLE app_model
MODIFY updated_at
DEFAULT CURRENT_TIMESTAMP
ON UPDATED CURRENT_TIMESTAMP
```

But, this doesn't translate well into a migration and doesn't work with sqlite3 and other db backends.

I have found a simple way to fix this which will work with any database backend, so you don't need to write migrations or alter your database tables. We can do this with a simple queryset method override. All we are doing here is adding the current timestamp to the update kwargs, so all instances in the queryset will get that update!

```python
from django.db.models import query, manager
from django.utils import timezone

class BaseQuerySet(query.QuerySet):

    def update(self, **kwargs):
        kwargs['updated_at'] = timezone.now()
        super().update(**kwargs)

class Manager(manager.BaseManager.from_queryset(BaseQuerySet)):
    pass
```

Then to use this, just add it to your model like so.

```python
from django.db import models
from .manager import Manager

class BaseModel(models.Model):
    objects = Manager()
    updated_at = models.DateTimeField()
```

Hope this helps, and thanks for reading!


Auto-updated timestamp fields in Django bulk querysets


Instead of rolling my own progress bars on new batch management commands, I've been using `tqdm` lately. Here's a quick snippet for multiprocessing a batch of items in a queryset.

You'll need tqdm to run this: `pip install tqdm`

**Note:** Since the function that I'm calling is using django's ORM, I close all db connections. Things tend to go wrong when forked processes are all using the same connection.

```python
from django.db import connection
from django.core.management.base import BaseCommand

from multiprocessing import Pool
from tqdm import tqdm

from project.models import Item

def func(item):
    connection.close()
    # Do some process
    return True


class Command(BaseCommand):

    def add_arguments(self, parser):
        parser.add_argument(
            '--processes',
            dest='processes',
            default=2,
            type=int,
            help='Number of processes to use to make run command on queryset',
        )

    def handle(self, *args, **options):
        items = Item.objects.all()

        with tqdm(total=items.count()) as bar, Pool(processes=options['processes']) as pool:
            for _ in pool.imap_unordered(func, items):
                bar.update()
```


Django Mulitprocessing with Progress Bar


Django querysets are really powerful. They can often be used to reduce expensive object iterations down to the database level. Let's take the following models and figure out which product has the least in stock.

```python
from django.db import models

class Product(models.Model):
inventory = models.IntegerField()

class Package(models.Model):
items = models.ManyToMany(Product)

class Order(models.Model):
customer = models.ForeignKey(Customer)
package = models.ForeignKey(Package)
```

The naiive implementation may look something like this. The loops and temp variables probably remind you of something you wrote in CS 101.

```python
def least_stock():
  # Which product has the least in stock?
  least = None
  least_stock = 100000

  for product in Product.objects.all():
      ordered = sum([p.orders.count() for p in product.packages.all()])
      in_stock = product.inventory - ordered
      if in_stock < least_stock:
          least = product
          least_stock = in_stock
  return least
```

This method is extremely expensive. Notice how it first queries for all of the packages that contain that product. Then for each of those packages, it runs a count to get the number of orders. This is `n + 1` database queries, where `n` is the number of packages that a product is in. I would run some benchmarking on this, but it is obvious that the running time would be horrible.

We can definitely do better, using a powerful queryset function built into django: `annotate`. Annotating allows you to add fields to the query using various functions that translate down to SQL. Using queryset functions, the sum becomes a count, the loop becomes an annotation, and the comparison beecomes an order_by. All of which is done at that database level.

```python
from django.db.models import Count, F

def least_stock():
    # Which product has the least in stock?
    return Product.objects.annotate(
        ordered=Count('packages__orders')
    ).annotate(
        in_stock=F('inventory') - F('ordered')
    ).order_by('in_stock').first()
```

This implementation uses annotate twice to significantly reduce the number of queries on the database. In fact, this results in a single query!

The [Django Docs page on queries](https://docs.djangoproject.com/en/1.11/topics/db/queries/) has many more examples. Using annotations and aggregates can siginificantly improve the performance of your django site.


Django Queryset Annotations for Increased Performance


A problem that many applications (hopefully) eventually run into is scalability. In a project I'm working on, we have a django management command that makes a time consuming API call to update a model. Let's take a look at the original code:

```python
class Command(BaseCommand):
    help = 'Create Shipping labels for upcoming shipments'

    def handle(self, *args, **options):
        start = datetime.date.today()
        end = start + datetime.timedelta(14)

        for shipment in Shipment.objects.filter(shipping_date__range=[start, end]):
            create_shipping_label(shipment)
```

At first, this seems like an innocent little script. It does exactly what it is supposed to do. For every shipment that is upcoming, let's create the shipping label.

Notice anything wrong with it though? What if we now have 20,000 shipments? Even if the API call takes only 1 second, that's still about 6 hours of processing time, for one seemingly innocent command.

Enter multiprocessing. As you may know, computers these days have anywhere from 2 to 16 cores in the CPU. When this script above is running, it's really only using one! By using the stdlib package for multiprocessing, we can run this script up to 16x faster. (8x faster on my i7, 4x faster on many common CPUs).

Here's the updated command to take advantage of multiprocessing:

from multiprocessing import cpu_count, Pool

```python
class Command(BaseCommand):
    help = 'Create Shipping labels for upcoming shipments'

    def handle(self, *args, **options):
        start = datetime.date.today()
        end = start + datetime.timedelta(14)

        shipments = Shipment.objects.filter(shipping_date__range=[start, end]):

        pool = Pool(processes=cpu_count())
        pool.map(create_shipping_label, shipments)
```

Those two lines of code create a thread pool of as many CPUs as possible, and simply map the iterator onto the function. Now that script can run 4 or more of those API calls at the same time, reducing the running time of the script significantly


Mutiprocessing in Python Django Management Commands


A problem I recently ran into was allowing for multiple themes to be used on the same project. I decided to set a django setting called `BRAND` in order to control the theme through separate css files.

For this example, we will assume that your static directory has sub-directories for each brand, with a file `site.css` in each.

Now, in order to inject our brand setting into the context, we need to add a context processor. Create a file called `context.py` in your project.

```python
from django.conf import settings

def brand_details(request):
    return {'BRAND': settings.BRAND}
```

This simple function injects the `BRAND` setting into the template context. Next we need to add the context processor to our template settings.

```python
BRAND = 'brand1'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': ['your-template-dir'],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                ...
                'project.context.brand_details',
            ],
        },
    },
]
```

Lastly, we need to use the injected variable to pull out the correct theme. In the head of your base.html file, we will use the variable to get the correct brand css file.

```html
{% load staticfiles %} {% with 'css/'|add:BRAND|add:'/site.scss' as brand_style
%}
<link rel="stylesheet" type="text/x-scss" href="{% static brand_style %}" />
{% endwith %}
```

There you have it! Django themes with a simple site setting. If you wanted to take this a step further, you could use the variable to pull in base templates. Also, that context processor function could run a query to get the theme, allowing you to edit the theme from the admin instead of the settings.py.


Using Template Variables to control Django Themes


As your Django app grows larger, your test suite starts piling up. In a project I was working on we had a test suite of 375 tests. Today I'll explain how I reduced the run time of that suite from 350+ seconds to 20 seconds.

### 1. Mock, mock, mock!

A HUGE bottleneck of our tests was in the billing logic. We had many tests that were actually hitting billing APIs (on test accounts of course, but still really bad). By mocking those calls you can significantly reduce the testing time. Take this test that mocks if a customer has a card on file. By mocking that `can_charge` call and setting the return value, we avoid an API call and can still test that our code works as expected.

```python
import mock
from django.test import Client

@mock.patch('billing.utils.can_charge')
def test_cant_charge_redirect(can_charge):
    can_charge.return_value =False
    response = Client().get('/checkout/')
    self.assertRedirects(response, '/checkout/add-card/')

@mock.patch('billing.utils.can_charge')
def test_can_charge_ok(can_charge):
    can_charge.return_value = True
    response = Client().get('/checkout/')
    self.assertEqual(response.status_code, 200)

```

You can also mock simple model unit tests. Instead of hitting the database by creating models, use Mock objects to simulate your model

```python
import mock
from models import User

def test_full_name():
    user = mock.Mock(spec=User)
    user.first, user.last = 'Test', 'Test'
    self.assertEqual(user.full_name, 'Test Test')
```

### 2. Remove unnecesary Apps

In my project, we had over 30 apps installed. By overriding settings for tests, I could remove unneeded apps like `django_debug_toolbar`, `django_extensions`, etc.

### 3. Turn debug off

You might be surprised, but setting `DEBUG = False` while running tests reduces the amount of debugging overhead that django takes and will improve your the speed of your test suite.

### 4. Remove unneeded middleware

We can assume that all of Django's middelware works correctly, so while testing, remove it to avoid all that overhead when making requests with the test client. Our final testing middleware settings looked like this.

```python
MIDDLEWARE_CLASSES = [
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
]
```

### 5. Only use one password hasher

This will save time when creating users or logging in with your test client. Md5 is the simplest password hasher, and since you don't need to worry about hashing strength during tests, you can get rid of all of the others.

```python
PASSWORD_HASHERS = [
    'django.contrib.auth.hashers.MD5PasswordHasher',
]
```

### 6. Disable logging

There's no need to add file I/O overhead to your testing suite, so disable it!

```python
import logging
logging.disable(logging.CRITICAL)
```

### 7. Use sqlite

Unless you are testing database specific things, just use sqlite for testing. It is the fastest DB for small amounts of data.

```python
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': 'test_database',
    }
}
```

## Putting it all together

All of this can be wrapped up nicely in `manage.py`! I also added another block looking for `--time` in the arguments. This bit of code overwrites the `setUp` and `tearDown` methods of django's `TestCase` to print out the total time taken to run a test (if it's over 0.5s). Using this code I uncovered multiple tests that took more than 10 seconds!

```python
#!/usr/bin/env python
import os
import sys

if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "marketplace.settings")

    from django.core.management import execute_from_command_line
    from django.conf import settings

    if 'test' in sys.argv:
        import logging
        logging.disable(logging.CRITICAL)
        settings.DEBUG = False
        settings.TEMPLATE_DEBUG = False
        settings.PASSWORD_HASHERS = [
            'django.contrib.auth.hashers.MD5PasswordHasher',
        ]
        settings.DATABASES = {
            'default': {
                'ENGINE': 'django.db.backends.sqlite3',
                'NAME': 'test_database',
            }
        }
        settings.MIDDLEWARE_CLASSES = [
            'django.contrib.sessions.middleware.SessionMiddleware',
            'django.middleware.csrf.CsrfViewMiddleware',
            'django.contrib.auth.middleware.AuthenticationMiddleware',
            'django.contrib.messages.middleware.MessageMiddleware',
        ]

    if 'test' in sys.argv and '--time' in sys.argv:
        sys.argv.remove('--time')
        from django import test
        import time

        def setUp(self):
            self.startTime = time.time()

        def tearDown(self):
            total = time.time() - self.startTime
            if total > 0.5:
                print("nt033[91m%.3fst%s033[0m" % (
                    total, self._testMethodName)

        test.TestCase.setUp = setUp
        test.TestCase.tearDown = tearDown

    execute_from_command_line(sys.argv)
```

I hope some of these tips help you to speed up your test suite. Happy coding!
