July 2024 - todnewman.com

July 26, 2024June 1, 2025

Thoughts on Motivation Management

AI generated Image “visualizing the management of worker motivation”

I mentioned in my last post on knowledge-work productivity that understanding how to manage motivation to do specific tasks will result in overall higher productivity. As someone who has thought about this (and experimented with approaches) for years, here are some of my thoughts:

Set up Spaces Focused on Improving Motivation: This is a general idea to increase your productivity overall. It also includes the notion from my last entry on investing in quality tools. I’ve learned that it is not smart to skimp on the cost of tools (computers, notebooks, monitors, other work infrastructure) that are higher quality and are likely to give you joy to work with. Obviously, one can’t always have the best of everything, but if there are key, strategic tools that you work with, my experience is it is smart to spend the extra $50 or 100 bucks. Especially for a tool you may spend years working with. Key examples of this are 1) my mechanical keyboard – typing on this keyboard is satisfying in some weird way and I find that I’m unhappy working without it. 2) My noise cancelling Sony wireless headphones – What a lucky thing that I spent the money (they’re not cheap) right before COVID to buy these things… I originally had intended them for use on flights, but of course, COVID ended that idea and it morphed into use during Zoom calls. Was an absolute lifesaver for remote work and I was constantly grateful for them.
1. In addition to quality tools, though, envisioning how your whole workspace could be “architected” to draw you in to work more effectively is time well spent. Things like KVM switches or USB hubs that allow the switching between computers are important for me, because they allow the quick, painless switching between a work computer and one or two personal computers. If you have a powerful personal computer (I use a Macbook Air and a powerful Linux workstation with big GPU’s) or two that are better for use with certain kinds of tasks, then the ability to switch common resources like the big monitors, mechanical keyboard, wireless mouse, or upgraded camera between them makes switching a breeze.
2. Obviously things like a good chair and desk are important. Some people revel in their standing desks because it allows them to work in completely different contexts. I also like my office decorations (lots of tall ships and John Wayne paintings) because they make me happy to be in the office, which can improve my motivation to work.
Track your Work: I think we all feel motivated when we make visible and measurable progress on a task. This, to me, is the chief value of a tracking schedule. I like checking off the boxes on a task! An example beyond the tracking schedule is the word count trackers that I use when I’m working on a book. I always have a daily goal to write some manageable number of words (usually like 100 or so) in the hope that I actually sit down, get in the zone, and crank out thousands of words. Knowing the value of visibility of work, I add to this a tracker where I have two columns, date and cumulative word count. I then do a simple line plot that shows the slope of my writing accomplishments. Some days, I write more and then get to see a steep slope of word count output for the day or week. This motivates me to keep it going! I refer to the word count tracker as a commitment device, but I also think of my IOS “Streaks” app (well worth the $3 or so on the app store) is also a commitment device that also helps build habits. And building habits gets us in that seat to do that writing (or gets us to the gym to lift, or reminds us to floss every day).
Manage an Overall Work List: I have found that having a quite varied list of things that need to be accomplished helps with the times when I just don’t feel like doing anything. I’ve learned that when I feel this way, that I can almost always find something on the overall work list to do. If I’m feeling more like refinishing a piece of furniture than building the AI classifier for the dataset I’ve been looking at, I am able to work without any sense of guilt on the furniture. My experience is that there will come a moment when I’m very motivated to explore that dataset and if I ride the wave of that motivation, I’ll do the work much more efficiently.
Know When You Work Well on Types of Tasks: I know that in the early morning, I’m much, much better at tasks that require creativity and challenge my thinking. Conversely, right around 2 PM after I’ve eaten lunch is NOT a good time for these tasks. I have found that 2 PM is better for walking around and doing gardening tasks or working on other more physical kinds of chores. Don’t fight these kinds of patterns!
Celebrate Completed Work: I have a strong bias to finishing tasks that I’m working on instead of just pushing them forward a few steps. I think this allows one to be much more productive. So to support this, I celebrate completions. Perhaps finishing a major task means that I drive to Dairy Queen for a sundae. Or maybe it just earns me a short nap (I’m a big fan of the 13 minute nap for rejuvenation). The act of celebrating a completion is important and helps you build enthusiasm for the next task.
Work to Identify how to fit your Motivation into an Employer’s Goals: OK, sometimes your employer doesn’t have the intimate knowledge of your motivation cycles and just wants you to work 9 hours straight on the most important tasks. If anyone is able to do this day after day, I’d love to know about you! Your employer isn’t really able to measure your productivity and assumes that measuring your hours-worked (and maybe how late you stay at the office) is the best proxy for productivity. Of course this is completely false. Any employer who cares at all about quality and productivity SHOULD be focused on tapping their employees’ best hours for the jobs. So many mistakes have been made by employees who were burned out, mentally exhausted, and working on activities they are far over-qualified for. Since the employer can’t manage this, somehow the employee needs to focus hard on maximizing their productivity through working tasks when they are most fit (and motivated) to work them. There is probably a whole lot more to be said about this, and possibly some of it would be controversial. I heard the quote once, “good employees do exactly what their employer tells them to do. Great employees conspire to make their employer astonishingly successful.” This is interesting to consider, especially in light of managing your motivation cycles.

Do you have any thoughts or insights on the management of motivation? We’d love to hear from you in the comments below!

If you want to be added to our newsletter (no worries, we won’t overwhelm you), we often create bonus material only for newsletter subscribers. Just sign up with your email address below.

July 19, 2024June 1, 2025

Rethinking Productivity in Knowledge Work

The Hidden Challenge of Measuring Knowledge Work

In my data-driven career, I’ve always been fascinated by one persistent question: how do we actually measure productivity when the output is primarily intellectual? While my normal posts focus on data analysis, today I want to explore something that stumped me for years—measuring productivity in design work and other knowledge-intensive fields.

Here’s the problem: when your work creates tangible products, productivity is straightforward. But what about when your “product” is an idea, a design, or a decision?

The Drafting Bottleneck: A Case Study

Let me share a concrete example from my experience. In system design, we could easily measure one particular bottleneck: drafting.

Why was drafting so critical? Because:

Factory assembly depends on these drawings
Supply chain operations use these documents as entry points
Quality assurance needs these specifications
The “recipe” for building complex systems lives in these documents

Drafting productivity was simple to define—one product, limited multitasking, clear start/finish times. We could calculate dollars per hour with reasonable accuracy.

But everything else in the design process? Nearly impossible to measure cleanly. Work spread across multiple teams with complex dependencies, where the “product” might be invisible—concepts, processes, models, and fragments of documentation.

A Better Approach to Knowledge Work Productivity

After wrestling with this challenge for years, I recently discovered a YouTube video by Cal Newport that resonated deeply with my experience. Newport offers three principles for improving knowledge work productivity that I find incredibly valuable:

1. Do Fewer Things at Once

We all know multitasking kills productivity, yet most of us still believe we’re the exception. The real insight isn’t just that you’re leaking efficiency—it’s that many of your activities may have zero impact on your results or satisfaction.

2. Work at a Natural Pace

This doesn’t mean “go slow.” It means finding your natural rhythm. Ask yourself: How do YOU work best? Would stepping away to think improve your outcome? Should you spend more time preparing before diving in (like building UML models before coding)?

3. Obsess Over Quality

Newport makes a brilliant observation here. When you invest in quality tools—like a premium notebook or a computer you love using—you signal to yourself that your work matters. It becomes psychologically harder to do sloppy work when you’re using tools you respect.

4. My Addition: Manage Your Motivation

Here’s my contribution to Newport’s list: Your productivity skyrockets when you’re genuinely motivated. I’ve found my work quality improves by an order of magnitude when I’m “all in” on a project.

Unfortunately, we can’t always work on the “blue widget” precisely when motivation strikes. But those obligation-driven sessions often correlate with lower quality and more distractions (hello, Reddit!).

The trick is creating “commitment devices” that help channel motivation into consistent work. As someone who loves writing, I’ve learned that my most creative passages emerge when inspiration hits—but I need systems to ensure that motivation translates into daily writing sessions.

The Bottom Line

Knowledge work productivity remains difficult to define and measure. While Newport’s principles (and my addition) won’t solve the measurement challenge, they offer something more valuable—practical ways to improve the quality and output of your most critical work, regardless of whether anyone is measuring it.

What strategies have you found most effective for your knowledge work productivity? Let us know in the comments below!

If you want to be added to our newsletter (no worries, we won’t overwhelm you), we often create bonus material only for newsletter subscribers. Just sign up with your email address below.

July 13, 2024July 25, 2025

Did Banning “The Shift” Actually Change Baseball? A Data Analysis

The Question That Started It All

In 2023, Major League Baseball introduced new rules restricting where defensive players could position themselves on the field. This change effectively banned “The Shift” – a defensive strategy that had become increasingly popular since 2006. At the time, I suspected this ban would be nothing more than a band-aid solution with little real impact on the game. Now, with a full season of data available, we can finally test that hypothesis.

Positioning of infielders under new rules' — image from https://www.mlb.com/glossary/rules/defensive-shift-limits

What Was “The Shift” Anyway?

The concept of shifting defensive players isn’t new – teams have been doing it sporadically throughout baseball history. However, the modern version of The Shift was pioneered by the Tampa Bay Rays in 2006 when manager Joe Maddon used advanced statistics (sabermetrics) to devise a strategy against Boston Red Sox slugger David “Big Papi” Ortiz.

The data showed that Ortiz, a left-handed power hitter, pulled nearly every ball to the right side of the field. So the Rays stacked extra defenders on that side, leaving the left side of the infield nearly empty. The strategy worked brilliantly – Ortiz’s batting average dropped from over .300 (2004-2006) to .265 by mid-2006 as other teams copied the approach.

The Shift became controversial because it was primarily used against baseball’s biggest stars and most exciting hitters, making the game feel less dynamic and offensive. Eventually, MLB decided enough was enough and implemented restrictions for the 2023 season.

My Approach to Testing the Impact

To determine whether banning The Shift actually changed anything, I needed to compare player performance before and after the ban. But this presented a challenge – what if other factors (like changes to the baseball itself or new pitching rules) also affected hitting during this period?

My solution was to analyze two groups of players:

Sluggers: The power hitters who were most likely to face The Shift (defined as players with above-average slugging percentage)

Non-sluggers: Regular hitters who rarely, if ever, faced The Shift

If banning The Shift was the primary driver of any performance changes, we should see significant improvements mainly among sluggers, with little change among non-sluggers.

The Data Collection Process

I used the Python library pybaseball to gather statistics from 2006 through 2024, focusing on key offensive metrics: at-bats, hits, doubles, triples, home runs, and walks. I divided this data into two eras:

“Shift Era”: 2006-2022
“Post-Shift Era”: 2023-2024

To ensure I was analyzing players who actually faced The Shift regularly, I set minimum at-bat thresholds (400, 600, and 800) and categorized sluggers by how far above average their performance was – from the top 50% down to the elite top 1%.

What the Numbers Revealed

The Initial Results Looked Promising

When I first compared the average performance between the two eras, the results seemed to support the idea that banning The Shift helped hitters. Across the board, offensive numbers were higher in the post-ban period. Problem solved, right?

Top 16% of Sluggers Compared Before and After the Shift was Banned. Also non-Sluggers for Comparison

Not so fast. Averages can be misleading, and I needed to determine whether these differences were statistically significant or just random variation.

The Statistical Reality Check

To get a more complete picture, I used the Kolmogorov-Smirnov test, which compares entire distributions rather than just averages. This test tells us whether two groups of data are fundamentally different or could reasonably come from the same underlying population.

Using a 95% confidence interval (the standard threshold for statistical significance), here’s what I found:

For Non-sluggers: Here’s where things got interesting. The non-sluggers showed statistically significant improvements across nearly all offensive categories – even more consistently than the sluggers did.

p-values comparing non-sluggers before and after the ban. The red line is our confidence interval. Any bars below the red line indicate that metric is statistically different before and after the Shift

For Sluggers: Only the top 16% of power hitters showed statistically significant improvements in most categories (hits, triples, home runs, and walks – but not doubles). The other slugger groups showed mixed results.

p-values comparing Sluggers before and after the ban. The red line is our confidence interval. Any bars below the red line indicate that metric is statistically different before and after the Shift

The Surprising Conclusion

This finding was the key to understanding what really happened. Since The Shift was rarely used against non-sluggers, it couldn’t be responsible for their improved performance. Yet these “regular” hitters showed more consistent statistical improvements than the power hitters who were supposedly being helped by the ban.

The evidence points to a clear conclusion: something other than banning The Shift was responsible for the improved offensive performance across baseball.

Whether it’s changes to the baseball itself, new rules affecting pitchers, evolving hitting approaches, or other factors, the data suggests that banning The Shift had minimal impact on the game. In fact, sluggers (the players the rule was designed to help) showed less consistent improvement than the players who were never affected by The Shift in the first place.

Final Thoughts

My initial skepticism about the Shift ban appears to have been justified. While offensive numbers did improve after 2023, this improvement affected all types of hitters equally – not just the power hitters who faced The Shift. This pattern strongly suggests that other factors were driving the change.

Sometimes the most interesting finding is discovering that the obvious explanation isn’t the right one. In this case, banning one of baseball’s most visible and controversial strategies appears to have been largely symbolic rather than transformative.

Note: This analysis covers data through 2024. As more seasons pass, we’ll get an even clearer picture of The Shift ban’s true impact – or lack thereof.

To see categories of my sports analytic posts, pick one of the below:

Is this kind of article interesting to you? Let us know in the comments below.

If you want to be added to our newsletter (no worries, we won’t overwhelm you), we often create bonus material only for newsletter subscribers. Just sign up with your email address below.