Gadgets. Tech. Data Warehouse. Business Intelligence. NFL. College Football.
889 stories

Power BI NFL Football Stats Comparisons and Analysis Report is now available!

1 Share

fantasy football data and analyticsFor the past few years I’ve combined my love of professional football and analytics by releasing a series of Power BI reports featuring player statistics. This year is no different. This year I’m finally able to release my NFL Football Stats, Comparisons, and Analysis reports featuring the stats for players at the quarterback, runningback, wide receiver, tightend, and kicker positions. Unlike previous years, this years reports are based on data from My goal for producing this report is, as nerdy as this is, to give me a leg up on my fantasy football drafts. If you’ve ever played fantasy football, you know the key to winning is having the deepest roster and I hope that these reports will allow me to identify middle and late round talent using the collected data in a readable and navigable format.

View the Power BI NFL Stats, Comparisons, and Analysis Reports here

I think this is probably my best version of the NFL stats report that I’ve released yet, and there’s a few reasons why I think so. First, I included comparisons between each individual player and league averages, which I hope will provide valuable insight into where each player stacks up compared to the rest of the league.

Secondly, I included colored KPIs on the Team Dashboard to indicate how a player compares to the league. For instance, if a running back has a green KPI for Yards Rushing, that means they rushed for 10% more yards compared to the rest of the league. Yellow would indicate the player rushed for between 90% and 110% of the league average, and red would mean they only rushed for less than 90% of the league average.

Also, I included filters on the analysis reports for

position groups allowing you to filter for player that obtained a certain number of yards, receptions, touchdowns, etc. So if you were interested in only looking at wide receivers that had at least 1000 yards receiving and caught at least 10 touchdowns, you could use the slider bars to accomplish that type of analysis.

The Team Dashboard

After the title page, the first page is the Team Dashboard and features an overall team analysis. You can use the filter box in the top right to filter down to one team at a time. Use the year filter to look at the stats for each team for the previous 3 seasons. The quarterback, runningback, and receiver position groups are displayed with KPIs below.

football team dashboard stats

Quarterback Analysis and Comparisons

The next two pages feature the QB Analysis and QB Comparison reports. Use the Quarterbacks Analysis page to identify top performing quarterback but also identify which quarterback might be considered above average but not top tier.

NFL football quarterbacks stats analysis

Once you’ve narrowed down your choices for a QB, use the QB Comparisons page the compare the stats between the two player.

NFL football quarterback comparison stats analysis

Runningback Analysis and Comparisons

The RB Analysis and RB Comparisons reports are very similar to the QB Analysis and Comparisons with the exception of course being that the information displayed is limited to the runningback position group. One of the interesting things easily seen below in the scatter plot is that there is no big difference between the players Howard, Murray, and McCoy when comparing rush yards and receiving yards.

NFL football runningbacks stats analysis

NFL football runningbacks comparison

Receiver Analysis and Comparisons

Then you have the Receiver Analysis and Comparisons reports displaying stats and information for receivers, including wide receivers and tight ends.

NFL receivers stats analysis

NFL football tightend stats

NFL football receivers tightends stats comparison

Kicker Analysis and Comparisons

And of course, you’ve got the Kickers, the most important position in ANY fantasy football draft.

NFL football kicker data analysis

NFL football kicker comparisons

I hope you find these Power BI reports useful. If you have any questions or feedback, feel free to leave a comment down below. If you happen to uncover and bugs, feel free to let me know!


View the Power BI NFL Football Stats Comparisons and Analysis Report here

Download the Power BI Desktop file here

Read the whole story
30 days ago
santa clara, CA
Share this story

The Moral Shambles That is Our President

3 Comments and 11 Shares

Denouncing Nazis and the KKK and violent white supremacists by those names should not be a difficult thing for a president to do, particularly when those groups are the instigators and proximate cause of violence in an American city, and one of their number has rammed his car through a group of counter-protestors, killing one and injuring dozens more. This is a moral gimme — something so obvious and clear and easy that a president should almost not get credit for it, any more than he should get credit for putting on pants before he goes to have a press conference.

And yet this president — our president, the current President of the United States — couldn’t manage it. The best he could manage was to fumble through a condemnation of “many sides,” as if those protesting the Nazis and the KKK and the violent white supremacists had equal culpability for the events of the day. He couldn’t manage this moral gimme, and when his apparatchiks were given an opportunity to take a mulligan on it, they doubled down instead.

This was a spectacular failure of leadership, the moral equivalent not only of missing a putt with the ball on the lip of the cup, but of taking out your favorite driver and whacking that ball far into the woods. Our president literally could not bring himself to say that Nazis and the KKK and violent white supremacists are bad. He sorely wants you to believe he implied it. But he couldn’t say it.

To be clear, when it was announced the president would address the press about Charlottesville, I wasn’t expecting much from him. He’s not a man to expect much from, in terms of presidential gravitas. But the moral bar here was so low it was on the ground, and he tripped over it anyway.

And because he did, no one — and certainly not the Nazis and the KKK and the violent white supremacists, who were hoping for the wink and nod that they got here — believes the president actually thinks there’s a problem with the Nazis and the KKK and the violent white supremacists. If he finally does get around to admitting that they are bad, he’ll do it in the same truculent, forced way that he used when he was forced to admit that yeah, sure, maybe Obama was born in the United States after all. An admission that makes it clear it’s being compelled rather than volunteered. The Nazis and the KKK and the violent white supremacists will understand what that means, too.

Our president, simply put, is a profound moral shambles. He’s a racist and sexist himself, he’s populated his administration with Nazi sympathizers and white supremacists, and is pursuing policies, from immigration to voting rights, that make white nationalists really very happy. We shouldn’t be surprised someone like him can’t pass from his lips the names of the hate groups that visited Charlottesville, but we can still be disappointed, and very very angry about it. I hate that my baseline expectation for the moral behavior of the President of the United States is “failure,” but here we are, and yesterday, as with previous 200-some days of this administration, gives no indication that this baseline expectation is unfounded.

And more than that. White supremacy is evil. Nazism is evil. The racism and hate we saw in Charlottesville yesterday is evil. The domestic terrorism that happened there yesterday — a man, motivated by racial hate, mowing down innocents — is evil. And none of what happened yesterday just happened. It happened because the Nazis and the KKK and the violent white supremacists felt emboldened. They felt emboldened because they believe that one of their own is in the White House, or at least, feel like he’s surrounded himself with enough of their own (or enough fellow travelers) that it’s all the same from a practical point of view. They believe their time has come round at last, and they believe no one is going to stop them, because one of their own has his hand on the levers of power.

When evil believes you are one of their own, and you have the opportunity to denounce it, and call it out by name, what should you do? And what should we believe of you, if you do not? What should we believe of you, if you do not, and you are President of the United States?

My president won’t call out evil by its given name. He can. But he won’t. I know what I think that means for him. I also know what I think it means for the United States. And I know what it means for me. My president won’t call out evil for what it is, but I can do better. And so can you. And so can everybody else. Our country can be better than it is now, and better than the president it has.

Read the whole story
36 days ago
santa clara, CA
37 days ago
Share this story
3 public comments
37 days ago
The golf analogy was icing on the cake.
Denver, CO
37 days ago
You are making hoops for the president to jump through, he denounced all side for the violence. He has denounced the violence you know it, you're response is weak and without substance. Why don't you talk about the cause of this and their numbers growing? Hint it is not Trump.
Bangkok, Thailand
37 days ago
“Trump comments were good. He didn’t attack us. He just said the nation should come together. Nothing specific against us….. There was virtually no counter-signaling of us at all. He said he loves us all…. No condemnation at all. When asked to condemn, he just walked out of the room. Really, really good. God bless him.” ~The Daily Stormer
37 days ago
There are not multiple sides to this.
36 days ago
There aren't multiple sides, care to explain, that seems a refutation of reality as much a commonsense. Seems both of you perhaps are living in a bubble, Quit it with the might is right reasoning in such attitudes and ignorance. Recognize the poison there on both sides.Ignoring this collectivism of any sort is going to end poorly for everyone.
37 days ago
GOP. Delenda. Est.

I Only Have 7 Trips Left. On Managing Work / Life Balance, Love & Family

1 Comment and 2 Shares

Like many people these days, I spent much of my 20’s and early 30’s thinking about work & fun and not too much about “the future.” Like characters from one of my favorite novels “The Unbearable Lightness of Being” life seemed very light.

My first son was born the day before my 35th birthday so the decade that followed was very heavy and consequential. Life mattered for more than my pure enjoyment — I had to be responsible for the futures of these two lovable, little boys. I still worked hard and the balance of my time and energy went into family. My relationships narrowed to a smaller set of people who really mattered to me, my number of frivolous hobbies dwindled to only the most valuable and time became my scarcest commodity. If you’ve lived a decade with young children you know that it’s both unbelievably rewarding and also physically and emotionally exhausting.

Many of my friends and colleagues also find themselves in the “sandwich years” of aging parents where responsibilities increase for your elders at the same time as for your kids and mortality becomes a reality. During this decade we lost a close family member we loved to cancer and realized that life is too short and if we didn’t take advantage of the blessings we had to spend time together we would be shortchanging ourselves and our children.

So for the past 7 years we have ramped up the amount of sibling, cousin, grandparent, extended family time we could and we have loved every minute of this. I started thinking about “how many Thanksgivings, July 4ths or holidays we really had all together” and when you do the math it is daunting.

I already had a sense of the heaviness (in a good way) of my forties when I came across this excellent post on one of my favorite blogs WaitButWhy entitled “The Tail End,” in which the author uses pictographs to bring the succinctness of life and family time to reality. The author was 34 when he wrote this and estimated that if he’s REALLY lucky he has at best 60 Super Bowls left

If I assume that I’m 10 years less lucky (and live to 84) and I happen to be 49 years old now that means the Eagles have only about 35 more tries to win their first Super Bowl. Now you can see the urgency of Carson Wentz fulfilling his full expectations! It’s on you, Carson. I’ll do my best to make it to 90 but I’m still counting on you.

But seriously I sent this Tail End article recently to my brothers and sister recently to remind them why it was so important that we all get together for Thanksgiving this year. In kid years I have just 4 more Thanksgivings until my eldest son goes to college so I don’t have many to spare. And while I fully expect my children to come back for family vacations post high school, I’m also a realist about life and an advocate of independence.

It took losing my wife’s brother to realize how little time we all had together and the importance of getting together every family vacation we could but I also look at this as the gift that Tom gave us all in our lifetimes. And I think about Tom at every family gathering whether it’s Tania’s family gathering or mine. I am now the age Tom was when he passed away (49) and I don’t take for granted the time I have on this Earth.

So last year I talked with my wife about how few “nuclear family” trips we had been able to take given all of the extended family trips that were so important to us and we committed to doing 2 nuclear family trips per year until Jacob is in college (and of course we plan to continue this for years after and we have Andy for 8 more years!).

I just returned this weekend from our 3rd of 10 trips (70% to go!) and this time I decided not to bring my computer. I put on an out-of-office notice (see below) and received some of the nicest emails and text messages including from my good friend Michael Broukhim who ensures me that he and his brother Danny still vacation with their parents and Mike & Danny are both in their 30s! (I vacationed with my parents, too, until I got engaged at 33).

I was reading the saddest story this morning about a Silicon Valley lawyer who struggled with work/life balance and stress and the pressures of modern life of keeping up with the Jones’s and competing at the top levels in tech startup life. It’s a really sad but important story that I hope you’ll read. It is written by the ex-wife of a corporate lawyer in Silicon Valley who struggled with drug addiction and trying to maintain his status atop his field with the stresses that go with this. She titled it “The Lawyer, the Addict.

I’m not perfect and like many of you still struggle with work / life balance. I was blessed in life not to have chemical dependency issues or depression but I’ve seen it all around me and take the live’s of some people I was close to. It’s why I try to write about and be available to people who suffer from depression. It’s why I try to be open about how stressful being a founder really was and how stressful being a VC is even for an obviously “privileged class” and how physically unhealthy being a founder was to me. As you will see if you read the Lawyer, Addict piece — even highly successful people can succumb to the pressures of peer expectations and relative performance that is entirely self made destruction but real nonetheless.

I love my wife and I love my children. I think some of our fondest memories will be the goofy time we spent during our travels as opposed to the planned itineraries. We’ll remember all of the games of Hearts. We’ll remember when Andy fell down the hill into the bushes (but was ok). We’ll remember throwing the football on the beach with Troy Aikman (the nicest pro football player you’ll ever meet who even with no cameras around and even once he found out we were Eagles fans was still so gracious to my boys). We’ll remember Daddy accidentally shoving an entire Serrano chili pepper into his mouth because it was dark outside and he thought it was a carrot. And we’ll remember how much time Mom spent meticulously planning with love so that our entire family could enjoy every moment.

If you’re caught on the hamster wheel, recognizing it and trying to take some actions is the first step. Having just gotten back from my first proper 2-week vacation (as opposed to extended family gathering) since 2009 I can tell you it was truly life fulfilling. I’m now ready to come back to work feeling really refreshed. As a side note if you’ve never been to Alaska it is truly one of the most beautiful, spectacular and awe-inspiring places I’ve visited. If you want to catch just a few moments of our trip you can find them on Instagram.


Below was my out-of-office reply in hopes of inspiring at least some of you to seek out your own work / life balanced vacations in the years ahead…

4 Years

Thank you for writing to me and forgive me for not responding right away.

About a year ago I was sitting down with my wife and talking about life and realizing that my two boys were about to pass me in height and in their minds they would soon be passing me, too, in worldly knowledge. As if! My eldest son was in 8th grade at the time. Like many of you, my wife & I worked our butts off in our 20s and 30s. When we had kids we did everything we could to balance daily existence, jobs, being great parents and, well, sleep. Every chance for a vacation was an opportunity to see grandparents, aunts & uncles, cousins and childhood friends. We love our families and cherish these visits but it’s different than nuclear-family downtime.

Now we face high school. And we realize we only have 4 years left as a nuclear family until we send Jacob to college. I’m even a bit verklempt as I type this. So Tania & I promised ourselves 2 great trips a year with just our nuclear family. 8 more nuclear-family vacations to create memories that we hope last beyond our time on this planet. We love our boys and our family and at this pivotal moment we also want to model good behavior where we don’t spend the entirety of our trip doing emails or checking Facebook.

So we’re off to Alaska. We won’t be 100% unplugged but we plan to as much as possible so we likely won’t see your email. When I get back I don’t plan to spend 50 hours processing old emails. So here are my asks

1. If it’s urgent please email xxxxxxxxxxx who will help. He really doesn’t mind — even if it’s just directing you to somebody else at Upfront who can help. If it’s future scheduling of a meeting for me please email xxxxxxxxxxx. If it really needs my attention please text me (I don’t mind) but know that we may not have perfect text messaging coverage. Jori has my itinerary and can find me. No, we’re not going on a cruise. Why does everybody always ask that when you tell them you’re going to Alaska?!?

2. If it can wait please email me again on July 15th. This is the single longest true vacation I have taken since 2009 and I can’t tell you how excited I am to recharge the batteries and crush my kids at Hearts.

3. If you find yourself today or in the future at the same life stage as I am, find a way to truly check out. You don’t get these days back. So I’m going to make the most of my 8 trips and 4 years. I hope if you’re able to you will one day, too.

I Only Have 7 Trips Left. On Managing Work / Life Balance, Love & Family was originally published in Both Sides of the Table on Medium, where people are continuing the conversation by highlighting and responding to this story.

Read the whole story
64 days ago
santa clara, CA
Share this story
1 public comment
62 days ago
I find this more depressing than uplifting. Take holidays, be unplugged from work. It's not that important, and you're not that important to it.

Data Virtualization: Unlocking Data for AI and Machine Learning

1 Comment

This post is authored by Robert Alexander, Senior Software Engineer at Microsoft.

For reliability, accuracy and performance, both AI and machine learning heavily rely on large sets. Because the larger the pool of data, the better you can train the models. That’s why it’s critical for big data platforms to efficiently work with different data streams and systems, regardless of the structure of the data (or lack thereof), data velocity or volume.

However, that’s easier said than done.

Today every big data platform faces these systemic challenges:

  1. Compute / Storage Overlap: Traditionally, compute and storage were never delineated. As data volumes grew, you had to invest in compute as well as storage.
  2. Non-Uniform Access of Data: Over the years, too much dependency on business operations and applications have led companies to acquire, ingest and store data in different physical systems like file systems, databases and data warehouses (e.g. SQL Server or Oracle), big data systems (e.g. Hadoop). This results in disparate systems, each with its own method of accessing data.
  3. Hardware-Bound Compute: You have your data in nice storage schema (e.g. SQL Server), but then you’re hardware constrained to execute your query as it takes several hours to complete.
  4. Remote Data: Data is either dispersed across geo-locations, or uses different underlying technology stacks (e.g. SQL Server, Oracle, Hadoop, etc.), and is stored on premises or in the cloud. This requires raw data to be physically moved to get processed, thus increasing network I/O costs.

With the advent of AI and ML, beating these challenges has become a business imperative. Data virtualization is rooted on this premise.

What’s Data Virtualization Anyway?

Data virtualization offers techniques to abstract the way we handle and access data. It allows you to manage and work with data across heterogenous streams and systems, regardless of their physical location or format. Data virtualization can be defined as a set of tools, techniques and methods that let you access and interact with data without worrying about its physical location and what compute is done on it.

For instance, say you have tons of data spread across disparate systems and want to query it all in a unified manner, but without moving the data around. That’s when you would want to leverage data virtualization techniques.

In this post, we’ll go over a few data virtualization techniques and illustrate how they make the handling of big data both efficient and easy.

Data Virtualization Architectures

Data virtualization can be illustrated using the lambda architecture implementation of the advanced analytics stack, on the Azure cloud:

Figure 1: Lambda architecture implementation using Azure platform services 

In big data processing platforms, tons of data are ingested per second, and this includes both data at rest and in motion. This big data is then collected in canonical data stores (e.g. Azure storage blob) and subsequently cleaned, partitioned, aggregated and prepared for downstream processing. Examples of downstream processing are machine learning, visualization, dashboard report generation, so forth.

This downstream processing is backed by SQL Server, and – based on the number of users – it can get overloaded when many queries are executed in parallel by competing services. To address such overload scenarios, data virtualization provides Query Scale-Out where a portion of the compute is offloaded to more powerful systems such as Hadoop clusters.

Another scenario, shown in Figure 1, involves ETL processes running in HDInsight (Hadoop) clusters. ETL transform may need access to referential data stored in SQL Server.

Data virtualization provides Hybrid Execution which allows you to query referential data from remote stores, such as on SQL Server.

Query Scale-out

What Is It?

Say you have a multi-tenant SQL Server running on a hardware constrained environment. You want to offload some of the compute to speed up queries. You also want to access the big data that won’t fit in SQL Server. These are situations where Query Scale-Out can be used.

Query Scale-out uses PolyBase technology, which was introduced in SQL Server 2016. PolyBase allows you to execute a portion of the query remotely on a faster, higher capacity big data system, such as on Hadoop clusters.

The architecture for Query Scale-out is illustrated below.

Figure 2: System-level illustration of Query Scale-Out

What Problems Does It Address?

  • Compute / Storage Overlap: You can delineate compute from storage by running queries in external clusters. You can extends SQL Server storage by enabling access of data in HDFS.
  • Hardware-Bound Compute: You can run parallel computations, leveraging faster systems.
  • Remote Data: You can keep the data where it is, only return the processed result set.

Further explore and deploy Query Scale-out using the one-click automated demo at the solution gallery.

Hybrid Execution

What Is It?

Say you have ETL processes which run on your unstructured data and then store the data in blobs. You need to join this blob data with referential data stored in a relational database. How would you uniformly access data across these distinct data sources? These are the situations in which Hybrid Execution would be used.

Hybrid Execution allows you to “push” queries to a remote system, such as to SQL Server, and access the referential data.

The architecture for Hybrid Execution is illustrated below.

Figure 3: System-level illustration of Hybrid Execution

What Problems Does It Address?

  • Non-Uniform Access of Data: You are no longer constrained by where and how data is stored.
  • Remote Data: You can access reference data from external systems, for use in downstream apps.

Further explore and deploy Hybrid Execution using the one-click automated demo at the solution gallery.

Performance Benchmarks: What Optimization Gains Can You Expect?

You may be asking yourself whether it’s worthwhile using these techniques.

Query Scale-Out makes sense when data already exists on Hadoop. Referring to Figure 1, you may not want to push all the data to HDInsight just to see the performance gain.

However, one can imagine a use case where lots of ETL processing happens in HDInsight clusters and the structured results are published to SQL Server for downstream consumption (for instance, by reporting tools). To give you an idea of the performance gains you can expect by using these techniques, here are some benchmark numbers based on the datasets used in our solution demo. These benchmarks were produced by varying the size of datasets and the size of HDInsight clusters.

Figure 4: Query execution time with and without scaling

The x axis shows the number of rows in the table used for benchmarking. The y axis shows the number of seconds the query took to execute. Note the linear increase in execution time with SQL Server only (blue line) versus when HDInsight is used with SQL Server to scale out the query execution (orange and grey lines). Another interesting observation is the flattening out of execution time of a four versus a two-worker node HDInsight cluster (grey vs. orange line).

Of course, these results are specific to the simplified dataset and schema we provide with the solution demo. With much larger real-world datasets in SQL Server, which typically runs multiple queries competing for resources, more dramatic performance gains can be expected.

The next question to ask is when does it become cost effective to switch over to using Query Scale-Out? The below chart incorporates the pricing of resources used in this experiment. You can see a detailed pricing calculation here.

Figure 5: Query execution time with and without scaling (with pricing)

You can see that with 40 million rows it’s cheapest to execute this query on SQL Server only. But by the time you are up to 160 million rows, Scale-Out becomes cheaper. This shows that as the number of rows increases, it could become cheaper to run with scaling out. You can use these types of benchmarks and calculations to help you deploy your resources with an optimal balance of performance and cost.

Try It Yourself, with One-Click Deployment

To try out the data virtualization techniques discussed in this blog post, deploy the solution demo in your Azure subscription today using the automated one-click deployment solution.

To gain a deeper understanding on how to implement data virtualization techniques, be sure to read our technical guide.


Read the whole story
74 days ago
Good explainer of a very important technology
santa clara, CA
Share this story

Refresh Types

3 Comments and 20 Shares
The hardest refresh requires both a Mac keyboard and a Windows keyboard as a security measure, like how missile launch systems require two keys to be turned at once.
Read the whole story
88 days ago
santa clara, CA
Share this story
3 public comments
88 days ago
Baltimore, MD
88 days ago
Hard Reset - PC reset button - causes SEGA to fight SOPA.
Moses Lake, WA
88 days ago
The hardest refresh requires both a Mac keyboard and a Windows keyboard as a security measure, like how missile launch systems require two keys to be turned at once.

Amazon’s New Customer

1 Comment

Back in 2006, when the iPhone was a mere rumor, Palm CEO Ed Colligan was asked if he was worried:

“We’ve learned and struggled for a few years here figuring out how to make a decent phone,” he said. “PC guys are not going to just figure this out. They’re not going to just walk in.” What if Steve Jobs’ company did bring an iPod phone to market? Well, it would probably use WiFi technology and could be distributed through the Apple stores and not the carriers like Verizon or Cingular, Colligan theorized.

I was reminded of this quote after Amazon announced an agreement to buy Whole Foods for $13.7 billion; after all, it was only two years ago that Whole Foods founder and CEO John Mackey predicted that groceries would be Amazon’s Waterloo. And while Colligan’s prediction was far worse — Apple simply left Palm in the dust, unable to compete — it is Mackey who has to call Amazon founder and CEO Jeff Bezos, the Napoleon of this little morality play, boss.

The similarities go deeper, though: both Colligan and Mackey made the same analytical mistakes: they mis-understood their opponents goals, strategies, and tactics. This is particularly easy to grok in the case of Colligan and the iPhone: Apple’s goal was not to build a phone but to build an even more personal computer; their strategy was not to add on functionality to a phone but to reduce the phone to an app; and their tactics were not to duplicate the carriers but to leverage their connection with customers to gain concessions from them.

Mackey’s misunderstanding was more subtle, and more profound: while the iPhone may be the most successful product of all time, Amazon and Jeff Bezos have their sights on being the most dominant company of all time. Start there, and this purchase makes all kinds of sense.

Amazon’s Goal

If you don’t understand a company’s goals, how can you know what the strategies and tactics will be? Unfortunately, many companies, particularly the most ambitious, aren’t as explicit as you might like. In the case of Amazon, the company stated in its 1997 S-1:’s objective is to be the leading online retailer of information-based products and services, with an initial focus on books.

Even if you picked up on the fact that books were only step one (which most people at the time did not), it was hard to imagine just how all-encompassing would soon become; within a few years Amazon’s updated mission statement reflected the reality of the company’s e-commerce ambitions:

Our vision is to be earth’s most customer centric company; to build a place where people can come to find and discover anything they might want to buy online.

“Anything they might want to buy online” was pretty broad; the advent of Amazon Web Services a few years later showed it wasn’t broad enough, and a few years ago Amazon reduced its stated goal to just that first clause: We seek to be Earth’s most customer-centric company. There are no more bounds, and I don’t think that is an accident. As I put it on a podcast a few months ago, Amazon’s goal is to take a cut of all economic activity.

This, then, is the mistake Mackey made: while he rightly understood that Amazon was going to do everything possible to win in groceries — the category accounts for about 20% of consumer spending — he presumed that the effort would be limited to e-commerce. E-commerce, though, is a tactic; indeed, when it comes to Amazon’s current approach, it doesn’t even rise to strategy.

Amazon’s Strategy

As you might expect, given a goal as audacious as “taking a cut of all economic activity”, Amazon has several different strategies. The key to the enterprise is AWS: if it is better to build an Internet-enabled business on the public cloud, and if all businesses will soon be Internet-enabled businesses, it follows that AWS is well-placed to take a cut of all business activity.

On the consumer side the key is Prime. While Amazon has long pursued a dominant strategy in retail — superior cost and superior selection — it is difficult to build sustainable differentiation on these factors alone. After all, another retailer is only a click away.

This, though, is the brilliance of Prime: thanks to its reliability and convenience (two days shipping, sometimes faster!), plus human fallibility when it comes to considering sunk costs (you’ve already paid $99!), why even bother looking anywhere else? With Prime Amazon has created a powerful moat around consumer goods that does not depend on simply having the lowest price, because Prime customers don’t even bother to check.

This, though, is why groceries is a strategic hole: not only is it the largest retail category, it is the most persistent opportunity for other retailers to gain access to Prime members and remind them there are alternatives. That is why Amazon has been so determined in the space: AmazonFresh launched a decade ago, and unlike other Amazon experiments, has continued to receive funding along with other rumored initiatives like convenience store and grocery pick-ups. Amazon simply hasn’t been able to figure out the right tactics.

Amazon’s Tactics

To understand why groceries are such a challenge look at how they differ from books, Amazon’s first product:

  • There are far more books than can ever fit in a physical store, which means an e-commerce site can win on selection; in comparison, there simply aren’t that many grocery items (a typical grocery store will have between 30,000 and 50,000 SKUs)
  • When you order a book, you know exactly what you are getting: a book from Amazon is the same as a book from a local bookstore; groceries, on the other hand, can vary in quality not just store-to-store but, particularly in the case of perishable goods, item-to-item
  • Books can be stored in a centralized warehouse indefinitely; perishable groceries can only be stored for a limited amount of time and degrade in quality during transit

As Mackey surely understood, this meant that AmazonFresh was at a cost disadvantage to physical grocers as well: in order to be competitive AmazonFresh needed to stock a lot of perishable items; however, as long as AmazonFresh was not operating at meaningful scale a huge number of those perishable items would spoil. And, given the inherent local nature of groceries, scale needed to be achieved not on a national basis but a city one.

Groceries are a fundamentally different problem that need a fundamentally different solution; what is so brilliant about this deal, though, is that it solves the problem in a fundamentally Amazonian way.

The First-And-Best Customer

Last year in The Amazon Tax I explained how the different parts of the company — like AWS and Prime — were on a conceptual level more similar than you might think, and that said concepts were rooted in the very structure of Amazon itself. The best example is AWS, which offered server functionality as “primitives”, giving maximum flexibility for developers to build on top of:1

The “primitives” model modularized Amazon’s infrastructure, effectively transforming raw data center components into storage, computing, databases, etc. which could be used on an ad-hoc basis not only by Amazon’s internal teams but also outside developers:

stratechery Year One - 274

This AWS layer in the middle has several key characteristics:

  • AWS has massive fixed costs but benefits tremendously from economies of scale
  • The cost to build AWS was justified because the first and best customer is Amazon’s e-commerce business
  • AWS’s focus on “primitives” meant it could be sold as-is to developers beyond Amazon, increasing the returns to scale and, by extension, deepening AWS’ moat

This last point was a win-win: developers would have access to enterprise-level computing resources with zero up-front investment; Amazon, meanwhile, would get that much more scale for a set of products for which they would be the first and best customer.

As I detailed in that article, this exact same framework applies to

Prime is a super experience with superior prices and superior selection, and it too feeds into a scale play. The result is a business that looks like this:

stratechery Year One - 275

That is, of course, the same structure as AWS — and it shares similar characteristics:

  • E-commerce distribution has massive fixed costs but benefits tremendously from economies of scale
  • The cost to build-out Amazon’s fulfillment centers was justified because the first and best customer is Amazon’s e-commerce business
  • That last bullet point may seem odd, but in fact 40% of Amazon’s sales (on a unit basis) are sold by 3rd-party merchants; most of these merchants leverage Fulfilled-by-Amazon, which means their goods are stored in Amazon’s fulfillment centers and covered by Prime. This increases the return to scale for Amazon’s fulfillment centers, increases the value of Prime, and deepens Amazon’s moat

As I noted in that piece, you can see the outline of similar efforts in logistics: Amazon is building out a delivery network with itself as the first-and-best customer; in the long run it seems obvious said logistics services will be exposed as a platform.

This, though, is what was missing from Amazon’s grocery efforts: there was no first-and-best customer. Absent that, and given all the limitations of groceries, AmazonFresh was doomed to be eternally sub-scale.

Whole Foods: Customer, not Retailer

This is the key to understanding the purchase of Whole Foods: to the outside it may seem that Amazon is buying a retailer. The truth, though, is that Amazon is buying a customer — the first-and-best customer that will instantly bring its grocery efforts to scale.

Today, all of the logistics that go into a Whole Foods store are for the purpose of stocking physical shelves: the entire operation is integrated. What I expect Amazon to do over the next few years is transform the Whole Foods supply chain into a service architecture based on primitives: meat, fruit, vegetables, baked goods, non-perishables (Whole Foods’ outsized reliance on store brands is something that I’m sure was very attractive to Amazon). What will make this massive investment worth it, though, is that there will be a guaranteed customer: Whole Foods Markets.

stratechery Year One - 270

In the long run, physical grocery stores will be only one of the Amazon Grocery Services’ customers: obviously a home delivery service will be another, and it will be far more efficient than a company like Instacart trying to layer on top of Whole Foods’ current integrated model.

I suspect Amazon’s ambitions stretch further, though: Amazon Grocery Services will be well-placed to start supplying restaurants too, gaining Amazon access to another big cut of economic activity. It is the AWS model, which is to say it is the Amazon model, but like AWS, the key to profitability is having a first-and-best customer able to utilize the massive investment necessary to build the service out in the first place.

I said at the beginning that Mackey mis-understood Amazon’s goals, strategies, and tactics, and while that is true, the bigger error was in misunderstanding Amazon itself: unlike Whole Foods Amazon has no desire to be a grocer, and contrary to conventional wisdom the company is not even a retailer. At its core Amazon is a services provider enabled — and protected — by scale.

Indeed, to the extent Waterloo is a valid analogy, Amazon is much more akin to the British Empire, and there is now one less obstacle to sitting astride all aspects of the economy.

  1. To be clear, AWS was not about selling extra capacity; it was new capability, and Amazon itself has slowly transitioned over time (as I understand it is still a hybrid) []
Read the whole story
89 days ago
Perhaps Ben's best piece yet (and there are many to choose from), especially because how most in the media have completely misunderstood the purchase.
santa clara, CA
Share this story
Next Page of Stories