skip to Main Content

Our Story (that Began in 2010!) Hits an Inflection Point

Buckle up folks, this is a longish post – but if you find the topic even HALF as interesting as I do, it will be worth your time to read.  Don’t think of it as a long blog post; maybe it’s better thought of as a short, informal whitepaper.

Today, I’m going to do two things:

  1. Give away our secrets.  (Seriously, I’m going to tell you how our entire business works, at least at a high level.)
  2. Explain what I believe the future of the BI consulting industry looks like – because the tools revolution we’re experiencing has far-reaching implications for how the entire industry works.

Let’s start here:  despite working in a hyper-rational industry, emotion is the only thing that really ever drives me forward.  I don’t think that makes me the least bit unique of course; in some sense I believe motivation is ALWAYS an emotional thing: one way or another, your “temperature” needs to rise before you take significant action.  In the early days after leaving Microsoft (story here if you’re interested), for instance, Fear was the initial motivator.  I had stepped off a cliff and had no idea where I was going to land.  That tends to spur action!

But Excitement, surprisingly, is actually FAR more motivating than Fear – at least for me.  Despite the obvious career pressure in late 2009, I had “stalled” on the idea of starting a blog for several months.  (Laziness trumps Fear perhaps).  But then, something happened.  I saw…  it.  And just like that, we were off and running.

What is It?  (It’s It, duh.)

imageThe band Faith No More once famously asked, “What Is It?”  And in this case, It is when you see, for the first time, that the world of data – your world, and others’ worlds – will never be the same again.  That first moment where it hits you that these new tools (Power Pivot and Power BI) are much better, DRAMATICALLY, overwhelmingly better, than any marketing message could ever convey.  You have to see it for yourself.  And once you have, your view of the world of data changes forever.  (Most of you reading this have seen it, and may even remember, as I do, the exact moment when “it” happened.  It is, ahem, truly Epic.)

Sharing that lightning-bolt moment with others, and seeing it on their faces, is one of the most Exciting things I will ever experience.  In fact, the entire team here at PowerPivotPro kinda lives for that – it’s what’s drawn us all together.  It simply never gets old, never loses its pulse-quickening power, to see it dawn on people’s faces.  Heck, even just hanging out with other members of the “Seen It” Club is pretty energizing, as the enthusiasm reflects back and forth in an endlessly magnifying process, like fission.

Re-inventing an Industry

But as if that weren’t enough Excitement, I also firmly believe our company is doing nothing less than inventing a new industry, or at bare minimum re-inventing an existing one.  And that, dear friends, is the persistent motivator, the one that occupies my thoughts as I fall asleep at night, and the one that returns the moment I wake.

In some sense, the question in the title of today’s post is the story of our entire company.  It’s a question that has fascinated me since 2009.  How many people EVER get what I got in 2009 – advanced notice of a massive economic shift?  It was a gift from heaven, the opportunity of many lifetimes, and I’ve never been able to even contemplate wasting it (in fact my decision to leave the startup I co-ran from 2010-2013 had a lot to do with the belief that we were wasting it).  I’ve been talking about this, noodling on it, and evolving my understanding of it for nearly seven years now.  A gift and an obsession simultaneously.

CAC and LTV:  The Language of Silicon Valley

Cost to Acquire Customer vs. Long Term Value in the Traditional BI Industry

To illustrate the changing industry, here’s a diagram I concocted awhile back for an internal team meeting.  It reflects a “lingo” I learned while visiting “angel” investors back in 2013 (like this guy, shortly before he did this.)   I was originally planning to take a bunch of money from one or more of these people before realizing that it wasn’t going to be necessary (more on this later).  But along the way I learned some very valuable concepts, like CAC and LTV!

To a lot of Angels and VC’s, CAC and LTV are the most succinct way to summarize a business – especially from the economic value standpoint:

  • CAC (Cost to Acquire Customer) – on average, how much does it cost your business to find and “land” a single new customer?
  • LTV (Long Term Value) – on average, how much money do you get from a single customer, over the duration of that customer’s “lifetime” with your business?

Of course, these are broad brushes which “hide” things like the cost to serve and maintain a customer, customer attrition rate, total addressable market size, etc., but at a high level, if it costs you $X to acquire a customer, and you “get” $Y from that customer over time…  If Y is a lot bigger than X, then “it’s just a simple matter of financing,” as some of my west coast friends like to say.  (For example, if you have the Midas touch, and in your biz, CAC = $10 and LTV = $2,000, go straight to the bank RIGHT NOW and take the biggest possible loan they will give you – and then press the gas pedal to the floor.  Better yet, press it through the floor.)

It’s quite fair to characterize traditional BI consulting as a “High CAC, High LTV” business.  The price of the software was significant, but paled in comparison to the costs of projects (typically 6-7 figures!).  Projects had to be expensive, because the software was so complex to use that it necessitated dedicated “Ivory Tower Professionals” – folks whose entire careers were devoted to said tools – in order to build something meaningful.  This, in turn, meant that no single brain could ever contain the richness of the business (both its needs and complexities) and the ability to execute.  And then, the communication cost KILLED YOU.

I’ve written extensively about Communication Cost as the “Dark Matter” of BI projects over the years – try here, here, here, here, here, and Matt added his voice here if you want to read more, so I won’t belabor that point in this post, but “Complex Software –> Dedicated Experts –> Massive Communication Cost –> Incredibly Slow and Expensive” was pretty much the entire game in a nutshell.

The Dark Ages

Yep, those were dark times.

Then “Self Service” Tools Entered the Scene!

Power Pivot was Microsoft’s first foray into the “self-service” BI tools market when it released in 2010, but it wasn’t the first or only kid on that particular block.  Qlikview for instance was probably the first big player in that space, followed by Tableau, with Microsoft perhaps as the third significant entry.  (Maybe TIBCO predated Power Pivot too, not sure).

These tools, despite their differences, all share two key traits:

  1. They have robust engines under the hood, enabling more responsible/scalable analysis than possible in traditional Excel.
  2. They are learnable by business users – so in theory, you don’t need the expensive experts anymore.

That’s the marketing message, anyway – your mileage may vary with the different tools.  But in my personal experience, in the experience of our team, and the experience of our clients, Power Pivot and its newer cousin Power BI both deliver, BIG TIME, on those promises.  (Otherwise we would have moved on to something else by now.)

So…  we can revisit that chain of consequences from before…

Contrasting Traditional BI Consulting with Consulting in the Self-Service Age



What does this mean for the consulting industry?

It means two things right off the bat, both of which are incompatible with the old business models:

  1. Projects become a lot shorter and cheaper overall, seriously undermining the old price points.
  2. Furthermore, in order for clients to reap the massive benefits conveyed by Power BI and/or Power Pivot, consultants can no longer do most of the work.  The client has to be hands-on, directly involved with the execution of the models and analyses!

Shorter, cheaper projects (smaller pie), AND most of the work should be done by the client (smaller fraction of the pie for the consultants)!  That’s a big problem right?

Well no, not really, as long as you’re willing to re-invent yourself.  Or if, like us, you have the luxury of a blank canvas, unencumbered by past successes in the old model, you can just create a brand-new type of firm from scratch.

Make no mistake:  “Self-Service” and “Dependent” are fundamentally at odds with one another.  You simply cannot shoe-horn the former into the latter’s pre-existing business model.  The market is going to force everyone’s hand in the next few years, and it’s going to be exciting.

Low Cost does NOT mean Low Value – Quite the Opposite.

Big Value Created from Small Spends

Don’t let the cartoon font fool you:  there is BIG value to be found in agile projects.

Counterintuitive at first but absolutely true:  using the new methodology, you get MORE value than the old way!  Bringing the Business itself right into the “galactic center” of the analytics world has the opposite of a cheapening effect:  it delivers massive value.  MASSIVE.  Much more value than the old methodology in fact – even BEFORE you consider the lower price point.  And honestly this shouldn’t surprise us once we think about it, even though it does surprise us at first.

For example, have you ever tried to debug a relative’s computer problem over the phone, versus in person?

Analogy:  Old World of Analytics Consulting (Left) vs. New (Right)

Phone Debugging versus Hands-On Debugging of a Relative’s Computer Problem:  Analogous to Traditional vs. Agile BI Consulting
The service at left is much slower, and you’d understandably charge a much bigger fee for it, but the service at right is more valuable to the “client,” at any price.

That analogy illustrates the Communication Cost problem quite well I think:  The problems themselves are not so bad.  They don’t require calculus, advanced degrees, or esoteric skills.  They just require low-friction clarity – a removal of the barriers between “I have a problem” and “I have the skills to solve it.”  Communication Cost is everything in the old model – 99% of the cost and time of any project.  The new model drops Communication Cost down close to zero.

This Just In:  The traditional firms are not really adapting yet.

Traditional Firms are Built to Only Do High-Billing ProjectsThere are plenty of traditional consulting firms who are “doing” Power BI.  But what we’ve long suspected is that they’re merely using Power BI as a new sales vector for the old model:  ride the excitement of the self-service tools wave, and once in the door, sell old-school expensive infrastructure projects and “consultant-dependent” BI that’s as heavily-bottlenecked on the consulting firm as it’s always been.

We’ve confirmed this twice in the past year via off-the-record conversations with a very popular and widely-published industry analyst, who told us “When we ask the larger firms whether they ‘do’ self-service BI, they all say yes, but then when we ask them for a case study, they don’t have any.  They say they do it, but they don’t actually practice it.”

And to be clear, I am not “accusing” these firms of anything.  This isn’t dirty or dishonest behavior on their part – nothing of the sort!  They simply have no choice but to behave this way.  They are built for the high-overhead, big project, dependent-on-external-experts world.  The only way they can help their clients is to apply the methodologies they are built for.

Honestly, in their shoes, I would be starting a new company, a subsidiary perhaps, rather than trying to adapt the existing firm to the new model.  And then, eventually, you let the new firm “eat” the parent.

When we ask the larger firms whether they ‘do’ self-service BI, they all say yes, but then when we ask them for a case study, they don’t have any.  They say they do it, but they don’t actually practice it.

-A leading industry analyst (who we cannot identify due to his/her firm’s policies)

“So, what’s the inflection point you speak of, Rob?”

Inflection Point is the Sum of Many Things

It’s Actually Lots of Things

It’s not one thing really, it’s a bunch of things.  And it wasn’t all recent – it’s been happening for a long time.  But the cumulative impact has come to a head recently, as if we found a second, larger gas pedal.  And the net result is that we’re about to become a lot more aggressive, a lot “louder” on the scene, so to speak.

Guess what?  The last year in which I personally performed > 50% of the client work at P3 was 2014, and we’ve recently taken me out of the rotation entirely (with the exception of our classes, like the one I am teaching this week in Chicago). This has freed me to focus my efforts elsewhere, in more of a leadership and mentoring capacity, but just as importantly, it’s given me the time to identify the talent we need to grow.  Being a geographically-distributed firm has allowed us to comb basically the entire country looking for elite talent, and I’m excited to “confess” that I’m not the most-capable consultant at my own firm.  Not even close actually – I might not even be in the top five.  Almost certainly not top five, actually, and my ranking falls every time we hire someone new – because the bar is basically “must be better than Rob.”  (Not kidding.)

This Spring alone, we’ve signed three new full-time consultants to the team.  Stand by for a blog post talking about these simply amazing people, probably titled something like “The People Who Shouldn’t Exist.”

And it’s not just consultants we’ve added.  We have slowly built an entire leadership team for instance, which has allowed us to expand our capabilities in many directions.  I recently took a bit of a sabbatical, for instance, to deal with some family matters, and the company didn’t miss a beat.  (In fact, it moved forward.  Hmmm.  Moving on.)

Also, in November we quietly created our first-ever “sales” department – which is something I explicitly never wanted to do (because if you’re pressuring people to buy, you’re doing it wrong IMO, and I still believe that).  But we also lacked account managers, which meant we were actually making it hard for our clients to use us even as much as THEY wanted to, so we corrected that.  Our clients have been better-served as a result – so much so that we’re adding another account manager this week to keep up.

I’m also personally spearheading some very exciting initiatives that we are not yet ready to disclose.  Stay tuned.

Operationally, what IS different about running a “new style” firm?

Cost to Acquire Customer vs. Long Term Value in the New BI/Analytics Industry

Keep in Mind:  “Small-Value” Clients run the Gamut from Small to MASSIVE Organizations

Here are the big differences, some of which are obvious, but others have only become clear to us over the years:

  1. Since the price point on a single project is FAR lower, you simply CANNOT spend nearly as much to “win” a project.  Cold-calling, for instance?  Forget it.  You have to have a more scalable, cost-effective means for reaching and winning potential clients.
  2. It’s very tricky to achieve that critical 70-80% utilization of your consulting staff, since you cannot “park” a team of 3-5 people at a single client for months on end like in the old model.  If you don’t solve this, you cannot staff with FTE consultants.
  3. “Teams” of consultants merely drive the price up while simultaneously increasing communication cost, so it’s crucially important that your consultants be able to operate as relatively autonomous individuals.  At any given point, the client is almost always only working with one person from our firm.
  4. To be more clear, you cannot have a separate Solutions Architect (good planner, communicator) paired with teams of “techies” who execute.  Consultants need fantastic comm/soft skills, paired with equally-good tech/hard skills, in a single person – and these people, in our experience, are more likely to have come up through the Business than through IT (perhaps for the simple reason that the Business outnumbers IT dramatically, as opposed to something more fundamental).
  5. You cannot treat Training as a sales funnel.  In the self-service world, Training must be a core part of your business, one of the primary values you provide to your clients.  All consultants in the new world are inherently trainers…  and also consultants!  Which merely reinforces point #4 – it’s a new skillset, and many people already on staff at traditional firms just aren’t suited for it.
  6. Overall, you have to think and operate as an Internet business rather than as a geographically-focused, brick-and-mortar firm.  Yes, you absolutely get on airplanes too, but you have to reach, engage, convert, and to a certain extent also service your clients with a mentality akin to that of an Internet startup…  while also maintaining a personal connection that’s foreign to the Internet world.  Oh, and the same thing goes for staffing your team.

Why am I willing to share all of that?

Why Give Away the Secrets?Why “give away” so much hard-won wisdom, and essentially provide a blueprint for competitors?  A few reasons.

  • First of all, I kinda just can’t help it. The whole thing is so fascinating that I want to engage others in the conversation.  And in some sense, it will be benefit us to have others join us in this space, because critical mass of the New Style in the market will grow demand for our services faster than we can on our own – counterintuitive but true I think.
  • Second, my colleagues and I really, truly, believe in this stuff.  We’re all here because we’ve seen this Better Way, and we want things to evolve.  We want the world to improve, to get smarter.  Sounds cheesy but that’s what we’re like in person, too.
  • Third, come on!  The world of opportunity is so massive in this new model that it’s silly to worry about competitors!  Everyone is a potential client, because everyone has data, and everyone can now afford some degree of help!  If you’re focusing on the competition rather than on the clients, you’re wasting energy.  (More proof that we practice what we preach: some of our clients are other consulting firms, and we help them learn to do what we do.)
  • And finally, yes, we DO think it’s hard to follow in our footsteps, even if you have the blueprint.  You can’t just snap your fingers and have the kind of team we do, nor can you develop a low-overhead, Internet-scale method for reaching customers overnight.  It’s taken us seven years.  Maybe you will be faster than that.  We kinda hope you will be faster than that.  We look forward to you keeping us on our toes.

Rob Collie

One of the original engineering leaders behind Power BI and Power Pivot during his 14-year career at Microsoft, Rob Collie founded a consulting company in 2013 that is 100% devoted to “the new way forward” made possible by Power BI and its related technologies. Since 2013, PowerPivotPro has rapidly grown to become the leading firm in the industry, pioneering an agile, results-first methodology never before seen in the Business Intelligence space. A sought-after public speaker and author of the #1-selling Power BI book, Rob and his team would like to help you revolutionize your business and your career.

This Post Has 34 Comments
  1. Simply brilliant!
    I cannot agree more on the shift you observe in the BI market.
    Power BI Consultants are genuine trainers, willing to work with business users.

    Chriss webb also said something interesting and linked to your “new strategy”: because projects are quicker and costs lower, because Power BI is agile, SME’s might be the biggest self-service bi client potential in the next 3-5 years!

    What do you think ?
    We are definitely embarked in an exciting journey… and I am SO happy to be part of it !
    Thanks for sharing your thoughts with such a level of details.

  2. I find your postings extremely motivating. I’ve been motivated ever since reading the following from page 232 of your first book, “In short, Microsoft has decided that the PowerPivot approach (DAX and tables) is a better way forward than the old SSAS language and approach (often referred to as “MOLAP” – MDX, Dimensions, etc.) PowerPivot – tables and DAX, which you will now see referred to as the “tabular approach” – is the future of their professional-grade product, and not just something for Excel Pros.”

    I attend the local SQL Saturday events every year. I asked about the Tabular model in 2015 and was quickly informed that the tabular model was inferior, no ifs, ands or buts. When I attended in 2016, tabular was the “new” way. To me, it was a seismic shift in attitude. I immediately recalled what you stated in your first book and smiled. Rob saw this coming years ago.

  3. Excellent post Rob. Hit the nose on the head with fear and excitement being extremely motivating. I can see how excitement is motivating now, but I believe initially fear (i.e. leaving a good job for the “unknown”) was probably the driving factor. Or I’m projecting my own thoughts here. Anyhow, great post

    1. Something very fundmental! An honest acknowledgement that some of the past expenditure on previous BI projects / software implementations should be written off. And that’s the difficult part, anyone who strongly recommended investment into niche data tools / BI projects on the back of the DOT.DATA frenzy will want to save face and not be so forthcoming with this admission. So a pioneering change of leadership and vision would be my answer

    2. I’m curious to hear your thoughts on this, as I am part of a large organisation with a huge bi stack (microstrategy plus Oracle plus exasol), the whole St the same time business teams are starting to self develop and deploy small efficient solutions based on power Bi and tableau

      1. Middle management (section heads in the company I worked for) are the key to exploding self-service BI adoption. Since most of these people are not interested in learning the technology for their own use, but rather how it can improve the bottom line in their groups, a seminar demonstrating real-world examples of cost benefit needs to be presented. I envision that this seminar would be free for all participants as an enticement to attend. Bottom-up adoption of new technology, especially in the information technology field, rarely occurs in large corporations, IME. So, the ideas for improvement and efficiency have to originate from the bosses in order to have a chance to be promoted and implemented.

        1. My approach was simply to go guerilla and develop tools delivering benefits, against a tide of resistance and bureaucratic IT nonsense I may add. You could call it a ground up approach

          As soon as you put a tool in front of a manager that can either free up manpower for them, mitigate risk or provide valuable insight they never had before you immediately have their ear. The fact you’ve delivered something that can be counted in days rather than months also commands attention

          Once they want your tools you’ve then got the buy-in to re-approach IT and say “remember when you said I couldn’t have 64 bit office or power query or power bi desktop, well you’re going to need to change your tune”

  4. Good to see you hammering the keys again. I know that Lightning Bolt Explosion #3ish through who knows how many, happened between Wednesday 07/30/14 and Friday 08/02/14. Just got all giddy over a measure that says “SPECIAL ALERT” in red when something has been discounted above and beyond the norm but, not at 100%…verb measure that says “Go look at the discounts on this item! Sales is giving away the farm to move this. So Simple. So Slick. Mr. Stanley of data, I thank you once again.

  5. Hi Rob, another insightful post. I agree that that this world is changing in making self service more available to end users. Its way awesome.. But I would argue that ,as a consultant, Power Pivot and BI allows us to “Value price” our projects more often as opposed to billing on an hourly rate basis. We have the ability to turn Analytics more effectively than before. It may be “lower cost” to produce these reports, but not lower knowledge base or value to customer. I find that Customers don’t want to learn how we do our magic, they want to focus on building their business with our insights and data analysis. Corporations may have inhouse staff for this but they also ought to encourage that “one” analyst that eats this stuff up.(that was me!). They can get huge value by giving that person/persons more rein.

  6. Rob, what do you see as the target person / department in an organization for these services? (accounting, finance, sales, management, CEO, CFO, CIO, etc.). AND what size of organization? We are a “traditional” Consulting and Training organization and you are correct, making the transition to the “new” model is tough, as moving from a few BIG projects to MANY small projects has its own challenges. You have been doing this for long enough and have become pretty “well-known,” so is it your reputation in this industry that is bringing P3 these projects? Any thoughts would be great!! Thanks.

  7. I think that you should separate on-line operational reporting, were you load data “as is” from the source system, from analytical reporting, were you clean data, apply business rules. apply history business rules, load starschemas, add smart small keys, do aggregations and so on. What about order balances, change in order balances? If the source system like AX Dynamics only keeps the last version of a record, that will complicate matters. Also in AX Dynamics the database is stupid and all logic resides in code outside of the database.

  8. The first thing that came to mind with PowerPoint v1 was indeed the ability to go from POC to production in a much faster fashion. Funny thing is this ability was tucked away in SSAS MD, but, to my surprise, so few companies used this capability. After finding a bug in the “sync descriptions” feature of BIDS Helper, I once asked Greg if there was ever consideration to make the “sync descriptions” work both ways (I.e. bring description metadata into the RDBMS from SSAS). At the time (2011), to my surprise, he stated no one had asked for it. My point is that there didn’t seem to be a critical mass of folks out there using SSAS as a potential POC tool….Taking it’s biz and tech metadata and bringing it to SQL Server. SSAS Tabular and Power BI bring that more to the forefront and enable the model about which you speak.

    I also agree that the ideal “consultant”, regardless of project role, is extremely polyvalent and is capable of handling implementation on his/her own. The advantage, of course, is the ability spend more on account management/client acquisition….

    There is so much more to say, but I agree with your points and see where you are coming from.

  9. Great blog post Rob and I totally agree, I also thought about 2 – 3 years ago that Power Pivot was a game changer, because back then it was integrated into Excel, and Excel has got 2 billion users. Now with the addition of Power BI, it removes the Office dependancy and this allows people to sit with the business and quickly solve business problems and let them act on their data that they have!

  10. Completely agree Rob, and this is why the Power BI Suite has been such a boost for our business.
    We were already offering rapid solutions (Excel and VBA) using autonomous consultants with combined soft skills, paired with equally-good tech/hard skills,
    Then this incredible new suite of tools comes along!
    We are growing and it’s the Power BI suite driving things forward

  11. Thanks for sharing your thoughts, Rob! It is really motivating!
    I have no tech background, just years of experience in business and a long-term passion to Excel & self-service BI (and I am still have full time job, not corresponding with tech, consulting or something close to it). And in last two years, with Power BI GA, I’ve got a tens of clients from the thin air, just because I love to share my knowledge. This “look, it’s easy” thing makes people to wonder. They sometimes don’t believe it is ever possible without some big solutions, months of implementation etc.
    Agreed, it is a really big shift. Market changes.

  12. Sorry to rain on your parade guys, but this article is way too misleading. While I’m a big user of PowerPivot and PowerBI, there’s some definite problems with the tools:

    – Like Excel, they are still buggy. Perhaps because they depend largely on Excel guts to operate, they are burdened with all of the nastiness built into Excel over the years, I’m not sure. But, for prime time BI, they have a ways to go yet.
    – It doesn’t scale well. As long as you can survive with the in memory model (typically < 1M records), then fine. But as you data grows and it grows more complex, it breaks down.
    – More semantic control in MSAS. Granted, most people lack the sophistication to create proper MSAS models, but there's things you can solve in MSAS that just can't be solved in PowerPivot.
    – Forced into the tabular model. These technologies just aren't ready for the superior cube model (maybe some day). Parent/Child hierarchies, many-to-many relationships, scope control, etc. just don't work properly.
    – Lack of data sophistication from most end users. Most people don't care if they get the data right, just so long as they get an answer. While yes, the business user is the be arbitrator if their problem is solved, they generally don't have the training to understand dimensional math and can get themselves tied up in knots.

    This doesn't mean that these tools don't have a place, it's just that they aren't ready to replace MSAS/SSIS, not by a long shot. And MSAS/SSIS projects don't have to be long and complex, that's just a failure of the consultants working on the project. Typically, I get cubes up and running in less than two weeks (includes data processing & cleansing), and most are a couple of days (standardized data formats). And yes, I have hundreds of calculated members (standardized scripts helps big time).

    90% of the overrun projects I've seen are because the BI consultants are learning on the job. (A hazard in the industry, when the proper skillsets are in short supply.)

    I'll continue to use these tools and build dynamic, flexible Excel based solutions (and have fun doing it). But I'll keep my MSAS/SSIS environment as well thank you very much.

    1. Apologies David, I held your comment in moderation until I was ready to respond.

      Short version: I mean you no harm. I don’t even want to “win” an argument with you, as I believe the market will decide, and IS deciding, and what you and I say to each other would just be verbal sparring.

      Longer version:

      1) My points are much less about existing/emerging technologies and more about this: subject matter experts (SME’s) who can operate powerful BI toolsets are the future – as opposed to dedicated technical experts floating around and executing projects on behalf of the SME’s in the business. I looked you up and it appears that you work at a firm that is focused on specific verticals as opposed to being a “general purpose BI technical firm.” If you know SSAS and SSIS, great! You’re participating in the trend I’m talking about. Most SME’s cannot and will not learn those more traditional toolsets – just too opaque, too difficult to learn.

      But the newer generation of tools, like PPVT/PBI, *are* learnable by a large number of SME’s, and that changes the game dramatically. What does it mean to operate a general-purpose consulting firm (like ours) in such an era? That is what we are finding out, and that is the trail we like to think we are blazing.

      2) In our experience, the “power” difference between Power Pivot and SSAS-MD, and the “power” difference between Power Query/M vs. SSIS, is not so large as to “protect” general-purpose, traditional BI consulting from the rising tide of empowered SME’s. And in fact, Power Pivot *IS* SSAS, and can therefore be upsized to SSAS-Tabular, so in practice the difference there is practically nil.

      (Also, I see many of the SSAS-MD “celebrities” of yesterday who have basically 100% converted their businesses to Tabular. Microsoft itself is doing its best to cease investing in SSAS-MD. The writing is on the wall here – the future of SSAS is DAX. And DAX is the language of Power Pivot and Power BI. The tools we are using are the absolute best in the biz, full stop. Although I will concede that M doesn’t scale like SSIS, at least not yet.)

      3) I looked around at our parade. No rain here 🙂 Maybe none on your parade either, in which case, great! But I’m not misleading anyone here. If anything, I’m providing (even to folks like you!) valuable food for thought when it comes to future-proofing your own careers and businesses. I am betting my own company on the things I am saying. And every month, the market sends us fresh signals that it’s an excellent bet. (Ex: our 2017 YTD revenue is up > 200% vs 2016 parallel YTD, and in 2016, we assisted approximately 250 different firms. we don’t need the old biz model to die in order to keep growing, but clearly, a new model is ascendant – in parallel at bare minimum).

      thanks for reading,

    2. Can you please elaborate on this “burdened with all of the nastiness built into Excel over the years” ?

      I’m genuinely interested in knowing your take on things because the individuals I’ve come across in the past saying similar have either delivered the grand sum of squat or by the time they have finished, the business question has either changed or their project has gone tits up and been quietly swept into the graveyard of other ‘enterprise BI’ endeavours

      I have worked in an organisation that spent millions setting up a so called replacement for Excel and guess what I was told today “were going to use Power BI to manipulate the ‘reports’ coming out of this system”. The fact this is happening means the ‘proper’ system simply doesn’t meet business need, it’s just expensive middleware for exporting to Excel / Power BI

    3. One more thing: we routinely work with many millions of rows of data in Power Pivot and Power BI.

      < 1 M rows is the traditional Excel limit, and that is GONE when you switch to DAX.

    4. Interesting David. I appreciate hearing your opinion, my experience has been really different. I hear regularly from people who recite some version of what you’re describing and I often wonder if maybe you’re right. But then I go run a training with 50 students, excel people, who’ve used these tools to transform their business, get promotions, and increase their overall life satisfaction. In those moments it doesn’t seem all that important that the tool is missing “scope control”. Seems like you haven’t had these experiences and so yes, you’re probably not going to agree with something that’s obvious to everyone that has.

    5. I think you may not be using PowerPivot optimally. We routinely build reports with 40 to 80 million rows of Medicaid data in the Oregon Health Authority. We have a genius that extracts, transforms and loads medical claims and Medicaid enrollment data each month. We link to the server and use some pretty simple SQL to create cube-like data tables. Once we post the reports on SharePoint, they update monthly. Our customers love the reports we create for them

  13. …and the little “golden nugget” is the importance of the office worker’s ability with Excel which contributes to a user’s ability to learn and adapt to Microsoft-PowerBI.
    When hiring managers are looking at resumes, just what really qualifies as Excellent Excel skills?
    There can still be a big gap between having a user and having a user with suitable knowledge, and that can become an additional role (fee) for the “new” consultancy.
    Most users I work with are unfamiliar with any Table features in Excel!

  14. Brilliant post Rob. It’s incredible to see you put in writing what we are experiencing in practice. We are a couple years behind you but with very similar values and approach to consulting which naturally led to the same expression in helping our clients with BI. Our team has learned so much from your team’s journey and we’d love to contribute back to what you’re doing with some of our learnings. What’s the best way to do that?

  15. Great post. I believe you are reaching a lot of end-users and are helping them to get more out of their data and be more effective in that. All enabled by Power Bi and (very important) the training in it. I work for a BI (IT) dept. of a large international firm and we are about stop delivering the ‘enterprise bi’ front ends and focus on the backend (data, infra, software, semantic layers etc.) with Power BI as a self service front end. We have had too many failures when trying to deliver the requirements. Biggest hurdle in my opinion is advanced data modelling and DAX which you can’t really do without in reality. We unfortunately have very few users who could master that (I don’t believe in self service Bi ‘for the masses’). Typically you need the ones who already work with Pivot tables and advanced formulas like Vlookup and even then specialist advice and training are a must to help them step by step.

    1. Tom we couldn’t possibly be MORE on the same page. For example, I also 100% agree – target the pivot and vlookup people for education and empowerment. You can damn-near ignore everyone else – both because the pivot/vlookup folks will admirably proxy for, and serve, all the other folks… and because the other folks will never learn the toolset, just like they never learned pivot and vlookup! 🙂

      In general, I recognize an uncommon wisdom and enlightenment even in the tone of what you say. You’ve seen “it,” and that fact leaps off of my screen. Can we do a case study writeup on your company’s journey? Serious question.

  16. Great post Rob, As someone who “get’s it” and has been running with the ball as a consultant here in Melbourne AU for a couple of years now (time flies!) I wholeheartedly agree with you. The industry shift (especially in the past few months) has been SEISMIC and I feel blessed that I discovered this technology when it was emerging (way after you guys, but early enough!) customers are now asking to keep my team on for longer and longer and when they see the value we are producing it’s incredible how quickly they find ways to re-direct funding into PowerBI projects. I think you guys will go absolutely BANANAS after June (or whenever the “On-premises” explosion happens) hopefuly I can keep up too!! – Greg

  17. Hi Rob,
    Love the PowerPivotPro team posts and read them faithfully. Our firm is completely bought into the PowerPivot/PowerBI platform for quick-wins and as the analytical/reporting platform for business users. What I’m struggling to understand is how you feel about situations where the fundamental data quality is poor. We’ve had several clients that have a history of failed BI projects (before us) because they believe the marketing spin on every BI solution in the marketplace and never look at the ‘boring’ problem of good-old master data management, data quality and business process consistency. We call these the ‘data foundation’. When the foundations are solid, PP/PBI are like the magic cherry on top. With poor foundations, we can get quick wins but they may or may not be sustainable. How do you think about this?

    1. Hi Rich. Great question. Let me throw out a few assorted thoughts in no particular order:

      1) Big infrastructure projects like MDM and DW are a great revenue source for the traditional BI industry. After all, building something “heavy” SHOULD carry a heavy price tag right? Psychologically, it fits. The traditional industry relies on these projects as their bread and butter, so we cannot fully trust their opinions on whether such projects are truly necessary.

      2) That said, crap data yields crap results. Many times, we are simply “fortunate” to walk into a situation where the data is already clean enough to yield good results. I’m cognizant of that. Whereas point #1 cautions against the default practice of too MUCH infra work, there absolutely IS such a thing as too LITTLE infra 🙂

      3) So… in the new model we need the “Goldilocks” amount of infra – neither too little nor too much.

      4) My personal favorite piece of guidance here is to work “backwards” – don’t build the infra and THEN build models/reports/dashboards. Instead, first sit down and start building something that attempts to be useful, and do so IMMEDIATELY, with the data sources at hand. When/if you encounter issues that simply cannot be mitigated at the DAX/M level, THEN you commission some infra work – but you keep it narrowly focused as opposed to a “boil the ocean,” “let’s fix everything and build ourselves a robust platform for all future pursuits both imagined and real.” The former can sometimes be done in a day or a week. The latter can drag for months or years.

      5) The metaphor I use for #4 is “build faucets, then run pipes to them,” as opposed to “building tons of plumbing in anticipation of future faucets.” Reactive infrastructure is actually a GOOD thing, because infra for its own sake is wasteful by definition.

      6) And you know what? Infra budgets often INCREASE in this methodology, because clear ROI was always missing in the old model. The C-Suite is much happier to green light tangible “I pay X, I get Y” projects as opposed to “just trust us, it’s over your head, but you’re gonna need it.”

      7) As an aside, my life insurance company recently and spontaneously decided to start sending ALL of my documents, statements, and other correspondence to my ex-wife’s house, after sending my stuff to my correct address for years. I’m absolutely certain that this is the work of some “smart” MDM script, and I cannot wait for them to run said script again 😉

      8) Bottom line, I think the future world has plenty of infra work in it. It’s just approached via a different philosophy (top-down rather than bottom-uo), and will have the same attribute of “more projects of smaller size” relative to the traditional world. And YES, there will still be occasions where big monolithic investments are justified, but that will stand in sharp contrast to today’s practice of doing them “just because we’re supposed to.”

Leave a Comment or Question