More Excel Services API Info

November 10, 2009

Two new links – one to the official Excel blog, and one to Shahar Prish’s personal blog.  Definitely check out Shahar’s blog – he’s a bit of a Renaissance man… one of the big brains behind Excel Services, but also one of the co-inventors of the method that enabled us to embed the PowerPivot db directly in the Excel file, which makes him a personal hero of mine :P

Excel blog post:  http://blogs.msdn.com/excel/archive/2009/11/09/excel-services-in-sharepoint-2010-rest-api-examples.aspx

Shahar’s blog:  http://blogs.msdn.com/cumgranosalis/


DAX: The =RELATED() Function

November 10, 2009

A quick update here on the football project (not really warranting a video).  The source db is awfully complex, and I’m in the process of wrapping my Excel-centric head around it.

For instance, in the Plays table, I’m trying to understand how many row constitute a single NFL play.  It’s not one row per play – no, that would be too simple.

Here, for instance, is a field that claims to be a unique play ID, but when I sort by it, it clearly isn’t:

                       PowerPivot PlayID Sorted   

With the table sorted, though, I notice that there are multiple SeasonNumber values corresponding to PlayUniqueID = 1:

                       PowerPivot Season Number 

…so I add a calc column concatenating the two, with renewed hopes for uniqueness:

PowerPivot New UniqueID

And…  curses, no such luck:

                     PowerPivot PlayGUID

At this point I realize that there really just are multiple rows per real event.  There are other columns in there that I suspect are crucial:  PlayerRole, PrimaryRole, etc. – all with integer values, sadly.

The good news, though, is that those integers are ID’s, and they reference into other tables that I are related to the Plays table.  Once those relationships are set up, I can exploit a new function added by PowerPivot.

To gain some more understanding about how this table works, I decide to lean on a familiar Excel trick and just add some more columns, VLOOKUP-style, but using the new DAX function named =RELATED()

PowerPivot =RELATED 1

            PowerPivot =RELATED 2

                           PowerPivot =RELATED 3

Be gone, VLOOKUP!

Check that out – it’s a single-parameter function!  I just say, “hey, go get me this column from this related table,” and off it goes to fetch the values that match the row from the current table, using the relationship.

The equivalent VLOOKUP would be something like:

    =VLOOKUP([PlayerID, CleanPlayers, 2, FALSE)

…and if the PlayerID column wasn’t the first column in the CleanPlayers table, well, it flat-out wouldn’t work.

Those three =RELATED() formulas yield:

PowerPivot =RELATED results

Ok, this gives me some hope of figuring things out.  (Steve Martin’s presence confuses me, though…)

I may or may not keep these columns in this table when I am done, since they are NOT required – I can build reports using the original fields in those other three tables, combined with fields from the Plays table, as demonstrated in previous posts.  This is just a temporary convenience for exploring this table.

Note that Excel users work like this all the time.  I kinda doubt DBA’s frewquently add columns like this, expecting to just remove them shortly thereafter.  But here I am, working with an environment that conveys real database advantages, but I’m still doing my informal, explore-and-experiment Excel shtick.  I love it.


Darinee’s PowerPivot Adventure – Postscript

November 10, 2009

Here’s the IM chat I had with Darinee right after I published her guest post:

  PowerPivot PostScript

9 am tomorrow was too generous.

Imagine how much things can change once we’ve empowered an army of Darinee’s :)


A 48 hour case study from Bing – Zero to Sixty :)

November 10, 2009

“When you can snatch this pebble from my hand, you will be re- HEY, GIVE THAT BACK!”

-Me

I promised a guest post, and a guest post I deliver :)  Darinee is a good friend that I met during my brief stint working on Bing, before Amir came calling and said “So, Rob, we’ve got this little project…”

I didn’t really think of her as an Excel person, and I definitely don’t think of her as a BI person.  To me, she is test programmer extraordinaire, cold-blooded Halo assassin, and relentless-pursuer-of-the-right-thing.  I guess it’s that latter quality that made her give me a call when she got assigned a reporting function…

Here, in her own words, is Darinee’s Fantastic 48 Hour Adventure with PowerPivot:

Hey all,

I’m Darinee, and I’m a test lead in Bing. But after a week of playing with PowerPivot, I’m seriously thinking that data analysis and visualization may be a future career move. J (Just kidding! No, really, don’t tell my manager…)

Every test lead at Microsoft, at some point, gets tasked with the job of analyzing bug metrics, and this always is done in Excel, because we don’t have time for any other approach. Excel can be great but mailing files around inevitably leads to out-of-sync files, and, frankly, storing the file in a Sharepoint document library doesn’t change things much since people save them off locally anyway.

Before PowerPivot, I had built an Excel file that dynamically pulled data from our bug database. It had an über-table with 70K rows of bug data and a bazillion calculated columns, many of which were VLOOKUPs to other tables. While all of this data allowed me to graph what I needed, this ultimately meant a sluggish Excel file that took forever to update – plus it was 25MB. :(

Here’s what it looked like:

PP_Old

PP_OldData

So, what did I do? Rob here worked with me in Bing for awhile before leaving us to go back to BI-land (traitor!), and I called him about PowerPivot because last time we talked, he said it was aimed at this sort of thing. By the end of our 30 minute conversation, I couldn’t get this dopey grin off my face as to what PowerPivot+Sharepoint could do for me. I installed the Excel add-in. That was Tuesday around lunchtime.

By late afternoon, the smile was even bigger. Switching over to PowerPivot took less time than updating my old spreadsheet! PowerPivot completely simplified my data model AND resulted in faster performance. I was able to throw those lookup tables each into their own PowerPivot table, connected by relationships (no VLOOKUP’s!), while still producing the same meaningful pivot tables. In fact, another table of data, which was prohibitive to add to my prior workbook, added hardly any size to my new PowerPivot workbook. Multi-table pivots are f.p.! (That stands for, ahem, frankly phenomenal.) And my new file, despite containing much more data than the old file, now was only 13MB!

Ok, I’m kind of a n00b for not knowing about Slicers before, but I love how PowerPivot automatically formats them around my charts. I was easily able to add Slicer controls to break down the data by six different fields without dealing with layout. A few tweaks to appearance, and I was now looking at a dashboard-type application, whereas before I had just had… a spreadsheet. And again, I couldn’t get this dopey grin off my face at work so my neighbors knew something extraordinary was up.  They were right:

     PP_New2

Wednesday morning, I set up my own Sharepoint server with a PowerPivot gallery, and boom! – I had the same usable reports via a URL that I could share. No more out of sync problems, because everyone just consumes via the web interface. Portal with thumbnails, too?  Money!

PP_NewSP

By Thursday mid-day I’d finished my tweaks. Ultimately, PowerPivot made organizing my data simple and let me create beautiful reporting apps quickly – reporting with Excel is now actually FUN! The official reporting team next door is eyeing my site now, saying ooh… we want that. And all in a few days’ work. :)

Rob here again.  Seriously, this was a blast for me.  I didn’t expect things to go this way, to be honest.  I figured I’d give her a quick summary on the phone, and maybe in a few weeks she’d have time to kick the tires a little, ask me more questions, etc.  This isn’t her main job, after all.

Nope, not Darinee.  About once an hour, every hour, after our phone chat, I’d get an update over IM.  “OMG, multi-table pivots!!!”  Then “Slicers rock!!!”  And “I can’t believe this is still Excel!”

And then the clincher, Wednesday morning – “Hey, I’ve got my SharePoint server set up, here’s a URL to a published report – whaddya think?”

Everyone, please do me a favor – when you blast through things at this speed and are having as much fun with it as Darinee is, drop me a note so I can share that joy.  Darinee’s dopey smile was nothing compared to mine :)

Summing up, here are the things that struck me:

  1. She no longer has a version control problem on her spreadsheets, thanks to web rendering
  2. Her next step is to configure auto-refresh of these reports for zero-maintenance updates – Darinee, I give you til 9 am tomorrow :)
  3. Her reports are now much more functional than before – they operate on more data and offer much more exploration capability thanks to slicer dashboards
  4. Her files, despite containing much more data, are half the size as before
  5. Her coworkers are all excited about this.  Excited.  About Excel reports.  Let that sink in.  And then remember that end user engagement pays all sorts of dividends.
  6. She literally went from knowing NOTHING about PowerPivot, to a full deployment on a server she set up, in two days time.

Awesome.  Darinee and I are still arguing over who should feel grateful to who :)