Note: The results of this comparison were shared online, a replay of which is available here.

Which Self-Service BI tool is the best?

It is a topic that has been discussed & debated quite often. While the question takes various forms — with a mix of tools chosen amongst Tableau, Power BI, TIBCO Spotfire, Qliksense, SAP Lumira and the like for consideration— the answers to any one question takes different forms depending on whom you speak to.

We often found that considerations used to assess these tools were either oversimplified (e.g. We chose Tableau as it delivers the best User Experience) or relied on a larger but yet seemingly limited set of factors at the high level (e.g. cost, data connectivity, data blending, visualizations, collaboration, mobility and data governance).

To be sure, we – the team at Visual BI – decided to figure it out on our own, using factors as granular as possible. And thus we set out on one of the most elaborate comparisons every done. We decided to focus on four products — Tableau, Power BI, SAP Lumira and SAP Analytics Cloud. (Note: We do intend to conduct another comparison exercise in the near future with additional products such as Qliksense and Tibco Spotfire in the mix)

The complete findings, compiled in a deck with 50+ slides, were presented online – a recording of which is available  here.

In this column, we’ll review our approach & a summary of the findings.

1. Primary Evaluation Criteria (Functional)

We decided to perform the evaluation across five (5) major areas-

• Data Acquisition
• Data Preparation
• Data Visualization
• Data Exploration
• Data Collaboration

And within each area, we decided to drill down further into more granular aspects as shown below.

For example, a comparison of maps & charts available gave the following findings. (Note: The visual for charts shown below is a partial list)

2. Supplementary Evaluation Criteria (Typical Enterprise Needs)

In addition to the functional criteria outlined above, we also decided to focus on an exhaustive set of enterprise considerations such as the following (in no particular order)-

1. Single Sign On
2. Security
3. OS & Browser Compatibility
4. Web Authoring
5. Mobile Compatibility
6. Notifications
7. Live Connectivity
8. Pricing
9. Need for buying (and thus additional pricing for) add-on components/licenses such as Tableau Extracts, Power BI Service & Enterprise Gateway
10. SDK Availability
11. Offline Analysis
12. Integration with on-premise & cloud platforms
13. … and more

A sample slide comparing collaboration & sharing features across the four products is shown below

3. Tertiary Evaluation Criteria (for SAP Customers)

We did a deep-dive into connectivity factors specific to enterprises running SAP, such as-

1. SAP HANA Connectivity (Import & Live connectivity modes)
2. SAP BW Connectivity (Import & Live connectivity modes)
3. SAP BW features (hierarchies, merge, conditions, exceptions, cell editors, etc)
4. SAP BusinessObjects Universe connectivity
5. Use of SAP BICS (using OLAP engine) vs. Non-SAP MDX Connectivity

For example, a comparison of SAP BW connectivity (Live mode) across the products is shown here

And finally, we also looked into implications of using BICS vs. MDX (the former used by SAP and the latter used by Tableau & Power BI)

Conclusion

No surprises here..

After an in-depth evaluation we have came to the conclusion that the devil is in the details. And there is no one answer that would suit everyone.

For an in-depth analysis & a list of evidences supporting this conclusion, view our replay here.

And finally, here are some snippets from our analysis that may interest you:

1. The turf on which we are playing — home vs. away — as decided by whether you are predominantly an SAP shop or Microsoft shop — is important. As a rule of thumb, you may not attain strong benefits by playing on an away turf. SAP Analytics Cloud works best for SAP HCP or Cloud-based SAP data sources and cloud applications; Power BI works best for Azure or MS stack out of the lot; Tableau is best-in-class and may appeal to you if you are not aligned heavily with either SAP or Microsoft. At the same time, Tableau does lack a home turf and it would be interesting to watch where they go from here.

2. Tableau is feature-rich, delivers the best UX and can connect to tons of data sources. However it also seems to be more expensive of the lot.

3. Power BI is closing the gap well with Tableau. It integrates best with Office365 and Sharepoint, provides intuitive Excel-like interface to deliver one of the best data preparation experience;

4. One drawback of Power BI & Tableau: SSO & Security would be a challenge

5. Tableau & Power BI connectivity to SAP sources is sub-optimal due to reliance on the older MDX connectivity vs. the newer BICS

6. SAP Analytics Cloud works best for SAP HCP or Cloud-based data sources; Support for Planning is a unique selling point. As a cloud-only system, we found it to be much more powerful than Tableau online or Power BI online.

7. SDKs for SAP Lumira & Power BI provide options for custom visualizations & features

8. Comparing pricing is not straightforward — there are add-on components, hidden costs, volume licensing, tiered plans & more

Some differences in capabilities are due to philosophical differences in the way the vendors approach Self Service BI. For example, SAP Lumira & SAP Analytics Cloud have minimal data blending capability for HANA and BW Live connections — as the belief is that any data wrangling has to be done at the back end — and not at the visualization or front-end layer. In a similar way, SAP Lumira advocates connecting to only HANA views using the native connector as compared to consuming tables through the Query with SQL option.

10. For the above reasons and more — just because a tool worked in a different firm (even at your previous firm), it is no guarantee that it would work for your firm today. There are simply many contextual considerations that come into play.

11. And finally, the products are in different stages of lifecycle & maturity — what applies today may not hold true 12 months from now.

That summarizes our approach and findings. If you would like our team to walk you through the detailed (and updated) findings, reach out to us here.