Today, I am adding the second post in my MicroStrategy Report Optimization series. This will be a multi-part series (I will leave it open-ended so I can continue to add to it).
Today, we will look at the components that comprise performance.
Source: MicroStrategy University, Deploying MicroStrategy High Performance BI, V9.3.1, MicroStrategy, Inc. September, 2013.
Components of Performance
There are five key layers or components that a typical BI query must go through. They are:
- Caching Options
- Data Transfer
- System Architecture and Configuration
- Client rendering or Data Presentation
- Data Warehouse Access
The components above are not listed in any specific order of access during the execution of a query. The image below illustrates the five components.
Caching and Intelligent Cubes
MicroStrategy’s memory technology is engineered to meet the increased demand for higher BI performance, which is driven by the rapid expansion of both data volumes and the number of BI users in organizations across industries. MicroStrategy accelerates performance by pre-calculating computations and placing the results into its memory acceleration engine to dramatically improve real-time query performance.
Data transfer over one or more networks are a very important component of a BI implementation. A slow or poorly tuned network performance in any of those transfers will translate into poor performance from a report or a dashboard execution perspective.
System Architecture and Configuration
Successful BI applications accelerate user adoption and enhance productivity, resulting in demand for more users, data, and reports. MicroStrategy provides the ability to adapt quickly to constant changes and evolve along with business requirements. MicroStrategy Intelligence Server hs been proven in real-world scenarios to deliver the highest performance at scale with the fewest servers and minimum IT overhead.
Dashboards provide graphical, executive views into KPIs, enabling quick business insights. MicroStrategy enables higher performing dashboards, averaging 30-45% faster execution and interactivity. Using new compression methods, MicroStrategy dashboards have a smaller footprint than ever before – up to to 55% smaller – resulting in faster delivery using less network bandwidth. Dashboards deliver ever more analysis and data for end-users.
Data Warehouse Access
High performance BI starts with optimizing SQL queries to retrieve results from the database as quickly as possible. BI performance is dependent largely on the time that the queries take to execute in the database. An average reporting request usually takes 40 seconds to complete, out of which 34 seconds, or 85% of the query time, is spent executing in the database.
Therefore, it is critical to optimize report queries to reduce database execution time.
This is from a blog post by Thomas Greco.
Thomas is a web developer / graphic designer living in New York City. When Thomas isn’t striving towards frontend perfection, he enjoys hanging with friends, going to concerts, and exploring through the wilderness!
The Dygraphs.js library allows developers to create interactive charts using the X and Y axis to display powerful diagrams. The more data being parsed, the higher the functionality of the graph. That being said, Dygraphs was built for these visualizations to contain a multitude of views. For example, Dygraphs.js makes it capable to analyze separate portions of a data-set, such as specific months, in addition to the timeframe in its entirety. Also, the Dygraphs.js library is compatible across all major web browsers, and can responds to touch sensitivity, making it a thoroughougly solid choice as a data visualization framework.
Eventually becoming the successor to Protovis.js, D3 is capable of creating stunning graphics via dynamically updating the DOM. An acronym for Data-Driven Document, D3.js makes use of chained methods when scripting visualizations, subsequently creating dynamic code that is also reusable. Due to its reliance on the DOM, D3 has been created in accordance with W3C web standards so that the library may render correctly across web browsers. Lastly, D3′s path generator function, defined as
d3.svg.line(), gives developers the capability to produce a handful of SVGs by defining different paths, and their properties.
The Google Visualization API
Hailing from the Google Developers Console (GDC), Google’s Visualization API can be called with barely any code. In addition to easy DOM modification, this Google API makes it easy for its user to easily define custom modifier functions that can then be placed into custom groups. Furthermore, this interface’s usability, matched with its support from the GDC’s open source network, place it among the top of the list of data visualization tools.
Polymaps.js makes use of SVGs to generate interactive web maps with cross browser compatibility in mind. At the heart of Polymaps lies vector tiles, which help ensure both optimal load speeds and optimal zoom functionality. Although it may come configured with components, Polymaps.js is easily customized, and is able to read data in the form of vector geometry, GeoJSON Files, and more. Check out the graph below of the U.S. created by the U.S. Census borough.
The Raphael.js library was created with an emphasis on browser compatibility. The framework follows the SVG W3C Recommendation, which is a set of standards that ensure images are completely scalable and without pixelation. In addition to the use of SVGs, Raphael.js even reverts to the Vector Model Language (VML) if rendered in Internet Explorer browsers prior to IE9. Although VML is very rarely used today, the support for it does a great job of showing the attention to detail that the Raphael.js team placed on this project when developing the library.
For those who already use the juggernaut that is Ember.js, the developers at Addepar Open Source have created a few add-on libraries to extend the Ember experience: Ember Table, Ember Widgets, and Ember Charts. A child of Ember.js and D3.js, Ember Charts utilizes the properties of flat-design. Although limited, the library does have a handful of options that deal with properties such as color and size, making it fairly simple to create impressive visualizations. Nonetheless, Ember’s presence in the front end could really help Ember Chart’s popularity in the future.
It has been a while since I have discussed some of the latest creative thoughts on data visualization from Stephen Few. I have read all of Steve’s books, attended several classes from him, and religiously follow his blog and newsletter on his website, Perceptual Edge.
For those of you who don’t know, Stephen Few is the Founder & Principal of Perceptual Edge. Perceptual Edge, founded in 2003, is a consultancy that was established to help organizations learn to design simple information displays for effective analysis and communication.
Steve has stated that his company will probably always be a company of one or two people, which is the perfect size for him. With 25 years of experience as an innovator, consultant, and educator in the fields of business intelligence and information design, he is now considered the leading expert in data visualization for data sense-making and communication.
Steve writes a quarterly Visual Business Intelligence Newsletter, speaks and teaches internationally, and provides design consulting. In 2004, he wrote the first comprehensive and practical guide to business graphics entitled Show Me the Numbers, now in its second edition. In 2006, he wrote the first and only guide to the visual design of dashboards, entitled Information Dashboard Design, also now in its second edition. In 2009, he wrote the first introduction for non-statisticians to visual data analysis, entitled Now You See It.
Here is his latest thoughts from his newsletter.
Why Do We Visualize Quantitative Data?
Per Stephen Few, we visualize quantitative data to perform three fundamental tasks in an effort to achieve three essential goals:
These three tasks are so fundamental to data visualization, Steve used them to define the term, as follows:
Data visualization is the use of visual representations to explore, make sense of, and communicate data.
Steve poses the question of why is it that we must sometimes use graphical displays to perform these tasks rather than other forms of representation? Why not always express values as numbers in tables? Why express them visually rather than audibly?
Essentially, there is only one good reason to express quantitative data visually: some features of quantitative data can be best perceived and understood, and some quantitative tasks can be best performed, when values are displayed graphically. This is so because of the ways our brains work. Vision is by far our dominant sense. We have evolved to perform many data sensing and processing tasks visually. This has been so since the days of our earliest ancestors who survived and learned to thrive on the African savannah. What visual perception evolved to do especially well, it can do faster and better than the conscious thinking parts of our brains. Data exploration, sensemaking, and communication should always involve an intimate collaboration between seeing and thinking (i.e., visual thinking).
Despite this essential reason for visualizing data, people often do it for reasons that are misguided. Steve dispels a few common myths about data visualization.
Myth #1: We visualize data because some people are visual learners.
While it is true that some people have greater visual thinking abilities than others and that some people have a greater interest in images than others, all people with normal perceptual abilities are predominantly visual. Everyone benefits from data visualization, whether they consider themselves visual learners or not, including those who prefer numbers.
Myth #2: We visualize data for people who have difficulty understanding numbers.
While it is true that some people are more comfortable with quantitative concepts and mathematics than others, even the brightest mathematicians benefit from seeing quantitative information displayed visually. Data visualization is not a dumbed-down expression of quantitative concepts.
Myth #3: We visualize data to grab people’s attention with eye-catching but inevitably less informative displays.
Visualizations don’t need to be dumbed down to be engaging. It isn’t necessary to sacrifice content in lieu of appearance. Data can always be displayed in ways that are optimally informative, pleasing to the eye, and engaging. To engage with a data display without being well-informed of something useful is a waste.
Myth #4: The best data visualizers are those who have been trained in graphic arts.
While training in graphic arts can be useful, it is much more important to understand the data and be trained in visual thinking and communication. Graphic arts training that focuses on marketing (i.e., persuading people to buy or do something through manipulation) and artistry rather than communication can actually get in the way of effective data visualization.
Myth #5: Graphics provide the best means of telling stories contained in data.
While it is true that graphics are often useful and sometimes even essential for data-based storytelling, it isn’t storytelling itself that demands graphics. Much of storytelling is best expressed in words and numbers rather than images. Graphics are useful for storytelling because some features of data are best understood by our brains when they’re presented visually.
We visualize data because the human brain can perceive particular quantitative features and perform particular quantitative tasks most effectively when the data is expressed graphically. Visual data processing provides optimal support for the following:
1. Seeing the big picture
Graphs reveal the big picture: an overview of a data set. An overview summarizes the data’s essential characteristics, from which we can discern what’s routine vs. exceptional.
The series of three bar graphs below provides an overview of the opinions that 15 countries had about America in 2004, not long after the events of 9/11 and the military campaigns that followed.
Steve first discovered this information in the following form on the website of PBS:
Based on this table of numbers, he had to read each value one at a time and, because working memory is limited to three or four simultaneous chunks of information at a time, he couldn’t use this display to construct and hold an overview of these countries’ opinions in his head. To solve this problem, he redisplayed this information as the three bar graphs shown above, which provided the overview that he wanted. Steve was able to use it to quickly get a sense of these countries’ opinions overall and in comparison to one another.
Bonus: Here is a link to where Steve discusses the example above on his website.
2. Easily and rapidly comparing values
Try to quickly compare the magnitudes of values using a table of numbers, such as the one shown above. You can’t, because numbers must be read one at a time and only two numbers can be compared at a time. Graphs, however, such as the bar graphs above, make it possible to see all of the values at once and to easily and rapidly compare them.
3. Seeing patterns among values
Many quantitative messages are revealed in patterns formed by sets of values. These patterns describe the nature of change through time, how values are distributed, and correlations, to name a few.
Try to construct the pattern of monthly change in either domestic or international sales for the entire year using the table below.
Difficult, isn’t it? The line graph below, however, presents the patterns of change in a way that can be perceived immediately, without conscious effort.
You can thank processes that take place in your visual cortex for this. The visual cortex perceives patterns and then the conscious thinking parts of our brains make sense of them.
4. Comparing patterns
Visual representations of patterns are easy to compare. Not only can the independent patterns of domestic and international sales be easily perceived by viewing the graph above, but they can also be compared to one another to determine how they are similar and different.
These four quantitative features and activities require visual displays. This is why we visualize quantitative data.
I have received an e-mail from a reader that wanted me to discuss how to optimize MicroStrategy Reports.
I think this is an excellent topic and will today start a new blog topic called MicroStrategy Report Optimization. This will be a multi-part series (I will leave it open-ended so I can continue to add to it).
Before we start the series, it is important to understand report execution flow in MicroStrategy. That is what I will start with today.
Source: MicroStrategy University, Deploying MicroStrategy High Performance BI, V9.3.1, MicroStrategy, Inc. September, 2013.
Report Execution Flow
To better understand how some of the components affect the report performance in a MicroStrategy environment, it is important to become familiar with the report execution flow.
The following image depicts the different steps involved in the report execution process.
The following steps and components are involved in the report execution flow.
Tips & Tricks #11: The URL path is not visible in the report properties dialog box in MicroStrategy Web
Each report and document has a URL path that could be used to directly execute the object when pasted in the browser address bar. The URL path can be found in the properties section when a user right-clicks on the object.
The expected view of that window is as follows:
The view observed by some users without the link feature:
The required feature is not enabled in the MicroStrategy Web Admin page.
- Navigate to the MicroStrategy Web Admin page.
- Select ‘Security’ on the left-hand side.
- Under ‘URL’, enable the feature – “Session information is included on the URL”.
If the session information setting should remain off for security reasons, use the Show Link feature as shown below.
MicroStrategy Knowledgebase Technical Note 39787, Updated July 22, 2013.
Mr. Dresner is Chief Research Officer of Dresner Advisory Services, LLC, an independent advisory firm and a well-known authority and author in the areas of Business Intelligence and Performance Management.
Howard has 32 years of IT industry experience with 24 years in the Business Intelligence market.
He spent 13 years at Gartner, where he was a Research Fellow and Lead Analyst for BI. He also served as Chief Strategy Officer at Hyperion Solutions prior to forming Dresner Advisory Services in 2007.
Howard is a frequent speaker around the globe and has published two books on the subject – including: Profiles in Performance – Business Intelligence Journeys and the Roadmap for Change (John Wiley & Sons, November 2009) and The Performance Management Revolution: Business Results through Insight and Action (John Wiley & Sons, November 2007).
Through the Wisdom of Crowds Business Intelligence market research reports, Howard engages with a global community to redefine how research is created and shared.
I thought I would share a peek into his report with you on the topic of Success with Business Intelligence and Technology Priorities.
Mr. Dresner notes that with a few exceptions, reports of success with BI deviate little based on specific technology priorities. Those claiming success are slightly more likely to favor in-memory analysis and data mining and advanced algorithms. Those that are less successful were more likely to favor big data, analytical applications, software-as-a-service / cloud and location intelligence.
As part of his survey, when asked for reasons why business intelligence fails, respondents point to shortfalls and constraints surrounding data that include “tools” and “time,” but also “business,” “organization” and “management”.
Primary reasons for failure include: a lack of management understanding or appreciation of BI, a predominant focus upon technology vs. solving business problems and a lack of skills and resources to deliver solutions.
To visualize the data associated with success and not being successful, Mr. Dresner uses a radar chart (see chart below). A radar chart is a graphical method of displaying multivariate data in the form of a two-dimensional chart of three or more quantitative variables represented on axes starting from the same point. The relative position and angle of the axes is typically uninformative.
The radar chart is also known as web chart, spider chart, star chart, star plot, cobweb chart, irregular polygon, polar chart, or kiviat diagram.
As I delve deeper into the report, I will share other insights with you.