Category Archives: Intelligence Server

Interview Question #11: History List Messages Stored in a Database

Question

Which of the following is possible only when History List messages are stored in a database?

A. History List Messages are available in a two-tier environment.

B. When deleting History List messages using a schedule, a single session is created on the Intelligence Server.

C. History List messages can be shared across cluster nodes.

D. You can duplicate History List messages on reprompt or refresh.

E. Both B and C.

 

 

 

 

 

Answer

B. When deleting History List messages using a schedule, a single session is created on the Intelligence Server.

Benefits of Using Database-Based History List Storage

  • Access additional information about each History List message, such as message and job execution statistics. As a result, you can effectively monitor and manage the History List based on its actual usage by the users.
  • The database backing of the History List provides greater scalability and improved performance. Instead of accessing a multitude of large files that usually reside on the server machine, you retrieve users’ inbox information from a database.
  • When the administrative task to delete History List messages is triggered, it creates only one session on the Intelligence Server rather than multiple separate sessions for each user.
  • Use the History List Messages monitor to monitor and manage messages for each MicroStrategy user.

 

MicroStrategy Course Where You Will Learn About This Topic

MicroStrategy Administration Course

Using the R Integration functionality, how to perform Text Mining on a MicroStrategy report and display the result

Readers:Jaime Perez

Here is another great post in the MicroStrategy Community from Jaime Perez (photo, right) and his team. A lot of work when into the preparation of this post and it shows some great ways to use the “R” integration with MicroStrategy.

Contributors from Jaime’s team include:

ssonobe  ssonobe

Lili 

Joanne A 

Ohingst  Ohingst

Enjoy!

Michael

Text Mining Using R Integration in MicroStrategy

Users may wish to perform text mining using R on the result of any arbitrary MicroStrategy report and display the result. One of the problems that hinders the users from achieving it is that the number of output elements is not always consistent. For example, a report may have three attributes named ‘Age groups’, ‘Reviewer’, and ‘Survey feedback’ and the report might display four rows of feedback as follows:

01.jpg

If the above report result is sent to R as an input and the R script breaks down each sentence of the feedback into the term frequency that is grouped by the age groups, it will have 18 rows.

02.jpg

Since the number of output elements is greater than the number of the MicroStrategy report rows, the report execution will fail. Using the objects in the Tutorial project, this technical note (TN207734) describes one way to display the result of text mining on a MicroStrategy report, using the R integration functionality.

PREMISE:
– Following the instructions in TN43665, the MicroStrategy R Integration Pack has already been installed on the Intelligence Server.

The Steps Involved

STEP 1: Decide on the input values that need to be sent to R via R metrics
The first step is to decide on which data you wish to perform text mining. In this technical note, the sample report will let users select one year element, the arbitrary number of category elements, and specify the Revenue amount in prompts. The report will then display the value of the normalized TF-IDF (term frequency and inverse document frequency) for every word showing up in the qualified Item attribute elements, grouped by the Category elements.

A user may select the following values for each prompt and the report may look as shown below.

  • Year: 2012
  • Category: Books, Movies, and Music
  • Revenue: greater than $15,000

03.jpg

Eventually, the user may want to see the normalized TF-IDF for every word showing up in the Item attribute elements as shown below:

04.jpg

Since the final output displays each word from the Item attribute and it is grouped by the Category elements, the necessary input values to R are as follows

  • The elements of the Category attribute.
  • The elements of the Item attribute.

 

STEP 2: Create metrics to pass the input values to R

The input values to R from MicroStrategy must be passed via metrics. Hence, on top of the current grid objects, additional metrics need to be created. For this sample report, since the inputs are the elements of two attributes, create two metrics with the following definitions so that the elements are displayed as metrics.

Max(Category@DESC) {~}

Max(Item@DESC) {~}

05.jpg

 

STEP 3: R script – Phase 1: Define input and output variables and write R script to obtain what you wish to display in a MicroStrategy report

In the R script, define (1) a variable that receives the inputs from MicroStrategy and (2) a variable that will be sent back to MicroStrategy as the output as depicted below. Since the number of output elements must match with the number of input elements, it is defined as “output = mstrInput2” to avoid the errors. In other words, this script executes R functions to obtain the data that you wish to display in a MicroStrategy report, but the output is the same as the input. More details about how to display the result in a MicroStrategy report will be followed up later in this technical note.

06.jpg

 

In this technical note, after manipulating the input value, we assume that the variable named ‘norm.TF.IDF’ in the R script holds the values of the TF-IDF for each term.

07.jpg

 

STEP 4: Create tables in the data warehouse to store the value of your R output

In order to display the values of the ‘norm.TF.IDF’ defined in a MicroStrategy report, tables to hold the result need to be created in the data warehouse. In other words, additional report will later have to be created in MicroStrategy and it will extract the data from the database tables, which are created in this section.

In this specific example, the variable ‘norm.TF.IDF’ has the elements of words (terms) and categories and the values of the normalized TF-IDF. Considering the types of data, the first two should be displayed as attributes and the values of the normalized TF-IDF should be presented in a metric. Hence, two lookup tables to hold the term and category elements and one fact table need to be created to store all the data. On top of these tables, one relationship table is also required since the relationship between words and categories is many-to-many.

 

STEP 5: R script – Phase 2: Populate the tables in your R script

As previously mentioned, the variable named ‘norm.TF.IDF’ contains the values, which a user wishes to display in a MicroStrategy report as shown below.

07.jpg

 

In this R script, four more variables are defined from ‘norm.TF.IDF’, each of which contains the subset of data that will be inserted into the database tables.

 

tm_Category holds the unique elements of the Category.

10.jpg

 

tm_Word holds the unique elements of the Word (Term).

11.jpg

 

tm_Word_Cat stores the values of the many-to-many relationship.

12.jpg

 

tm_Fact contains the values of TF-IDF for every Word-Category combination.

13.jpg

 

In the R script, populate the database tables with the above four subsets of ‘norm.TF.IDF’.

# Load RODBC
library(RODBC)

# RODBC package: assign ch the connectivity information
ch <- odbcConnect("DSN_name")

# Delete all the rows of the tables
sqlClear(ch, "tm_Category", errors = TRUE)
sqlClear(ch, "tm_Word",     errors = TRUE)
sqlClear(ch, "tm_Word_Cat", errors = TRUE)
sqlClear(ch, "tm_Fact",     errors = TRUE)

# SQL: insert the data into tables; use parameterized query
sqlSave(ch, tm_Category, tablename = "tm_Category", rownames=FALSE, append=TRUE, fast = TRUE)
sqlSave(ch, tm_Word,  tablename = "tm_Word", rownames=FALSE, append=TRUE, fast = TRUE)
sqlSave(ch, tm_Word_Cat, tablename = "tm_Word_Cat", rownames=FALSE, append=TRUE, fast = TRUE)
sqlSave(ch, tm_Fact, tablename = "tm_Fact", rownames=FALSE, append=TRUE, fast = TRUE)

#Close the channel
odbcClose(ch)

 

STEP 6: Create and add an R metric, which implements the R script

The R script is done. It is time to implement this R script from MicroStrategy by creating an R script. In the deployR interface, open the R script and define the input and output that you specify in Step 3 as follows. Since the elements of the Category and Item attributes are characters, choose “String” as its data type. Likewise, since the output is the same as the mstrInput2, its data type is also set to string.

14.jpg

 

Create a stand-alone metric and paste the metric definition of the deployR utility. Then, replace the last parameters by the Category and Item metrics that you created in Step 2.

15.jpg

 

Add the R metric to the report.

15.2.png

 

The report and R will perform the following actions after adding the R metric
i. The report lets users select the prompt answers
ii. MicroStrategy sends the Category and Item elements to R via the R metric
iii. R performs text mining to calculate the TF-IDF based on the inputs
iv. R generates subsets of the TF-IDF
v. R truncates the database tables and populates them with the subset of the TF-IDF
vi. R sends the output(which is actuary the input) to MicroStrategy
vii. The report displays the values of all object including the R metric

 

STEP 7: Create MicroStrategy objects to display the data

From the tables created in Step 4, create the Word and Category attributes and the fact named weight. The object relationship is as depicted below.

08.jpg

09.jpg

 

Now, create a new report with these objects. This report will obtain and display the data from the database tables.

16.jpg

 

STEP 8: Utilize the report level VLDB properties to manipulate the order of the report execution jobs

There are currently two reports and let each of which to be named R1 and R2 as described below

  • R1: A report which prompts users to specify the report requirements and implements the R script executing text mining
  • R2: This report obtains the result of text mining from the database and display it

 

If the two reports are placed in a document as datasets as shown below, there is one problem: R2 may start its execution before R1 populates the database tables with the result of text mining.

17.jpg

 

In order to force R2 to execute its job after the completion of R1, the VLDB properties PRE/POST statements along with additional database table may be used. The table tm_Flag contains the value of 0 or 1. R2 is triggered when R1 sets the value of completeFlag to 1. The detailed steps are described below with the script for SQL Server.

 

i. Create another table in the database, which holds the value of 1 or 0

CREATE TABLE tm_Flag
(
   completeFlag int
)


INSERT INTO tm_Flag VALUES(0)

 

ii. In the VLDB property ‘Report Post Statement 1” of the R1 report, defines a Transact-SQL statement that changes the value of completeFlag to the value of 1.

DECLARE @query as nvarchar(100)
SET @query = 'UPDATE tm_Flag SET completeFlag = 1'
EXEC sp_executesql @query

 

iii. Define the VLDB property ‘Report Pre Statement 1’ in R2 so that it will check the value of completeFlag every second and loop until it turns to 1. After the loop, it will revert the value of completeFlag back to 0. After this Report Pre Statement, R2 will obtain data from the database, which has been populated by R1.

DECLARE @intFlag INT
SET @intFlag = (select max(completeFlag) from tm_Flag)

WHILE(@intFlag = 0)
BEGIN
	WAITFOR DELAY '00:00:01'
	SET @intFlag = (select max(completeFlag) from tm_Flag)
END

DECLARE @query as nvarchar(100)
SET @query = 'UPDATE tm_Flag SET completeFlag = 0'
EXEC sp_executesql @query

 

Activity Diagram

18_revised.png

 

 

Overall execution flow

  1. Answer prompts

19.png

 

2. Only the text mining result is displayed to users

20.png

 

Third Party Software Installation:

WARNING: The third-party product(s) discussed in this technical note is manufactured by vendors independent of MicroStrategy. MicroStrategy makes no warranty, express, implied or otherwise, regarding this product, including its performance or reliability.

 

Tips & Tricks #2: In MicroStrategy Web, Report Execution Fails with Error ‘Results for this message cannot be retrieved from MicroStrategy Intelligence Server. This might be because the execution has failed. Please contact your administrator.’

I had this error the other day. Fortunately, I had just been reading about working set governor settings the other night to prepare for the CPA and MCE exams.

First, let’s discuss the issue.

Issue/Problem

In MicroStrategy Web, when I executed a report, I received the following error message (also, see screenshot below).

(Results for this message cannot be retrieved from MicroStrategy Intelligence Server. This might be because the execution has failed. Please contact your administrator.)

Results cannot be retrieved error message

Basically, what happened was my report request could not be processed. Looking through the MicroStrategy KnowledgeBase, it basically tells you to try to run the report again. If it still throws this error, contact your Administrator.

If you look at the Web Log on the Web Server, you will see an error similar to the following:

<record reset=`true`>
 <package>com.microstrategy.webapi</package>
 <level>SEVERE</level>
 <miliseconds>1157577081154</miliseconds>
 <timestamp>09/06/2006 14:11:21:154</timestamp>
 <thread>154</thread>
 <class>CDSSXMLReportServer</class>
 <method>GetExecutionResultsEx</method>
 <message>(Results for this message cannot be retrieved from 
MicroStrategy Intelligence Server. This might be because the
execution has failed. Please contact your administrator.) 
(com.microstrategy.webapi.MSTRWebAPIException)</message>
 <exception>com.microstrategy.webapi.MSTRWebAPIException: 
(Results for this message cannot be retrieved from 
MicroStrategy Intelligence Server. This might be because
the execution has failed. Please contact your 
administrator.)
 at com.microstrategy.webapi.
CDSSXMLReportServer.GetExecutionResultsCommon(Unknown Source)
 at com.microstrategy.webapi.
CDSSXMLReportServer.GetExecutionResultsEx(Unknown Source)
 at com.microstrategy.web.objects.
WebReportInstanceImpl.
getExecutionResults(Unknown Source)
 at com.microstrategy.web.objects.
WebReportInstanceImpl.pollForResults(Unknown Source)
 at com.microstrategy.web.objects.
WebReportInstanceImpl.populateUserReportCache(Unknown Source)
 at com.microstrategy.web.objects.
WebReportInstanceImpl.pollStatus(Unknown Source)
 at com.microstrategy.web.beans.
ReportInstanceProxy.pollStatus(Unknown Source)
 at com.microstrategy.web.beans.
ReportBeanImpl.localCollectData(Unknown Source)
 at com.microstrategy.web.beans.
ReportBeanImpl.doCollectData(Unknown Source)
 at com.microstrategy.web.beans.
AbstractWebBean.collectData(Unknown Source)
 at com.microstrategy.web.app.beans.
AbstractAppComponent.collectData(Unknown Source)
 at com.microstrategy.web.app.beans.
ReportFrameBeanImpl.collectData(Unknown Source)
 at com.microstrategy.web.app.beans.
AbstractAppComponent.collectData(Unknown Source)
 at com.microstrategy.web.app.beans.
PageComponentImpl.collectData(Unknown Source)
 at com.microstrategy.web.app.
MSTRWebController.processRequest(Unknown Source)
 </exception>
 <userName>Administrator</userName>
 <clientID>172.19.19.2</clientID>
 </record>

The DSSErrors.log file on the MicroStrategy Intelligence Server contains the following errors:

[Kernel][Error] MsiWorkingSet::PersistMsg(): 
failed to attach RI to msg, error 0x80003F79
 [Kernel][Error] CDSSServerMessage::put_OriginalRI: 
WSResultPool->AddRI for msg xxxx return error 0x80003F79
 [Kernel][Error] CDSSServerMessage::put_OriginalRI: 
WSResultPool->AddRI for msg xxxx return error 0x80003F79
 [Kernel][Error] CDSSServerMessage::GetReportInstance():
get_OriginalRI() return error 0x1

Now what?

Solution

The size of the report that was to be executed has 40MB report cache size while the ‘Maximum RAM for Working Set cache size’ is set as 102,400KB, as shown in the image below:

Governing Rules - Default - Working Set

The MicroStrategy Intelligence Server was unable to swap out the report instances of 40MB in the Working Set.  To resolve this issue, I needed to increase the size of Maximum RAM for Working Set cache to a higher value, for example 512,000KB.

What is the ‘Working Set’ Governor Setting?

When a user runs a report from MicroStrategy Web or Web Universal, the results from the report are added to what is called the working set for that user’s session and stored in memory on the Intelligence Server. The working set is a collection of messages that reference in-memory report instances. A message is added to the working set when a user executes a report or retrieves a message from his or her Inbox.

The purpose of the working set is to:

  1. Allow the efficient use of the web browser’s Back button.
  2. Improve web performance for manipulations.
  3. Allow users to manually add messages to the History List.

This setting is accessible via the MicroStrategy Intelligence Server Configuration as shown below:

Governing Rules - Default - Working Set

Working Set Governors

The ‘Working Set file directory’ is the location in the filesystem where the Report Instances may be persisted on disk. A report instance will be persisted on disk in binary format if its size exceeds the limit set by the ‘Maximum RAM for Working Set cache’ governor or none of the report instances in memory can be swapped to make room for the new report instance. The persisted report instance will be persisted as the <filename(GUID)>.po and may be reused if the report is invoked again.

The ‘Maximum RAM for Working Set cache’ is the governor that modulates the size of the WorkingSet Result Pool. The Maximum value is: 2147483647 MB, the Minimum value depends on version and is 200 MB in 9.3.1, and the Default value is: 200 MB. Note that if the value specified is greater then the machines memory it uses the default of 200 MB. The default value is usually sufficient, but if memory issues arise, as noted above, the setting can be increased. Increasing this setting means that the MicroStrategy Intelligence Server may be operating with a higher average memory footprint during its lifecycle, so proper tuning may be needed if memory usage becomes an issue.

Important Notes

  • There is no Working Set (WS) for a session created by the MicroStrategy Desktop client.
  • This is a MicroStrategy Intelligence Server configuration level setting, so it applies to all the projects and all the users and is not specific to a project. If these settings are changed, MicroStrategy Intelligence Server may need to be restarted.
  • The MicroStrategy Working Set is not the same as the Microsoft Windows Operating System Working Set.