BI Publisher provides a set of features to protect Oracle BI Publisher server instances from out-of-memory errors by blocking report requests that generate excessive amounts of data. These "memory guard" features consist of a set of properties that are set in the xmlp-server-config .xml configuration file and which control the behavior of BI Publisher when running reports in online and scheduled modes. Basically, the functionality of “Memory Guard” is intended to prevent excessive consumption of memory by execution of reports that attempt to return large volumes of data on BI Server.
Previously, a report which consumed excessive amounts of memory could cause the Weblogic Server hosting the BI Domain to encounter an “Out Of Memory” condition which would in turn cause it to shutdown and restart, triggering an outage of BI functionality. Often the user running the report would be unaware that they had triggered the outage but repeated execution of the report would result in repeated outages and adverse impact on the user base. As a result the Memory Guard was developed to impose restrictions on the amount of memory and size of data that a report could use and thereby prevent these repeated outages of the server.
Note: This feature has been release as part of the Fusion Applications Release 8 Patch Bundle 8.
If a custom report is repeatedly failing then there are a number of steps which can be taken to try and determine whether or not the Memory Guard is preventing its execution.
Most reports which fail encounter Memory Guard restrictions because they are generating large amounts of XML data which would require excessive amounts of memory to process. Often this excessive XML generation and memory consumption stems from a poorly understood or designed report which is running a custom SQL query against the database. With that in mind if you find that a report you are running is repeatedly failing then look to test with reduced amounts of data by adding filtering to the report, if it then completes then it is likely that you have a long running query or excessive data to process.
Numbers of different types of error message are encountered; the more commonly seen are as below:
--------------------------------------ERRORS-------------------------------------------------
Jul 22, 2014 5:00:59 PM oracle.xdo.enterpriseScheduler.util.ProcessorUtil logException
WARNING: !BurstingJobProcessor.getReportData :: ::ERROR_GETTING_REPORT_DATA::[INSTANCE_ID=adc00cfa.1406056167500] IOException:[INSTANCE_JOB_ID=1054] oracle.xdo.dataengine.diagnostic.XMLSizeLimitException: XML Output ( 200446bytes ) generated exceeds specified file size limit (200000bytes )..!
Please contact system administrator to increase file size limit.
oracle.xdo.XDOException: oracle.xdo.dataengine.diagnostic.XMLSizeLimitException: XML Output ( 200446bytes ) generated exceeds specified file size limit (200000bytes )..!
Please contact system administrator to increase file size limit.
at oracle.xdo.dataengine.XMLPGEN.processMergedDataSet(XMLPGEN.java:1652)
at oracle.xdo.dataengine.DataProcessor.processData(DataProcessor.java:376)
Jul 22, 2014 4:46:45 PM oracle.xdo.servlet.XDOServlet WARNING: javax.servlet.ServletException: oracle.xdo.XDOException: oracle.xdo.dataengine.diagnostic.XMLSizeLimitException: XML Output ( 105801bytes ) generated exceeds specified file size limit (100000bytes )..!!!!!!!
Please contact system administrator to increase file size limit
Jul 22, 2014 4:43:34 PM oracle.xdo.online.ViewerIO WARNING: oracle.xdo.XDOException: oracle.xdo.XDOException: oracle.xdo.XDOException: java.io.IOException: Memory above memory limit:257426
at oracle.xdo.online.DataCache.getDataFile(DataCache.java:430)
at oracle.xdo.online.ViewerIO.createReportViewData(ViewerIO.java:981)
Error occurred on retrieving data xml
Report data size exceeds the maximum limit (524288000 bytes). Stopped processing.
--------------------------------------ERRORS-------------------------------------------------
Following points needs to be considered in order to resolve the issue:
1)Ensure that you are not running 'blind' wildcard queries against database tables. There is a temptation to perform this type of query for data validation or to review the data that the tables contain but this can result in very large amounts of data being retrieved.
2)Ensure that any custom query is restricted to a maximum of 60 columns as documented in the Data Modeling Guide.
3)Ensure that queries are not simply pulled from the OBTI nqquery.log and copied into a BI Publisher report. Review the SQL being issued and ensure that any sub queries are necessary and required for the query being issued
4)Ensure that you do not simply join large numbers of tables without restriction. Various custom reports have been seen where >10 tables have been joined together with primary key joins and no filtering. This can mean that very large result sets are generated.
5)If the report is at the design stage steps can be taken to review the data model and establish if actions can be taken to reduce the amount of data being returned. Columns can be truncated for display in the report, checks made to establish if columns can be retrieved from alternate locations, and a review made to establish if grouping, sorting, etc can be performed within the data model to reduce the amount of data being sent before any calculations are performed.
6)If the full set of data is required then the efforts can be made to run the report as a scheduled job rather than synchronously from the Catalog. The expectation is that reports run synchronously are expected to return in a relatively short period of time as they would cause the users session to hang whilst the report is generating, therefore they would also be expected to have a small data set to process. If a large data set is to be processed then schedule the report in this way ensuring that only the XML_DATA_SIZE_LIMIT is being applied.
7)As a final option discussions could be had with the BI Administrator to establish if changes could be made to the Memory Guard parameters in order to allow the report to complete. This should be considered a last resort as increasing any of the Memory Guard parameter settings will increase the scope for other reports and users to run larger reports and ultimately could lead to OutOfMemory scenarios and BIDomain outages if not properly managed.
Note: There are nine properties that comprise the memory guard features. Some properties work together to fully define the conditions to terminate processing. The key settings are:
Also note that the XML_DATA_SIZE_LIMIT does not get applied when data for a data model comes from a non-SQL based source such as a pre-existing XML file, webservice/http output, or Excel file.
ONLINE_REPORT_MAX_DATA_SIZE
This feature enables you to specify a maximum data size allowed for online report viewing. When you set a maximum data size, the following occurs when a user opens a report for online viewing:
1.A user submits a report to view online in the browser.
2.The data engine generates the data for the report.
3.Before document generation, the size of the data (in bytes) is inspected.
4.If the data generated is larger than the maximum setting, the report processing is ended. The user gets the following message:
The report you are trying to run exceeds the data limit set for this server. Either re-run with parameters that reduce the data or schedule this report. Contact your Administrator if you have questions.
The user can then either set parameters (if available for the report) to limit the data and resubmit online; or use the BI Publisher scheduler to submit the report.
To enable this feature, set the property ONLINE_REPORT_MAX_DATA_SIZE.
ONLINE_REPORT_FREE_MEMORY_THRESHOLD
This setting enables you to specify a minimum value for free JVM space. This enables you to control whether to run a report based on two factors: current usage and the size of the report data. This feature requires the setting of several properties that work together. You specify the threshold JVM space, the report maximum report size that will be allowed when the JVM falls below the threshold, and the maximum wait time to pause the report to wait for more JVM free space to become available.
When you set these properties, the following occurs when a user opens a report for online viewing:
1.A user submits a report to view online in the browser.
2.The data engine generates the data for the report.
3.JVM memory is inspected. If the available JVM memory is above the threshold value, the report processes as usual and there is no system intervention.
If the available JVM memory is below the threshold value, the size of the report data is inspected and compared to the setting online report data size under free memory. If the report data is below this threshold, then the report continues processing.
If the report data size exceeds the threshold, then the report is paused to wait for free memory to become available. The report will wait for the time specified in the wait period setting. If the free memory does not rise back above the minimum in the wait period specified, the report request is rejected.
ONLINE_REPORT_MAX_DATA_SIZE_UNDER_FREE_MEMORY_THRESHOLD
This setting sets an absolute limit to the data that can be generated from the execution of a data model. This setting applies to both online report requests and to requests submitted through the scheduler. When the size of the file generated by the data engine exceeds the limit, the data engine terminates execution of the data model and throws the exception:
"oracle.xdo.dataengine.diagnostic.XMLSizeLimitException: XML Output (NNNNNNbytes) generated exceeds specified file size limit (NNNNNbytes)".
If the report request was submitted through the scheduler, the job will show as failed in the Report Job History page. The exception error noted above displays when you rest your cursor over the status. If the report request was submitted online, the user will get the error "Unable to retrieve the data XML."
DB_FETCH_SIZE_AUTO
This setting calculates and sets database fetch size at run time depending on total number of data set columns and total number of query columns. Setting this property will override the server-level and data model-level database fetch size properties. When set, this property takes effect for all data models and can significantly slow processing time. This setting is recommended for implementations of BI Publisher that frequently process complex queries of hundreds of columns, such as Oracle Fusion Applications implementations. This setting is not recommended for most general implementations of BI Publisher.
Changes using GUI: The parameters are available through the BI Publisher > Administration > Runtime Configuration > Properties:
![Memory Guard Settings 1]()
Changes from Configuration File:
Implement the memory guard features by adding this set of properties to the xmlp-server-config.xml file located at:
{BI Publisher Repository}/Admin/Configuration/xmlp-server-config.xml
The following shows sample entries for the memory guard properties in the xmlp-server-config.xml file
({BI Publisher Repository}/Admin/Configuration/xmlp-server-config.xml):
![Memory Guard Settings 2]()