Oracle "Hyperion EPMA Server" service is not coming online

Recently I have encountered this issue with Oracle Hyperion. When we restarted Hyperion server, the “Hyperion EPMA Server” service is not coming online and application is down.

Here are the Errors from event.log

EPMA_Server
[EPMA Server Startup] ERROR SVR_ERR_PROCESSMGR_CANT_INIT_SESSIONMGR:Cannot initialize the Session Manager.Hyperion.DimensionServer.Interface.Exceptions.EPMAServiceException: Cannot initialize the Session Manager. ---> Hyperion.CommonServices.Exceptions.SessionManagerException: An error was encountered by the CAS Security Provider: Class: java.lang.NullPointerException
StackTrace:

at Hyperion.DimensionServer.SessionManager.CasSecurityProvider.GetDataBaseDetails(String& vendor, String& serverName, Int32& dbPort, String& dbName, String& username, String& password, String& jdbcUrl)
at Hyperion.DimensionServer.SessionManager.SessionManager.InitializeSqlConnectionString()
at Hyperion.DimensionServer.SessionManager.SessionManager.Initialize(String configFileName, Boolean restorePastInstanceSessions, Boolean enableCaching)
--- End of inner exception stack trace ---


Hyperion EPMA DimmensionServer
Service cannot be started. Hyperion.DimensionServer.Interface.Exceptions.EPMAServiceException: Cannot initialize the Session Manager. ---> Hyperion.CommonServices.Exceptions.SessionManagerException: An error was encountered by the CAS Security Provider: Class: java.lang.NullPointerException
StackTrace:
at Hyperion.DimensionServer.SessionManager.CasSecurityProvider.GetDataBaseDetails(String& vendor, String& serverName, Int32& dbPort, String& dbName, String& username, String& password, String& jdbcUrl)
at Hyperion.DimensionServer.SessionManager.SessionManager.InitializeSqlConnectionString()
at Hyperion.DimensionServer.SessionManager.SessionManager.Initialize(String configFileName, Boolean restorePastInstanceSessions, Boolean enableCaching)
--- End of inner exception stack trace ---

The event log is no much helpful. But note that the Hyperion EPMA Service will not come up if you have any issues with database. Check the logs to see any “ORA- “errors .

I found below error in EssbaseAdminServices0.log
#### <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1329242417925> ORA-28001: the password has expired

Now it’s clear that Database user password expired.

SQL> select USERNAME, ACCOUNT_STATUS from dba_users where USERNAME = ‘SHARED_SERVICES’;

USERNAME ACCOUNT_STATUS
------------------------------ --------------------------------
SHARED_SERVICES EXPIRED(GRACE)

Do not change the password. If you change the password then you need to re-configure the Hyperion service in order to use the new password.

If you know the password then set the using below command.
SQL> Alter user SHARED_SERVICES identified by ‘xxxxxx’;

If you don’t know the password then reset the password following below steps
SQL> select password from sys.user$ where name='SHARED_SERVICES';

PASSWORD
------------------------------
61266722B44D5BG418

SQL> alter user QWERTY identified by values '61266722B44D5BG418’;
User altered.

Also make sure to modify PASSWORD_LIFE_TIME to “UNLIMITED” for DEFAULT or User profile that you are using

SQL> alter profile default limit PASSWORD_LIFE_TIME unlimited;
Profile altered.

Regards,
Satishbabu Gunukula
http://www.oracleracexpert.com

Oracle "Hyperion EPMA Server" service is not coming online

I have come across below issue wtih Oracle Hyperion. When we restarted Hyperion server, the “Hyperion EPMA Server” service is not coming online and application is down.

Here are the Errors from event.log

EPMA_Server

[EPMA Server Startup] ERROR SVR_ERR_PROCESSMGR_CANT_INIT_SESSIONMGR:Cannot initialize the Session Manager.Hyperion.DimensionServer.Interface.Exceptions.EPMAServiceException: Cannot initialize the Session Manager. ---> Hyperion.CommonServices.Exceptions.SessionManagerException: An error was encountered by the CAS Security Provider: Class: java.lang.NullPointerException

StackTrace:

at Hyperion.DimensionServer.SessionManager.CasSecurityProvider.GetDataBaseDetails(String& vendor, String& serverName, Int32& dbPort, String& dbName, String& username, String& password, String& jdbcUrl)

at Hyperion.DimensionServer.SessionManager.SessionManager.InitializeSqlConnectionString()

at Hyperion.DimensionServer.SessionManager.SessionManager.Initialize(String configFileName, Boolean restorePastInstanceSessions, Boolean enableCaching)

--- End of inner exception stack trace ---


Hyperion EPMA DimmensionServer

Service cannot be started. Hyperion.DimensionServer.Interface.Exceptions.EPMAServiceException: Cannot initialize the Session Manager. ---> Hyperion.CommonServices.Exceptions.SessionManagerException: An error was encountered by the CAS Security Provider: Class: java.lang.NullPointerException
StackTrace:
at Hyperion.DimensionServer.SessionManager.CasSecurityProvider.GetDataBaseDetails(String& vendor, String& serverName, Int32& dbPort, String& dbName, String& username, String& password, String& jdbcUrl)
at Hyperion.DimensionServer.SessionManager.SessionManager.InitializeSqlConnectionString()
at Hyperion.DimensionServer.SessionManager.SessionManager.Initialize(String configFileName, Boolean restorePastInstanceSessions, Boolean enableCaching)
--- End of inner exception stack trace ---


The event log is no much helpful. But note that the Hyperion EPMA Service will not come up if you have any issues with database. Check the logs to see any “ORA- “errors .

I found below error in EssbaseAdminServices0.log

#### <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1329242417925> ORA-28001: the password has expired

Now it’s clear that Database user password expired.

SQL> select USERNAME, ACCOUNT_STATUS from dba_users where USERNAME = ‘SHARED_SERVICES’;
USERNAME ACCOUNT_STATUS
------------------------------ --------------------------------
SHARED_SERVICES EXPIRED(GRACE)

Do not change the password. If you change the password then you need to re-configure the Hyperion service in order to use the new password.

If you know the password then set the using below command.
SQL> Alter user SHARED_SERVICES identified by ‘xxxxxx’;

If you don’t know the password then reset the password following below steps
SQL> select password from sys.user$ where name='SHARED_SERVICES';

PASSWORD
------------------------------
61266722B44D5BG418

SQL> alter user QWERTY identified by values '61266722B44D5BG418’;
User altered.

Also make sure to modify PASSWORD_LIFE_TIME to “UNLIMITED” for DEFAULT or User profile that you are using

SQL> alter profile default limit PASSWORD_LIFE_TIME unlimited;
Profile altered.

Regards,
Satishbabu Gunukula
http://www.oracleracexpert.com

BEA-002936 maximum thread constraint or BEA-000337 [STUCK] ExecuteThread in WebLogic

I come across an issue and found below error message in server log file. As per oracle note Doc ID 1356278.1 this message is only informational and can be safely ignored. But our planning application crashed and we had to restart the services.

####<Sep 8, 2022 11:06:22 AM PDT> <Info> <WorkManager> <SCLWPA2188> <Planning0> <[ACTIVE] ExecuteThread: '29' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1662660382282> <BEA-002936> <maximum thread constraint ASYNC_REP_FLUSH_WM is reached>
####<Sep 8, 2022 11:08:39 AM PDT> <Info> <WorkManager> <SCLWPA2188> <Planning0> <[ACTIVE] ExecuteThread: '31' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1662660519981> <BEA-002936> <maximum thread constraint ASYNC_REP_FLUSH_WM is reached>

In many cases the error may clear after some time. After research I found that WebLogic Server associates some internal work managers whenever a JMS server is created and below are the parameters

weblogic.jms.<JMSServer_NAME>.AsyncPush
Min threads = 1, max threads = unlimited, high fair share

weblogic.jms.<JMSServer_NAME>.Limited
Min threads = 1, max threads = 8, normal fair share.

weblogic.jms.<JMSServer_NAME>.System
Min threads = 1, max threads = unlimited, high fair share.

The work managers will expand up to the max limit and will shrink back when the load returns to normal, however shrinking doesn’t start unless there are few idle state threads.

As you see the work managers have a limited thread count of 8, but this limit is internal and cannot be changed without contacting Oracle support.  

Users may also see STUCK thread issues

Sep 3, 2022 11:23:31 AM EDT> <Error> <WebLogicServer> <abcsystems.orass.com> <server-cc113sp> <[ACTIVE] ExecuteThread: '33' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <45443435326649> <BEA-000337> <[STUCK] ExecuteThread: 45 for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for “550" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 678511 ms

The root cause Thread gets stuck if it is continually working (not idle) for a set period of time. This time is called the StuckThreadMaxTime, and has a default value of 600 seconds. The stuck thread cannot complete its current work or accept any new work the server logs the message. Note that if all threads gets stuck then server health state will change to either “warning” or “critical”

The stuck thread errors may disappear after some time if you see these errors frequently then you should start tuning to avoid issues. Note that restart will clear these errors.

I would recommend you go to WebLogic configuration and finetune your parameters based upon the current settings such as “Stuck thread max time” and “Stuck Thread Timer Interval”

To fine tune the parameter go to weblogic console à Domain à Environment à Server à <Managed Server> à Configuration > Tuning

Make sure you apply changes and reboot the server for new changes to take into effect. You can also refer below WLS documentation and Metalink note for more details.

WLS documentation : https://docs.oracle.com/cd/E23943_01/apirefs.1111/e13952/taskhelp/tuning/TuningExecuteThreads.html
Metalink Note: Note 1302472.1 - Information about STUCK Threads Condition and Tuning in Weblogic Server

Thanks & Regards

http://oracleracexpert.com, Oracle ACE Pro

Smart View timeout error message

If users are working with planning data forms on excel and receiving any of the below timeout error then the issue is with IE Settings.

Cannot connect to provider because: The request timed out. Contact your administrator to increase NetRetryCount and NetRetryInterval

The requeset timed out. Contact your administrator to increase netRetryCount and netRetryInterval.

When trying to save the data then users might see below error

“This form contains unsaved data. Select OK to continue without saving data, or Cancel this operation”

Follow below steps to fix the issue

Step1:- Create an empty text file and copy below contents

Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings]
"KeepAliveTimeout"=dword:00180000
"ReceiveTimeout"=dword:00dbba00
"ServerInfoTimeout"=dword:00180000

Step2: - Rename the file as LogOut.reg” and right click on the file  click on merge to update your registry

Step3:- Restart your computer (optional)

You can manually update/ add registry entries but it may corrupt the registry and this is the recommended approach.

Regards,
Satishbabu Gunukula
http://www.oracleracexpert.com

User Account Control is enabled. Installation may lead to unpredictable results and thus will be stopped

You need to disable User Account Control (UAC) before you install Hyperion Essbase, otherwise you will receive below error.

 User Account Control is enabled. Installation may lead to unpredictable results and thus will be stopped. EPMINS-01001: One or more preinstallation checks failed. Correct the errors before continuing with the installation. Press Enter to exit

You can disable the User access Control (UAC) by following below steps

In case of Windows 2008 Server
1. Start -> Control Panel
2. Click User Accounts in Control Panel
3. In the User Accounts window, click User Accounts
4. Click on Turn User Account Control on or off
5. Uncheck the "Use User Account Control (UAC) to help protect your computer", and then click OK. 6. Restart the computer to apply the changes

In case of Windows7
1. Start -> Control Panel
2. Click User Accounts in Control Panel
3. In the User Accounts window, click User Accounts
4. Click on “Change User Account Control settings”
5. Move the bar to “Never Notify”
6. Restart the computer to apply the changes

Now launch the installTool.cmd to install Hyperion and you should receive below error message.

User Account Control is disabled. EPMINS-01002: All installation prerequisites have been met. Starting EPM System Installer.

If you are running the installation over network share then you will see below error message

EPMINS-01001: One or more preinstallation checks failed. Correct the errors before continuing with the installation. Installer Path check... Failed! Installation from shared folder is not supported, please mount a network drive.

Make sure you map the network drive to local computer and launch the installTool.cmd and you should be able to install successfully.

Regards
Satishbabu Gunukula
http://oracleracexpert.com