ORA-29548: Java system class reported: release of Java system classes in the database (12.1.0.2.190716 1.6) does not match that of the oracle executable (12.1.0.2.200114 1.6) - USER ( APPS ) has been detected in FND_WEB_SEC.GET_OP_VALUE." then follow below steps.

Issue Details:

We have applied Java VM PSU (July 2019) on our 12c (12.1.0.2) database in 2019.  Now we applied JavaVM PSU (Jan 2020).  After patch one of the user reported their functionality failing with below error.

 "Oracle error -29548: ORA-29548: Java system class reported: release of Java system classes in the database (12.1.0.2.190716 1.6) does not match that of the oracle executable (12.1.0.2.200114 1.6) - USER ( APPS ) has been detected in FND_WEB_SEC.GET_OP_VALUE."


Solution:

It is always better to check below two queries after we apply JavaVM patches.

SQL> select dbms_java.get_jdk_version() from dual;
select dbms_java.get_jdk_version() from dual
                                        *
ERROR at line 1:
ORA-29548: Java system class reported: release of Java system classes in the
database (12.1.0.2.190716 1.6) does not match that of the oracle executable
(12.1.0.2.200114 1.6)

SQL> select dbms_java.longname('TEST') from dual;
select dbms_java.longname('TEST') from dual
                                       *
ERROR at line 1:
ORA-29548: Java system class reported: release of Java system classes in the
database (12.1.0.2.190716 1.6) does not match that of the oracle executable
(12.1.0.2.200114 1.6)
SQL>

 To fix the above error, run the below command as SYS user.

SQL> @?/javavm/install/update_javavm_db.sql
SQL> SET FEEDBACK 1
SQL> SET NUMWIDTH 10
SQL> SET LINESIZE 80
SQL> SET TRIMSPOOL ON
SQL> SET TAB OFF
SQL> SET PAGESIZE 100
SQL>
SQL> alter session set "_ORACLE_SCRIPT"=true;

Session altered.

SQL>
SQL> -- If Java is installed, do CJS.
SQL>
SQL> -- If CJS can deal with the SROs inconsistent with the new JDK,
SQL> -- the drop_sros() call here can be removed.
SQL> call initjvmaux.drop_sros();

Call completed.

SQL>
SQL> create or replace java system;
  2  /

Java created.

SQL>
SQL> update dependency$
  2    set p_timestamp=(select stime from obj$ where obj#=p_obj#)
  3    where (select stime from obj$ where obj#=p_obj#)!=p_timestamp and
  4          (select type# from obj$ where obj#=p_obj#)=29  and
  5          (select owner# from obj$ where obj#=p_obj#)=0;

0 rows updated.

SQL>
SQL> commit;

Commit complete.

SQL>


Now run the above two queries and test the issue still persists or not.

SQL> alter session set "_ORACLE_SCRIPT"=false;

Session altered.

SQL>
SQL> select dbms_java.get_jdk_version() from dual;

DBMS_JAVA.GET_JDK_VERSION()
--------------------------------------------------------------------------------
1.6.0_211

1 row selected.

SQL> select dbms_java.longname('TEST') from dual;

DBMS_JAVA.LONGNAME('TEST')
--------------------------------------------------------------------------------
TEST

1 row selected.

SQL>

datapatch -verbose failed.

Issue Details:

We have applied JAN 2020 PSU patch 30340202 on our 12.1.0.2 database.

As post patch step, we need to run datapatch -versbose, when we run that it failed with below error.


cd $ORACLE_HOME/OPatch
./datapatch -verbose

oracle@mydbserver /ora/db/tech_st/12.1.0.2/OPatch
ORCL-DB: ./datapatch -verbose
SQL Patching tool version 12.1.0.2.0 Production on Wed Feb 19 09:52:12 2020
Copyright (c) 2012, 2016, Oracle.  All rights reserved.

Log file for this invocation: /ora/db/tech_st/12.1.0.2/cfgtoollogs/sqlpatch/sqlpatch_10280_2020_02_19_09_52_12/sqlpatch_invocation.log

Connecting to database...OK
Bootstrapping registry and package to current versions...done

Queryable inventory could not determine the current opatch status.
Execute 'select dbms_sqlpatch.verify_queryable_inventory from dual'
and/or check the invocation log
/ora/db/tech_st/12.1.0.2/cfgtoollogs/sqlpatch/sqlpatch_10280_2020_02_19_09_52_12/sqlpatch_invocation.log
for the complete error.
Prereq check failed, exiting without installing any patches.

Please refer to MOS Note 1609718.1 and/or the invocation log
/ora/db/tech_st/12.1.0.2/cfgtoollogs/sqlpatch/sqlpatch_10280_2020_02_19_09_52_12/sqlpatch_invocation.log
for information on how to resolve the above errors.

SQL Patching tool complete on Wed Feb 19 09:52:40 2020



Upon checking we got the below error in the database.

SQL> select dbms_sqlpatch.verify_queryable_inventory from dual;

VERIFY_QUERYABLE_INVENTORY
--------------------------------------------------------------------------------
ORA-20001: Latest xml inventory is not loaded into table



Solution:

This is due ot java, java is pointing to operating system local java(/usr/bin/java) not to the $ORACLE_HOME/jdk/jre/bin/java.

 oracle@mydbserver /ora/db/tech_st/12.1.0.2/OPatch
ORCL-DB: which java
/usr/bin/java
oracle@mydbserver /ora/db/tech_st/12.1.0.2/OPatch
ORCL-DB: export PATH=$ORACLE_HOME/jdk/jre/bin:$PATH:.
oracle@mydbserver /ora/db/tech_st/12.1.0.2/OPatch
ORCL-DB: which java
/ora/db/tech_st/12.1.0.2/jdk/jre/bin/java
oracle@mydbserver /ora/db/tech_st/12.1.0.2/OPatch


Retested the issue and again it failed with same error.

When we check with Oracle Support team, they suggest to run datapatch verbose as shown below.

datapatch -apply 30340202 -force -verbose -bundle_series DBPSU

Issue fixed !!!

ORA-25153: Temporary Tablespace is Empty has been detected in FND_GSM_UTIL.APPEND_CTX_FRAGMENT.

Issue:

Got the below error while running autoconfig on dbTier.

 Attempting upload of Context file and templates to database...ERROR: InDbCtxFile.uploadCtx() : Exception : Error executng BEGIN fnd_gsm_util.append_ctx_fragment(:1,:2,:3); END;: 1; Oracle error -25153: ORA-25153: Temporary Tablespace is Empty has been detected in FND_GSM_UTIL.APPEND_CTX_FRAGMENT.
oracle.apps.ad.autoconfig.oam.InDbCtxFileException: Error executng BEGIN fnd_gsm_util.append_ctx_fragment(:1,:2,:3); END;: 1; Oracle error -25153: ORA-25153: Temporary Tablespace is Empty has been detected in FND_GSM_UTIL.APPEND_CTX_FRAGMENT.
        at oracle.apps.ad.autoconfig.oam.InDbCtxFile.uploadCtx(InDbCtxFile.java:220)
        at oracle.apps.ad.autoconfig.oam.CtxSynchronizer.uploadToDb(CtxSynchronizer.java:328)
        at oracle.apps.ad.tools.configuration.FileSysDBCtxMerge.updateDBCtx(FileSysDBCtxMerge.java:721)
        at oracle.apps.ad.tools.configuration.FileSysDBCtxMerge.updateDBFiles(FileSysDBCtxMerge.java:226)
        at oracle.apps.ad.context.CtxValueMgt.processCtxFile(CtxValueMgt.java:1690)
        at oracle.apps.ad.context.CtxValueMgt.main(CtxValueMgt.java:763)
FAILED


Solution:

The error itself tells us issue with TEMP tablespace, so just checked files or present or not, found no files.  Added a temp file and re-run the autoconfig on dbTier. 

SQL> select * from v$tempfile;

no rows selected


SQL> alter tablespace TEMP add tempfile '/data/ORCL/tem01.dbf' size 5000m;

Tablespace altered.

SQL> 

AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.

Issue:

Got the below error while running autoconfig on AppTier.

Updating s_tnsmode to 'generateTNS'
UpdateContext exited with status: 0
AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.
Error generating tnsnames.ora from the database, temporary tnsnames.ora will be generated using templates
Instantiating Tools tnsnames.ora
Tools tnsnames.ora instantiated
Web tnsnames.ora instantiated

adgentns.pl exiting with status 2
ERRORCODE = 2 ERRORCODE_END


Background of Environment:

This is a cloned environment from production.

Solution:

All entries in the FND_NODES tables belongs to prod instance, no entry for current non-prod environment. Which means clone steps did not perform completely.  Follow below steps to fix the above issue.

1) Connect to database using apps and do fnd_nodes clean up.
  
  sqlplus apps/<apps_password>
  SQL> exec fnd_conc_clone.setup_clean;

2) Run autoconfig on dbTier
3) Run autoconfig on appsTier

Find Concurrent Request ID from SID

Find Concurrent Request id from SID:

SELECT s.inst_id,a.request_id,s.sid,s.serial#,c.spid
  FROM apps.fnd_concurrent_requests a, gv$process c, gv$session s
   WHERE s.sid in ('&sid')
  AND s.paddr = c.addr
  AND a.oracle_process_id = c.spid
  AND a.phase_code = UPPER ('R');

Output:

   INST_ID REQUEST_ID        SID    SERIAL# SPID
---------- ---------- ---------- ---------- ------------------------
         1   20602689      15647       2673 5617