Adop phase=fs_clone failing with error [UNEXPECTED]Error 1 occurred while Executing txkADOPValidation script on

Issue Details: - 

adop phase=fs_clone failed.

 applmgr@oraebsdemo ~

EBSPOC: adop phase=fs_clone

Enter the APPS password:

Enter the SYSTEM password:

Enter the WLSADMIN password:

Validating credentials.

Warning: WebLogic Admin Server is not running.

Trying to start Admin Server.

You are running adadminsrvctl.sh version 120.10.12020000.11

Starting WLS Admin Server...

Refer /u01/apps/fs1/inst/apps/EBSPOC_oraebsdemo/logs/appl/admin/log/adadminsrvctl.txt for details

AdminServer logs are located at /u01/apps/fs1/FMW_Home/user_projects/domains/EBS_domain_EBSPOC/servers/AdminServer/logs

adadminsrvctl.sh: exiting with status 0

adadminsrvctl.sh: check the logfile /u01/apps/fs1/inst/apps/EBSPOC_oraebsdemo/logs/appl/admin/log/adadminsrvctl.txt for more information ...

Initializing.

    Run Edition context  : /u01/apps/fs1/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml

    Patch edition context: /u01/apps/fs2/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml

    Patch file system free space: 858.38 GB

Validating system setup.

    Node registry is valid.

    [WARNING]   MTCC has not been run. Apply Patch 17537119 and run checkMTpatch.sh.

Checking for existing adop sessions.

    No pending session exists.

    Starting new adop session.

===========================================================================

ADOP (C.Delta.12)

Session ID: 3

Node: oraebsdemo

Phase: fs_clone

Log: /u01/apps/fs_ne/EBSapps/log/adop/3/20211009_180424/adop.log

===========================================================================

Validating configuration on node: [oraebsdemo].

    Log: /u01/apps/fs_ne/EBSapps/log/adop/3/20211009_180424/fs_clone/validate/oraebsdemo

    [UNEXPECTED]Error occurred running "perl /u01/apps/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl  -contextfile=/u01/apps/fs1/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml -patchctxfile=/u01/apps/fs2/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml -phase=fs_clone -logloc=/u01/apps/fs_ne/EBSapps/log/adop/3/20211009_180424/fs_clone/validate/oraebsdemo -promptmsg=hide"

    [UNEXPECTED]Error 1 occurred while Executing txkADOPValidation script on oraebsdemo

[STATEMENT] Please run adopscanlog utility, using the command

"adopscanlog -latest=yes"

to get the list of the log files along with snippet of the error message corresponding to each log file.

adop exiting with status = 1 (Fail)

applmgr@oraebsdemo ~

EBSPOC:

Cause:- 

Run below to verify the patch context file entry, if no rows selected then follow the solution section.

SQL> select distinct(PATH) from FND_OAM_CONTEXT_FILES where NAME not in ('TEMPLATE','METADATA','config.txt') and CTX_TYPE='A' and (status is null or upper(status) in ('S','F')) and EXTRACTVALUE(XMLType(TEXT),'//file_edition_type') = 'patch';

no rows selected

SQL> 

Solution:-

To fix this issue test the following to upload the missing PATCH context file to the database:

1. Get the value of your patch context file.

    1.1. Source your PATCH file system

    1.2. Echo the value of $CONTEXT_FILE and take note of it. You will use this value in step #3.

2. Source the RUN filesystem

3. Execute the following command on the run filesystem to upload the patch context file to the database.

$ADJVAPRG oracle.apps.ad.autoconfig.oam.CtxSynchronizer \

action=upload \

contextfile=/u01/apps/fs2/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml \

logfile=/tmp/patchctxupload.log

Note: the context file parameter should point to the physical location of the patch context file in your system

4. Run the following query and it should return one entry for the recently uploaded context file.

SQL> select distinct(PATH) from FND_OAM_CONTEXT_FILES where NAME not in ('TEMPLATE','METADATA','config.txt') and CTX_TYPE='A' and (status is null or upper(status) in ('S','F')) and EXTRACTVALUE(XMLType(TEXT),'//file_edition_type') = 'patch';

PATH

--------------------------------------------------------------------------

/u01/apps/fs2/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml

-- Rerun the adop phase=fs_clone, it completed successfully.

OPatch failed with error code 73, The Central Inventory is corrupted:

Details of error as follows - 

oracle@orademodb01 ~

ORCL: opatch lsinv

Oracle Interim Patch Installer version 12.2.0.1.21

Copyright (c) 2021, Oracle Corporation.  All rights reserved.

Oracle Home       : /ora/db/tech_st/12.1.0.2

Central Inventory : /home/oracle/oraInventory

   from           : /ora/db/tech_st/12.1.0.2/oraInst.loc

OPatch version    : 12.2.0.1.21

OUI version       : 12.1.0.2.0

Log file location : /ora/db/tech_st/12.1.0.2/cfgtoollogs/opatch/opatch2021-09-28_16-16-01PM_1.log

List of Homes on this system:

  Home name= ORCL_Home, Location= "/ora/db/tech_st/12.1.0.2"

LsInventorySession failed: RawInventory gets null OracleHomeInfo

OPatch failed with error code 73

oracle@orademodb01 ~

Solution:

Turned out my inventory had not been updated with my new Oracle Home. When I looked in the inventory.xml file on my server, there was no entry for the installation there.

To fix this, add the new OH to your inventory. From your OH that is missing, do the following:

oracle@orademodb01 /ora/db/tech_st/12.1.0.2

ORCL: cat oraInst.loc

inventory_loc=/home/oracle/oraInventory

inst_group=oinstall

oracle@orademodb01 /ora/db/tech_st/12.1.0.2

ORCL: cat /etc/oraInst.loc

inventory_loc=/home/oracle/oraInventory

inst_group=oinstall

oracle@orademodb01 /ora/db/tech_st/12.1.0.2

ORCL: cd $ORACLE_HOME/oui/bin

oracle@orademodb01 /ora/db/tech_st/12.1.0.2/oui/bin

ORCL: ./runInstaller -invPtrLoc /ora/db/tech_st/12.1.0.2/oraInst.loc -attachHome ORACLE_HOME=/ora/db/tech_st/12.1.0.2 ORACLE_HOME_NAME="ORCL"

Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 15517 MB    Passed

The inventory pointer is located at /ora/db/tech_st/12.1.0.2/oraInst.loc

'AttachHome' was successful.

oracle@orademodb01 /ora/db/tech_st/12.1.0.2/oui/bin

EBS 12.2.10 Upgrade (Patch 30399999) error

 While applying EBS 12.2.10 patch 30399999 got the below error.

ATTENTION: All workers either have failed or are waiting:

FAILED: file fnd_web_resource.xdf on worker 1.

FAILED: file fnd_web_product_family.xdf on worker 4.

FAILED: file fnd_web_used_resource.xdf on worker 5.

FAILED: file fnd_web_product_family_tl.xdf on worker 7.

FAILED: file fnd_web_application.xdf on worker 8.

Fix:

Login to sqlplus as apps user and execute below :

exec AD_ZD_SEED.UPGRADE('FND_WEB_PRODUCT_FAMILY');

exec AD_ZD_SEED.UPGRADE('FND_WEB_APPLICATION');

exec AD_ZD_SEED.UPGRADE('FND_WEB_RESOURCE');

exec AD_ZD_SEED.UPGRADE('FND_WEB_PRODUCT_FAMILY_TL');

exec AD_ZD_SEED.UPGRADE('FND_WEB_USED_RESOURCE');

Once you applied the fix, restart all failed workers and the patch got completed successfully. 

How to check online patching is enabled from backend ?

 If you wanted to check online patching is enabled or not from backend, run the below query as apps user.

sqlplus apps/apps

SQL> select decode(editions_enabled, 'Y','Yes','No') "Online Patching Enabled?"

from fnd_oracle_userid au, dba_users du

where au.oracle_username = du.username and au.read_only_flag = 'U';  

Onl

---

Yes

DB Startup issue after memory change in the server level

Faced an issue after changing the DB memory parameters (SGA and PGA) and got the below error while startup.

ORA-27106: System Pages Not Available To Allocate Memory

Issue: Found that the "USE_LARGE_PAGES" setting to "ONLY"

Fix: Parameter USE_LARGE_PAGES set to "TRUE" and started database, DB came up.