Adop phase=fs_clone failing with error [UNEXPECTED]Error 1 occurred while Executing txkADOPValidation script on

Issue Details: - 

adop phase=fs_clone failed.

 applmgr@oraebsdemo ~

EBSPOC: adop phase=fs_clone

Enter the APPS password:

Enter the SYSTEM password:

Enter the WLSADMIN password:

Validating credentials.

Warning: WebLogic Admin Server is not running.

Trying to start Admin Server.

You are running adadminsrvctl.sh version 120.10.12020000.11

Starting WLS Admin Server...

Refer /u01/apps/fs1/inst/apps/EBSPOC_oraebsdemo/logs/appl/admin/log/adadminsrvctl.txt for details

AdminServer logs are located at /u01/apps/fs1/FMW_Home/user_projects/domains/EBS_domain_EBSPOC/servers/AdminServer/logs

adadminsrvctl.sh: exiting with status 0

adadminsrvctl.sh: check the logfile /u01/apps/fs1/inst/apps/EBSPOC_oraebsdemo/logs/appl/admin/log/adadminsrvctl.txt for more information ...

Initializing.

    Run Edition context  : /u01/apps/fs1/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml

    Patch edition context: /u01/apps/fs2/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml

    Patch file system free space: 858.38 GB

Validating system setup.

    Node registry is valid.

    [WARNING]   MTCC has not been run. Apply Patch 17537119 and run checkMTpatch.sh.

Checking for existing adop sessions.

    No pending session exists.

    Starting new adop session.

===========================================================================

ADOP (C.Delta.12)

Session ID: 3

Node: oraebsdemo

Phase: fs_clone

Log: /u01/apps/fs_ne/EBSapps/log/adop/3/20211009_180424/adop.log

===========================================================================

Validating configuration on node: [oraebsdemo].

    Log: /u01/apps/fs_ne/EBSapps/log/adop/3/20211009_180424/fs_clone/validate/oraebsdemo

    [UNEXPECTED]Error occurred running "perl /u01/apps/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl  -contextfile=/u01/apps/fs1/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml -patchctxfile=/u01/apps/fs2/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml -phase=fs_clone -logloc=/u01/apps/fs_ne/EBSapps/log/adop/3/20211009_180424/fs_clone/validate/oraebsdemo -promptmsg=hide"

    [UNEXPECTED]Error 1 occurred while Executing txkADOPValidation script on oraebsdemo

[STATEMENT] Please run adopscanlog utility, using the command

"adopscanlog -latest=yes"

to get the list of the log files along with snippet of the error message corresponding to each log file.

adop exiting with status = 1 (Fail)

applmgr@oraebsdemo ~

EBSPOC:

Cause:- 

Run below to verify the patch context file entry, if no rows selected then follow the solution section.

SQL> select distinct(PATH) from FND_OAM_CONTEXT_FILES where NAME not in ('TEMPLATE','METADATA','config.txt') and CTX_TYPE='A' and (status is null or upper(status) in ('S','F')) and EXTRACTVALUE(XMLType(TEXT),'//file_edition_type') = 'patch';

no rows selected

SQL> 

Solution:-

To fix this issue test the following to upload the missing PATCH context file to the database:

1. Get the value of your patch context file.

    1.1. Source your PATCH file system

    1.2. Echo the value of $CONTEXT_FILE and take note of it. You will use this value in step #3.

2. Source the RUN filesystem

3. Execute the following command on the run filesystem to upload the patch context file to the database.

$ADJVAPRG oracle.apps.ad.autoconfig.oam.CtxSynchronizer \

action=upload \

contextfile=/u01/apps/fs2/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml \

logfile=/tmp/patchctxupload.log

Note: the context file parameter should point to the physical location of the patch context file in your system

4. Run the following query and it should return one entry for the recently uploaded context file.

SQL> select distinct(PATH) from FND_OAM_CONTEXT_FILES where NAME not in ('TEMPLATE','METADATA','config.txt') and CTX_TYPE='A' and (status is null or upper(status) in ('S','F')) and EXTRACTVALUE(XMLType(TEXT),'//file_edition_type') = 'patch';

PATH

--------------------------------------------------------------------------

/u01/apps/fs2/inst/apps/EBSPOC_oraebsdemo/appl/admin/EBSPOC_oraebsdemo.xml

-- Rerun the adop phase=fs_clone, it completed successfully.

EBS 12.2.10 Upgrade (Patch 30399999) error

 While applying EBS 12.2.10 patch 30399999 got the below error.

ATTENTION: All workers either have failed or are waiting:

FAILED: file fnd_web_resource.xdf on worker 1.

FAILED: file fnd_web_product_family.xdf on worker 4.

FAILED: file fnd_web_used_resource.xdf on worker 5.

FAILED: file fnd_web_product_family_tl.xdf on worker 7.

FAILED: file fnd_web_application.xdf on worker 8.

Fix:

Login to sqlplus as apps user and execute below :

exec AD_ZD_SEED.UPGRADE('FND_WEB_PRODUCT_FAMILY');

exec AD_ZD_SEED.UPGRADE('FND_WEB_APPLICATION');

exec AD_ZD_SEED.UPGRADE('FND_WEB_RESOURCE');

exec AD_ZD_SEED.UPGRADE('FND_WEB_PRODUCT_FAMILY_TL');

exec AD_ZD_SEED.UPGRADE('FND_WEB_USED_RESOURCE');

Once you applied the fix, restart all failed workers and the patch got completed successfully. 

How to check online patching is enabled from backend ?

 If you wanted to check online patching is enabled or not from backend, run the below query as apps user.

sqlplus apps/apps

SQL> select decode(editions_enabled, 'Y','Yes','No') "Online Patching Enabled?"

from fnd_oracle_userid au, dba_users du

where au.oracle_username = du.username and au.read_only_flag = 'U';  

Onl

---

Yes

How to check who logged in Oracle Apps(EBS) Front End ?

 It is very common for DBA to check who logged into EBS application in order to take application down for some maintenance ... etc and make sure the system is idle.

Here is the query to see who is there on the system from backend, connect to database as APPS user and run the query.

I ran this query to see who logged in last one day, based on your requirement you can change it for hours as well.

 col user_name for a40

 select USER_NAME, to_char(LAST_LOGON_DATE, 'DD-MON-YYYY HH24:MI:SS') from fnd_user where last_logon_date > sysdate -1 order by USER_NAME;

USER_NAME        TO_CHAR(LAST_LOGON_DATE,'D

-----------      --------------------------

ABC.DEF          21-JUN-2021 09:48:28

GHI.JKL          21-JUN-2021 08:51:31

MNO.PQR          21-JUN-2021 10:18:14

Workflow Override Email Address

It is very common requirement to stop workflow notifications going out of the system, particularly from Development instances and Test instances. Oracle have given a functionality “workflow override email address” to achieve this requirement.

In many companies this task is done by Database Administrators (DBA) but incase you need to do it yourself you follow the below steps.  There are two ways to setup this: 

1. Login to Oracle Apps preferably with a user who is "Workflow System Administrator".

Navigate to “Workflow Administrator Web Applications” Responsibility
Expand "Oracle Applications Manager"

Click "Workflow Manager"

Click "Service Components"

Click “Workflow Notification Mailer”

Click "Set Override Address"

Once you provide the email address you will receive a verification code for the first time which you need to enter it back in the above page to complete the process. Verification field will be enabled after you provide the email address.
2.  From backend, connect to the database using apps user and run below update statement.

update fnd_svc_comp_param_valsset    parameter_value = '<email address>'where  parameter_value = 'NONE'and    parameter_id =( select parameter_idfrom   fnd_svc_comp_params_tlwhere  display_name = 'Test Address');
commit;

OTA – not up and Running in 12.1.3

 

We were in the process of creating Purchase Orders from iProc Requisitions and to send the Auto-approved Purchase Orders through OSN. We created OSN accounts and then realized that the PO’s didn’t hit the OSN. We did check the Work-flow agents and listeners were Enabled and running fine. As part of the trouble-shooting the issue, we found that the OTA (OXTA) was not up and running.

SQL –

select machine,action, decode(count(*),0,’Error: OTA is Not Running’,’OTA is Running’)
from gv$session
where action like ‘%OXTA%’
group by machine, action;

The above query returned no rows.

Reason –

1. ASADMIN password didn’t either match in the system-jazn-data.xml ($INST_TOP/ora/10.1.3/j2ee/oafm/config/system-jazn-data.xml) as of ASADMIN set through the User Management
2. If there was an Upgrade the passwords got changed

Fix –

1. Check and confirm that dbc file specified in $INST_TOP/ora/10.1.3/j2ee/oafm/application-deployments/oafm/data-sources.xml file exists and is correct.
2.  Confirm that the ASADMIN user setup is correct as below

a. Set profile Applications SSO Login Types  (APPS_SSO_LOCAL_LOGIN) for ASADMIN user to ‘Local’.

b. Change the ASADMIN password from User Management as follows (note: in this example password is set to welcome)

– Log onto Oracle E-Business Suite using sysadmin/<sysadmin_pw>.
– Select the User Management responsibility in the Navigator.
– Click the Users link from the navigation menu to open the User Maintenance window.
– Locate ‘ASADMIN’ user by entering information in the search area to retrieve the ‘ASADMIN’ user.
– Click the Update icon next to the ASADMIN user to open the Update User window.
– Remove the Active to date field and click Apply in case is not already removed
– Click the Reset Password icon next to the ASADMIN user to open the Reset Password window.
– Enter new password twice and click Submit

c. Confirm the definition for ASADMIN user in $INST_TOP/ora/10.1.3/j2ee/oafm/application-deployments/oafm/data-sources.xml is defined correctly:

user=”ASADMIN”
password=”->ASADMIN”

Note: The password above must be set with exact value as above because ->ASADMIN is an indirection, it indicates AS 10.1.0.3 that the password of ASADMIN must be taken from system-jazn-data.xml

d. Change password for ASADMIN user in $ORA_CONFIG_HOME/10.1.3/j2ee/oafm/config/system-jazn-data.xml to be !(! in front is mandatory)

Mention: ! in front of password is mandatory and this sign means that password will be encrypted

e. Shutdown and restart the OAFM container and re-test:

sh $ADMIN_SCRIPTS_HOME/adoafmctl.sh stop

sh $ADMIN_SCRIPTS_HOME/adoafmctl.sh start

References –

1. NOTE: 1O87499.1 – Cannot Start OXTA After Upgrade To R12.1.x
2. NOTE:418926.1 – How to Configure OTA For XML Gateway in Release 12
3. NOTE:419839.1 – How to enable Apache, OC4J and OPMN logging in Oracle Applications R12

Checking Status of all Concurrent Managers from backend.

 There are two ways you can check the concurrent managers running in Oracle apps server.

1. From Oracle Apps Front End :-

Below is the navigation to check currently running concurrent managers in the server:

System Administrator >> Concurrent : Manager >> Administer.

2. Query to get the info of all the concurrent managers running in the Oracle apps DB:-

SELECT DECODE (
          CONCURRENT_QUEUE_NAME,
          'FNDICM', 'Internal Manager',
          'FNDCRM', 'Conflict Resolution Manager',
          'AMSDMIN', 'Marketing Data Mining Manager',
          'C_AQCT_SVC', 'C AQCART Service',
          'FFTM', 'FastFormula Transaction Manager',
          'FNDCPOPP', 'Output Post Processor',
          'FNDSCH', 'Scheduler/Prereleaser Manager',
          'FNDSM_AQHERP', 'Service Manager: AQHERP',
          'FTE_TXN_MANAGER', 'Transportation Manager',
          'IEU_SH_CS', 'Session History Cleanup',
          'IEU_WL_CS', 'UWQ Worklist Items Release for Crashed session',
          'INVMGR', 'Inventory Manager',
          'INVTMRPM', 'INV Remote Procedure Manager',
          'OAMCOLMGR', 'OAM Metrics Collection Manager',
          'PASMGR', 'PA Streamline Manager',
          'PODAMGR', 'PO Document Approval Manager',
          'RCVOLTM', 'Receiving Transaction Manager',
          'STANDARD', 'Standard Manager',
          'WFALSNRSVC', 'Workflow Agent Listener Service',
          'WFMLRSVC', 'Workflow Mailer Service',
          'WFWSSVC', 'Workflow Document Web Services Service',
          'WMSTAMGR', 'WMS Task Archiving Manager',
          'XDP_APPL_SVC', 'SFM Application Monitoring Service',
          'XDP_CTRL_SVC', 'SFM Controller Service',
          'XDP_Q_EVENT_SVC', 'SFM Event Manager Queue Service',
          'XDP_Q_FA_SVC', 'SFM Fulfillment Actions Queue Service',
          'XDP_Q_FE_READY_SVC', 'SFM Fulfillment Element Ready Queue Service',
          'XDP_Q_IN_MSG_SVC', 'SFM Inbound Messages Queue Service',
          'XDP_Q_ORDER_SVC', 'SFM Order Queue Service',
          'XDP_Q_TIMER_SVC', 'SFM Timer Queue Service',
          'XDP_Q_WI_SVC', 'SFM Work Item Queue Service',
          'XDP_SMIT_SVC', 'SFM SM Interface Test Service')
          AS "Concurrent Manager's Name",
       max_processes AS "TARGET Processes",
       running_processes AS "ACTUAL Processes"
  FROM apps.fnd_concurrent_queues
 WHERE CONCURRENT_QUEUE_NAME IN
          ('FNDICM',
           'FNDCRM',
           'AMSDMIN',
           'C_AQCT_SVC',
           'FFTM',
           'FNDCPOPP',
           'FNDSCH',
           'FNDSM_AQHERP',
           'FTE_TXN_MANAGER',
           'IEU_SH_CS',
           'IEU_WL_CS',
           'INVMGR',
           'INVTMRPM',
           'OAMCOLMGR',
           'PASMGR',
           'PODAMGR',
           'RCVOLTM',
           'STANDARD',
           'WFALSNRSVC',
           'WFMLRSVC',
           'WFWSSVC',
           'WMSTAMGR',
           'XDP_APPL_SVC',
           'XDP_CTRL_SVC',
           'XDP_Q_EVENT_SVC',
           'XDP_Q_FA_SVC',
           'XDP_Q_FE_READY_SVC',
           'XDP_Q_IN_MSG_SVC',
           'XDP_Q_ORDER_SVC',
           'XDP_Q_TIMER_SVC',
           'XDP_Q_WI_SVC',
           'XDP_SMIT_SVC');

Query to find Concurrent Requests submitted by a particular user

Query to find Concurrent Requests submitted by a particular User:

SELECT
    request_id,
    request_date,
    user_concurrent_program_name,
    responsibility_name,   
    argument_text,   
    phase_code,
    status_code,
    logfile_name,
    outfile_name,
    output_file_type
FROM
    fnd_concurrent_requests fcr,
    fnd_concurrent_programs_tl fcp,
    fnd_responsibility_tl fr,
    fnd_user fu
WHERE
    fcr.CONCURRENT_PROGRAM_ID = fcp.concurrent_program_id
    and fcr.responsibility_id = fr.responsibility_id
    and fcr.requested_by = fu.user_id
    and user_name = upper('&USERNAME')
ORDER BY REQUEST_DATE DESC;

How to check product patch level or code level oracle apps?

To obtain the current patchset levels for each specific E-Business Suite functional product use the following steps:

1. Log into OAM.

(Responsibility "Oracle Applications Manager", menu "OAM Support Cart")
and Navigate:

2. Support Cart --> Applications Signature --> Collect --> Check "Product Information" box
--> Click on "View" (eyeglasses)

This will display the following patchset information:

Application Name
Current Patch Level
Product Version
Status (Installed, Shared Product, Inactive)

3. The patchset level information can be retrieved directly from sqlplus with the following diagnostic script:

$AD_TOP/sql/adutconf.sql

4. Use the script below to find patchset level.

SELECT app_short_name, MAX(patch_level)

FROM apps.ad_patch_driver_minipks

GROUP BY app_short_name ;

Followed the below Metalink Document:

E-Business Suite Applications Manager Steps To Get The Patchset Level of Each Specific Oracle Applications Functional Product in R12 (Doc ID 550654.1)

How to open a .JNLP file (Launch forms using Java Web Start) ?

Assumptions:
1. Java is already installed on your machine.
2. Your EBS form sessions are JWS enabled.

How to open a JNLP file:

1. Login to E-Business Suite.
2. Open a form.
3. It downloads frmservlet.jnlp file into your downloads folder.
4. Right-click on the file and choose Open with then click Choose another app.
5. If you see Java (TM) Web Start Launcher in this list (you may need to click More apps to expand the list), select it and check the box that says Always use this app to open .jnlp files then click the OK button.
6. If Java (TM) Web Start Launcher is not in the list, check the box that says Always use this app to open .jnlp files then click Look for another app on this PC.
    Navigate to the following location:
    C:\Program Files (x86)\Java\jreXXXX\bin
    XXXX represents a number that will vary based on the version of Java installed on the computer, for example: jre1.8.0_181.
    Select the file named javaws.exe and click Open.
7. Going forward, all .jnlp files will open using Java Web Start.

adoacorectl.sh start, failed with 'ERROR : Timed out( 100000 ): Interrupted Exception'

Issue Details:

$INST_TOP/admin/scripts/adoacorectl.sh start
Timeout specified in context file: 100 second(s)


ERROR : Timed out( 100000 ): Interrupted Exception
You are running adoacorectl.sh version 120.13


--> Upon checking logs, found the issue.

Logs:
$LOG_HOME/appl/admin/log/adoacorectl.txt
$LOG_HOME/ora/10.1.3/opmn/default_group~oacore~default_group~1.log


Issue:
Java HotSpot(TM) Server VM warning: CodeCache is full. Compiler has been disabled.
Java HotSpot(TM) Server VM warning: Try increasing the code cache size using -XX:ReservedCodeCacheSize=
Code Cache  [0xf3e00000, 0xf6e00000, 0xf6e00000)
total_blobs=6685 nmethods=6295 adapters=331 free_code_cache=1843Kb


Solution:

Java HotSpot(TM) 64-Bit Server VM Warning: CodeCache Is Full (Doc ID 2579830.1)

Reboot the server on which OVM Manager is installed.

This activity will not have any effect on the running VMs.

FRM-92095: Oracle JInitiator version is too low. Please install version 1.1.8.2 or higher.

Issue Details:

When ever you launch forms, it may end up with below error.

FRM-92095: Oracle JInitiator version is too low.  Please install version 1.1.8.2 or higher.

Solution:

Window 7 Users:

(Windows Key > Right click on My Computer > Properties > Advanced System Settings > Environment Variables > Add either a User or System attribute > Click on the New button > Enter "JAVA_TOOL_OPTIONS" under Parameter Name and -Djava.vendor="Sun Microsystems Inc." under Parameter Values)

IE issue when launching Java forms for EBS R12.1

Issue Details:

We installed EBS 12.1 and when we tried to open forms in IE11, it threw an error message "In order to access this application, you must install the J2SE Plugin version 1.5.0_13. To install this plugin, click here to download the oaj2se.exe executable. Once the download is complete, double-click the oaj2se.exe file to install the plugin. You will be prompted to restart your browser when the installation ".

Solution:

The steps we performed to resolve the issue are:

1. Go to tools>compatibility settings>and add the domain. My domain was oracle.com so i added it.

2. Tools>internet options>security>internet>custom level>disable xss filter

3. Under internet options>security>Local Intranet>sites>advance option>added the site http://myebs.myappserver.com

4. internet options>security>Local Intranet>custom level>disable xss filter

5. The final step was to apply the RPM-openmotif21-2.1.30-11.EL5.i386.rpm, Shut down ebs services including database and install the RPM. Once installed, start the services and test it.

ORA-25153: Temporary Tablespace is Empty has been detected in FND_GSM_UTIL.APPEND_CTX_FRAGMENT.

Issue:

Got the below error while running autoconfig on dbTier.

 Attempting upload of Context file and templates to database...ERROR: InDbCtxFile.uploadCtx() : Exception : Error executng BEGIN fnd_gsm_util.append_ctx_fragment(:1,:2,:3); END;: 1; Oracle error -25153: ORA-25153: Temporary Tablespace is Empty has been detected in FND_GSM_UTIL.APPEND_CTX_FRAGMENT.
oracle.apps.ad.autoconfig.oam.InDbCtxFileException: Error executng BEGIN fnd_gsm_util.append_ctx_fragment(:1,:2,:3); END;: 1; Oracle error -25153: ORA-25153: Temporary Tablespace is Empty has been detected in FND_GSM_UTIL.APPEND_CTX_FRAGMENT.
        at oracle.apps.ad.autoconfig.oam.InDbCtxFile.uploadCtx(InDbCtxFile.java:220)
        at oracle.apps.ad.autoconfig.oam.CtxSynchronizer.uploadToDb(CtxSynchronizer.java:328)
        at oracle.apps.ad.tools.configuration.FileSysDBCtxMerge.updateDBCtx(FileSysDBCtxMerge.java:721)
        at oracle.apps.ad.tools.configuration.FileSysDBCtxMerge.updateDBFiles(FileSysDBCtxMerge.java:226)
        at oracle.apps.ad.context.CtxValueMgt.processCtxFile(CtxValueMgt.java:1690)
        at oracle.apps.ad.context.CtxValueMgt.main(CtxValueMgt.java:763)
FAILED


Solution:

The error itself tells us issue with TEMP tablespace, so just checked files or present or not, found no files.  Added a temp file and re-run the autoconfig on dbTier. 

SQL> select * from v$tempfile;

no rows selected


SQL> alter tablespace TEMP add tempfile '/data/ORCL/tem01.dbf' size 5000m;

Tablespace altered.

SQL> 

AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.

Issue:

Got the below error while running autoconfig on AppTier.

Updating s_tnsmode to 'generateTNS'
UpdateContext exited with status: 0
AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.
Error generating tnsnames.ora from the database, temporary tnsnames.ora will be generated using templates
Instantiating Tools tnsnames.ora
Tools tnsnames.ora instantiated
Web tnsnames.ora instantiated

adgentns.pl exiting with status 2
ERRORCODE = 2 ERRORCODE_END


Background of Environment:

This is a cloned environment from production.

Solution:

All entries in the FND_NODES tables belongs to prod instance, no entry for current non-prod environment. Which means clone steps did not perform completely.  Follow below steps to fix the above issue.

1) Connect to database using apps and do fnd_nodes clean up.
  
  sqlplus apps/<apps_password>
  SQL> exec fnd_conc_clone.setup_clean;

2) Run autoconfig on dbTier
3) Run autoconfig on appsTier

Find Concurrent Request ID from SID

Find Concurrent Request id from SID:

SELECT s.inst_id,a.request_id,s.sid,s.serial#,c.spid
  FROM apps.fnd_concurrent_requests a, gv$process c, gv$session s
   WHERE s.sid in ('&sid')
  AND s.paddr = c.addr
  AND a.oracle_process_id = c.spid
  AND a.phase_code = UPPER ('R');

Output:

   INST_ID REQUEST_ID        SID    SERIAL# SPID
---------- ---------- ---------- ---------- ------------------------
         1   20602689      15647       2673 5617

Oracle Apps R12.1.3 iHelp Page Prompting for Credentials.

Issue Details:-

EBS iHelp page issue, when user clicks on help page, it is prompting for credentials, which not a normal behaviour.

Once we pass the credentials, it suppose to take that to help page, but it is going to users home page.

Environment Details:

We have one dbTier and two appTiers and login is load balanced url.

Solution:

We reviewd below metalink document, but that didn't help us.

R12: iHelp Not Working After Implementing A Hardware Load Balancer, SSL Acceleration, Or DMZ (Doc ID 728463.1)

Worked with Oracle Support, and they found that the issue occurred due to hardware load balancer implementation.  Oracle recommended to keep NULL for HELP_WEB_AGENT in the $CONTEXT_FILE and run autoconfig.

<HELP_WEB_AGENT oa_var="s_help_web_agent">https://myerp.domain.com</HELP_WEB_AGENT>

To

<HELP_WEB_AGENT oa_var="s_help_web_agent"></HELP_WEB_AGENT>

Run Autoconfig.

This fixed our issue and able to launch help page without prompting for username and password.

Concurrent Program failed with ORACLE error 20100 in FDPSTP

FND_FILE creation failed.

In the concurrent program log file, we see below message.

ORACLE error 20100 in FDPSTP 
 Cause: FDPSTP failed due to ORA-20100: ORA-20100: File o2710130.tmp creation failed. File could not be opened or operated on as requested. 
 Action: Make sure the directory - /applptmp - is a valid directory with write permiss 


Checks for this error:

Option - 1:  Check your applptmp variable

applmgr@oracledemo ~
EBS: echo $APPLPTMP
/applptmp
applmgr@oracledemo ~
EBS:

Option - 2: Go to the applptmp directory and try to create or modify a file with applmgr.

applmgr@oracledemo ~
EBS: touch file
applmgr@oracledemo ~
EBS:

I'm able to create a file, so no issue with this.

Option - 3: Check utl_file_dir parameter in the database.

SQL> show parameter utl_file_dir

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
utl_file_dir                         string      /applptmp

This variable is pointing to right directory.


Option - 4: Create a test file using FND_FILE.

SQL> exec FND_FILE.PUT_LINE(FND_FILE.LOG, 'THIS IS A TEST');
BEGIN FND_FILE.PUT_LINE(FND_FILE.LOG, 'THIS IS A TEST'); END;

*
ERROR at line 1:
ORA-20100: ORA-20100: File o2711023.tmp creation failed.
File could not be opened or operated on as requested.
Action: Make sure the directory - /applptmp - is a valid directory with write
permissions and is accessible from the database server node for write
operations. This
ORA-06512: at "APPS.FND_FILE", line 319
ORA-06512: at "APPS.FND_FILE", line 364
ORA-06512: at "APPS.FND_FILE", line 421
ORA-06512: at line 1

This has failed and it written the error message clearly, /applptmp is a valid directory with write permissions and is accessible from the database server node for write.

About our environment, we have database on one server and application on another server.  On verification, somehow /applptmp is mounted on AppTier but not on DBTier, hence concurrent requests are failing.  Once we mounted the /applptmp on DBTier all requests got completed successfully.

SQL>exec FND_FILE.PUT_LINE(FND_FILE.LOG, 'THIS IS A TEST');

PL/SQL procedure successfully completed.

Find different components (Apache, Java, JRE, Forms...etc) version for Oracle E-Business Suite R12

Use below commands to find different components version in Oracle Apps R12.

1) Apache Version:

$IAS_ORACLE_HOME/Apache/Apache/bin/httpd -v

2) Java Version:

sh -c "`awk -F= '$1 ~ /^JSERVJAVA.*$/ {print $2}' $ADMIN_SCRIPTS_HOME/java.sh` -version;"

3) Client JRE Version:

cat $FORMS_WEB_CONFIG_FILE|grep sun_plugin_version| cut -c 1-35
(or)
grep plugin $CONTEXT_FILE

4) Forms Version:

$ORACLE_HOME/bin/frmcmp_batch|grep Forms| grep Version

5) Perl Version:

$IAS_ORACLE_HOME/perl/bin/perl -v|grep built

6) PL/SQL Version:

$ORACLE_HOME/bin/frmcmp_batch|grep PL/SQL|grep Version

Cancel Concurrent Request From Backend

Run this as 'APPS' user:

1) Concurrent Request Status:

select status_code,phase_code from fnd_concurrent_requests where  request_id=&1;  

2) Cancel Concurrent Request:

update fnd_concurrent_requests set status_code='D', phase_code='C' where request_id=&1; 
commit; 

3)  Update Concurrent request as 'Complete Normal'

update fnd_concurrent_requests set status_code='C',phase_code='C' where request_id=&1;

commit;

4)  Cancel All Pending Concurrent Requests

Updating base tables directly is not supported or recommended by Oracle – this update would be OK to execute in a non-production instance. In a production instance, use APIs or cancel the pending requests through the forms interface.

update FND_CONCURRENT_REQUESTS set phase_code='C',status_code='D' where phase_code='P' ;

commit;