RMAN-10035 ORA-19502 ORA-27030

RDBMS Version: 8.1.7.2
Operating System & Version: AIX 4.3.2
Error Number (if applicable):
Product (i.e. SQL*Loader, Import, etc.): RMAN w/ TDP
Product Version:
RMAN + TDP backup failure
Hi,
I have Oracle 8.1.7.3 running on an IBM AIX 4.3.2 OS. TDP has been configured w/ RMAN for complete hotbackups. This
setup was working fine for the last 4 months before I started receiving the following error during backup. The failures occur @ haphazard intervals.
On Nov 22 I got the following error:
RMAN-00571: ==========================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ====================================================
RMAN-03007: retryable error occurred during execution of command: backup
RMAN-07004: unhandled exception during command execution on channel e1
RMAN-10035: exception raised in RPC: ORA-19502: write error on file "/backup/ELASPROD/hbkp_ELASPROD_478596261_1135_1", blockno 14370561 (blocksize=512)
ORA-27030: skgfwrt: sbtwrite2 returned error
ORA-19511: ANS1017E (RC-50) Session rejected: TCP/IP connection failure
RMAN-10031: ORA-19624 occurred during call to DBMS_BACKUP_RESTORE.BACKUPPIECECREATE
On Nov 23 it was
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03007: retryable error occurred during execution of command: backup
RMAN-07004: unhandled exception during command execution on channel e2
RMAN-10035: exception raised in RPC: ORA-19502: write error on file "/backup/ELASPROD/hbkp_ELASPROD_478682643_1139_1", blockno 9867009 (blocksize=512)
ORA-27030: skgfwrt: sbtwrite2 returned error
ORA-19511: ANS1301E (RC1) Server detected system error
RMAN-10031: ORA-19624 occurred during call to DBMS_BACKUP_RESTORE.BACKUPPIECECREATE
On Dec 12 it was
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03007: retryable error occurred during execution of command: backup
RMAN-07004: unhandled exception during command execution on channel e1
RMAN-10035: exception raised in RPC: ORA-19502: write error on file "/backup/ELASPROD/archive/arch_ELASPROD_480413812_1330_1", blockno 1574913 (blocksize=512)
ORA-27030: skgfwrt: sbtwrite2 returned error
ORA-19511: ANS1017E (RC-50) Session rejected: TCP/IP connection failure
RMAN-10031: ORA-19624 occurred during call to DBMS_BACKUP_RESTORE.BACKUPPIECECREATE
On Jan 21 it was
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03007: retryable error occurred during execution of command: backup
RMAN-07004: unhandled exception during command execution on channel e1
RMAN-10035: exception raised in RPC: ORA-19502: write error on file "/backup/ELASPROD/archive/arch_ELASPROD_483781354_1696_1", blockno 1763457 (blocksize=512)
ORA-27030: skgfwrt: sbtwrite2 returned error
ORA-19511: ANS1329S (RC29) Server out of data storage space
RMAN-10031: ORA-19624 occurred during call to DBMS_BACKUP_RESTORE.BACKUPPIECECREATE
I have another database which gets backed up after this 1 is done. i have not faced any such problems w/ the other database.
What should be done?
Thanks in advance.


From: RAMESH VENKATAKRISHNAN 21-Jan-03 18:47
Subject: Re : RMAN + TDP backup failure

Hi

Log Nov 22 and Dec 12 :-
- ORA-19511: ANS1017E (RC-50) Session rejected: TCP/IP connection failure
This might be because of the failure in tcp/ip communications when connecting to the server.
You can try playing with "IDLETIMEOUT" TSM parameter.

Log Jan 21 :-
- ORA-19511: ANS1329S (RC29) Server out of data storage space
This is because your storage pool is full

Log Nov 23 :-
- ORA-19511: ANS1301E (RC1) Server detected system error
This is a generic message. you can get more information from sbtio.log

Ramesh...







From: Oracle, Rowena Serna 23-Jan-03 16:29
Subject: Re : RMAN + TDP backup failure

Thanks to Ramesh for the suggestion, as you can see,
the errors you are seeing are media management errors and you should contact your vendor to address them.


Error: ORA-19511
Text: %s
---------------------------------------------------------------------------
Cause: An error occurred in the media management software which is linked with the Oracle server to perform backup and restore in cooperation with Recovery Manager.
Action: If the text of message 19511 does not provide enough information to resolve the problem, then you should contact the vendor of the media management software.


Regards,
Rowena Serna
Oracle Corporation

From: Jos Doldersum 16-Jun-03 14:58
Subject: Re : RMAN + TDP backup failure

I had to increase the COMMTIMEOUT parameter from 60 to 180 to solve the RC-50 problem.

Starting RMAN backup based on Archive File System Usage

Starting RMAN backup based on Archive File System Usage:
Starting RMAN Backup Based on Archive File System
by Avnish Rastogi

Often when DBAs do export/import to load data from one database to another database, it gerenartes archive logs at very high pace and fills up archive file system in couple of hours. Below script will help DBA in scheduling RMAN backup based on file system usage. Please change the location of logfile, cmdfile and RMAN catalog user/password before using this script.

##########################################################################
# Description: Purpose of this script is to kick off RMAN backup for archive logs when archive file
#
##########################################################################
#!/bin/ksh -x

if [ $# != 1 ]
then
echo 'Usage: feeSpace.ksh %Used' 1>&2
exit 1
fi

FSUsed=$1

#GET LOG ARCHIVE DEST FROM V$PARAMETER

LogArchiveDest=`$ORACLE_HOME/bin/sqlplus -s '/as sysdba' SELECT value FROM v\\$parameter WHERE name='log_archive_dest';
exit;
!`

#Define Command Variables used by RMAN

MSGLOG=/tmp/arch_RMAN.log
CMDFILE=/tmp/arch_RMAN.rcv
RMANCatalog=user/passwd@service.org

set `df -P $LogArchiveDest |grep -v Filesystem`
PercentUsed=`echo $5|cut -f1 -d '%'`
FS=$6

if [ '$PercentUsed' -gt '$FSUsed' ]
then
$ORACLE_HOME/bin/arman target / catalog $RMANCatalog cmdfile $CMDFILE msglog $MSGLOG
fi


Rman Script for kicking off archive backups in Oracle 9I

run {
BACKUP ARCHIVELOG ALL DELETE INPUT
FILESPERSET 6
FORMAT '%d_%D%M%Y.%s';
}

Rman Script for kicking off archive backups in Oracle 8I

run {
allocate channel t1 type 'SBT_TAPE';
#backup all archive logs
sql 'alter system archive log current';
backup
filesperset 5
format 'al_%s_%p_%t'
(archivelog all delete input);
sql 'alter system archive log current';
sql 'alter database backup controlfile to trace';
resync catalog;
}

SQL Server - attach all the datababses

attach all the datababses on a SQL Server:
If you are trying to attach all the files to a simillar location as the current server, you can use the script below. It will print out the script with sp_attach_db statements for all the databases. It doesn't look for secondary files and you can update the script accordingly.
--Script to attach all User databases to original location.
declare curname cursor for select dbid from sysdatabases where dbid > 4
open curname
declare @dbid int
fetch next from curname into @dbid
while @@fetch_status = 0
begin
declare @sql varchar(500)
declare @datafile varchar(100)
declare @logfile varchar(100)
select @datafile=filename from sysaltfiles where dbid = @dbid and groupid = 1
select @logfile=filename from sysaltfiles where dbid = @dbid and groupid = 0
--print @logfile
set @sql = 'sp_attach_db ' db_name(@dbid) ',''' ltrim(rtrim(@datafile)) ''',''' ltrim(rtrim(@logfile)) ''' ;'
print @sql
fetch next from curname into @dbid
end
close curname
deallocate curname