环境:
Oracle 10.2.0.4, Red Hat Linux4.5 32bit
3.8妇女节这天,一台DB server因硬件故障挂掉。固通过备份恢复后的DB到新server上,恢复的DB开启没用多久硬盘就满了,检查DB bdump目录写了大量trc文件,且open后alert.log不断的报错:
Fri Mar 9 04:19:59 2018
dispatcher 'D000' encountered error getting listening address
Fri Mar 9 04:19:59 2018
Errors in file /u01/product/admin/umlaut/bdump/umlaut_ora_20942.trc:
ORA-07445: exception encountered: core dump [kslgetl()+80] [SIGSEGV] [Address not mapped to object] [0x130] [] []
ORA-00108: failed to set up dispatcher to accept connection asynchronously
Fri Mar 9 04:20:02 2018
found dead dispatcher 'D000', pid = (13, 219)
Fri Mar 9 04:20:02 2018
dispatcher 'D000' encountered error getting listening address
Fri Mar 9 04:20:02 2018
Errors in file /u01/product/admin/umlaut/bdump/umlaut_ora_20944.trc:
ORA-07445: exception encountered: core dump [kslgetl()+80] [SIGSEGV] [Address not mapped to object] [0x130] [] []
ORA-00108: failed to set up dispatcher to accept connection asynchronously
....
trc文件内容:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /u01/product/oracle
System name: Linux
Node name: dthistory02
Release: 2.6.9-55.ELsmp
Version: #1 SMP Fri Apr 20 17:03:35 EDT 2007
Machine: i686
Instance name: umlaut
Redo thread mounted by this instance: 1
Oracle process number: 13
Unix process pid: 18998, image: oracle@dthistory02 (D000)
Warning: keltnfy call to ldmInit failed with error 46
*** 2018-03-09 03:42:51.581
network error encountered getting listening address:
NS Primary Error: TNS-12533: TNS:illegal ADDRESS parameters
NS Secondary Error: TNS-12560: TNS:protocol adapter error
NT Generic Error: TNS-00503: Illegal ADDRESS parameters
OPIRIP: Uncaught error 108. Error stack:
ORA-00108: failed to set up dispatcher to accept connection asynchronously
Exception signal: 11 (SIGSEGV), code: 1 (Address not mapped to object), addr: 0x130, PC: [0x831f388, kslgetl()+80]
Registers:
%eax: 0x00000000 %ebx: 0x00000007 %ecx: 0x00001768
%edx: 0x00003980 %edi: 0x20098a44 %esi: 0x00000000
%esp: 0xbffff188 %ebp: 0xbffff194 %eip: 0x0831f388
%efl: 0x00010246
kslgetl()+57 (0x831f371) jnz 0x831f5a2
kslgetl()+63 (0x831f377) mov 0xce77488,%esi
kslgetl()+69 (0x831f37d) mov 0xc(%edi),%ebx
kslgetl()+72 (0x831f380) movzw 0xc1e0d20(,%ebx,2),%edx
> kslgetl()+80 (0x831f388) test %edx,0x130(%esi)
kslgetl()+86 (0x831f38e) lea 0x9c(%esi),%eax
kslgetl()+92 (0x831f394) jnz 0x831f58c
kslgetl()+98 (0x831f39a) test %ebx,%ebx
kslgetl()+100 (0x831f39c) jl 0x831f4e7
*** 2018-03-09 03:42:51.583
ksedmp: internal or fatal error
ORA-07445: exception encountered: core dump [kslgetl()+80] [SIGSEGV] [Address not mapped to object] [0x130] [] []
ORA-00108: failed to set up dispatcher to accept connection asynchronously
Current SQL information unavailable - no session.
查到MOS文章: ORA-07445: [kslgetl()+80] Followed by ORA-108: failed to set up dispatcher to accept connection asynchronously(ID 1298804.1)和报错基本一致,发现此问题主要和hostname及hosts设置相关。
检查配置:
# hostname
dthistory02
# cat /etc/hosts
10.10.10.191 dt history02 localhost
果然 hostname和hosts中不一致,修改/etc/hosts后再无报错(不需要重启DB),至此问题解决。
附:文档 ID 1298804.1
ORA-07445: [kslgetl()+80] Followed by ORA-108: failed to set up dispatcher to accept connection asynchronously
APPLIES TO:
Oracle Net Services - Version 11.1.0.7 and later
Information in this document applies to any platform.
SYMPTOMS
The following errors are seen in the trace file written by an ORA-7445 [kslgetl]:
network error encountered getting listening address:
NS Primary Error: TNS-12533: TNS:illegal ADDRESS parameters
NS Secondary Error: TNS-12560: TNS:protocol adapter error
NT Generic Error: TNS-00503: Illegal ADDRESS parameters
OPIRIP: Uncaught error 108. Error stack:
ORA-00108: failed to set up dispatcher to accept connection asynchronously
Exception signal: 11 (SIGSEGV), code: 1 (Address not mapped to object), addr: 0x130, PC: [0x82f09dc, kslgetl()+80]
The trace file indicates that there is no session:
Current SQL information unavailable - no session.
The Call Stack Trace in the ORA-7445 trace file contains a function list similar to:
kslgetl <- PGOSF57_ksfglt
<- kghfre <- kmnsbf <- nsbfr <- nsiofrrg <- nsiocancel
<- nsopen_shutitdown <- nsclose <- nsgblclose <- nsgblTRMHelper <- nsgblRealTerm
<- nlstdstp <- npinlt <- ksuabt <- opidrv <- sou2o
<- opimai_real <- main <- libc_start_main
CHANGES
None.
CAUSE
The trace file first reports: Warning: keltnfy call to ldmInit failed with error 46
The ORA-7445 is not the starting point here. This exception is just a spin-off from ORA-108 and it is possible that different internal errors may be seen, such as ORA-600 [504], depending on what is happening when the ORA-108 is encountered.
The cause for the ORA-108 is related to the inital message at the beginning of the trace file: "keltnfy call to ldmInit failed with error 46" and this is followed by: "network error encountered getting listening address:"
The error code (here: 46) is the key for solving the issue.
This warning says that ldmInit() returned error 46 which is LDMERR_HOST_NOT_FOUND (host not found).
This error is returned if the OS call gethostbyname() fails with an error. So these appears to be a network specific issue.
SOLUTION
1) Check permission on /etc/hosts
$ ls -l /etc/hosts -rw-r--r-- 2 root root 194 Oct 17 2006 /etc/hosts
Check if /etc/hosts file is correctly configured
( all of this on one line ).
2) Check the hostname:
$ hostname
$ ping `hostname`
Make sure you are able to ping the hostname
3) Check if /etc/nodename is correctly configured
If you have DNS setup, ping is not a tool to diagnose DNS problem. A better tool to use is nslookup, dnsquery, or dig.
$ nslookup $ nslookup $ nslookup
The forward and reverse lookup should succeed and return consistent address/info.
4) Check nsswitch.conf
$ more nsswitch.confhosts: files dnsMake sure host lookup is also done through the /etc/hosts file and not just dns. It is recommended that FILES come first before DNS.
Also, check the resolv.conf. This makes sure that the DNS is working properly.