[hobbit] dbcheck for RAC connections refused
Martin Flemming
martin.flemming at desy.de
Wed Aug 5 21:34:16 CEST 2009
Hi !
.. was some days on holiday .. ;-)
Thanks a lot for help !
.. indeed, something is going wrong, with the communication to the rac-database ..
you see, i'm not a oracle-expert ...
i will contact my second oracle-admin and will ask him,
whats going wrong with the configuration from my first oracle-admin ...
Thanks, i will report ...
cheers,
martin
On Fri, 31 Jul 2009, Francesco Duranti wrote:
> This is really strange.
> Regarding the RAC test it cannot be done without the tnsnames.ora configuration.
> What's the problem regarding the tnsnames.ora? does it exists?
>
> The timeout problem seems also strange because it's like your client could not reach the database.
> You should try to connect to the oracle client (sqlplus).
>
> You can also try with this perl script
>
> #/usr/bin/perl
> use DBI;
> $ENV{'ORACLE_HOME'}="/usr/local/oracle/10.2.0/client/";
> $dbhost="hostname";
> $port=1521;
> $sid="SID";
> $user="user";
> $pw="password";
> $data_source="DBI:Oracle:host=$dbhost;port=$port;sid=$sid";
> $dbh = DBI->connect($data_source,$user,$pw ,{ RaiseError => 1, PrintError =>1});
> $dbh->disconnect;
> exit 0;
>
> Let me know if there's any problem...
>
> Francesco
>
>
> -----Original Message-----
> From: Martin Flemming [mailto:martin.flemming at desy.de]
> Sent: venerdì 31 luglio 2009 0.28
> To: hobbit at hswn.dk
> Subject: RE: [hobbit] dbcheck for RAC connections refused
>
>
> Hi !
>
> Unfortunatley, i've got no luck till yet ...
>
> I've cleaned up my dbcheck.ini like this, also without any spaces ...
>
>
> # cat etc/dbcheck.ini|grep -v \#
>
>
>
>
> [default]
> warningout = send
> conn = conn
> include =
> defaultgroup =
> timeoutdbconn = 20s
> timeoudbtselect = 1h
> oraclehome = /opt/products/oracle-client/10.2g/
> fastcheck = 30m
> mediumcheck = 2h
> slowcheck = 6h
> bbdisplay = localhost
> bbsendtype = multi
> singlesendashost = no
> bbconncolumn = ChkConn::fast
> bbauditcolumn = Audit::fast
> bbtablespacecolumn = TblSpace::fast
> bbextentcolumn = Extent::slow
> bbhitcachecolumn = HitCache::fast
> bbmemreqcolumn = MemReq::fast
> bbrollbackcolumn = RollBack::slow
> bbinvobjcolumn = InvObj::fast
> bblockscolumn = Locks::fast
> bbsessioncolumn = Session::fast
> bbsessionuserlist = yes
> username = XXXXXXXXXX
> password = XXXXXXXXXXXXXXxx
> tblspcheck = 90:95
> tblspreportautoext = 90:95
> pinlibcheck = 95:90
> blbuffcheck = 95:90
> hitcachecheck = 95:90
> rowcachecheck = 95:90
> extentcheck = 90:95
> rollbackcheck = 90:95
> sessioncheck = 90:95
> processcheck = 90:95
> rb.SYSTEM = 101:101
> invobjwarn = :::
> invobjignore =
> dbtype = Oracle
>
>
>
> [atlast1]
> hostname = lcg3d-a-v-1
> port = 1521
> username = XXXXXXXXXXX
> password = XXXXXXXXXXXXX
> bbauditcolumn = Audit:notest
> bbtablespacecolumn = TblSpace:notest
> bbextentcolumn = Extent:notest
> bbrollbackcolumn = RollBack:notest
> bblockscolumn = Locks:notest
> bbinvobjcolumn = InvObj:notest
>
>
>
> [atlast2]
> hostname = lcg3d-a-v-2
> port = 1521
> bbauditcolumn = Audit:notest
> bbtablespacecolumn = TblSpace:notest
> bbextentcolumn = Extent:notest
> bbrollbackcolumn = RollBack:notest
> bblockscolumn = Locks:notest
> bbinvobjcolumn = InvObj:notest
> username = XXXXXXXXXXx
> password = XXXXXXXXXXXXXX
>
>
> [atlast3]
> hostname = lcg3d-a-v-3
> port = 1521
> bbauditcolumn = Audit:notest
> bbtablespacecolumn = TblSpace:notest
> bbextentcolumn = Extent:notest
> bbrollbackcolumn = RollBack:notest
> bblockscolumn = Locks:notest
> bbinvobjcolumn = InvObj:notest
> username = XXXXXX
> password = XXXXXXXXXX
>
>
> [atlast4]
> hostname = lcg3d-a-v-4
> port = 1521
> username = XXXXXXX
> password = XXXXXXXXXXX
> bbauditcolumn = Audit:notest
> bbtablespacecolumn = TblSpace:notest
> bbextentcolumn = Extent:notest
> bbrollbackcolumn = RollBack:notest
> bblockscolumn = Locks:notest
> bbinvobjcolumn = InvObj:notest
>
>
>
>
> [atlas_tag_db]
> oraclerac = yes
> port = 1521
> username = XXXXXXXX
> password = XXXXXXXXX
> bbhitcachecolumn = HitCache:notest
> bbmemreqcolumn = MemReq:notest
>
>
> The output of the test is :
>
> l Stats
> Test Time = 00:00:01
> Host Checked = 5
> Status Msg = 29
>
> Colors Colors
> clear 24
> red 5
>
> Events Events
> Audit 1
> ChkConn 5
> Extent 1
> HitCache 4
> InvObj 1
> Locks 1
> MemReq 4
> RollBack 1
> Session 5
> TblSpace 1
> uptime 5
>
> Types Types
> status+30m 22
> status+60 5
> status+6h 2
>
> Hosts Summary
> Hosts clear red Number Times
> atlas_tag_db 8 1 9 00:00:00
> atlast1 4 1 5 00:00:00
> atlast2 4 1 5 00:00:00
> atlast3 4 1 5 00:00:00
> atlast4 4 1 5 00:00:00
>
> Hosts Summary
> Hosts Audit ChkConn Extent HitCache
> InvObj Locks MemReq RollBack Session TblSpace uptime
> atlas_tag_db 1 1 1 0
> 1 1 0 1 1 1 1
> atlast1 0 1 0 1
> 0 0 1 0 1 0 1
> atlast2 0 1 0 1
> 0 0 1 0 1 0 1
> atlast3 0 1 0 1
> 0 0 1 0 1 0 1
> atlast4 0 1 0 1
> 0 0 1 0 1 0 1
>
> Errors
> No General Errors
> atlas_tag_db
> Fri Jul 31 00:02:15 2009:ERROR: Timeout connecting to DBI:Oracle:atlas_tag_db!
> atlast1
> Fri Jul 31 00:02:15 2009:ERROR: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-1;port=1521;sid=atlast1!
> atlast2
> Fri Jul 31 00:02:15 2009:ERROR: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-2;port=1521;sid=atlast2!
> atlast3
> Fri Jul 31 00:02:15 2009:ERROR: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-3;port=1521;sid=atlast3!
> atlast4
> Fri Jul 31 00:02:15 2009:ERROR: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-4;port=1521;sid=atlast4!
>
> Warnings
> No General Warnings
>
>
>> In tnsnames.ora (under ORACLE_HOME/network/admin) I've those configuration:
>
> This is a good point, i can't use the tnsnames.ora, because my
> oracle-software is placed in the global filesystem AFS ...
>
> So, could this be a problem, if i have to got all informations only from dbcheck.ini ?
>
>
>
>> You can also try to connect with a telnet to the hostname and listener port
>
>
> Yep, telnet to port 1521 is working .. to lcg3d-a-v-1/4
>
> here my enviroment:
>
> bin/bbcmd ext/dbcheck.pl -v
> 2009-07-31 00:04:28 Using default environment file /usr/lib/hobbit/server/etc/hobbitserver.cfg
> Program Information
> dbcheck.pl version 1.09
> Hobbit_fd_lib.pm version 1.16
>
> OS Information
> Linux it-wgs02 2.6.9-78.0.22.ELsmp #1 SMP Thu Apr 30 23:30:54 CDT
> 2009 i686 i686 i386 GNU/Linux
> Perl version is v5.8.5
>
> Module Information
> FindBin 1.44
> POSIX 1.08
> Text::Wrap 2001.09292
> Time::Local 1.1
> Sys::SigAction 0.04
> Config::IniFiles 2.39
> DBI 1.605
> DBD::DBM 0.03
> DBD::ExampleP 12.010007
> DBD::File 0.35
> DBD::Gofer 0.011377
> DBD::Oracle 1.23
> DBD::Proxy 0.2004
> DBD::Sponge 12.010002
>
>
> Hmm, misterious ..
>
> martin
>
>
>
>
>
> On Thu, 30 Jul 2009, Francesco Duranti wrote:
>
>> This is what i normally do to check a RAC DB + single RAC instance in the configuration:
>>
>> oraclehome = /usr/local/oracle/10.2.0/client/
>> bbdisplay = localhost
>> bbconncolumn = ChkConn::fast
>> bbauditcolumn = Audit::fast
>> bbtablespacecolumn = TblSpace::fast
>> bbextentcolumn = Extent::slow
>> bbhitcachecolumn = HitCache::fast
>> bbmemreqcolumn = MemReq::fast
>> bbrollbackcolumn = RollBack::slow
>> bbinvobjcolumn = InvObj::fast
>> bblockscolumn = Locks::fast
>> bbsessioncolumn = Session::fast
>> bbsessionuserlist = yes
>> username = userxxx
>> password = pwxxxxx
>>
>> [CL10G]
>> oraclerac = yes
>> bbhitcachecolumn = HitCache:notest
>> bbmemreqcolumn = MemReq:notest
>>
>> [CL10G1]
>> hostname = itromdb21
>> bbauditcolumn = Audit:notest
>> bbtablespacecolumn = TblSpace:notest
>> bbextentcolumn = Extent:notest
>> bbrollbackcolumn = RollBack:notest
>> bblockscolumn = Locks:notest
>> bbinvobjcolumn = InvObj:notest
>>
>> [CL10G2]
>> hostname = itromdb22
>> bbauditcolumn = Audit:notest
>> bbtablespacecolumn = TblSpace:notest
>> bbextentcolumn = Extent:notest
>> bbrollbackcolumn = RollBack:notest
>> bblockscolumn = Locks:notest
>> bbinvobjcolumn = InvObj:notest
>>
>>
>> In tnsnames.ora (under ORACLE_HOME/network/admin) I've those configuration:
>>
>> CL10G =
>> (DESCRIPTION =
>> (ADDRESS = (PROTOCOL = TCP)(HOST = ITROMDB21)(PORT = 1521))
>> (ADDRESS = (PROTOCOL = TCP)(HOST = ITROMDB22)(PORT = 1521))
>> (LOAD_BALANCE = no)
>> (CONNECT_DATA =
>> (SERVER = DEDICATED)
>> (SERVICE_NAME = cl10g)
>> )
>> )
>>
>> CL10G2 =
>> (DESCRIPTION =
>> (ADDRESS = (PROTOCOL = TCP)(HOST = ITROMDB22)(PORT = 1521))
>> (CONNECT_DATA =
>> (SERVER = DEDICATED)
>> (SERVICE_NAME = cl10g)
>> (INSTANCE_NAME = cl10g2)
>> )
>> )
>>
>> CL10G1 =
>> (DESCRIPTION =
>> (ADDRESS = (PROTOCOL = TCP)(HOST = ITROMDB21)(PORT = 1521))
>> (CONNECT_DATA =
>> (SERVER = DEDICATED)
>> (SERVICE_NAME = cl10g)
>> (INSTANCE_NAME = cl10g1)
>> )
>> )
>>
>>
>> With this setting I normally use the tnsnames.ora configuration for RAC (I was able to check RAC database only using tnsnames.ora not directly.
>> You can try to remove the hostname and port from the db configuration for the RAC.
>> If the SID is equal to the name of the DB to check (the one in []) you can also remove the sid line from the configuration.
>>
>> You can also try to connect with a telnet to the hostname and listener port to see if it answer (or better you can configure on the client the tnsnames.ora and do a tnsping to see if the instances are answering).
>>
>> As you can see I don't do some checks on the RAC itselft or on the single instances... For example I don't test memreq and hitcache on the RAC and I don't check TblSpace, Extent, Rollback, locks, invalid objects on the single instances.
>>
>>
>>
>> Please check also on the configuration file and remove all ending space after host or port or sid. I don't know if it can create problems (I'll check this also and remove those from the configuration option for the next version).
>>
>>
>> Let me know if this work for you or you have other problems.
>>
>> Francesco
>>
>>
>>
>> -----Original Message-----
>> From: Martin Flemming [mailto:martin.flemming at desy.de]
>> Sent: giovedì 30 luglio 2009 11.47
>> To: hobbit at hswn.dk
>> Subject: RE: [hobbit] dbcheck for RAC connections refused
>>
>>
>> Hi, Franceso !
>>
>> Thanks for the hint, it looks better/cleaner now, but the error is still exist ...
>>
>> "
>> /usr/lib/hobbit/server # bin/bbcmd ext/dbcheck.pl -t atlast1
>> 2009-07-30 08:08:28 Using default environment file /usr/lib/hobbit/server/etc/hobbitserver.cfg
>> Thu Jul 30 08:08:29 2009:ERROR on atlast1: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-1 ;port=1521 ;sid=atlast1 !
>> "
>>
>>
>>
>>
>> [it-wgs02] /usr/lib/hobbit/server # bin/bbcmd perl -d ext/dbcheck.pl -t atlast1
>> 2009-07-30 08:05:59 Using default environment file /usr/lib/hobbit/server/etc/hobbitserver.cfg
>>
>> Loading DB routines from perl5db.pl version 1.27
>> Editor support available.
>>
>> Enter h or `h h' for help, or `man perldebug' for more help.
>>
>> Hobbit_fd_lib::CODE(0x9505830)(/usr/lib/hobbit/server/ext/Hobbit_fd_lib.pm:1222):1222: &load_config ; DB<1> n
>> Hobbit_fd_lib::CODE(0x9505830)(/usr/lib/hobbit/server/ext/Hobbit_fd_lib.pm:1223):1223: &get_disabled_hosts; DB<1> n
>> Hobbit_fd_lib::CODE(0x9505830)(/usr/lib/hobbit/server/ext/Hobbit_fd_lib.pm:1224):1224: my $pv=sprintf "%vd", $^V; DB<1> n
>> Hobbit_fd_lib::CODE(0x9505830)(/usr/lib/hobbit/server/ext/Hobbit_fd_lib.pm:1225):1225: my $sv=$Sys::SigAction::VERSION; DB<1> n
>> Hobbit_fd_lib::CODE(0x9505830)(/usr/lib/hobbit/server/ext/Hobbit_fd_lib.pm:1226):1226: if (($pv =~ /5\.8\.0/) && ($sv !~ /0\.09/)) { DB<1> n
>> main::(ext/dbcheck.pl:39): my ($orahome, @listahost, $dbhost, $port, $username, $password, $dbtype, $sid,$onlycustomcheck); DB<1> n
>> main::(ext/dbcheck.pl:40): my (%warnhash,%alerthash,$invobjignore,$invobjwarn, at invobjwlist, at invobjilist,$checkuserlist,%customcheck); DB<1> n
>> main::(ext/dbcheck.pl:41): my ($sendtv,$singlesendas,%selcmd,$raccheck, @KeyToCheck, at CustomKeyToCheck); DB<1> n
>> main::(ext/dbcheck.pl:43): my %DBCHECK = (main::(ext/dbcheck.pl:44): oracle => [ "fulltest","conn","audit","tablespace","extent","hitcache","memreq",
>> main::(ext/dbcheck.pl:45): "rollback","invobj","locks","session" ],
>> main::(ext/dbcheck.pl:46): oracledef => [ "Database","ChkConn::fast","Audit::fast","TblSpace::fast","Extent::slow",
>> main::(ext/dbcheck.pl:47): "HitCache::fast","MemReq::fast", "RollBack::slow","InvObj::fast",
>> main::(ext/dbcheck.pl:48): "Locks::fast","Session::fast" ],
>> main::(ext/dbcheck.pl:49): informix => [ "fulltest","conn","tablespace" ],
>> main::(ext/dbcheck.pl:50): informixdef => [ "Database","ChkConn::fast","TblSpace::fast" ],
>> main::(ext/dbcheck.pl:51): sqlserver => [ "fulltest","conn","tablespace" ],
>> main::(ext/dbcheck.pl:52): sqlserverdef => [ "Database","ChkConn::fast","TblSpace::fast" ],
>> main::(ext/dbcheck.pl:53): mysql => [ "fulltest","conn","mysqlperf" ],
>> main::(ext/dbcheck.pl:54): mysqldef => [ "MySQL","ChkConn::fast","mysqlperf::fast" ],
>> main::(ext/dbcheck.pl:55): db2 => [ "fulltest","conn","tablespace" ],
>> main::(ext/dbcheck.pl:56): db2def => [ "DB2","ChkConn::fast","TblSpace::fast" ],
>> main::(ext/dbcheck.pl:57): ); DB<1> n
>> main::(ext/dbcheck.pl:59): my $CHECKLIST = [ "pinlib:95:90", "blbuff:95:90", "rowcache:95:90", "tblsp:90:95", "hitcache:95:90", "rollback:90:95",
>> main::(ext/dbcheck.pl:60): "session:90:95", "process:90:95", "memreq:90:95", "extent:90:95" ]; DB<1> n
>> main::(ext/dbcheck.pl:63): @listahost=&Hobbit_fd_lib::get_server_to_test($TEST) if (!@listahost); DB<1> n
>> main::(ext/dbcheck.pl:64): &Hobbit_fd_lib::execute_thread(\&execute_db_test , at listahost); DB<1> n
>> Thu Jul 30 08:06:27 2009:ERROR on atlast1: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-1 ;port=1521 ;sid=atlast1 !
>>
>> Debugged program terminated. Use q to quit or R to restart,
>> use O inhibit_exit to avoid stopping after program termination,
>> h q, h R or h O to get additional info.
>> main::(ext/dbcheck.pl:65): exit;
>> DB<1>
>>
>> ... and the logfile e.g.
>>
>> Thu Jul 30 11:45:37 2009:ERROR on atlas_tag_db: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-4 ;port=1521 ;sid=atlas_tag_db !
>> Thu Jul 30 11:45:37 2009:ERROR on atlast1: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-1 ;port=1521 ;sid=atlast1 !
>> Thu Jul 30 11:45:37 2009:ERROR on atlast3: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-3 ;port=1521 ;sid=atlast3 !
>> Thu Jul 30 11:45:37 2009:ERROR on atlast2: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-2 ;port=1521 ;sid=atlast2 !
>> Thu Jul 30 11:45:37 2009:ERROR on atlast4: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-4 ;port=1521 ;sid=atlast4 !
>>
>>
>>
>> Any suggestions for further debugging ?
>>
>> thanks & cheers
>>
>> martin
>>
>>
>> On Thu, 30 Jul 2009, Francesco Duranti wrote:
>>
>>> Hi,
>>> dbcheck.pl (and the other scripts in the package) use the CONFIG::IniFiles to read configuration data. CONFIG::Inifiles allow comment line starting with "#" and ";" but I think it will not recognize the "#" comment at the middle of a line so if you need comments you should put them alone on the line before or next to the line to comment on.
>>>
>>> On some of the sample config file I've put some comment next to parameters and this could probably generate confusion. I'll change the sample config file and put this behavious also in the documentation I'm writing for the next version (the actual documentation is not really well written and organized).
>>>
>>>
>>> Francesco
>>>
>>>
>>> On Wed, July 29, 2009 15:29, martin.flemming at desy.de wrote:
>>>>
>>>> Hi !
>>>>
>>>> I want to use dbcheck.pl from
>>>>
>>>> http://sourceforge.net/projects/hobbit-perl-cl/
>>>>
>>>> .. but unfortunatley something is going wrong, i get only
>>>> "Connection Refused" Messages and no data .. :-(
>>>>
>>>> My config is below:
>>> [SNIP]
>>>>
>>>> dbcheck.ini:
>>>>
>>>> oraclehome = /opt/products/oracle-client/10.2g/
>>>> username = XXXX
>>>> password = XXXXXXXXXXXXXXXXXXXXXXX
>>>>
>>>>
>>>> [atlas_tag_db]
>>>> dbtype = Oracle
>>>> oraclerac = yes
>>>> hostname = lcg3d-a-v-4 # hostname of the rac instance n.2
>>>> sid = atlas_tag_db # sid of the rac instance n.2
>>>> port = 1521 # port of listener on rac instance
>>>
>>>
>>> I don't use this code, but your error output suggests that it's trying to
>>> use your hostname, sid, and port from dbcheck.ini exactly as written,
>>> comments and all. I don't see anything in the sample dbcheck.ini file
>>> that suggests that you can use trailing comments in the way that you are
>>> doing, so perhaps you should try without those.
>>>
>>>
>
> To unsubscribe from the hobbit list, send an e-mail to
> hobbit-unsubscribe at hswn.dk
>
More information about the Xymon
mailing list