Bacula-users

[Bacula-users] Bacula Director and SD crashing

2011-03-13 17:12:18
Subject: [Bacula-users] Bacula Director and SD crashing
From: "Carlson, Mike" <carlson39 AT llnl DOT gov>
To: "bacula-users AT lists.sourceforge DOT net" <bacula-users AT lists.sourceforge DOT net>
Date: Sun, 13 Mar 2011 14:07:49 -0700
Upfront details of the environemnt:

# uname -a
FreeBSD write.llnl.gov 8.2-RELEASE FreeBSD 8.2-RELEASE #3: Tue Mar  8 11:32:05 
PST 2011     root AT write.llnl DOT gov:/usr/obj/usr/src/sys/GENERIC  amd64

# pkg_info |grep bacula
bacula-client-5.0.3 The network backup solution (client)
bacula-server-5.0.3 The network backup solution (server)
# pkg_info | grep postgresql
postgresql-client-9.0.3 PostgreSQL database (client)
postgresql-contrib-9.0.3 The contrib utilities from the PostgreSQL distribution
postgresql-server-9.0.3 The most advanced open-source database available 
anywhere

#  kldstat
Id Refs Address            Size     Name
 1   15 0xffffffff80100000 c9fe20   kernel
 2    1 0xffffffff80da0000 f080     aio.ko
 3    1 0xffffffff80db0000 16ea40   ispfw.ko
 4    1 0xffffffff80f20000 4fc8     geom_multipath.ko
 5    1 0xffffffff81022000 104a7a   zfs.ko
 6    1 0xffffffff81127000 1a22     opensolaris.ko

System memory summary
Total real memory available:    12264 MB
Logically used memory:          7605 MB
Logically available memory:     4659 MB

The Bacula director, database, and storage node are all currently on one server.

The problem:

I have a farily large Bacula environment (~1000 clients and servers), and there 
are plans to keep expanding our clients (and storage, and the database, 
etc...). I believe my lack of fully understanding the storage and director 
resources are at the root of it, because it appears that the director is 
running more jobs that the storage node can handle, however, it is not failing 
gracefully. Also, when the VirtualFull job is ran (a LOT run at once, between 
90 and 100), it almost immediately crashes the SD.

As far as job scheduling, I have 10 timeslots setup, from 7pm to 4am, each 
start time has around 90 - 110 clients.

I have the director configured for 1000 concurrent jobs:

Director {
        Name = write.llnl.gov-dir
        DIRport = 9101
        QueryFile = "/usr/local/share/bacula/query.sql"
        WorkingDirectory = "/var/db/bacula"
        PidDirectory = "/var/run"
        Maximum Concurrent Jobs = 1000
        Password = "XXXXXXXXXXXXXXXXXXXXXXXXXX"
        Messages = Daemon
}

I have a few different storage node (all File storage, no tapes here), but the 
primary one that is responsible for 900 clients is configured for 1000 
concurrent jobs, and 1000 separate devices.

Storage {
        Name = write.llnl.gov-sd
        SDAddress = write.llnl.gov
        SDPort = 9103
        WorkingDirectory = "/var/db/bacula"
        Pid Directory = "/var/run"
        Maximum Concurrent Jobs = 1001
}
Device {
        Name = FileStorageD1
        Media Type = File
        Archive Device = /filevol001/drive01
        LabelMedia = yes;
        Random Access = Yes;
        AutomaticMount = yes;
        RemovableMedia = no;
        AlwaysOpen = no;
        Maximum Concurrent Jobs = 4
        Maximum File Size = 5G
        Maximum Spool Size = 370g
        Maximum Job Spool Size = 20g
        Maximum Block Size = 262144
        Maximum Network Buffer Size = 262144
}
Device {
        Name = VF_FileStorageD1
        Media Type = File
        Archive Device = /filevol001/VirtualFull/vf_drive01
        LabelMedia = yes;
        Random Access = Yes;
        AutomaticMount = yes;
        RemovableMedia = no;
        AlwaysOpen = no;
        Maximum Concurrent Jobs = 4
        Maximum File Size = 5G
        Maximum Spool Size = 370g
        Maximum Job Spool Size = 20g
        Maximum Block Size = 262144
        Maximum Network Buffer Size = 262144
}

There are 500 FileStorageD### and 500 VF_FileStorageD###, all have the same 
above settings.

500 are for Incremental/Full jobs, the other 500 are for Virtual Fulls.

Since I hit the 500 or so client mark, I've started to see this error message 
pop up in the logs::
13-Mar 09:31 W1.llnl.gov JobId 15841: Fatal error: 
/tmp/bacula/bacula/src/filed/backup.c:1019 Network send error to SD. 
ERR=Input/output error

And then the sd deamon will crash.
(no debugging symbols found)...ptrace: Operation not permitted.
/var/db/bacula/1275: No such file or directory.
$1 = 0
/usr/local/share/bacula/btraceback.gdb:2: Error in sourced command file:
No symbol table is loaded.  Use the "file" command.

And the director follows shortly afterwards:
[New Thread 81ad04d40 (LWP 102349)]
[New Thread 81e889500 (LWP 102334)]
[New Thread 822935280 (LWP 102228)]
[New Thread 82649b540 (LWP 102039)]
[New Thread 821819140 (LWP 101905)]
[New Thread 827372f40 (LWP 103458)]
[New Thread 819963f00 (LWP 103457)]
[New Thread 8267c2840 (LWP 103456)]
[New Thread 8276135c0 (LWP 103454)]
[New Thread 8136d1180 (LWP 103453)]
[New Thread 8170ab540 (LWP 103452)]
[New Thread 81b446c80 (LWP 103451)]
[New Thread 81ca45f00 (LWP 103450)]
[New Thread 825b7dfc0 (LWP 103449)]
[New Thread 80533b8c0 (LWP 103442)]
[New Thread 81de9af40 (LWP 103441)]
[New Thread 811f4ad40 (LWP 103440)]
[New Thread 814ed6c40 (LWP 103436)]
[New Thread 824954a40 (LWP 103434)]
[New Thread 8226ff340 (LWP 103432)]
[New Thread 8268a16c0 (LWP 103431)]
[New Thread 81d7c15c0 (LWP 103430)]
[New Thread 81a0c1900 (LWP 103426)]
[New Thread 8245ec840 (LWP 103423)]
[New Thread 81d2ac540 (LWP 103421)]
[New Thread 81eea00c0 (LWP 102284)]
[New Thread 822beba80 (LWP 103483)]
[New Thread 822b17a80 (LWP 101351)]
[New Thread 81fa81a00 (LWP 103464)]
[New Thread 8249d1c00 (LWP 103463)]
[New Thread 81d7165c0 (LWP 103461)]
[New Thread 8111e1880 (LWP 103460)]
[New Thread 80a249700 (LWP 103459)]
[New Thread 81a274340 (LWP 103361)]
[New Thread 823ea4780 (LWP 103359)]
[New Thread 825d4a100 (LWP 103357)]
[New Thread 82565ac40 (LWP 103355)]
[New Thread 8070e8900 (LWP 103354)]
[New Thread 8045a9380 (LWP 103351)]
[New Thread 826d12300 (LWP 103350)]
[New Thread 821d4cf00 (LWP 103349)]
[New Thread 805175b40 (LWP 103348)]
[New Thread 819245a80 (LWP 103343)]
[New Thread 826298140 (LWP 103341)]
[New Thread 827073dc0 (LWP 103339)]
[New Thread 822f028c0 (LWP 103337)]
[New Thread 8238ae0c0 (LWP 103336)]
[New Thread 8077cfe00 (LWP 103333)]
[New Thread 82306d8c0 (LWP 103332)]
[New Thread 826a1bb80 (LWP 103455)]
[New Thread 8063e7e40 (LWP 103206)]
[New Thread 827005f80 (LWP 103204)]
[New Thread 8190a7a80 (LWP 103205)]
[New Thread 8093d7840 (LWP 103445)]
[New Thread 821818f80 (LWP 103202)]
[New Thread 821818dc0 (LWP 103443)]
[New Thread 827005dc0 (LWP 103329)]
[New Thread 8172d1740 (LWP 103195)]
[New Thread 822d25d80 (LWP 103192)]
[New Thread 811efcb80 (LWP 103193)]
[New Thread 8267c2680 (LWP 103328)]
[New Thread 8249d1a40 (LWP 103435)]
[New Thread 810c33700 (LWP 103433)]
[New Thread 806788380 (LWP 103306)]
[New Thread 824bf08c0 (LWP 103155)]
[New Thread 8269c7500 (LWP 103158)]
[New Thread 824e81b40 (LWP 103427)]
[New Thread 80bdcb480 (LWP 103425)]
[New Thread 822aa5e00 (LWP 103422)]
[New Thread 826a1b9c0 (LWP 103134)]
[New Thread 826d85140 (LWP 100835)]
[New Thread 824954340 (LWP 103326)]
[New Thread 822a04fc0 (LWP 103360)]
[New Thread 81d845e00 (LWP 103358)]
[New Thread 8268a1500 (LWP 103356)]
[New Thread 8249d1880 (LWP 103115)]
[New Thread 8245ec680 (LWP 103352)]
[New Thread 81a98e8c0 (LWP 103107)]
[New Thread 805b15400 (LWP 103098)]
[New Thread 8268a1340 (LWP 103100)]
[New Thread 82565aa80 (LWP 103344)]
[New Thread 824e81980 (LWP 103342)]
[New Thread 81057e740 (LWP 103324)]
[New Thread 813c52b80 (LWP 103340)]
[New Thread 8071288c0 (LWP 103322)]
[New Thread 806f66c80 (LWP 103321)]
[New Thread 82565a8c0 (LWP 103338)]
[New Thread 81ec18600 (LWP 103069)]
[New Thread 81a205500 (LWP 103334)]
[New Thread 8213faf00 (LWP 103318)]
[New Thread 8042ab900 (LWP 103317)]
[New Thread 815bec5c0 (LWP 103067)]
[New Thread 822efb8c0 (LWP 103330)]
[New Thread 81ad04b80 (LWP 103314)]
[New Thread 8172f0580 (LWP 103312)]
[New Thread 810c338c0 (LWP 103310)]
[New Thread 826a1b800 (LWP 103308)]
[New Thread 81ad049c0 (LWP 102273)]
[New Thread 82167c380 (LWP 103327)]
[New Thread 8255a4340 (LWP 103304)]
[New Thread 81a1bd3c0 (LWP 103302)]
[New Thread 81b65d500 (LWP 103299)]
[New Thread 81b98f8c0 (LWP 103297)]
[New Thread 80fb3ffc0 (LWP 103295)]
[New Thread 825d43100 (LWP 103294)]
[New Thread 81d7c1240 (LWP 103293)]
[New Thread 811f4b0c0 (LWP 103289)]
[New Thread 806f66e40 (LWP 103279)]
[New Thread 80a595c80 (LWP 102852)]
[New Thread 815d781c0 (LWP 102849)]
[New Thread 808171a80 (LWP 101739)]
[New Thread 81e5be880 (LWP 100915)]
[New Thread 81a48a0c0 (LWP 102264)]
[New Thread 821818c00 (LWP 103325)]
[New Thread 8081718c0 (LWP 102257)]
[New Thread 81cd26e00 (LWP 102237)]
[New Thread 817d25540 (LWP 103323)]
[New Thread 826a1b640 (LWP 103030)]
[New Thread 8268a1180 (LWP 103319)]
[New Thread 81dadf7c0 (LWP 103027)]
[New Thread 82177a1c0 (LWP 103315)]
[New Thread 806045280 (LWP 103313)]
[New Thread 825b7de00 (LWP 103311)]
[New Thread 824e817c0 (LWP 103309)]
[New Thread 823ea45c0 (LWP 103307)]
[New Thread 820ffa180 (LWP 103305)]
[New Thread 814705b00 (LWP 103303)]
[New Thread 824954880 (LWP 102993)]
[New Thread 82565a700 (LWP 103300)]
[New Thread 823655180 (LWP 103298)]
[New Thread 8228a2280 (LWP 103296)]
[New Thread 8093d0140 (LWP 102982)]
[New Thread 826297f80 (LWP 102988)]
[New Thread 825d49f40 (LWP 103290)]
[New Thread 824e81600 (LWP 103288)]
[New Thread 825f7d800 (LWP 103217)]
[New Thread 80778ba80 (LWP 102851)]
[New Thread 80d14f380 (LWP 102280)]
[New Thread 825f7d640 (LWP 102231)]
[New Thread 811f4af00 (LWP 102850)]
[New Thread 81016f700 (LWP 102405)]
[New Thread 819441580 (LWP 101184)]
[New Thread 8097d7dc0 (LWP 100697)]
[New Thread 824e81440 (LWP 102239)]
[New Thread 81eec8280 (LWP 101974)]
[New Thread 819963d40 (LWP 103287)]
[New Thread 8213eeb80 (LWP 101543)]
[New Thread 8252924c0 (LWP 100926)]
[New Thread 80951f2c0 (LWP 103283)]
[New Thread 808fdd500 (LWP 103281)]
[New Thread 8093c9140 (LWP 102396)]
[New Thread 81f582c80 (LWP 103286)]
[New Thread 8226c6fc0 (LWP 103284)]
[New Thread 824e81280 (LWP 103285)]
[New Thread 8255a4180 (LWP 102222)]
[New Thread 81afc3980 (LWP 103277)]
[New Thread 804e85900 (LWP 103282)]
[New Thread 820457f00 (LWP 102326)]
[New Thread 8245ec4c0 (LWP 103280)]
[New Thread 815242f40 (LWP 103278)]
[New Thread 81f8f6fc0 (LWP 102371)]
[New Thread 809517f40 (LWP 102360)]
[New Thread 825292300 (LWP 103276)]
[New Thread 825139800 (LWP 103270)]
[New Thread 820d8a180 (LWP 103268)]
[New Thread 804010c80 (LWP 103267)]
[New Thread 821fe6140 (LWP 103264)]
[New Thread 8109e9c40 (LWP 103263)]
[New Thread 824e5e440 (LWP 103262)]
[New Thread 819441040 (LWP 103261)]
[New Thread 81f1aa180 (LWP 103255)]
[New Thread 82508a800 (LWP 103254)]
[New Thread 821811340 (LWP 103251)]
[New Thread 80712bb80 (LWP 102809)]
[New Thread 825117640 (LWP 103248)]
[New Thread 820b72080 (LWP 103247)]
[New Thread 825405340 (LWP 103246)]
[New Thread 821e12300 (LWP 103241)]
[New Thread 81fde3880 (LWP 103239)]
[New Thread 8248b7500 (LWP 103237)]
[New Thread 80e76b640 (LWP 103236)]
[New Thread 81c8c7e00 (LWP 103235)]
[New Thread 8052f61c0 (LWP 103234)]
[New Thread 8212f0d40 (LWP 103233)]
[New Thread 81d4300c0 (LWP 103232)]
[New Thread 8181023c0 (LWP 103231)]
[New Thread 8212e8d40 (LWP 102167)]
[New Thread 8220a3fc0 (LWP 101913)]
[New Thread 8251d19c0 (LWP 102974)]
[New Thread 824bf0700 (LWP 102218)]
[New Thread 824d36700 (LWP 103275)]
[New Thread 80b9c3fc0 (LWP 103274)]
[New Thread 81c008f40 (LWP 103271)]
[New Thread 820a10080 (LWP 102971)]
[New Thread 80951f100 (LWP 103272)]
[New Thread 80951ef40 (LWP 102376)]
[New Thread 803f39540 (LWP 102212)]
[New Thread 8210e6a80 (LWP 103226)]
[New Thread 824592840 (LWP 103225)]
[New Thread 804b324c0 (LWP 103219)]
[New Thread 824d2c1c0 (LWP 103201)]
[New Thread 810324fc0 (LWP 103198)]
[New Thread 81e04ac00 (LWP 103190)]
[New Thread 809498100 (LWP 103189)]
[New Thread 8093cfdc0 (LWP 103151)]
[New Thread 8244a4a00 (LWP 103059)]
[New Thread 81be1cfc0 (LWP 103029)]
[New Thread 804bfc940 (LWP 102985)]
[New Thread 80458fe40 (LWP 102966)]
[New Thread 81ffe86c0 (LWP 102953)]
[New Thread 824d36540 (LWP 103269)]
[New Thread 8251d1800 (LWP 102958)]
[New Thread 8251d1640 (LWP 103265)]
[New Thread 80fe0a040 (LWP 103257)]
[New Thread 81e9ab500 (LWP 102905)]
[New Thread 820f1c340 (LWP 102956)]
[New Thread 8249546c0 (LWP 102926)]
[New Thread 824d36380 (LWP 103256)]
[New Thread 823ea4400 (LWP 102901)]
[New Thread 825139640 (LWP 103252)]
[New Thread 824bcb8c0 (LWP 103250)]
[New Thread 823e9b240 (LWP 103249)]
[New Thread 82510f640 (LWP 102878)]
[New Thread 818f07d00 (LWP 102872)]
[New Thread 81a955380 (LWP 102823)]
[New Thread 8093df100 (LWP 102814)]
[New Thread 80cd50100 (LWP 102811)]
[New Thread 81f07e340 (LWP 102879)]
[New Thread 825139480 (LWP 102865)]
[New Thread 823138700 (LWP 102868)]
[New Thread 822f858c0 (LWP 103242)]
[New Thread 81addc280 (LWP 102590)]
[New Thread 81852a340 (LWP 103240)]
[New Thread 82062c280 (LWP 102577)]
[New Thread 808aed700 (LWP 102565)]
[New Thread 80bdcaf40 (LWP 102562)]
[New Thread 820c98340 (LWP 102560)]
[New Thread 822aa5c40 (LWP 102556)]
[New Thread 8245b4680 (LWP 102114)]
[New Thread 8230a58c0 (LWP 102083)]
[New Thread 825292140 (LWP 103238)]
[New Thread 81f5f8e40 (LWP 101888)]
[New Thread 806908700 (LWP 103228)]
[New Thread 81fdd9a40 (LWP 102856)]
[New Thread 82071c0c0 (LWP 103227)]
[New Thread 805548100 (LWP 102854)]
[New Thread 81e17ea40 (LWP 102202)]
[New Thread 824d361c0 (LWP 102192)]
[New Thread 823ea4240 (LWP 102169)]
[New Thread 824954500 (LWP 102165)]
[New Thread 81f582e40 (LWP 100696)]
[New Thread 815d7ee40 (LWP 103224)]
[New Thread 825291f80 (LWP 103221)]
[New Thread 825405180 (LWP 102838)]
[New Thread 824d32e40 (LWP 103199)]
[New Thread 807f1d380 (LWP 103092)]
[New Thread 81ba20180 (LWP 102834)]
[New Thread 810302180 (LWP 102832)]
[New Thread 825404fc0 (LWP 102828)]
[New Thread 823ea4080 (LWP 103065)]
[New Thread 81ca46280 (LWP 102720)]
[New Thread 80f34c3c0 (LWP 102986)]
[New Thread 8200aaa80 (LWP 102607)]
[New Thread 825291dc0 (LWP 102954)]
[New Thread 825117480 (LWP 102904)]
[New Thread 8096dda80 (LWP 102631)]
[New Thread 8063aa900 (LWP 102625)]
[New Thread 813b77f00 (LWP 102500)]
[New Thread 81f8f6e00 (LWP 102498)]
[New Thread 821818a40 (LWP 102505)]
[New Thread 825404e00 (LWP 102605)]
[New Thread 824e5e280 (LWP 102585)]
[New Thread 805549b40 (LWP 102332)]
[New Thread 8093def40 (LWP 102341)]
[New Thread 823654fc0 (LWP 102345)]
[New Thread 8248c86c0 (LWP 102342)]
[New Thread 80631c580 (LWP 102526)]
[New Thread 81c07af40 (LWP 102282)]
[New Thread 8250749c0 (LWP 101997)]
[New Thread 8248c8500 (LWP 101992)]
[New Thread 822dfbbc0 (LWP 100739)]
[New Thread 817c99700 (LWP 100785)]
[New Thread 81f8c9380 (LWP 101645)]
[New Thread 815078a80 (LWP 101167)]
[New Thread 81995d0c0 (LWP 102533)]
[New Thread 824e5e0c0 (LWP 102139)]
[New Thread 8248b76c0 (LWP 102531)]
[New Thread 80712a680 (LWP 102124)]
[New Thread 820a09400 (LWP 102072)]
[New Thread 81e5c5880 (LWP 102063)]
[New Thread 8093debc0 (LWP 102515)]
[New Thread 81e980340 (LWP 102306)]
[New Thread 82364e180 (LWP 102529)]
[New Thread 815bf4400 (LWP 102302)]
[New Thread 825074800 (LWP 102516)]
[New Thread 82506c9c0 (LWP 102452)]
[New Thread 811c34c80 (LWP 101549)]
[New Thread 81c99c8c0 (LWP 101629)]
[New Thread 8238b6280 (LWP 102508)]
[New Thread 81943a3c0 (LWP 102178)]
[New Thread 822bf3a80 (LWP 102475)]
[New Thread 808aed540 (LWP 102494)]
[New Thread 8202b26c0 (LWP 102485)]
[New Thread 8092e2f80 (LWP 102482)]
[New Thread 815bec780 (LWP 102480)]
[New Thread 804e85ac0 (LWP 102478)]
[New Thread 824610fc0 (LWP 102471)]
[New Thread 813c0df00 (LWP 102465)]
[New Thread 81f8c9540 (LWP 102461)]
[New Thread 824898880 (LWP 102457)]
[New Thread 824610c40 (LWP 102454)]
[New Thread 81bb4fc00 (LWP 102254)]
[New Thread 8204580c0 (LWP 102248)]
[New Thread 8093dea00 (LWP 101990)]
[New Thread 80fb3fa80 (LWP 101155)]
[New Thread 81f8f6c40 (LWP 102262)]
[New Thread 8238b60c0 (LWP 103215)]
[New Thread 821e12140 (LWP 102961)]
[New Thread 81998eb80 (LWP 102247)]
[New Thread 819e53fc0 (LWP 102848)]
[New Thread 81918ca80 (LWP 102250)]
[New Thread 819389180 (LWP 102235)]
[New Thread 821b21f80 (LWP 102243)]
[New Thread 821818880 (LWP 102846)]
[New Thread 82364dfc0 (LWP 102368)]
[New Thread 822f85700 (LWP 100935)]
[New Thread 8224d2a40 (LWP 102035)]
[New Thread 804f7a540 (LWP 100573)]
[New Thread 81e891500 (LWP 101898)]
[New Thread 8224d2500 (LWP 102435)]
[New Thread 823d16d40 (LWP 102825)]
[New Thread 81d2ac8c0 (LWP 102431)]
[New Thread 823432380 (LWP 102817)]
[New Thread 8192d1180 (LWP 102608)]
[New Thread 80c082bc0 (LWP 102583)]
[New Thread 81f84c700 (LWP 102428)]
[New Thread 810f36e80 (LWP 102426)]
[New Thread 8214f3880 (LWP 102430)]
[New Thread 8093cff80 (LWP 101434)]
[New Thread 81995cf00 (LWP 102180)]
[New Thread 81998f280 (LWP 102840)]
[New Thread 80921fc00 (LWP 102836)]
[New Thread 81c52b500 (LWP 102421)]
[New Thread 822bf38c0 (LWP 102425)]
[New Thread 823d8db80 (LWP 102415)]
[New Thread 823644180 (LWP 102402)]
[New Thread 813e28900 (LWP 102819)]
[New Thread 82342a1c0 (LWP 102399)]
[New Thread 804189c80 (LWP 102387)]
[New Thread 824610e00 (LWP 102621)]
[New Thread 821a11f80 (LWP 102155)]
[New Thread 821dccf00 (LWP 101427)]
[New Thread 8055458c0 (LWP 101241)]
[New Thread 80e69c800 (LWP 102446)]
[New Thread 821dccb80 (LWP 102429)]
[New Thread 8218186c0 (LWP 102492)]
[New Thread 8221a0fc0 (LWP 102420)]
[New Thread 81057e580 (LWP 102407)]
[New Thread 81f72a440 (LWP 102400)]
[New Thread 81f8f6a80 (LWP 102389)]
[New Thread 8093de840 (LWP 102386)]
[New Thread 822dd5bc0 (LWP 101379)]
[New Thread 822dfba00 (LWP 102441)]
[New Thread 81b79a880 (LWP 102364)]
[New Thread 8230a5700 (LWP 102303)]
[New Thread 813c0db80 (LWP 102256)]
[New Thread 80ccc2f40 (LWP 102204)]
[New Thread 821b21dc0 (LWP 101593)]
[New Thread 820b7b400 (LWP 101296)]
[New Thread 805bd0780 (LWP 103223)]
[New Thread 821e11f80 (LWP 103222)]
[New Thread 821dccd40 (LWP 103220)]
[New Thread 821a11dc0 (LWP 103218)]
[New Thread 803ebd740 (LWP 103216)]
[New Thread 822b17c40 (LWP 102017)]
[New Thread 822edaa80 (LWP 103213)]
[New Thread 8238b5f00 (LWP 103211)]
[New Thread 8234321c0 (LWP 103209)]
[New Thread 823430e40 (LWP 103207)]
[New Thread 81ea9dbc0 (LWP 103197)]
[New Thread 81df0cd80 (LWP 102795)]
[New Thread 81d83e700 (LWP 102804)]
[New Thread 822f02700 (LWP 102784)]
[New Thread 821e11dc0 (LWP 102783)]
[New Thread 81f8f68c0 (LWP 103196)]
[New Thread 822c62d80 (LWP 103194)]
[New Thread 821818500 (LWP 102627)]
[New Thread 81fbd2d80 (LWP 102626)]
[New Thread 809856c40 (LWP 103188)]
[New Thread 80b28cd80 (LWP 103172)]
[New Thread 8226ff180 (LWP 103159)]
[New Thread 8093d7680 (LWP 102597)]
[New Thread 80ffc9200 (LWP 102586)]
[New Thread 803854300 (LWP 103147)]
[New Thread 80c0f7a00 (LWP 103139)]
[New Thread 821e11c00 (LWP 103136)]
[New Thread 8093d74c0 (LWP 101192)]
[New Thread 816658900 (LWP 101479)]
[New Thread 81dad2600 (LWP 103128)]
[New Thread 820b7b240 (LWP 103125)]
[New Thread 821818340 (LWP 103119)]
[New Thread 8093d7300 (LWP 103111)]
[New Thread 81f8f6700 (LWP 102552)]
[New Thread 8226fefc0 (LWP 103101)]
[New Thread 815d081c0 (LWP 103094)]
[New Thread 8210efc40 (LWP 102548)]
[New Thread 80edb2900 (LWP 102544)]
[New Thread 81d4ac280 (LWP 103091)]
[New Thread 821780e40 (LWP 103089)]
[New Thread 8224d2880 (LWP 103086)]
[New Thread 81c0f6100 (LWP 103075)]
[New Thread 821d4cd40 (LWP 102525)]
[New Thread 822d25bc0 (LWP 103063)]
[New Thread 803855f00 (LWP 103015)]
[New Thread 8221a0e00 (LWP 103004)]
[New Thread 822c5abc0 (LWP 103044)]
[New Thread 8182d72c0 (LWP 103000)]
[New Thread 809856a80 (LWP 102406)]
[New Thread 819df86c0 (LWP 103037)]
[New Thread 821a11c00 (LWP 103034)]
[New Thread 821a11a40 (LWP 103032)]
[New Thread 806da7480 (LWP 102542)]
[New Thread 81c909c40 (LWP 103028)]
[New Thread 8229350c0 (LWP 102493)]
[New Thread 822c62bc0 (LWP 103018)]
[New Thread 822c62a00 (LWP 103010)]
[New Thread 81d4a3f00 (LWP 103007)]
[New Thread 811efcf00 (LWP 103005)]
[New Thread 8093d7140 (LWP 103003)]
[New Thread 8224d26c0 (LWP 102999)]
[New Thread 8210efa80 (LWP 102997)]
[New Thread 80ce66f00 (LWP 102994)]
[New Thread 8070e8e40 (LWP 102992)]
[New Thread 820b7b080 (LWP 102990)]
[New Thread 81f8f6540 (LWP 102451)]
[New Thread 81c3c77c0 (LWP 102439)]
[New Thread 818f07ec0 (LWP 102984)]
[New Thread 819c78180 (LWP 102830)]
[New Thread 8213fad40 (LWP 102419)]
[New Thread 819e4ce00 (LWP 101239)]
[New Thread 80b288e80 (LWP 101374)]
[New Thread 8226fee00 (LWP 101521)]
[New Thread 82167c1c0 (LWP 102196)]
[New Thread 8228a20c0 (LWP 101862)]
[New Thread 808171c40 (LWP 101878)]
[New Thread 821e11a40 (LWP 102980)]
[New Thread 81be1ca80 (LWP 102978)]
[New Thread 80f703c80 (LWP 102977)]
[New Thread 821a11880 (LWP 102975)]
[New Thread 8228a1f00 (LWP 102190)]
[New Thread 80ce582c0 (LWP 102972)]
[New Thread 81f8f6380 (LWP 102970)]
[New Thread 81c901c40 (LWP 102969)]
[New Thread 81aa28a00 (LWP 102967)]
[New Thread 81fbd2bc0 (LWP 102112)]
[New Thread 810acfa80 (LWP 102963)]
[New Thread 82166de40 (LWP 101948)]
[New Thread 821e11880 (LWP 102964)]
[New Thread 821a116c0 (LWP 102416)]
[New Thread 8213fab80 (LWP 102960)]
[New Thread 805558180 (LWP 102414)]
[New Thread 8213fa9c0 (LWP 102410)]
[New Thread 82166dc80 (LWP 102409)]
[New Thread 820b7aec0 (LWP 102952)]
[New Thread 821263f00 (LWP 102403)]
[New Thread 81dd4bc40 (LWP 102903)]
[New Thread 80e841440 (LWP 102898)]
[New Thread 81ff83880 (LWP 102873)]
[New Thread 81faa2bc0 (LWP 102391)]
[New Thread 81fc8bc00 (LWP 102401)]
[New Thread 81842efc0 (LWP 102871)]
[New Thread 818ffe5c0 (LWP 102867)]
[New Thread 821818180 (LWP 102863)]
[New Thread 81fb27840 (LWP 102857)]
[New Thread 8214f36c0 (LWP 102845)]
[New Thread 81b484540 (LWP 102844)]
[New Thread 804b31180 (LWP 102380)]
[New Thread 81ad23f00 (LWP 102383)]
[New Thread 8063e73c0 (LWP 102385)]
[New Thread 8172f03c0 (LWP 102384)]
[New Thread 820c98180 (LWP 102382)]
[New Thread 81fb27d80 (LWP 102379)]
[New Thread 81f07e180 (LWP 102074)]
[New Thread 8070e4140 (LWP 102060)]
[New Thread 80b7fee40 (LWP 101984)]
[New Thread 80e784f00 (LWP 101925)]
[New Thread 81fb27bc0 (LWP 100720)]
[New Thread 81f5f8c80 (LWP 100686)]
[New Thread 81be9efc0 (LWP 102372)]
[New Thread 816eed100 (LWP 102839)]
[New Thread 807924340 (LWP 102369)]
[New Thread 8214f3500 (LWP 102835)]
[New Thread 820a10240 (LWP 102358)]
[New Thread 8210ef8c0 (LWP 102337)]
[New Thread 804b32680 (LWP 102338)]
[New Thread 803f3f740 (LWP 102339)]
[New Thread 811037180 (LWP 102782)]
[New Thread 81f07dfc0 (LWP 102314)]
[New Thread 8212f0b80 (LWP 102609)]
[New Thread 82033e500 (LWP 102276)]
[New Thread 805d58900 (LWP 102581)]
[New Thread 806788e00 (LWP 102551)]
[New Thread 81901c8c0 (LWP 102205)]
[New Thread 81ab1ff40 (LWP 102209)]
[New Thread 81558bbc0 (LWP 102353)]
[New Thread 80e7c3280 (LWP 102198)]
[New Thread 819222e00 (LWP 102143)]
[New Thread 81fb27a00 (LWP 102141)]
[New Thread 820c97fc0 (LWP 102351)]
[New Thread 81dd4ba80 (LWP 102347)]
[New Thread 814ab0480 (LWP 102094)]
[New Thread 803634740 (LWP 102092)]
[New Thread 80e83a440 (LWP 102300)]
[New Thread 80a595e40 (LWP 102090)]
[New Thread 81e9ab340 (LWP 102097)]
[New Thread 81faa2a00 (LWP 102086)]
[New Thread 81dd4b8c0 (LWP 102298)]
[New Thread 810f36b00 (LWP 101657)]
[New Thread 8049b4c80 (LWP 101681)]
[New Thread 820c97e00 (LWP 101545)]
[New Thread 81f7ec280 (LWP 101391)]
[New Thread 82062c0c0 (LWP 101943)]
[New Thread 813b780c0 (LWP 101776)]
[New Thread 81d1ca700 (LWP 101771)]
[New Thread 81e9ab180 (LWP 102266)]
[New Thread 806cc5bc0 (LWP 102050)]
[New Thread 81dd4b700 (LWP 101225)]
[New Thread 81f57c1c0 (LWP 102277)]
[New Thread 81e04aa40 (LWP 102274)]
[New Thread 80d150340 (LWP 102271)]
[New Thread 81b65d6c0 (LWP 102269)]
[New Thread 81b3fce40 (LWP 102005)]
[New Thread 81dd4b540 (LWP 102232)]
[New Thread 8093c8f80 (LWP 102226)]
[New Thread 81b4401c0 (LWP 101885)]
[New Thread 81f029fc0 (LWP 102040)]
[New Thread 81fc8ba40 (LWP 102829)]
[New Thread 81d1ca540 (LWP 102827)]
[New Thread 803852e00 (LWP 102030)]
[New Thread 81998ed40 (LWP 102032)]
[New Thread 8170ab8c0 (LWP 102159)]
[New Thread 806d5bcc0 (LWP 102026)]
[New Thread 81e04a880 (LWP 102021)]
[New Thread 8092e2dc0 (LWP 102027)]
[New Thread 81e5c56c0 (LWP 102019)]
[New Thread 815b9e5c0 (LWP 101998)]
[New Thread 81c52ba40 (LWP 101639)]
[New Thread 81bb4fa40 (LWP 100833)]
[New Thread 8070e8ac0 (LWP 101906)]
[New Thread 815426f40 (LWP 102397)]
[New Thread 806925380 (LWP 100917)]
[New Thread 809df4700 (LWP 102378)]
[New Thread 8097d7f80 (LWP 102215)]
[New Thread 8042abac0 (LWP 102289)]
[New Thread 80f15c540 (LWP 102375)]
[New Thread 81faa2840 (LWP 102211)]
[New Thread 8077c96c0 (LWP 102336)]
[New Thread 8092e2c00 (LWP 102320)]
[New Thread 80ecef900 (LWP 102137)]
[New Thread 81bb4f880 (LWP 102207)]
[New Thread 81eec80c0 (LWP 102193)]
[New Thread 819246180 (LWP 102184)]
[New Thread 81f029e00 (LWP 102168)]
[New Thread 81b3f61c0 (LWP 102166)]
[New Thread 81bb4f6c0 (LWP 102164)]
[New Thread 81f029c40 (LWP 100673)]
[New Thread 808172340 (LWP 102128)]
[New Thread 81df05d80 (LWP 101810)]
[New Thread 818ae5880 (LWP 102077)]
[New Thread 81eec7f00 (LWP 102012)]
[New Thread 81bb4f500 (LWP 101996)]
[New Thread 80dde0c80 (LWP 101918)]
[New Thread 819685100 (LWP 101535)]
[New Thread 81c52b880 (LWP 101195)]
[New Thread 81c52b6c0 (LWP 102826)]
[New Thread 81e04a6c0 (LWP 102824)]
[New Thread 80e69c640 (LWP 102820)]
[New Thread 80fe0a3c0 (LWP 102818)]
[New Thread 81a2a5a40 (LWP 102816)]
[New Thread 81d7c1400 (LWP 102815)]
[New Thread 81d1ca380 (LWP 102813)]
[New Thread 81e0436c0 (LWP 102810)]
[New Thread 81a1b2740 (LWP 102808)]
[New Thread 819deffc0 (LWP 102805)]
[New Thread 81a1bde40 (LWP 102802)]
[New Thread 818a586c0 (LWP 102715)]
[New Thread 812898040 (LWP 102343)]
[New Thread 803ebd3c0 (LWP 102340)]
[New Thread 81d845c40 (LWP 102666)]
[New Thread 81d845a80 (LWP 102634)]
[New Thread 81e891340 (LWP 102629)]
[New Thread 80b287ec0 (LWP 102327)]
[New Thread 81a4820c0 (LWP 102328)]
[New Thread 81d2ac700 (LWP 102329)]
[New Thread 81e04a500 (LWP 102606)]
[New Thread 8170a48c0 (LWP 102322)]
[New Thread 80ccbbd80 (LWP 102592)]
[New Thread 81df0cbc0 (LWP 102318)]
[New Thread 80921fa40 (LWP 102317)]
[New Thread 80778c500 (LWP 102579)]
[New Thread 8137f7340 (LWP 102568)]
[New Thread 80921f880 (LWP 100772)]
[New Thread 81dadf600 (LWP 102566)]
[New Thread 815b96940 (LWP 102564)]
[New Thread 805e0ae40 (LWP 102561)]
[New Thread 81bb4f340 (LWP 102559)]
[New Thread 81d4ac0c0 (LWP 102555)]
[New Thread 81d716400 (LWP 102297)]
[New Thread 81bb4f180 (LWP 102549)]
[New Thread 8040f0700 (LWP 102294)]
[New Thread 80ccad680 (LWP 102292)]
[New Thread 808ae61c0 (LWP 102293)]
[New Thread 80ccacdc0 (LWP 102290)]
[New Thread 81dadf440 (LWP 102534)]
[New Thread 81c99ca80 (LWP 102532)]
[New Thread 81130f340 (LWP 102530)]
[New Thread 8111e16c0 (LWP 102527)]
[New Thread 81b729a40 (LWP 102278)]
[New Thread 81cd26fc0 (LWP 102521)]
[New Thread 805559bc0 (LWP 102517)]
[New Thread 8063e1380 (LWP 101299)]
[New Thread 81cc37180 (LWP 101204)]
[New Thread 81a2a5880 (LWP 102469)]
[New Thread 81a98e700 (LWP 102514)]
[New Thread 815aa7b00 (LWP 102511)]
[New Thread 81d8458c0 (LWP 102507)]
[New Thread 81d845700 (LWP 102261)]
[New Thread 81a1bdc80 (LWP 102496)]
[New Thread 804bfb280 (LWP 102258)]
[New Thread 805f831c0 (LWP 102486)]
[New Thread 81d845540 (LWP 102484)]
[New Thread 81b729880 (LWP 102481)]
[New Thread 81d4abf00 (LWP 102479)]
[New Thread 81897ef00 (LWP 102477)]
[New Thread 8068fa700 (LWP 102472)]
[New Thread 81d4abd40 (LWP 102470)]
[New Thread 81c909a80 (LWP 102468)]
[New Thread 80921f6c0 (LWP 102233)]
[New Thread 81cc36fc0 (LWP 102458)]
[New Thread 805dad900 (LWP 102456)]
[New Thread 81d1ca1c0 (LWP 102224)]
[New Thread 81a2a56c0 (LWP 102152)]
[New Thread 817d1e540 (LWP 102440)]
[New Thread 8188e5f00 (LWP 102438)]
[New Thread 8188e5b80 (LWP 102437)]
[New Thread 81a98e540 (LWP 101764)]
[New Thread 8111e1500 (LWP 101151)]
[New Thread 80edb2740 (LWP 102197)]
[New Thread 810469900 (LWP 102191)]
[New Thread 81cc36e00 (LWP 101901)]
[New Thread 813c0d9c0 (LWP 101642)]
[New Thread 80e883440 (LWP 101817)]
[New Thread 810b2cc40 (LWP 101853)]
[New Thread 81be1cc40 (LWP 102433)]
[New Thread 805bcf980 (LWP 102219)]
[New Thread 81a2a5500 (LWP 101900)]
[New Thread 81bb4efc0 (LWP 102427)]
[New Thread 80b168180 (LWP 101897)]
[New Thread 81943a580 (LWP 100599)]
[New Thread 81d046380 (LWP 102424)]
[New Thread 81a2a5340 (LWP 100919)]
[New Thread 817f32240 (LWP 102206)]
[New Thread 80b981180 (LWP 102422)]
[New Thread 81a1bdac0 (LWP 102200)]
[New Thread 81a98e380 (LWP 102418)]
[New Thread 81c0f5f40 (LWP 102199)]
[New Thread 81b7296c0 (LWP 102194)]
[New Thread 803856b40 (LWP 102411)]
[New Thread 81afc37c0 (LWP 102188)]
[New Thread 81afc3600 (LWP 102187)]
[New Thread 81c0f5d80 (LWP 102404)]
[New Thread 814621e80 (LWP 102172)]
[New Thread 80fb3fe00 (LWP 102186)]
[New Thread 81a98e1c0 (LWP 102175)]
[New Thread 81558bd80 (LWP 102183)]
[New Thread 80cd50f00 (LWP 102392)]
[New Thread 81ca460c0 (LWP 102390)]
[New Thread 8111e1340 (LWP 102388)]
[New Thread 808500500 (LWP 102163)]
[New Thread 808b5d8c0 (LWP 102162)]
[New Thread 80b28c140 (LWP 102156)]
[New Thread 81afc3440 (LWP 101755)]
[New Thread 819441200 (LWP 101601)]
[New Thread 817f8b240 (LWP 101005)]
[New Thread 80fe0a200 (LWP 101540)]
[New Thread 815aa75c0 (LWP 101421)]
[New Thread 8147ee940 (LWP 101396)]
[New Thread 818ffe400 (LWP 100571)]
[New Thread 818ffe240 (LWP 101307)]
[New Thread 8055571c0 (LWP 102140)]
[New Thread 80974f380 (LWP 102370)]
[New Thread 81c07ad80 (LWP 102135)]
[New Thread 81addc0c0 (LWP 102359)]
[New Thread 80f703ac0 (LWP 102324)]
[New Thread 8101768c0 (LWP 102123)]
[New Thread 80b393880 (LWP 102113)]
[New Thread 81b484380 (LWP 100945)]
[New Thread 81a1bd900 (LWP 102315)]
[New Thread 80544fe80 (LWP 102071)]
[New Thread 80f703900 (LWP 102288)]
[New Thread 806c381c0 (LWP 102047)]
[New Thread 81c008d80 (LWP 102229)]
[New Thread 81be9ee00 (LWP 102217)]
[New Thread 810d32e00 (LWP 102120)]
[New Thread 804a44c40 (LWP 102002)]
[New Thread 818102740 (LWP 102118)]
[New Thread 806788c40 (LWP 101988)]
[New Thread 81addbf00 (LWP 101983)]
[New Thread 81be1ce00 (LWP 102115)]
[New Thread 81afc3280 (LWP 102100)]
[New Thread 8109e9fc0 (LWP 101839)]
[New Thread 81afc30c0 (LWP 102056)]
[New Thread 818f87400 (LWP 101801)]
[New Thread 81addbd40 (LWP 101814)]
[New Thread 811316fc0 (LWP 101822)]
[New Thread 8047a3380 (LWP 101957)]
[New Thread 81b664880 (LWP 101576)]
[New Thread 81ba1ffc0 (LWP 101358)]
[New Thread 81a2a5180 (LWP 101274)]
[New Thread 8081361c0 (LWP 100807)]
[New Thread 819df8500 (LWP 102065)]
[New Thread 806214900 (LWP 102062)]
[New Thread 80544d800 (LWP 102061)]
[New Thread 81ad23d40 (LWP 100711)]
[New Thread 81a1bd740 (LWP 102052)]
[New Thread 815bf45c0 (LWP 102048)]
[New Thread 81b4841c0 (LWP 101792)]
[New Thread 81b6646c0 (LWP 102006)]
[New Thread 807923a80 (LWP 102004)]
[New Thread 81ab1fd80 (LWP 101975)]
[New Thread 811f4b440 (LWP 100746)]
[New Thread 81b446e40 (LWP 101907)]
[New Thread 81b664500 (LWP 102366)]
[New Thread 8079238c0 (LWP 102362)]
[New Thread 81318cf40 (LWP 102104)]
[New Thread 8134c5580 (LWP 101850)]
[New Thread 815f2e8c0 (LWP 101860)]
[New Thread 819388fc0 (LWP 101861)]
[New Thread 819df8340 (LWP 101873)]
[New Thread 80b7b41c0 (LWP 101857)]
[New Thread 80554b580 (LWP 102080)]
[New Thread 80ea0da80 (LWP 100595)]
[New Thread 8176b4e40 (LWP 101537)]
[New Thread 805c59cc0 (LWP 102181)]
[New Thread 805557fc0 (LWP 102179)]
[New Thread 80ccb44c0 (LWP 102161)]
[New Thread 81ad23b80 (LWP 102158)]
[New Thread 806214740 (LWP 101989)]
[New Thread 80d010700 (LWP 102081)]
[New Thread 819df8180 (LWP 102127)]
[New Thread 819df7fc0 (LWP 102111)]
[New Thread 81ab1fbc0 (LWP 102107)]
[New Thread 815b9e940 (LWP 101724)]
[New Thread 8192d0fc0 (LWP 102025)]
[New Thread 8066d7540 (LWP 101982)]
[New Thread 805bd0940 (LWP 101963)]
[New Thread 8197a3d80 (LWP 101932)]
[New Thread 8040f6e40 (LWP 101927)]
[New Thread 819e53e00 (LWP 101922)]
[New Thread 80660a380 (LWP 101911)]
[New Thread 806c3c600 (LWP 100616)]
[New Thread 81918c8c0 (LWP 101788)]
[New Thread 806788540 (LWP 101637)]
[New Thread 8111e1180 (LWP 101237)]
[New Thread 818ae56c0 (LWP 101397)]
[New Thread 81ac2df00 (LWP 101329)]
[New Thread 81aa28bc0 (LWP 100782)]
[New Thread 818f87240 (LWP 102365)]
[New Thread 81ab1fa00 (LWP 102363)]
[New Thread 819795f40 (LWP 102361)]
[New Thread 80533ba80 (LWP 101348)]
[New Thread 81a1bd580 (LWP 102357)]
[New Thread 80bcf82c0 (LWP 102356)]
[New Thread 819a08280 (LWP 102354)]
[New Thread 80660a1c0 (LWP 102352)]
[New Thread 819c05180 (LWP 102350)]
[New Thread 8051c7940 (LWP 102348)]
[New Thread 819df7e00 (LWP 102346)]
[New Thread 819245fc0 (LWP 102344)]
[New Thread 80b528e40 (LWP 102133)]
[New Thread 8155e3f40 (LWP 102132)]
[New Thread 807b6b380 (LWP 102129)]
[New Thread 818a58340 (LWP 102130)]
[New Thread 81a0c1740 (LWP 102335)]
[New Thread 81a0c1580 (LWP 102333)]
[New Thread 807f1d540 (LWP 102331)]
[New Thread 81a0c13c0 (LWP 102117)]
[New Thread 806047e40 (LWP 102116)]
[New Thread 80a11f840 (LWP 102323)]
[New Thread 80554ae80 (LWP 102109)]
[New Thread 81a489f00 (LWP 102319)]
[New Thread 803f39700 (LWP 102313)]
[New Thread 81a205340 (LWP 102103)]
[New Thread 80e570800 (LWP 102105)]
[New Thread 818f08080 (LWP 102310)]
[New Thread 80bd6cf40 (LWP 102307)]
[New Thread 81a205180 (LWP 100691)]
[New Thread 80ccb4680 (LWP 102304)]
[New Thread 815eb28c0 (LWP 101201)]
[New Thread 817d25380 (LWP 102301)]
[New Thread 806047c80 (LWP 102299)]
[New Thread 819a080c0 (LWP 102082)]
[New Thread 81995d280 (LWP 102295)]
[New Thread 819245e00 (LWP 102287)]
[New Thread 811170340 (LWP 102041)]
[New Thread 819245c40 (LWP 102286)]
[New Thread 818f87080 (LWP 102075)]
[New Thread 818636fc0 (LWP 102285)]
[New Thread 80b28ac40 (LWP 102283)]
[New Thread 81998f0c0 (LWP 102281)]
[New Thread 8199640c0 (LWP 102279)]
[New Thread 819e53c40 (LWP 102136)]
[New Thread 8190a78c0 (LWP 102275)]
[New Thread 809856e00 (LWP 101287)]
[New Thread 811170180 (LWP 102272)]
[New Thread 8185f3180 (LWP 102270)]
[New Thread 804382a80 (LWP 102267)]
[New Thread 806047ac0 (LWP 101089)]
[New Thread 805b155c0 (LWP 101371)]
[New Thread 80ce66d40 (LWP 102265)]
[New Thread 812897e80 (LWP 102263)]
[New Thread 8050607c0 (LWP 102260)]
[New Thread 81998ef00 (LWP 102259)]
[New Thread 819c77fc0 (LWP 102037)]
[New Thread 814ed6700 (LWP 102255)]
[New Thread 8155dbf40 (LWP 102253)]
[New Thread 819a07f00 (LWP 102251)]
[New Thread 818f86ec0 (LWP 102249)]
[New Thread 814b4e100 (LWP 102246)]
[New Thread 80b7b4540 (LWP 102245)]
[New Thread 806da7640 (LWP 102240)]
[New Thread 819c04fc0 (LWP 102238)]
[New Thread 8176b4c80 (LWP 102236)]
[New Thread 818ae5500 (LWP 102234)]
[New Thread 815c2b540 (LWP 102014)]
[New Thread 810c33c40 (LWP 102015)]
[New Thread 80400a700 (LWP 102013)]
[New Thread 81116ffc0 (LWP 102227)]
[New Thread 819a07d40 (LWP 102225)]
[New Thread 80554a5c0 (LWP 101968)]
[New Thread 808171fc0 (LWP 101924)]
[New Thread 80b3d6dc0 (LWP 101919)]
[New Thread 8130f2100 (LWP 101893)]
[New Thread 81897ed40 (LWP 101880)]
[New Thread 8077d0a40 (LWP 100903)]
[New Thread 80b28ca00 (LWP 101728)]
[New Thread 81982e440 (LWP 102221)]
[New Thread 80ea0d8c0 (LWP 101993)]
[New Thread 818a58500 (LWP 102216)]
[New Thread 8109f8340 (LWP 102213)]
[New Thread 818636e00 (LWP 102214)]
[New Thread 8069088c0 (LWP 100745)]
[New Thread 81107e180 (LWP 101159)]
[New Thread 8194413c0 (LWP 102210)]
[New Thread 8190a7700 (LWP 100609)]
[New Thread 80b3106c0 (LWP 101985)]
[New Thread 8148a0e00 (LWP 102208)]
[New Thread 80544db80 (LWP 101977)]
[New Thread 8182d7100 (LWP 101976)]
[New Thread 8188e5d40 (LWP 102203)]
[New Thread 808b5da80 (LWP 101970)]
[New Thread 804ad6680 (LWP 101972)]
[New Thread 8136d1500 (LWP 101971)]
[New Thread 813e28e40 (LWP 102195)]
[New Thread 814621cc0 (LWP 101964)]
[New Thread 80e76bd40 (LWP 101962)]
[New Thread 805ca3200 (LWP 102189)]
[New Thread 804b32a00 (LWP 101935)]
[New Thread 8113176c0 (LWP 101959)]
[New Thread 817f32400 (LWP 101933)]
[New Thread 80a2f6c40 (LWP 101947)]
[New Thread 81016f8c0 (LWP 102177)]
[New Thread 81901c700 (LWP 102176)]
[New Thread 805b14d00 (LWP 102174)]
[New Thread 818636c40 (LWP 102171)]
[New Thread 807180a00 (LWP 101915)]
[New Thread 8130f22c0 (LWP 101917)]
[New Thread 806ea3e40 (LWP 101912)]
[New Thread 80b2d18c0 (LWP 101909)]
[New Thread 80b289c80 (LWP 101832)]
[New Thread 80a590500 (LWP 101215)]
[New Thread 81842f500 (LWP 101189)]
[New Thread 807af1540 (LWP 101182)]
[New Thread 80b28a1c0 (LWP 101161)]
[New Thread 80b289e40 (LWP 101156)]
[New Thread 803853a40 (LWP 101075)]
[New Thread 81842f340 (LWP 100946)]
[New Thread 80e7c30c0 (LWP 102149)]
[New Thread 80e7c2f00 (LWP 101879)]
[New Thread 805d58740 (LWP 102126)]
[New Thread 804bf9680 (LWP 101868)]
[New Thread 8038559c0 (LWP 102079)]
[New Thread 8147eeb00 (LWP 101856)]
[New Thread 8182d6f40 (LWP 101866)]
[New Thread 80cd50d40 (LWP 101869)]
[New Thread 805546f80 (LWP 101867)]
[New Thread 80a07e500 (LWP 102073)]
[New Thread 807340380 (LWP 101800)]
[New Thread 80cd50b80 (LWP 102059)]
[New Thread 805d58580 (LWP 101758)]
[New Thread 81842f180 (LWP 101471)]
[New Thread 80e784d40 (LWP 100702)]
[New Thread 8038560c0 (LWP 101940)]
[New Thread 805547a00 (LWP 101931)]
[New Thread 813e25b80 (LWP 101630)]
[New Thread 805558a40 (LWP 101620)]
[New Thread 80f15c8c0 (LWP 101619)]
[New Thread 805d583c0 (LWP 101930)]
[New Thread 80cd509c0 (LWP 101914)]
[New Thread 80bcf7f40 (LWP 101714)]
[New Thread 80bc7f480 (LWP 101426)]
[New Thread 80fb9ee00 (LWP 101388)]
[New Thread 818102580 (LWP 101369)]
[New Thread 80e76bb80 (LWP 101676)]
[New Thread 8172f0740 (LWP 101234)]
[New Thread 8158ddfc0 (LWP 101211)]
[New Thread 805545700 (LWP 100932)]
[New Thread 80b9c3540 (LWP 100713)]
[New Thread 805ca3040 (LWP 101186)]
[New Thread 815d08380 (LWP 101430)]
[New Thread 816eecf40 (LWP 101667)]
[New Thread 8070e4300 (LWP 101844)]
[New Thread 80d150180 (LWP 101811)]
[New Thread 80fb9efc0 (LWP 101777)]
[New Thread 80d14f540 (LWP 101708)]
[New Thread 810325180 (LWP 101531)]
[New Thread 8136d1340 (LWP 101910)]
[New Thread 810302340 (LWP 101789)]
[New Thread 80cd50800 (LWP 101575)]
[New Thread 811317500 (LWP 101554)]
[New Thread 816eecd80 (LWP 102150)]
[New Thread 80cd50640 (LWP 102145)]
[New Thread 8077c9500 (LWP 101967)]
[New Thread 80b289200 (LWP 101896)]
[New Thread 805ca2e80 (LWP 101954)]
[New Thread 80d14ffc0 (LWP 101953)]
[New Thread 811317340 (LWP 101876)]
[New Thread 810399500 (LWP 101875)]
[New Thread 80b28a540 (LWP 101923)]
[New Thread 815a2c940 (LWP 101908)]
[New Thread 810d32c40 (LWP 101660)]
[New Thread 80bc0ee40 (LWP 101903)]
[New Thread 80e76b9c0 (LWP 101902)]
[New Thread 80ee30540 (LWP 100341)]
[New Thread 81398e280 (LWP 101872)]
[New Thread 813c0dd40 (LWP 101405)]
[New Thread 80e76b800 (LWP 101836)]
[New Thread 80b288b00 (LWP 101855)]
[New Thread 80cd50480 (LWP 101849)]
[New Thread 810f37040 (LWP 101794)]
[New Thread 80b570c80 (LWP 101741)]
[New Thread 80d010540 (LWP 101410)]
[New Thread 8170ab700 (LWP 101383)]
[New Thread 805175d00 (LWP 101366)]
[New Thread 805c59b00 (LWP 101345)]
[New Thread 8048abc80 (LWP 100576)]
[New Thread 810d32a80 (LWP 101596)]
[New Thread 805b15780 (LWP 101362)]
[New Thread 80ef1fe40 (LWP 101250)]
[New Thread 80555ab80 (LWP 101176)]
[New Thread 80d010380 (LWP 100934)]
[New Thread 8163ccf40 (LWP 101222)]
[New Thread 803852700 (LWP 101094)]
[New Thread 80f15c700 (LWP 100954)]
[New Thread 80b7b4380 (LWP 100768)]
[New Thread 803f82380 (LWP 101404)]
[New Thread 80554a400 (LWP 102153)]
[New Thread 80d14f8c0 (LWP 102151)]
[New Thread 8158dde00 (LWP 102148)]
[New Thread 80bda0100 (LWP 102146)]
[New Thread 810597900 (LWP 101265)]
[New Thread 815f2e700 (LWP 102142)]
[New Thread 8158ddc40 (LWP 102068)]
[New Thread 8077d0c00 (LWP 102138)]
[New Thread 80e16f480 (LWP 101785)]
[New Thread 811317180 (LWP 102134)]
[New Thread 809715c40 (LWP 102125)]
[New Thread 8055e0800 (LWP 101736)]
[New Thread 80b7b4700 (LWP 102122)]
[New Thread 807e42700 (LWP 101769)]
[New Thread 8163ccd80 (LWP 101752)]
[New Thread 8051c7780 (LWP 101738)]
[New Thread 80739c700 (LWP 102121)]
[New Thread 80b44f700 (LWP 102119)]
[New Thread 80dde0e40 (LWP 101713)]
[New Thread 816658740 (LWP 101695)]
[New Thread 80544f780 (LWP 101691)]
[New Thread 80b288780 (LWP 101378)]
[New Thread 80f2bd740 (LWP 102110)]
[New Thread 80bdcb2c0 (LWP 101689)]
[New Thread 8051c75c0 (LWP 102106)]
[New Thread 80b28c840 (LWP 102102)]
[New Thread 80eec9380 (LWP 102101)]
[New Thread 809c75700 (LWP 102099)]
[New Thread 80bdcb100 (LWP 102095)]
[New Thread 808500340 (LWP 101321)]
[New Thread 8051c7400 (LWP 102093)]
[New Thread 8051c7080 (LWP 102091)]
[New Thread 8063aa580 (LWP 102089)]
[New Thread 8134c5740 (LWP 102088)]
[New Thread 811efcd40 (LWP 102085)]
[New Thread 80bd9ff40 (LWP 102084)]
[New Thread 80b28ae00 (LWP 101942)]
[New Thread 80ce5ff00 (LWP 101941)]
[New Thread 80b28a8c0 (LWP 101607)]
[New Thread 815b9e780 (LWP 102078)]
[New Thread 80ccb4a00 (LWP 102070)]
[New Thread 8055497c0 (LWP 101939)]
[New Thread 809990fc0 (LWP 101937)]
[New Thread 805060980 (LWP 101938)]
[New Thread 805547d80 (LWP 102069)]
[New Thread 80ccb4840 (LWP 102067)]
[New Thread 80a11d700 (LWP 102064)]
[New Thread 8134cc900 (LWP 102058)]
[New Thread 8098b7a80 (LWP 101848)]
[New Thread 80a450a80 (LWP 101435)]
[New Thread 80a445540 (LWP 102057)]
[New Thread 8052f6700 (LWP 102055)]
[New Thread 80544cd80 (LWP 102053)]
[New Thread 8134cc740 (LWP 102051)]
[New Thread 803f398c0 (LWP 102049)]
[New Thread 808d27340 (LWP 101671)]
[New Thread 815aa7940 (LWP 102023)]
[New Thread 80705cf00 (LWP 102009)]
[New Thread 8134cc580 (LWP 102044)]
[New Thread 80ccad4c0 (LWP 102042)]
[New Thread 804a07c00 (LWP 100830)]
[New Thread 806daa740 (LWP 102038)]
[New Thread 803ebdac0 (LWP 102036)]
[New Thread 815aa7780 (LWP 102033)]
[New Thread 8155e3d80 (LWP 102031)]
[New Thread 80ccad300 (LWP 102029)]
[New Thread 80a445700 (LWP 101889)]
[New Thread 806961540 (LWP 102024)]
[New Thread 805e048c0 (LWP 102022)]
[New Thread 80ccad140 (LWP 102020)]
[New Thread 80ccacf80 (LWP 102018)]
[New Thread 80678bd40 (LWP 102016)]
[New Thread 80a2f6e00 (LWP 101854)]
[New Thread 814ed6a80 (LWP 102011)]
[New Thread 8155dbd80 (LWP 102010)]
[New Thread 80dddb340 (LWP 102007)]
[New Thread 80b28c300 (LWP 102003)]
[New Thread 8109e98c0 (LWP 101858)]
[New Thread 80b167fc0 (LWP 102000)]
[New Thread 804a45dc0 (LWP 101999)]
[New Thread 80bcf8100 (LWP 101995)]
[New Thread 804b355c0 (LWP 101881)]
[New Thread 8109f8500 (LWP 101882)]
[New Thread 8109f1340 (LWP 101991)]
[New Thread 8155dbbc0 (LWP 100737)]
[New Thread 804bfb0c0 (LWP 100519)]
[New Thread 80b7f88c0 (LWP 101698)]
[New Thread 80692b900 (LWP 101587)]
[New Thread 80705c9c0 (LWP 101500)]
[New Thread 814705cc0 (LWP 101034)]
[New Thread 806c3bd40 (LWP 100759)]
[New Thread 80b3d6c00 (LWP 101411)]
[New Thread 804bfaf00 (LWP 101987)]
[New Thread 80842be40 (LWP 101981)]
[New Thread 804f7a380 (LWP 101408)]
[New Thread 808500180 (LWP 101401)]
[New Thread 814ed68c0 (LWP 101980)]
[New Thread 808d56a40 (LWP 101400)]
[New Thread 80bc0ad80 (LWP 101385)]
[New Thread 81191fac0 (LWP 101973)]
[New Thread 815243100 (LWP 101966)]
[New Thread 806f66900 (LWP 101381)]
[New Thread 804bfad40 (LWP 101380)]
[New Thread 804bfab80 (LWP 101965)]
[New Thread 80b310500 (LWP 101952)]
[New Thread 814b4e2c0 (LWP 101364)]
[New Thread 804ad9e80 (LWP 101951)]
[New Thread 805aceec0 (LWP 101357)]
[New Thread 8100ef380 (LWP 101353)]
[New Thread 80a083900 (LWP 101950)]
[New Thread 80b310340 (LWP 101936)]
[New Thread 80b310180 (LWP 101934)]
[New Thread 804bfa9c0 (LWP 101346)]
[New Thread 804b32f40 (LWP 101343)]
[New Thread 803852c40 (LWP 101341)]
[New Thread 80544fb00 (LWP 101335)]
[New Thread 805558f80 (LWP 101337)]
[New Thread 803854d80 (LWP 101333)]
[New Thread 804bf8180 (LWP 101318)]
[New Thread 804b32d80 (LWP 101314)]
[New Thread 810b2ce00 (LWP 101272)]
[New Thread 80ce5e300 (LWP 100592)]
[New Thread 807128a80 (LWP 101084)]
[New Thread 8047a31c0 (LWP 100999)]
[New Thread 80b289580 (LWP 100912)]
[New Thread 808172180 (LWP 101887)]
[New Thread 80bc7f2c0 (LWP 101300)]
[New Thread 80bc7f100 (LWP 101870)]
[New Thread 80bc7ef40 (LWP 101298)]
[New Thread 804b32bc0 (LWP 101288)]
[New Thread 814688e80 (LWP 100956)]
[New Thread 807340540 (LWP 101835)]
[New Thread 80b288080 (LWP 101290)]
[New Thread 805545540 (LWP 101826)]
[New Thread 814688cc0 (LWP 101275)]
[New Thread 80bc7ed80 (LWP 101781)]
[New Thread 80660e0c0 (LWP 101270)]
[New Thread 80554bc80 (LWP 101744)]
[New Thread 805bcc500 (LWP 101701)]
[New Thread 80b2d1700 (LWP 101690)]
[New Thread 810b2c8c0 (LWP 101651)]
[New Thread 80554b200 (LWP 101223)]
[New Thread 8055482c0 (LWP 101230)]
[New Thread 8038552c0 (LWP 101229)]
[New Thread 803e2fac0 (LWP 101228)]
[New Thread 813e28c80 (LWP 101573)]
[New Thread 808d56880 (LWP 101546)]
[New Thread 808d566c0 (LWP 101190)]
[New Thread 804ad6300 (LWP 101451)]
[New Thread 805d56600 (LWP 101339)]
[New Thread 803f3f580 (LWP 101179)]
[New Thread 804ad6140 (LWP 101175)]
[New Thread 813e28ac0 (LWP 101332)]
[New Thread 8100ef8c0 (LWP 101164)]
[New Thread 805172340 (LWP 101178)]
[New Thread 80813ce40 (LWP 101330)]
[New Thread 8143c7640 (LWP 100951)]
[New Thread 8100ef540 (LWP 100939)]
[New Thread 814447480 (LWP 100695)]
[New Thread 8055586c0 (LWP 101269)]
[New Thread 813c52d40 (LWP 101152)]
[New Thread 80a11ea40 (LWP 101210)]
[New Thread 80739cfc0 (LWP 101532)]
[New Thread 80717efc0 (LWP 101360)]
[New Thread 80a868700 (LWP 101442)]
[New Thread 8097d84c0 (LWP 101354)]
[New Thread 806da5500 (LWP 101865)]
[New Thread 806da56c0 (LWP 101859)]
[New Thread 806da5880 (LWP 101843)]
[New Thread 806da5a40 (LWP 101837)]
[New Thread 806da5dc0 (LWP 101808)]
[New Thread 806da5f80 (LWP 101783)]
[New Thread 806da6140 (LWP 101761)]
[New Thread 806da6300 (LWP 101723)]
[New Thread 806da64c0 (LWP 101709)]
[New Thread 806da6680 (LWP 101685)]
[New Thread 806da6840 (LWP 101673)]
[New Thread 806da6a00 (LWP 101634)]
[New Thread 806da6bc0 (LWP 101622)]
[New Thread 806da6d80 (LWP 101608)]
[New Thread 806da6f40 (LWP 101600)]
[New Thread 806da7100 (LWP 101599)]
[New Thread 806da72c0 (LWP 101569)]
[New Thread 806da7800 (LWP 101547)]
[New Thread 806da79c0 (LWP 101524)]
[New Thread 806da7b80 (LWP 101516)]
[New Thread 806da7d40 (LWP 101514)]
[New Thread 806da7f00 (LWP 101468)]
[New Thread 806da8440 (LWP 101416)]
[New Thread 806da8600 (LWP 101414)]
[New Thread 806da87c0 (LWP 101395)]
[New Thread 806da8980 (LWP 101393)]
[New Thread 806da8b40 (LWP 101377)]
[New Thread 8084ffa80 (LWP 101327)]
[New Thread 8052fcc80 (LWP 101323)]
[New Thread 804ad4fc0 (LWP 101311)]
[New Thread 8055e1b40 (LWP 101303)]
[New Thread 80bda3040 (LWP 100685)]
[New Thread 80b4db380 (LWP 101280)]
[New Thread 8070e8c80 (LWP 101208)]
[New Thread 8068fa8c0 (LWP 101273)]
[New Thread 805545e00 (LWP 101244)]
[New Thread 80973f700 (LWP 101218)]
[New Thread 808d57d80 (LWP 101148)]
[New Thread 80b28c680 (LWP 101162)]
[New Thread 80b5221c0 (LWP 101158)]
[New Thread 8087bde00 (LWP 101079)]
[New Thread 80c8c5100 (LWP 101003)]
[New Thread 80bdcde80 (LWP 101213)]
[New Thread 80b28b880 (LWP 101011)]
[New Thread 804e7f540 (LWP 101000)]
[New Thread 80bd26580 (LWP 100751)]
[New Thread 804f63e40 (LWP 100943)]
[New Thread 803f83180 (LWP 101823)]
[New Thread 80978b340 (LWP 101525)]
[New Thread 805450200 (LWP 101961)]
[New Thread 8055474c0 (LWP 101960)]
[New Thread 80b28bc00 (LWP 101958)]
[New Thread 80bc0a140 (LWP 101955)]
[New Thread 80582ee40 (LWP 101390)]
[New Thread 80b289ac0 (LWP 101928)]
[New Thread 80544d640 (LWP 101320)]
[New Thread 805dab280 (LWP 101384)]
[New Thread 80bc09f80 (LWP 101921)]
[New Thread 80544e0c0 (LWP 101895)]
[New Thread 804a06c40 (LWP 100625)]
[New Thread 80b287d00 (LWP 101894)]
[New Thread 80ef191c0 (LWP 101890)]
[New Thread 80876b540 (LWP 101883)]
[New Thread 807340700 (LWP 101871)]
[New Thread 80c081c00 (LWP 101863)]
[New Thread 80544eb40 (LWP 101805)]
[New Thread 80be8e700 (LWP 101361)]
[New Thread 806ea3c80 (LWP 101798)]
[New Thread 80a07f140 (LWP 101356)]
[New Thread 804f7aa80 (LWP 101787)]
[New Thread 80876ba80 (LWP 101745)]
[New Thread 804e85c80 (LWP 101313)]
[New Thread 8096ddc40 (LWP 101729)]
[New Thread 80f34c580 (LWP 101325)]
[New Thread 80ecafac0 (LWP 101665)]
[New Thread 807f58540 (LWP 101652)]
[New Thread 80705d440 (LWP 101949)]
[New Thread 80b30f8c0 (LWP 101944)]
[New Thread 81191f580 (LWP 101643)]
[New Thread 809c758c0 (LWP 101751)]
[New Thread 80a11fa00 (LWP 101790)]
[New Thread 805174800 (LWP 101946)]
[New Thread 80b30f700 (LWP 101530)]
[New Thread 80b28b340 (LWP 101631)]
[New Thread 8097d7a40 (LWP 101284)]
[New Thread 80974fe00 (LWP 101515)]
[New Thread 80973fe00 (LWP 101548)]
[New Thread 804a07dc0 (LWP 101557)]
[New Thread 805450e40 (LWP 101534)]
[New Thread 80a868540 (LWP 101904)]
[New Thread 80ce5fd40 (LWP 101899)]
[New Thread 80554a940 (LWP 101766)]
[New Thread 80b30f540 (LWP 101285)]
[New Thread 80b49b540 (LWP 101719)]
[New Thread 80544d100 (LWP 101874)]
[New Thread 80472d8c0 (LWP 101386)]
[New Thread 80554a080 (LWP 101310)]
[New Thread 8098f98c0 (LWP 101615)]
[New Thread 8047a3540 (LWP 101564)]
[New Thread 804b35cc0 (LWP 101589)]
[New Thread 8049ae1c0 (LWP 101319)]
[New Thread 805559d80 (LWP 101550)]
[New Thread 80631c740 (LWP 101884)]
[New Thread 805549440 (LWP 101142)]
[New Thread 804a45f80 (LWP 100636)]
[New Thread 80ce5fb80 (LWP 100582)]
[New Thread 808d27180 (LWP 101926)]
[New Thread 8097d76c0 (LWP 101929)]
[New Thread 8097d7500 (LWP 101920)]
[New Thread 807b72540 (LWP 101627)]
[New Thread 80c0f7bc0 (LWP 101852)]
[New Thread 806da8d00 (LWP 101851)]
[New Thread 806da8ec0 (LWP 101847)]
[New Thread 80c082d80 (LWP 101846)]
[New Thread 806da9080 (LWP 101845)]
[New Thread 806da9240 (LWP 101842)]
[New Thread 80a11fbc0 (LWP 101841)]
[New Thread 806da9400 (LWP 101840)]
[New Thread 806da95c0 (LWP 101838)]
[New Thread 80a10a840 (LWP 101834)]
[New Thread 806da9780 (LWP 101833)]
[New Thread 806da9940 (LWP 101831)]
[New Thread 80a07f300 (LWP 101830)]
[New Thread 806da9b00 (LWP 101829)]
[New Thread 806da9cc0 (LWP 101828)]
[New Thread 8077d0dc0 (LWP 101825)]
[New Thread 806da9e80 (LWP 101824)]
[New Thread 806daa040 (LWP 101821)]
[New Thread 8077c9880 (LWP 101820)]
[New Thread 806daa200 (LWP 101818)]
[New Thread 806daa3c0 (LWP 101815)]
[New Thread 806c3c980 (LWP 101813)]
[New Thread 806c3cb40 (LWP 101812)]
[New Thread 806c3cd00 (LWP 101809)]
[New Thread 80bf4a700 (LWP 101807)]
[New Thread 806c3cec0 (LWP 101806)]
[New Thread 806c3d080 (LWP 101804)]
[New Thread 806c3d240 (LWP 101803)]
[New Thread 806c3d400 (LWP 101799)]
[New Thread 80be8e8c0 (LWP 101797)]
[New Thread 806c3d5c0 (LWP 101795)]
[New Thread 806c3d780 (LWP 101793)]
[New Thread 806c3d940 (LWP 101791)]
[New Thread 809c75a80 (LWP 101603)]
[New Thread 806c3db00 (LWP 101602)]
[New Thread 806c3dcc0 (LWP 101597)]
[New Thread 806c3de80 (LWP 101595)]
[New Thread 806c3e040 (LWP 101594)]
[New Thread 806c3e200 (LWP 101786)]
[New Thread 806c3e3c0 (LWP 101784)]
[New Thread 806c3e580 (LWP 101782)]
[New Thread 809df48c0 (LWP 101780)]
[New Thread 806c3e740 (LWP 101778)]
[New Thread 806c3e900 (LWP 101775)]
[New Thread 806fb8700 (LWP 101774)]
[New Thread 806c3eac0 (LWP 101773)]
[New Thread 806c3ec80 (LWP 101770)]
[New Thread 806f86540 (LWP 101768)]
[New Thread 806c3ee40 (LWP 101767)]
[New Thread 806cc31c0 (LWP 101763)]
[New Thread 80ecefac0 (LWP 101760)]
[New Thread 806cc3380 (LWP 101759)]
[New Thread 806cc3540 (LWP 101756)]
[New Thread 80ecafc80 (LWP 101754)]
[New Thread 806cc3700 (LWP 101753)]
[New Thread 806cc38c0 (LWP 101749)]
[New Thread 80b570e40 (LWP 101748)]
[New Thread 806cc3e00 (LWP 101747)]
[New Thread 806cc3fc0 (LWP 101746)]
[New Thread 806cc4180 (LWP 101742)]
[New Thread 806cc4340 (LWP 101740)]
[New Thread 8042abc80 (LWP 101735)]
[New Thread 806cc4500 (LWP 101732)]
[New Thread 80b56a1c0 (LWP 101731)]
[New Thread 806cc46c0 (LWP 101730)]
[New Thread 806cc4880 (LWP 101727)]
[New Thread 80b522380 (LWP 101726)]
[New Thread 806cc4a40 (LWP 101725)]
[New Thread 806cc4c00 (LWP 101722)]
[New Thread 80b4db540 (LWP 101721)]
[New Thread 806cc4dc0 (LWP 101720)]
[New Thread 806cc4f80 (LWP 101718)]
[New Thread 80b49b700 (LWP 101716)]
[New Thread 806cc5140 (LWP 101715)]
[New Thread 806cc5300 (LWP 101712)]
[New Thread 80b44f8c0 (LWP 101711)]
[New Thread 806cc54c0 (LWP 101710)]
[New Thread 806cc5680 (LWP 101707)]
[New Thread 8098f9a80 (LWP 101706)]
[New Thread 806cc5840 (LWP 101705)]
[New Thread 806cc5a00 (LWP 101702)]
[New Thread 8098b7c40 (LWP 101700)]
[New Thread 806cc5d80 (LWP 101699)]
[New Thread 806cc5f40 (LWP 101697)]
[New Thread 807264e00 (LWP 101694)]
[New Thread 806cc6100 (LWP 101693)]
[New Thread 806cc62c0 (LWP 101688)]
[New Thread 809856fc0 (LWP 101687)]
[New Thread 806cc6480 (LWP 101686)]
[New Thread 806cc6640 (LWP 101683)]
[New Thread 806cc6800 (LWP 101679)]
[New Thread 809991180 (LWP 101678)]
[New Thread 806cc69c0 (LWP 101677)]
[New Thread 806cc6b80 (LWP 101675)]
[New Thread 806cc6d40 (LWP 101672)]
[New Thread 8087bdfc0 (LWP 101669)]
[New Thread 806cc6f00 (LWP 101668)]
[New Thread 806cc7280 (LWP 101666)]
[New Thread 80876bfc0 (LWP 101664)]
[New Thread 806cc7980 (LWP 101662)]
[New Thread 806cc7b40 (LWP 101659)]
[New Thread 80739c8c0 (LWP 101654)]
[New Thread 806cc7d00 (LWP 101653)]
[New Thread 806cc7ec0 (LWP 101650)]
[New Thread 8073408c0 (LWP 101649)]
[New Thread 806cc8080 (LWP 101648)]
[New Thread 806cc8240 (LWP 101644)]
[New Thread 8068faa80 (LWP 101641)]
[New Thread 808d57f40 (LWP 101640)]
[New Thread 808d58100 (LWP 101636)]
[New Thread 80ea0dc40 (LWP 101633)]
[New Thread 808d58480 (LWP 101632)]
[New Thread 808d58640 (LWP 101628)]
[New Thread 806961c40 (LWP 101625)]
[New Thread 808d58800 (LWP 101623)]
[New Thread 808d589c0 (LWP 101621)]
[New Thread 806925540 (LWP 101618)]
[New Thread 808d58b80 (LWP 101617)]
[New Thread 808d58d40 (LWP 101614)]
[New Thread 804f7ac40 (LWP 101613)]
[New Thread 80b286b80 (LWP 101612)]
[New Thread 80b286f00 (LWP 101606)]
[New Thread 80b2870c0 (LWP 101605)]
[New Thread 80b287280 (LWP 101604)]
[New Thread 805f83380 (LWP 101556)]
[New Thread 80b287440 (LWP 101555)]
[New Thread 804e85e40 (LWP 101553)]
[New Thread 80b287600 (LWP 101552)]
[New Thread 80b287980 (LWP 101551)]
[New Thread 804183a80 (LWP 101541)]
[New Thread 804ad5a40 (LWP 101538)]
[New Thread 805557c40 (LWP 101536)]
[New Thread 805558c00 (LWP 101533)]
[New Thread 803f828c0 (LWP 101529)]
[New Thread 808d58f00 (LWP 101528)]
[New Thread 804b35b00 (LWP 101526)]
[New Thread 80554b900 (LWP 101523)]
[New Thread 803853dc0 (LWP 101520)]
[New Thread 80b2869c0 (LWP 101517)]
[New Thread 80554acc0 (LWP 101506)]
[New Thread 806da80c0 (LWP 101503)]
[New Thread 805548f00 (LWP 101473)]
[New Thread 805176240 (LWP 101470)]
[New Thread 8049ae380 (LWP 101469)]
[New Thread 805549ec0 (LWP 101458)]
[New Thread 8084ff1c0 (LWP 101456)]
[New Thread 803633b00 (LWP 101455)]
[New Thread 80739c540 (LWP 101449)]
[New Thread 807791ac0 (LWP 101448)]
[New Thread 805f2fe40 (LWP 101445)]
[New Thread 805ca17c0 (LWP 101443)]
[New Thread 805545fc0 (LWP 101441)]
[New Thread 805547bc0 (LWP 101436)]
[New Thread 80705b140 (LWP 100596)]
[New Thread 803853340 (LWP 101429)]
[New Thread 803f88580 (LWP 101424)]
[New Thread 806cc3a80 (LWP 101422)]
[New Thread 803e7e900 (LWP 100710)]
[New Thread 805c556c0 (LWP 101425)]
[New Thread 8063e7200 (LWP 101413)]
[New Thread 803852540 (LWP 101180)]
[New Thread 803856d00 (LWP 100620)]
[New Thread 803f883c0 (LWP 101412)]
[New Thread 80554b740 (LWP 100927)]
[New Thread 80631c900 (LWP 101403)]
[New Thread 80544d2c0 (LWP 101081)]
[New Thread 803f39380 (LWP 101402)]
[New Thread 8054503c0 (LWP 101392)]
[New Thread 80712c600 (LWP 101163)]
[New Thread 80a1088c0 (LWP 101149)]
[New Thread 803634040 (LWP 101387)]
[New Thread 80544cf40 (LWP 101372)]
[New Thread 8055489c0 (LWP 101370)]
[New Thread 803634c80 (LWP 101368)]
[New Thread 805546a40 (LWP 101072)]
[New Thread 806d5a280 (LWP 100608)]
[New Thread 805548d40 (LWP 100587)]
[New Thread 80b28d9c0 (LWP 101367)]
[New Thread 803853500 (LWP 101365)]
[New Thread 803856280 (LWP 101202)]
[New Thread 8070e44c0 (LWP 101355)]
[New Thread 80555bb40 (LWP 101188)]
[New Thread 805547140 (LWP 101185)]
[New Thread 80b289740 (LWP 101324)]
[New Thread 805557380 (LWP 101316)]
[New Thread 80bc081c0 (LWP 101154)]
[New Thread 8051c7240 (LWP 101315)]
[New Thread 805450580 (LWP 101114)]
[New Thread 8038536c0 (LWP 101093)]
[New Thread 806f861c0 (LWP 100929)]
[New Thread 8097dc740 (LWP 100931)]
[New Thread 80544d9c0 (LWP 100918)]
[New Thread 8041831c0 (LWP 100876)]
[New Thread 80a07d700 (LWP 100493)]
[New Thread 80544eec0 (LWP 100742)]
[New Thread 806cc7600 (LWP 101309)]
[New Thread 80b28cbc0 (LWP 101304)]
[New Thread 80544e440 (LWP 101302)]
[New Thread 80b28b500 (LWP 101277)]
[New Thread 80b289900 (LWP 101086)]
[New Thread 80712bd40 (LWP 100767)]
[New Thread 80b2893c0 (LWP 101276)]
[New Thread 804010e40 (LWP 101092)]
[New Thread 805c58980 (LWP 100941)]
[New Thread 804a4a040 (LWP 100677)]
[New Thread 806cc70c0 (LWP 100681)]
[New Thread 8077c88c0 (LWP 101001)]
[New Thread 804a456c0 (LWP 101271)]
[New Thread 8047a9ac0 (LWP 101214)]
[New Thread 804a07500 (LWP 100817)]
[New Thread 80a11d8c0 (LWP 101240)]
[New Thread 80554be40 (LWP 101233)]
[New Thread 806214c80 (LWP 101166)]
[New Thread 805548800 (LWP 101171)]
[New Thread 805546c00 (LWP 101169)]
[New Thread 8097d6540 (LWP 101174)]
[New Thread 805557e00 (LWP 101193)]
[New Thread 80bd70200 (LWP 101115)]
[New Thread 805549280 (LWP 101132)]
[New Thread 80b28aa80 (LWP 101181)]
[New Thread 80544fcc0 (LWP 101105)]
[New Thread 803633780 (LWP 101017)]
[New Thread 80505d180 (LWP 101088)]
[New Thread 806d57180 (LWP 101019)]
[New Thread 803634ac0 (LWP 100994)]
[New Thread 803855800 (LWP 100972)]
[New Thread 80544e7c0 (LWP 100787)]
[New Thread 803856980 (LWP 100747)]
[New Thread 8084ff540 (LWP 100740)]
[New Thread 80778b1c0 (LWP 100679)]
[New Thread 80544f5c0 (LWP 100815)]
[New Thread 80b28bf80 (LWP 101611)]
[New Thread 80554a240 (LWP 101542)]
[New Thread 80b288940 (LWP 101680)]
[New Thread 80b289040 (LWP 101674)]
[New Thread 80b2d3f40 (LWP 101670)]
[New Thread 80b28a700 (LWP 101656)]
[New Thread 80978a540 (LWP 101626)]
[New Thread 80a109880 (LWP 101610)]
[New Thread 806da5c00 (LWP 101544)]
[New Thread 805176400 (LWP 101444)]
[New Thread 806da8280 (LWP 101432)]
[New Thread 807553d40 (LWP 100921)]
[New Thread 8036335c0 (LWP 101359)]
[New Thread 805557a80 (LWP 101336)]
[New Thread 80554a780 (LWP 101658)]
[New Thread 808d29f00 (LWP 101635)]
[New Thread 805547680 (LWP 100561)]
[New Thread 803855100 (LWP 101382)]
[New Thread 805546500 (LWP 101373)]
[New Thread 803633940 (LWP 100760)]
[New Thread 8038567c0 (LWP 101278)]
[New Thread 807553800 (LWP 101242)]
[New Thread 8070080c0 (LWP 101340)]
[New Thread 80555bd00 (LWP 101289)]
[New Thread 803857080 (LWP 100473)]
[New Thread 803857240 (LWP 100357)]
[New Thread 802a041c0 (LWP 100394)]
[Switching to Thread 81ad04d40 (LWP 102349)]
0x0000000801ddbfea in read () from /lib/libc.so.7
$1 = 1953067639
$2 = 44097640
$3 = 44085944
$4 = 47441064
$5 = 14848638
$6 = 14848681
$7 = 14848706
$8 = 14848669
$9 = 1953067639
$10 = 14848661
/usr/local/share/bacula/btraceback.gdb:11: Error in sourced command file:
No symbol table is loaded.  Use the "file" command.

I'm still having some problems getting enough information out of the btraceback 
script, both bacula-dir and sd are running with "-d 3", but I don;t know if 
that is an appropriate level. Also, I have to use gdb 7.2 from Ports, the build 
in 6.x version of GDB would just hang, waiting for input ( some sort of y/n 
question ).

I'm sorry for the long email, please let me know if more or less information 
would be useful, I'd like to understand how I can properly throttle Bacula, or 
find out why it is crashing.

Thanks!
Mike C

------------------------------------------------------------------------------
Colocation vs. Managed Hosting
A question and answer guide to determining the best fit
for your organization - today and in the future.
http://p.sf.net/sfu/internap-sfd2d
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

<Prev in Thread] Current Thread [Next in Thread>