Crash on openSUSE 10.2

[gmichelson]

Hi !

After last update I installed on openSUSE, mt-daapd is crashing if I try to access web admin page. I tried to go back few versions, but got same result. This is what I get in my message.log file :

Apr 22 18:10:51 blimp mt-daapd[24295]: Checking /mt-daapd for admin
Apr 22 18:10:51 blimp mt-daapd[24295]: Checking NULL/NULL for admin
Apr 22 18:10:51 blimp mt-daapd[24295]: Checking /mt-daapd for admin
Apr 22 18:10:51 blimp mt-daapd[24295]: Checking NULL/NULL for admin
Apr 22 18:10:51 blimp mt-daapd[24295]: Checking /mt-daapd for admin
Apr 22 18:10:51 blimp mt-daapd[24295]: Checking NULL/NULL for admin
Apr 22 18:10:51 blimp mt-daapd[24295]: Checking /mt-daapd for admin
Apr 22 18:10:51 blimp kernel: mt-daapd[24316]: segfault at 000000007302653a rip 00002ae37292bb9e rsp 0000000041ffecd0 error 4

I tried to remove db – it’s getting re-created successfully, but that did not help.

I compile server with following settings :

./configure –enable-sqlite –enable-oggvorbis –enable-flac –enable-rsp

Thanks !
George

[rpedde]

@gmichelson wrote:

Hi !

After last update I installed on openSUSE, mt-daapd is crashing if I try to access web admin page. I tried to go back few versions, but got same result.

Can you get a stack backtrace in gdb with that? Should be able to run it under gdb like:

Code:

[email protected]$ gdb /path/to/mt-daapd
(gdb) run -d9 -f -c /path/to/mt-daapd.conf -y
[code]

then make it crash, however you normally do it.

Then, back in gdb:

[code]
(gdb) bt

I need the stuff it dumps after you do “bt”.

[gmichelson]

Okay, after running under debugger it crashes with following message :

Status inquiry
Processing rendezvous message
select(11, 5.000000)
Returning status 0

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1090525504 (LWP 987)]
0x00002afbff88bb9e in __find_specmb () from /lib64/libc.so.6
(gdb) select(11, 0.813476)
select(11, 0.100585)

and here is trace :

#0 0x00002afbff88bb9e in __find_specmb () from /lib64/libc.so.6
#1 0x00002afbff8706e1 in vfprintf () from /lib64/libc.so.6
#2 0x00002afbff89574a in vsnprintf () from /lib64/libc.so.6
#3 0x00000000004170cc in xml_output (pxml=0x677a00, section=0x445ed4 “name”,
fmt=0xffffffffffffffa0

) at xml-rpc.c:307
#4 0x000000000041739f in xml_get_stats (pwsc=) at xml-rpc.c:549
#5 0x000000000040a147 in config_handler (pwsc=0x677830) at configfile.c:438
#6 0x00000000004088c0 in ws_dispatcher (arg=) at webserver.c:986
#7 0x00002afbfe82609e in start_thread () from /lib64/libpthread.so.0
#8 0x00002afbff8f04cd in clone () from /lib64/libc.so.6
#9 0x0000000000000000 in ?? ()

Let me know if I can provide any other information !

George

[gimpbully]

I’ve experienced similar crashes in svn-1549. It will run just fine until I attempt to load the web page with these errors:

2007-04-24 16:03:43 (700f15ce): Session 0: Streaming file ’09 Sparkle.mp3′ to 10.0.0.30 (offset 0)
2007-04-24 16:03:57 (700f15ce): Write error: Connection reset by peer
2007-04-24 16:03:58 (700f15ce): Session 0: Streaming file ’01 Search and Destroy.m4p’ to 10.0.0.30 (offset 0)
2007-04-24 16:04:11 (0f68c66f): Checking NULL/NULL for admin
2007-04-24 16:04:11 (0f68c66f): Checking root/[delted] for admin
2007-04-24 16:04:11 (0f68c66f): Thread 6: Entering ws_returnerror (302: Moved)
2007-04-24 16:04:11 (0f68c66f): Checking NULL/NULL for admin
2007-04-24 16:04:11 (0f68c66f): Checking root/[delted]for admin
2007-04-24 16:04:11 (b48a5a80): Checking NULL/NULL for admin
2007-04-24 16:04:11 (b48a5a80): Checking root/[delted]for admin
2007-04-24 16:04:11 (b48a5a80): Checking NULL/NULL for admin
2007-04-24 16:04:11 (b48a5a80): Checking root/[delted] for admin
2007-04-24 16:04:11 (b48a5a80): Checking NULL/NULL for admin
2007-04-24 16:04:11 (b48a5a80): Checking root/[delted]for admin
2007-04-24 16:04:11 (b48a5a80): Checking NULL/NULL for admin
2007-04-24 16:04:11 (b48a5a80): Checking root/[delted]for admin
2007-04-24 16:04:12 (b48a5a80): Checking NULL/NULL for admin
2007-04-24 16:04:12 (b48a5a80): Checking root/[delted]for admin
2007-04-24 16:04:12 (b48a5a80): Checking NULL/NULL for admin
2007-04-24 16:04:12 (b48a5a80): Checking root/[delted]for admin
2007-04-24 16:04:12 (b48a5a80): Checking NULL/NULL for admin
2007-04-24 16:04:12 (b48a5a80): Checking root/[delted]for admin
2007-04-24 16:04:12 (b48a5a80): Checking NULL/NULL for admin
2007-04-24 16:04:12 (b48a5a80): Checking root/[delted]for admin
2007-04-24 16:04:12 (0f68c66f): Checking NULL/NULL for admin
2007-04-24 16:04:12 (0f68c66f): Checking root/[delted]for admin
2007-04-24 16:04:12 (0f68c66f): Checking NULL/NULL for admin
2007-04-24 16:04:12 (0f68c66f): Checking root/[delted]for admin
2007-04-24 16:04:12 (c8021c84): Rendezvous socket closed (daap server crashed?) Aborting.
2007-04-24 16:04:12: Aborting

also from /var/log/messages:
Apr 24 16:04:12 eduardo mt-daapd[29097]: Checking NULL/NULL for admin
Apr 24 16:04:12 eduardo mt-daapd[29097]: Checking root/[deleted] for admin
Apr 24 16:04:12 eduardo mt-daapd[29097]: Checking NULL/NULL for admin
Apr 24 16:04:12 eduardo mt-daapd[29097]: Checking root/[deleted] for admin
Apr 24 16:04:12 eduardo mt-daapd[29135]: segfault at 000000006bf5e008 rip 00002b986a6d342c rsp 00000000417fdd50 error 4
Apr 24 16:04:12 eduardo mt-daapd[29096]: Rendezvous socket closed (daap server crashed?) Aborting.

[w1ll14m]

I have the same problem here,

It seems that index page triggers something that causes a Segmentation fault somewhere (i think it is only on 64 systems 😥 as i don’t have this problem on my 32bit system…)

rpedde ever seen this before ?

Thread 23: Entering ws_dispatcher (Connection from 127.0.0.1)

Thread 23: got request

Request: GET /xml-rpc?method=stats HTTP/1.1

Thread 23: Read: Host: 127.0.0.1:8080

Thread 23: Adding header *Host=127.0.0.1:8080*

Added *Host=127.0.0.1:8080*

Thread 23: Read: User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; nl; rv:1.8.1.3) Gecko/20070427 Firefox/2.0.0.3

Thread 23: Adding header *User-Agent=Mozilla/5.0 (X11; U; Linux x86_64; nl; rv:1.8.1.3) Gecko/20070427 Firefox/2.0.0.3*

Added *User-Agent=Mozilla/5.0 (X11; U; Linux x86_64; nl; rv:1.8.1.3) Gecko/20070427 Firefox/2.0.0.3*

Thread 23: Read: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

Thread 23: Adding header *Accept=text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5*

Added *Accept=text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5*

Thread 23: Read: Accept-Language: nl,en-us;q=0.7,en;q=0.3

Thread 23: Adding header *Accept-Language=nl,en-us;q=0.7,en;q=0.3*

Added *Accept-Language=nl,en-us;q=0.7,en;q=0.3*

Thread 23: Read: Accept-Encoding: gzip,deflate

Thread 23: Adding header *Accept-Encoding=gzip,deflate*

Added *Accept-Encoding=gzip,deflate*

Thread 23: Read: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Thread 23: Adding header *Accept-Charset=ISO-8859-1,utf-8;q=0.7,*;q=0.7*

Added *Accept-Charset=ISO-8859-1,utf-8;q=0.7,*;q=0.7*

Thread 23: Read: Keep-Alive: 300

Thread 23: Adding header *Keep-Alive=300*

Added *Keep-Alive=300*

Thread 23: Read: Connection: keep-alive

Thread 23: Adding header *Connection=keep-alive*

Added *Connection=keep-alive*

Thread 23: Read: X-Requested-With: XMLHttpRequest

Thread 23: Adding header *X-Requested-With=XMLHttpRequest*

Added *X-Requested-With=XMLHttpRequest*

Thread 23: Read: X-Prototype-Version: 1.4.0

Thread 23: Adding header *X-Prototype-Version=1.4.0*

Added *X-Prototype-Version=1.4.0*

Thread 23: Read: Referer: http://127.0.0.1:8080/index.html

Thread 23: Adding header *Referer=http://127.0.0.1:8080/index.html*

Added *Referer=http://127.0.0.1:8080/index.html*

Thread 23: Read: Authorization: Basic [password-hash]

Thread 23: Adding header *Authorization=Basic [password-hash]*

Added *Authorization=Basic [password-hash]*

Thread 23: Read:

Thread 23: Headers parsed!

Checking to see if connection matches close

And it does NOT

Thread 23: Connection type HTTP/1.1

: Connection: persist

Thread 23: parsing GET args

Thread 23: Entering ws_getgetvars (method=stats)

Thread 23: Adding arg method = stats

Added *method=stats*

Thread 23: Done parsing GET/POST args!

Thread 23: Original URI: /xml-rpc

Thread 23: Translated URI: /xml-rpc

Thread 23: Preparing to find handler

Checking /xml-rpc against handler for /

Thread 23: URI Match!

Thread 23: Time is 1177731934 seconds after epoch

Thread 23: Setting time header

Added *Date=Sat, 28 Apr 2007 03:45:34 GMT*

Added *Connection=keep-alive*

Added *Server=mt-daapd/svn-1549*

Added *Content-Type=text/html*

Added *Content-Language=en_us*

Thread 23: Using non-default handler

in main_auth

Checking url /xml-rpc

Checking url /xml-rpc

Dispatching auth for /xml-rpc to config auth

Checking NULL/NULL for admin

Checking if pw required for /xml-rpc as admin

Yep

Preparing to decode [password-hash]

Decoded firefly:[password]

Decoded user=firefly, pw=[password]

in main_auth

Checking url /xml-rpc

Checking url /xml-rpc

Dispatching auth for /xml-rpc to config auth

Checking firefly/[password] for admin

Added *HTTP_USER=firefly*

Added *HTTP_PASSWD=[password]*

in main_handler

Checking url /xml-rpc

Checking url /xml-rpc

Dispatching /xml-rpc to config handler

Entering config_handler

Entering config_set_status

Exiting config_set_status

Updating Connection from keep-alive to close

Entering config_set_status

Exiting config_set_status

Added *Cache-Control=no-cache*

Added *Expires=-1*

Updating Content-Type from text/html to text/xml; charset=utf-8

Emitting reponse header Expires: -1

Emitting reponse header Cache-Control: no-cache

Emitting reponse header Content-Language: en_us

Emitting reponse header Content-Type: text/xml; charset=utf-8

Emitting reponse header Server: mt-daapd/svn-1549

Emitting reponse header Connection: close

Emitting reponse header Date: Sat, 28 Apr 2007 03:45:34 GMT

Status inquiry

Processing rendezvous message

select(12, 5.000000)

Returning status 0



Program received signal SIGSEGV, Segmentation fault.

[Switching to Thread 1107310912 (LWP 10321)]

0x00002b6de0dae031 in ?? () from /lib/libc.so.6

(gdb) kill

Kill the program being debugged? (y or n) y

Processing rendezvous message

Rendezvous socket closed (daap server crashed?) Aborting.

Aborting

(gdb) run -d9 -f -c /etc/mt-daapd.conf -y

Backtrace:

#0  0x00002b280f662031 in ?? () from /lib/libc.so.6

#1  0x00002b280f6474b1 in vfprintf () from /lib/libc.so.6

#2  0x00002b280f66ba1a in vsnprintf () from /lib/libc.so.6

#3  0x00000000004176bc in xml_output (pxml=0x5785e0, section=0x446bdc "name",

fmt=0xffffffffffffffc0 
) at xml-rpc.c:307

#4  0x0000000000417993 in xml_get_stats (pwsc=) at xml-rpc.c:549

#5  0x000000000040a7b7 in config_handler (pwsc=0x579e00) at configfile.c:438

#6  0x0000000000408f2f in ws_dispatcher (arg=) at webserver.c:986

#7  0x00002b280e40a135 in start_thread () from /lib/libpthread.so.0

#8  0x00002b280f6c52ce in clone () from /lib/libc.so.6

#9  0x0000000000000000 in ?? ()

(gdb) select(12, 2.164062)

select(12, 0.100585)

select(12, 0.100585)

select(12, 5.000000)

Regards,
William

[rpedde]

@w1ll14m wrote:

rpedde ever seen this before ?

Well, I see the backtrac, but I don’t see at all why. 🙁

Can you change line 549 of xml-rcp.c from:

xml_output(pxml,"name",plugin_get_description(phandle));

to

xml_output(pxml,"name","%s",plugin_get_description(phandle));

And see what that does? Shouldn’t make a difference, but I’d be interested to know.

[w1ll14m]

@rpedde wrote:

@w1ll14m wrote:

rpedde ever seen this before ?

Can you change line 549 of xml-rcp.c from:

xml_output(pxml,"name",plugin_get_description(phandle));

to

xml_output(pxml,"name","%s",plugin_get_description(phandle));

And see what that does? Shouldn’t make a difference, but I’d be interested to know.

I Will try 🙂
I’ll keep you posted 🙂

[w1ll14m]

Ok,

I used mt-daapd-1549:
Edited xml-rpc.c as said above

./configure --enable-sqlite3 -enable-=avahi -enable-oggvorbis --enable-flac --enable-musepack --enable-ffmpeg --with-ffmpeg==/usr/include/ffmpeg/ --enable-upnp --with-id3tag

Source compiles just fine.

Debugging again:

Program received signal SIGSEGV, Segmentation fault.

[Switching to Thread 1098918208 (LWP 23210)]

0x00002b75a9932bd0 in strlen () from /lib/libc.so.6

Backtrace:

(gdb) bt

#0  0x00002b75a9932bd0 in strlen () from /lib/libc.so.6

#1  0x00002b75a9904b60 in vfprintf () from /lib/libc.so.6

#2  0x00002b75a9925a1a in vsnprintf () from /lib/libc.so.6

#3  0x00000000004176bc in xml_output (pxml=0x579e40, section=0x446bdc "name", fmt=0x417fe601 "æ177A") at xml-rpc.c:307

#4  0x0000000000417997 in xml_get_stats (pwsc=) at xml-rpc.c:549

#5  0x000000000040a7b7 in config_handler (pwsc=0x578760) at configfile.c:438

#6  0x0000000000408f2f in ws_dispatcher (arg=) at webserver.c:986

#7  0x00002b75a86c4135 in start_thread () from /lib/libpthread.so.0

#8  0x00002b75a997f2ce in clone () from /lib/libc.so.6

#9  0x0000000000000000 in ?? ()

(gdb) select(12, 5.000000)

Also tested 1571:
Same changes in xml-rpc.c
same ./configure used, and compiles fine.

unpatched:

Program received signal SIGSEGV, Segmentation fault.

[Switching to Thread 1098918208 (LWP 30635)]

0x00002ae19cb29031 in ?? () from /lib/libc.so.6

(gdb) select(12, 0.100585)

Backtrace:

bt

#0  0x00002ae19cb29031 in ?? () from /lib/libc.so.6

#1  0x00002ae19cb0e4b1 in vfprintf () from /lib/libc.so.6

#2  0x00002ae19cb32a1a in vsnprintf () from /lib/libc.so.6

#3  0x00000000004176dc in xml_output (pxml=0x579d10, section=0x446c1c "name", fmt=0xffffffffffffffc0 
) at xml-rpc.c:307

#4  0x00000000004179b3 in xml_get_stats (pwsc=) at xml-rpc.c:549

#5  0x000000000040a7d7 in config_handler (pwsc=0x578840) at configfile.c:443

#6  0x0000000000408f2f in ws_dispatcher (arg=) at webserver.c:986

#7  0x00002ae19b8d1135 in start_thread () from /lib/libpthread.so.0

#8  0x00002ae19cb8c2ce in clone () from /lib/libc.so.6

#9  0x0000000000000000 in ?? ()

(gdb) select(12, 5.000000)

Patched:

Updating Content-Type from text/html to text/xml; charset=utf-8

Emitting reponse header Expires: -1

Emitting reponse header Cache-Control: no-cache

Emitting reponse header Content-Language: en_us

Emitting reponse header Content-Type: text/xml; charset=utf-8

Emitting reponse header Server: mt-daapd/svn-1571

Emitting reponse header Connection: close

Emitting reponse header Date: Tue, 1 May 2007 11:05:12 GMT

Status inquiry

Processing rendezvous message

select(12, 2.257812)

Returning status 0



Program received signal SIGSEGV, Segmentation fault.

[Switching to Thread 1098918208 (LWP 29044)]

0x00002b8613296bd0 in strlen () from /lib/libc.so.6

(gdb) select(12, 0.091796)

Backtrace:

bt

#0  0x00002b8613296bd0 in strlen () from /lib/libc.so.6

#1  0x00002b8613268b60 in vfprintf () from /lib/libc.so.6

#2  0x00002b8613289a1a in vsnprintf () from /lib/libc.so.6

#3  0x00000000004176dc in xml_output (pxml=0x579cc0, section=0x446c1c "name", fmt=0x417fe601 "æ177A") at xml-rpc.c:307

#4  0x00000000004179b7 in xml_get_stats (pwsc=) at xml-rpc.c:549

#5  0x000000000040a7d7 in config_handler (pwsc=0x578870) at configfile.c:443

#6  0x0000000000408f2f in ws_dispatcher (arg=) at webserver.c:986

#7  0x00002b8612028135 in start_thread () from /lib/libpthread.so.0

#8  0x00002b86132e32ce in clone () from /lib/libc.so.6

#9  0x0000000000000000 in ?? ()

Seems to make no difference (in stability).
It seems you can view playlist page (where is no playlist except dynamic playlist from mt-daapd) and if you go further, it crashes again.

I can see some changes in the backtrace from the unpatched and patched version, but i can’t tell what it is except that is indeed somewhere in xml-rpc.

[rpedde]

@w1ll14m wrote:

bt

#0  0x00002b8613296bd0 in strlen () from /lib/libc.so.6

#1  0x00002b8613268b60 in vfprintf () from /lib/libc.so.6

#2  0x00002b8613289a1a in vsnprintf () from /lib/libc.so.6

#3  0x00000000004176dc in xml_output (pxml=0x579cc0, section=0x446c1c "name", fmt=0x417fe601 "æ177A") at xml-rpc.c:307

That’s just not possible.

Do you have a [plugins] section in your config file, as well as a plugin_dir? And when you start it, does it show it loading the same version of the plugins as your main program?

It looks like mismatched plugins or something.

[gmichelson]

Hi !

I recompiled once more and verified plugin versions – they are correct, this is snipped of the log file :

blimp:/usr/local/share/mt-daapd/plugins # tail /var/log/messages
May 5 09:31:21 blimp mt-daapd[8958]: Firefly Version svn-1571: Starting with debuglevel 1
May 5 09:31:21 blimp mt-daapd[8958]: Plugin loaded: rsp/svn-1571
May 5 09:31:21 blimp mt-daapd[8958]: Plugin loaded: ssc-script/svn-1571
May 5 09:31:21 blimp mt-daapd[8958]: Plugin loaded: daap/svn-1571
May 5 09:31:21 blimp mt-daapd[8958]: Starting rendezvous daemon
May 5 09:31:21 blimp mt-daapd[8958]: Starting signal handler
May 5 09:31:21 blimp mt-daapd[8960]: Initializing database
May 5 09:31:23 blimp mt-daapd[8960]: Starting web server from /usr/local/share/mt-daapd/admin-root on port 3689
May 5 09:31:23 blimp mt-daapd[8960]: Registering rendezvous names
May 5 09:31:23 blimp mt-daapd[8960]: Serving 10465 songs. Startup complete in 2 seconds

I can access smart play list edit page and create smart list, but accessing any other page in admin site crashes the server.

Hope that helps !

George

[Author]

Posts