Server (svn-1130) crashed after 60mins or so

[richo132]

Upgraded to 1130.
Started well:
Uptime > 1 hour
3 PCs connected
32 songs streamed
Then server crashed as before.

[richo132]

Have tried reverting to nightly 1050
I noticed a couple of things straight away:

1. With 3 PCs connected, there’s no delay to start streaming new tunes
– with 1082 and 1130 I was seeing long delays often 1 min+
2. The web app status screen refreshes more smoothly.

[rpedde]

@richo132 wrote:

Have tried reverting to nightly 1050
I noticed a couple of things straight away:

1. With 3 PCs connected, there’s no delay to start streaming new tunes
– with 1082 and 1130 I was seeing long delays often 1 min+
2. The web app status screen refreshes more smoothly.

I’ve replicated this… it’s a memory leak (fortunately) not a deadlock. I haven’t tracked it down yet, and I probably won’t until I dump a nightly for tonight, but I’ll find it in the next couple days.

— Ron

[richo132]

Thanks for the feedback.
I don’t know whether it helps or not (maybe part of same issue) but the 1050 server crashed for me as well after 80mins or so.
Different symptom though, as can be seen in attached log output
“..Dispatcher: accept failed: Too many open files”

2006-05-30 23:19:15: Starting rendezvous daemon
2006-05-30 23:19:15: Starting signal handler
2006-05-30 23:19:15: Initializing database
2006-05-30 23:20:53: Starting web server from /opt/share/mt-daapd/admin-root on port 3689
2006-05-30 23:20:53: Registering rendezvous names
2006-05-30 23:20:55: Scanned 9349 songs in 98 seconds
2006-05-30 23:24:23: Session 0: Streaming file ‘2-11 Agitated.m4a’ to 192.168.0.250 (offset 0)
2006-05-30 23:25:08: Session 0: Streaming file ’01 Less Than A Pearl.m4a’ to 192.168.0.243 (offset 0)
2006-05-30 23:28:28: Session 0: Streaming file ’01 Stockholm Syndrome.m4a’ to 192.168.0.250 (offset 0)
2006-05-30 23:28:46: Session 0: Streaming file ’02 Amarantine.m4a’ to 192.168.0.243 (offset 0)
2006-05-30 23:31:53: Session 0: Streaming file ’03 It’s In The Rain.m4a’ to 192.168.0.243 (offset 0)
2006-05-30 23:35:35: Session 0: Streaming file ’02 New Born.m4a’ to 192.168.0.250 (offset 0)
2006-05-30 23:35:55: Session 0: Streaming file ’04 If I Could Be Where You Are.m4a’ to 192.168.0.243 (offset 0)
2006-05-30 23:39:50: Session 0: Streaming file ’05 The River Sings.m4a’ to 192.168.0.243 (offset 0)
2006-05-30 23:40:51: Rescanning database
2006-05-30 23:42:04: Session 0: Streaming file ’03 Muscle Museum.m4a’ to 192.168.0.250 (offset 0)
2006-05-30 23:42:49: Session 0: Streaming file ’06 Long Long Journey.m4a’ to 192.168.0.243 (offset 0)
2006-05-30 23:46:31: Session 0: Streaming file ’04 Hysteria.m4a’ to 192.168.0.250 (offset 0)
2006-05-30 23:46:31: Session 0: Streaming file ’07 Sumiregusa.m4a’ to 192.168.0.243 (offset 0)
2006-05-30 23:50:53: Session 0: Streaming file ’05 Bliss.m4a’ to 192.168.0.250 (offset 0)
2006-05-30 23:51:10: Session 0: Streaming file ’08 Someone Said Goodbye.m4a’ to 192.168.0.243 (offset 0)
2006-05-30 23:55:27: Session 0: Streaming file ’09 A Moment Lost.m4a’ to 192.168.0.243 (offset 0)
2006-05-30 23:55:28: Session 0: Streaming file ’04 Sing For Absolution.m4a’ to 192.168.0.250 (offset 0)
2006-05-30 23:58:46: Session 0: Streaming file ’10 Drifting.m4a’ to 192.168.0.243 (offset 0)
2006-05-31 00:00:45: Session 0: Streaming file ’05 Stockholm Syndrome.m4a’ to 192.168.0.250 (offset 0)
2006-05-31 00:03:03: Session 0: Streaming file ’11 Amid The Falling Snow.m4a’ to 192.168.0.243 (offset 0)
2006-05-31 00:05:42: Session 0: Streaming file ’07 Interlude.m4a’ to 192.168.0.250 (offset 0)
2006-05-31 00:06:48: Session 0: Streaming file ’08 Hysteria.m4a’ to 192.168.0.250 (offset 0)
2006-05-31 00:06:49: Session 0: Streaming file ’12 Water Shows The Hidden Heart.m4a’ to 192.168.0.243 (offset 0)
2006-05-31 00:10:41: Session 0: Streaming file ’10 Butterflies & Hurricanes.m4a’ to 192.168.0.250 (offset 0)
2006-05-31 00:15:51: Session 0: Streaming file ’07 Everything Goes.m4a’ to 192.168.0.250 (offset 0)
2006-05-31 00:20:11: Session 0: Streaming file ’01 Torn.m4a’ to 192.168.0.250 (offset 0)
2006-05-31 00:22:24: Session 0: Streaming file ’01 Less Than A Pearl.m4a’ to 192.168.0.243 (offset 0)
2006-05-31 00:24:29: Session 0: Streaming file ’10 Intuition.m4a’ to 192.168.0.250 (offset 0)
2006-05-31 00:26:26: Session 0: Streaming file ’02 Amarantine.m4a’ to 192.168.0.243 (offset 0)
2006-05-31 00:29:29: Session 0: Streaming file ’01 Pas de deux – Le Prince et la fee.m4a’ to 192.168.0.250 (offset 0)
2006-05-31 00:29:38: Session 0: Streaming file ’03 It’s In The Rain.m4a’ to 192.168.0.243 (offset 0)
2006-05-31 00:34:12: Session 0: Streaming file ’04 If I Could Be Where You Are.m4a’ to 192.168.0.243 (offset 0)
2006-05-31 00:34:24: Session 0: Streaming file ’03 Dance of the Sugar-Plum Fairy.m4a’ to 192.168.0.250 (offset 0)
2006-05-31 00:37:07: Session 0: Streaming file ’04 Coda.m4a’ to 192.168.0.250 (offset 0)
2006-05-31 00:38:09: Session 0: Streaming file ’05 The River Sings.m4a’ to 192.168.0.243 (offset 0)
2006-05-31 00:40:06: Session 0: Streaming file ’06 Le Papillon, Prelude; Act 1 Scene.m4a’ to 192.168.0.250 (offset 0)
2006-05-31 00:41:18: Session 0: Streaming file ’06 Long Long Journey.m4a’ to 192.168.0.243 (offset 0)
2006-05-31 00:43:48: Dispatcher: accept failed: Too many open files
2006-05-31 00:43:48: Dispatcher: Aborting
2006-05-31 00:43:48: Aborting
2006-05-31 00:43:48: Rendezvous socket closed (daap server crashed?) Aborting.
2006-05-31 00:43:48: Aborting

[rpedde]

@richo132 wrote:

Thanks for the feedback.
I don’t know whether it helps or not (maybe part of same issue) but the 1050 server crashed for me as well after 80mins or so.
Different symptom though, as can be seen in attached log output
“..Dispatcher: accept failed: Too many open files”

Probably the same deal. not cleaning up after serving the files. I’ll find it, but it won’t be in tonight’s drop. I’ll post back here when I have it.

— Ron

[richo132]

There is no reference in your upgrade notes to the suspected memory leak.
However, I have upgraded to the 1197 nightly and things are better.
In three trials my NSLU2 server ran for about 1.5hrs before crashing.
Is there anything I can do as a user to help further pinpoint the problem ?
Cheers.

[rpedde]

@richo132 wrote:

There is no reference in your upgrade notes to the suspected memory leak.
However, I have upgraded to the 1197 nightly and things are better.
In three trials my NSLU2 server ran for about 1.5hrs before crashing.
Is there anything I can do as a user to help further pinpoint the problem ?
Cheers.

Someone on another thread noticed it seemed to be worse on a multi processor machine. I’m in the process of loading one so I can test it and see if I can nail it down easier.

— Ron

[Author]

Posts