GIANT Library Loading and Browsing Problems

[ratbag21]

I have searched the forums high and low, but have not found a clear answer or example for my issues.

I have an NSLU2 unslung using the 6.8 firmware to disk2 [2GB flash drive], with a Western Dig My Book 500GB connected to disk1. I am using the 0.2.4 mt-daapd as my music server. The NSLU2 boots great, Firefly starts great, and works great I might add!!!! [big thanks to all involved in the project].

My issue is that I have a 45,000+ library and cannot browse [even by artist – 2600+ different artists] at all. Small playlists [.m3u] will load in an instant, large playlists [8,000 – 12,000 entries] will load SOMETIMES.

My question: is there a solution to possibly correct this behavior, or am I looking for something that does not exist? I would like to be able to browse songs etc, and load large playlists.

thanks,
Rat

[rpedde]

@ratbag21 wrote:

I have searched the forums high and low, but have not found a clear answer or example for my issues.

I have an NSLU2 unslung using the 6.8 firmware to disk2 [2GB flash drive], with a Western Dig My Book 500GB connected to disk1. I am using the 0.2.4 mt-daapd as my music server. The NSLU2 boots great, Firefly starts great, and works great I might add!!!! [big thanks to all involved in the project].

My issue is that I have a 45,000+ library and cannot browse [even by artist – 2600+ different artists] at all. Small playlists [.m3u] will load in an instant, large playlists [8,000 – 12,000 entries] will load SOMETIMES.

My question: is there a solution to possibly correct this behavior, or am I looking for something that does not exist? I would like to be able to browse songs etc, and load large playlists.

thanks,
Rat

Is this against a soundbridge, I assume?

If so, then nightlies will probably work better. At least faster, and people have reported with with sqlite3 as a backend, it browses better, too.

Right now, the slug nightlies don’t support sqlite3, but I’ll have that in nightlies soon. Possibly as soon as this weekend.

— Ron

[ratbag21]

Yes this is with a soundbridge M1001.

So, this is related to db access, speed of slug vs. time-out on the soundbridge?

so, you recommend that I wait to load nightlies until after the sqlite3 is enabled? …or will nightlies without sqlite3 also possibly/probably give a performance improvement?

thanks for the response,
Rat

[mas]

I think there are 2 problems:

1) Timout. After about 60 secs or so it wont go on. With a slug its not easy to deliver the entries fast enough. Latest nightly plus sqlite3 selfcompiled is likely best. 4.3k work in about 15-20 sec so at around 12-15k it should time out.

2) Memory in the Soundbridge. They never stated a maximum precisely but the thing has only 32MB memory of which only so many is free. I think over 10k you get a problem with that anyway.

So basically, much more than 10k songs in one select wont work. You need to either split databases or only use playlists/selections of <10k.

[rpedde]

@mas wrote:

Latest nightly plus sqlite3 selfcompiled is likely best. 4.3k work in about 15-20 sec so at around 12-15k it should time out.

I’ve double-checked my build chain for unslung, and it’s configured for both sqlite and sqlite3.

You should be able to just change the dbtype from “sqlite” to “sqlite3” in your config file and see what that does.

— Ron

[ratbag21]

That change in DBtype could only be done via a “nightly” build, correct? The existing stable-build does not support this option, correct?

One additional question and I am off to make some changes…

That config change causes the App to create a DB file in a SQLite3 format and then reads that file directly? …Or is there a SQLite App that needs to be running as well on the slug?

thanks again!!!

BTW, the whole Firefly setup is by compare the best option out there for having music on demand via soundbridge.

[rpedde]

@ratbag21 wrote:

That change in DBtype could only be done via a “nightly” build, correct? The existing stable-build does not support this option, correct?

Correct.

One additional question and I am off to make some changes…

That config change causes the App to create a DB file in a SQLite3 format and then reads that file directly? …Or is there a SQLite App that needs to be running as well on the slug?

Nope, sqlite is a “serverless” sql engine. Everything it needs is built into the daap server.

BTW, the whole Firefly setup is by compare the best option out there for having music on demand via soundbridge.

Well, once it gets stable again. 🙂

— Ron

[ratbag21]

Ron,

Thanks again for your response and for the maintenance of the Firefly.

Would you recommend the 1668 build or the 1586 for the SQLite and stability?

thanks,
Rat

[rpedde]

@ratbag21 wrote:

Ron,

Thanks again for your response and for the maintenance of the Firefly.

Would you recommend the 1668 build or the 1586 for the SQLite and stability?

thanks,
Rat

1668 seems stableish, outside of some corner cases. 1671 is cooking now. There is really only one issue still outstanding, and that’s segfaults when connecting via rsp on mipsel boxes. Afaik.

But were I you, and just wanted to test it, it makes sense to test it on a stable apples-to-apples basis, so I’d probably go 1586.

— Ron

[ratbag21]

Ron,

Ok I did the install on 1586. It seems to have gone smoothly, however the I get an “error on reading the config file”. I did a full edit on the new config file so all the location etc. parameters matched my previous config file.

Do I need to install any other pkgs/files beyond “mt-daapd_svn-1586-1_armeb.ipk”?

Do I need to do a cold/clean reboot of the slug/server after install?

Can 1586 do sqlite3 ok [vs. other options] or do recommend another choice?

Do I need to remove the old version/type db file?

Can the sqlite3 db [or any db selected] location be the same as the db parameter location?

Do the options [settings] in the config file need to have “=” between the option name and the setting, or can there just be spaces as in previous config format?

I did an update of my “/etc/ipkg/firefly.conf” file [confirmed that the file contents read as directed] , however when I run the ipkg update command, the system says it is ignoring an unknown parameter, and then the package update comes from the “nslu2-linux.org”. The update to the file was to “src firefly http://ipkg.fireflymediaserver.org/armeb&#8221;, is this the correct syntax in the firefly.conf file for that location?

sorry about all the questions and I appreciate the responses…I think that covers all the odds and ends…

rat

[Author]

Posts