Preventing full reload

[rpedde]

@blamm wrote:

I disagree. If a large percentage (configurable?) of the files are missing then clearly something may be wrong and Firefly should ask first before clearing the DB.

Again, I’ll take that patch. I imagine that if you start working on it you’ll see the difficulties inherent in it, particularly for those machines that are headless, or embedded, or basically aren’t windows machines.

And for what gain? It doesn’t change the fact that the server doesn’t work when someone tries to play a song that doesn’t exist anymore, and in fact works no better than the solutions (skip first scan & quick startup) enumerated above.

And even if it were to work spectacularly and simply, it violates the dictum of mechanism not policy. I don’t want software that stops me from shooting myself in the foot. Someday I may just want to do that. If I wanted software that stopped me from doing what I wanted, I’d just run windows and WMC and be done with it.

[S80_UK]

@rpedde wrote:

If I wanted software that stopped me from doing what I wanted, I’d just run windows and WMC and be done with it.

Fantastic! Can I have that for my email signature?

[CCRDude]

To give NeomaD a point, on Windows you could check for each drive what type it is, and Windows could tell you its a removable drive. But then, that’s only Windows, and to finally take that point away again, what about CDs you had in the drive only once etc. 😉

A general idea for speeding up such rescans would be to not remove songs from the database, but flag them, in a new boolean field (“filexists” or something), as N/A. Any functions processing the database should then ignore entries where this is set (a simple additional parameter to the queries). This way, on a rescan it would only have to remove the N/A flag instead of parsing the same file again.

That combined with a “lastseen” database column could allow to only remove entries finally from the DB after time X, e.g. when the songs haven’t been seen for a week or two, making accidental short outages of an external or network drive less time-consuming on the rescan.

As Ron said: if you feel this is important, feel free to provide a patch for that 😀

PS: My song database is not even half that big, for more files. Are SQLite2 databases three times as big?

PPS: One occasion where its normal that such a large number of files “disappear”: if you retag your files and change your directory and/or filename structure, for example.
For a server application, “asking the user” about decisions is always bad (unless its when the user aka admin actively is configuring the server), since who knows if he’ll access the user interface the next day or three months from now? I can go for weeks without looking at Firefly, and if it wouldn’t find new files I moved to its network drive, I would think something is wrong actually, while it just sits there and waits for me to give it an answer what it should do about those missing files…
Hmmm. One way to combine the last situation with my example above: when > 50% of all files are not found, 1. set the N/A flag, 2. set a global flag that would, if you access the web interface, show you the “problem” and allow you to remove those entries with one click (contrary to the “after time X” described above).

[NeomaD]

Ok, I get the messege. Ill try to not start the server when my NAS is offline.

I was just thinking, Its allright that it starts over again if the media isnt there, BUT if it cant contact the place where the media should be eg D: or my NAS, then it maybe should react in another way.

And my second question still is a mystery? Why dont it index all my songs??
Is there some advice for me?

Thx again for great job with the server and answering my questions!
//Martin

[rpedde]

@NeomaD wrote:

Ok, I get the messege. Ill try to not start the server when my NAS is offline.

Sorry. Just hit me wrong somehow. This is software I’ve been working on for something like three years, off and on. Mostly on. I spend somewhere around 12 to 15 hours a week answering forum posts and emails. I *try* to do some coding every day on it (although forum volume lately has slow me down significantly).

My point is that I have a lot of time and effort invested in this, and I take it very personally. So when I read your post, I somehow internalized something like: “This isn’t even good enough for free software. It sucks. You suck.”

I realize you probably didn’t mean it that way, but I manged to spin myself up a bit. Sorry. So perhaps I was a bit snippy. 😳

I was just thinking, Its allright that it starts over again if the media isnt there, BUT if it cant contact the place where the media should be eg D: or my NAS, then it maybe should react in another way.

Maybe. But it’s a much harder problem that it appears. In the case of a windows machine, maybe it’s fairly easy. But to generalize that to a headless nas box sitting in a closet. Thats harder.

Plus, my point about mechanism versus policy meant that you never know what the right thing is.

What if someone were using it to plug and unplug an iPod. Such that when they plugged in the iPod, it indexed it and the songs were available in the music server. But when they undocked it, they disappeared. It would be a misfeature to keep the songs around in that case, or to bug them about whether or not they really wanted to remove the files.

Or (like me), some people might have an “incoming” folder, where stuff gets put when it gets… uh… ripped from an appropriately purchased CD, and digitized only for personal archival use, or other fair use. (eek!)

Once it gets tagged, arted, and verified, then I move it into my collection. But I don’t want songs I delete (or move into my main collection) still in the database from their old location in the /incoming folder.

So to me, mechanism versus policy would suggest that the computer doesn’t know how you are using it, and that you have to make that decision yourself. In this case, a reasonable thing to do might be to suppress scanning at all, and only scan when you know the device is online and has updated music (skip_initial set to 1, scan_interval set to 0, scan via the web interface). Or use a script to determine when the disk goes online, and start the server and kick off a manual scan. Then stop the server when the disk goes offline (again, via a script).

Or any number of other solutions. But I don’t like solving problems based on a single use-case that isn’t shared by everyone else. That’s all I was really getting at. (plus extra snippiness, of course).

And my second question still is a mystery? Why dont it index all my songs??
Is there some advice for me?

It’s either that the user the server is running as doesn’t have permissions to the files in question (hint: change the user it runs as in control panel -> administrative tools -> services -> firefly media server -> properties -> Logon)

or…

It’s a file type (.flac, .ogg, .wma, etc) that isn’t in the “extensions” config variable.

Thx again for great job with the server and answering my questions!
//Martin

np. Sorry to be a dick. Didn’t really mean to, just sometimes stuff just hits me wrong. 🙂

Peace.

— Ron

[NeomaD]

Yeh, I got the impression you where insulted by my question. I appologise for bad english…
(This was the best swedish teachers could do with me in nine years of english-studies! :-))

I fully understand that this is a great undertaking and that you not can implement functions and features just because I think something isnt allright.

But my second question is still a mystery…

It seems like Firefly is “serving” all my songs (according to the filecount)…
…but in my soundbridge not all are listed…

In my musicfolder I have an artist folder and in the artistfolder I have a folder for each record. And for Artist A, only Vol 2 can be found by Soundbridge…
..strange ey?

All mp3, all with nifty looking ID-tags…

Eg.
Artist A
– Vol 1
– Vol 2
– Vol 3

Have this happend before?

//Martin

[rpedde]

@NeomaD wrote:

Artist A
– Vol 1
– Vol 2
– Vol 3

Have this happend before?

//Martin

Usually, thats a permissions thing. You can increase your debuglevel through the web interface to 5 or more. Once you have that, do a full scan and look at the log in c:program filesfirefly media serverfirefly.log.

You should be able to see where it looks in the “artist a” directory, then just look as it walks through the files in that folder. It should say something about having problems with the folder or the files… maybe permission denied, or a problem reading the tags, or something of that nature. That will probably give you the hint you need.

— Ron

[NeomaD]

Im right now doing some reserach in this matter…

Its no permission thing though the files are in the same directory…

It dont say there are any problem, but I found something peculiar…

When I look at the tags in “MP3-tag”, “Windows explorer” and “Winamp” they all look good. (Some of them differed between v1 and v2, but that shuoldnt be a problem=? Right?)

But Firefly doesnt index the Artist-tag, (This is an assumption though I cant brows these songs by artist, but the are displayd in genre or if I serch for their song name.)

As a test, I tried to restate the Artist, with the exact same name, and saved the ID-tags…
…and voila! Firefly now also indexed the Artist-tag.
Almost everything solved this way, with a new resaved ID-tag. Just almost though, I shall try to resave the files that are badly iondexed again.
And see if that works.

Thought you could find yhis info intresting.

//Martin Stegmark

[CCRDude]

If it doesn’t destroy any important information, try to remove the ID3v2 completely, save, then re-add it. I’ve experienced a few tags were broken things (some taggers have problem with unsynchronized tags, for example) changed tags in a way that resulted in only parts were read, and just resaving didn’t always help (since broken tags in between would have been conserved).

[rpedde]

@NeomaD wrote:

But Firefly doesnt index the Artist-tag, (This is an assumption though I cant brows these songs by artist, but the are displayd in genre or if I serch for their song name.)

As ccrdude mentioned, I found stuff like this to happen when the id3v1 and id3v2 tags are different.

Firefly prefers id3v2 tags, and ignores id3v1 tags in favor of them. Other taggers may do thing differently. Or perhaps a tag that other applications read correctly, firefly (or rather libid3tag) doesn’t read, and so Firefly falls back to the id3v1 tag, since it doesn’t see a (valid) id3v2 tag.

I’ve only seen the second case once, but it’s possible that’s what you are seeing as well. Corrupt id3v2 tags, or unsynchronized id3v1 and id3v2 tags.

[Author]

Posts