some ratings not getting read from iTunes library

[js]

Hi, first off thanks for the great software! I’m enjoying using it on my new NSLU2.

I’m running the pre-built svn-1523 on an NSLU2 running Unslung 6.8

I copied my iTunes library from my Mac. All music files sit under /public/mp3 on the NSLU2, and my library file is at /public/mp3/iTunes Music Library.xml. There are about 3000 songs in my library.

When rescanning the library, some of the songs aren’t getting their ratings imported correctly. Here is a snippet from the log (at debug level 9) of two songs, one of which got the rating set correctly, and one that did not:

2007-04-09 16:09:24 (00000400): Scanning iTunes id #954

2007-04-09 16:09:24 (00000400): scan_path: file://localhost/Users/jason/Music/iTunes/iTunes Music/Beck/One Foot in

the Grave/04 Burnt Orange Peel.mp3

2007-04-09 16:09:24 (00000400): Mapping

file://localhost/Users/jason/Music/iTunes/iTunes Music/Beck/One Foot in

the Grave/04 Burnt Orange Peel.mp3 to /public/mp3/Beck/One Foot in the

Grave/04 Burnt Orange Peel.mp3

2007-04-09 16:09:24 (00000400): Executing: select * from songs where

path='/public/mp3/Beck/One Foot in the Grave/04 Burnt Orange

Peel.mp3' and idx=0

2007-04-09 16:09:24 (00000400): Executing: INSERT INTO updated VALUES (175)

2007-04-09 16:09:24 (00000400): Rows: 1

2007-04-09 16:09:24 (00000400): Executing: select count(*) from songs

where path='/public/mp3/Beck/One Foot in the Grave/04 Burnt Orange

Peel.mp3' and idx=0

2007-04-09 16:09:24 (00000400): Executing: UPDATE songs SET title='Burnt Orange Peel',artist='Beck',album='One Foot in the Grave',genre='',comment='',type='mp3',composer='',orchestra='',

conductor='',grouping='',url='',bitrate=192,samplerate=44100,

song_length=98977,file_size=2375599,year=0,track=4,total_tracks=0,

disc=0,total_discs=0,time_modified=1176046565,db_timestamp=1176160164,

bpm=0,disabled=0,compilation=0,rating=80,sample_count=0,

codectype='mpeg',album_artist='' WHERE path='/public/mp3/Beck/One Foot in the Grave/04 Burnt Orange Peel.mp3' and idx=0

2007-04-09 16:09:24 (00000400): Rows: 1

2007-04-09 16:09:24 (00000400): Executing: insert into updated (id) select id from songs where path='/public/mp3/Beck/One Foot in the Grave/04 Burnt Orange Peel.mp3' and idx=0

2007-04-09 16:09:24 (00000400): Rows: 1

2007-04-09 16:09:24 (00000400): Scanning iTunes id #955

2007-04-09 16:09:24 (00000400): scan_path: file://localhost/Users/jason/Music/iTunes/iTunes Music/Beck/One Foot In

The Grave/Cyanide Breath Mint.mp3

2007-04-09 16:09:24 (00000400): Mapping file://localhost/Users/jason/Music/iTunes/iTunes Music/Beck/One Foot In

The Grave/Cyanide Breath Mint.mp3 to /public/mp3/Beck/One Foot In The

Grave

2007-04-09 16:09:24 (00000400): Executing: select * from songs where path='/public/mp3/Beck/One Foot In The Grave' and idx=0

2007-04-09 16:09:24 (00000400): Error statting: No such file or directory

2007-04-09 16:09:24 (00000400): Executing: select * from songs where path='/public/mp3/Beck/One Foot In The Grave' and idx=0

(sorry if there are some weird linebreaks in there — had a little trouble getting it formatted right).

You’ll see that on the second song, the mapping for the local file looks to be truncated.

Permissions on both files are the same:
-rwxrw-r– 1 guest everyone 2375599 Apr 8 08:36 /public/mp3/Beck/One Foot in the Grave/04 Burnt Orange Peel.mp3
-rwxrw-r– 1 guest everyone 2328578 Apr 8 08:36 /public/mp3/Beck/One Foot in the Grave/Cyanide Breath Mint.mp3

Both songs show up in the firefly database, and are able to be played — it’s just the one doesn’t get updated correctly when the library is scanned.

I also just checked, and noticed that the songs who don’t get their ratings updated, also don’t appear in the playlists imported from iTunes.

Any other information I could provide that might be helpful?

[dougyp]

Hello,

This isn’t a direct answer but I suspect you have a similar problem that I’ve been trying to trudge through (I haven’t had time to focus on it to submit it to the forums but maybe someone knows what’s going on?).

I suspect that you’ve got a case-sensitivity issue/incompatibility between iTunes and the Unix path/filenames. I’ve taken 2 samples from your log:

1) path=’/public/mp3/Beck/One Foot in the Grave/04 Burnt Orange Peel.mp3′ and idx=0

2) file://localhost/Users/jason/Music/iTunes/iTunes Music/Beck/One Foot In The Grave/Cyanide Breath Mint.mp3

Notice that the directory for one of the files is “…In The Grave…” and the other one is “…in the Grave…”.

Unix is of course case-sensitive and as near as I can tell iTunes is not (well – windows certainly isn’t). I think when iTunes is running on your PC (I assume windows – ouch!). That it doesn’t matter “in the” versus “In The” becuase Windows will find the file either way. But Unix won’t…

Now, that said – I do think that Firefly is somewhat inteligent here – at least it seems to find the files sometimes – but I have definitely found that many of my MP3 files end up missing from all the playlists and in the couple of samples that I’ve debugged to the lowest level -it’s ended up being a case-sensitive issue…

I also noticed that when I upgraded to a more recent version (maybe a month ago), things were much better but still not 100%.

To top it off – Windows and iTunes under windows aren’t very cooperative when you try to fix the case of the folder or MP3 file. Sometimes it takes, sometimes not. And when you are dealing with 12,000 songs it’s almost impossible to debug all the problems.

Anyone else seen case-sensitivity issues and found ways to fix? I HAVE searched around but not come up with a thread that sounded like my issue. But there’s always that good chance that I missed something.

[js]

Ah, I think you’re right. I knew another pair of eyeballs would be able to spot a difference. I’m actually running iTunes on OS X — but since it’s not case-sensitive either, it would have the same problems.

I think I need to just fix the file path in my iTunes library but I’m not sure how to do that, short of deleting and re-importing the songs.

Will try later and let you know if that fixes things. Thanks!

[rpedde]

@js wrote:

Ah, I think you’re right. I knew another pair of eyeballs would be able to spot a difference. I’m actually running iTunes on OS X — but since it’s not case-sensitive either, it would have the same problems.

I think I need to just fix the file path in my iTunes library but I’m not sure how to do that, short of deleting and re-importing the songs.

Will try later and let you know if that fixes things. Thanks!

There is a bit to twiddle for case-insensitive file systems… it’s scanning/case_sensitive. Set that to 0 for non-case sensitive file systems (like osx, which is case preserving but not sensitive on a default format).

You’ll need to do a full scan after that, though.

— Ron

[dougyp]

Thanks Ron,

I found that setting in the web-admin page (advanced settings) but since my admin page is non-writable I can’t set it (I assume I could start as root and it’d work). But – before I do that, is there a startup option or a new parm I can add to the mt-daapd.conf file instead?

I searched the forums for “case AND scan” and couldn’t find reference to anything ‘cept the admin web-page. I checked my mt-daapd.conf file and the only 2 scan parameters that I see are “rescan_interval” and “scan_type”.

Hoping it’s a startup option or new parm I can add?

thanks,
-Doug

[rpedde]

@dougyp wrote:

Thanks Ron,

I found that setting in the web-admin page (advanced settings) but since my admin page is non-writable I can’t set it (I assume I could start as root and it’d work). But – before I do that, is there a startup option or a new parm I can add to the mt-daapd.conf file instead?

I searched the forums for “case AND scan” and couldn’t find reference to anything ‘cept the admin web-page. I checked my mt-daapd.conf file and the only 2 scan parameters that I see are “rescan_interval” and “scan_type”.

Hoping it’s a startup option or new parm I can add?

thanks,
-Doug

chown guest /opt/etc/mt-daapd/mt-daapd.conf

should get you a writeable web config. As far as the option goes, it’s in the “show advanced” seciton of the web config, under scanning, or if you are editing the config file directly, look for a section like:

[scanning]

and add the line:

case_sensitive=0

under it. If there is no scanning section already, just add it at the bottom:

[scanning]

case_sensitive=0

— Ron

[js]

OK, got my missing ratings.

The case_sensitive actually didn’t help. The actual files getting scanned are on an ext3 filesystem, though they come originally from a Mac.

I did go through and make sure the case on all directories was the same on both in the firefly library directories, and under the original itunes library directories, and that the case was correct in the itunes xml file.

Once I had everything consistent, I copied the xml file over to the firefly library directory, and rescanned. Ratings and playlists then got populated correctly.

The only things I have out of sync now are songs with non-ASCII characters in the title, but I think that just requires turning on utf-8 support on the NSLU2, right?

Whew! Thanks, guys.

[rpedde]

@js wrote:

The case_sensitive actually didn’t help. The actual files getting scanned are on an ext3 filesystem, though they come originally from a Mac.

Now that I think of it, that’s probably true… while it was on the mac it was case insensitive, but when it’s on ext3, it’s case sensitive. So yeah, I’m not sure how to programatically fix that.

The only things I have out of sync now are songs with non-ASCII characters in the title, but I think that just requires turning on utf-8 support on the NSLU2, right?

Yeah, but it probably has to be before you copy the files over. You can probably use iconv to convert from utf-8 (which the xml file is) to codepage, though.

[Author]

Posts