Feedback on 1236

Thanks for the pointer to the BugDB. Good to know that things are in such good hands!

That leave 1, 4, 6, 8, 9, 10. Anyone else seeing these?

M

2. iTunes tmp files

If you set “supports update” to “no” in the config page, does it still happen?

I can’t find a “supports update” in the config file or in either UI. Do you mean “Always Scan”? If you turn that off then it still happens.

I watched it a little yesterday (as it generated 300 TMP files) and it’s definitely Firefly causing the problem because if I stop the server the temp files stop too.

It appears that this version of iTunes creates xml temp files (when the library’s being updated) and then renames them once they’re complete. While ripping CDs I’m seeing the XML file timestamp update every minute or two (and the temp files are generated every minute or two).

The fact that it’s taking so long to rescan won’t be helping matters.

4. not picking up iTunes content regularly

Don’t know if that’s an iTunes thing or a me thing. I’ll try and replicate it. This is ticket #171.

This could be a symptom of the same problem. If the refresh is taking an hour to complete (according to the log) then that would explain why it’s not picking up the changes.

8. error adding iTunes playlist

That comes from duplicate playlists. If there is a backup copy of the iTunes xml file or something, then it tries to add another playlist with the same name — the second try at adding it fails.

Arguably, there could be multiple playlists with the same name, but I just haven’t allowed it. I’m not sure why. Confusion, maybe?

There aren’t multiple playlists in the XML file but if it was picking up some of the content from the TMP files, that could cause the problem. Perhaps the same TMP issue again?

9. memory leak

Aaaaah… that makes me think it’s in the itunes xml parsing. Great. No wonder I couldn’t find it on valgrind – my linux box with valgrind doesn’t have (duh) iTunes, so I’m not grinding against a real xml file. I’ll run this to ground. Ticket #173.

I came home today to find the server running at about 200M when I hadn’t seen it like that for days. I also found a lot of “unknown meta” items in the log (1212 of them). I hadn’t seen any memory problems or meta messages for days? A coincidence?

10. missing files

You can turn logging up to 5, then watch the logs on scan. You should get some sense of it there.

Otherwise, you can dump the database with something like

sqlite pathtosongs.db “.dump songs”

I’ve got the logging on at level 5 but it only gives me the final filename, not the full path which makes it harder to match. I’ll probably turn on debug 9 and then rebuild and see what’s being missed and why.

Cheers,
Mark

Hi there. I’ve been running in heavy debug mode for the last couple of days (and have a nice 130M log file to show for it that I need to do some grawking on).

Grommet, thanks for the config addition. On the meta errors, I know that’s a known bug, but I thought it was interesting that they seem to coincide with the memory leak. I’ll continue monitoring that one.

A couple of tidbits…
I’ve been running with the following settings:

always_scan = 0

scan_type = 2

supports_update = 0

…and with iTunes listening turned off as recommended. But when I rip CDs and the iTunes XML is being updated regularly, I’m still getting lots of TMP files because of the lock on the XML file. Would it be reasonable to take a temporary copy of the XML file for digestion leaving the main file unlocked?

While running in -9 mode I noticed the following behaviour during updates:

...

2006-06-22 : Rows: 506883

2006-06-22 : Adding itunes track #756

2006-06-22 : Executing: select type from playlists where id=443

2006-06-22 : Executing: select count(*) from songs where id=714

2006-06-22 : Executing: insert into playlistitems (playlistid, songid) values (443,714)

2006-06-22 : Rows: 506884...

This appears to be what’s taking all the time on the rescans (taking up to an hour). Surely it shouldn’t have managed to generate 500k rows. I don’t have that many playlists.

Also, when the server restarted, it seemed to decide that doing a full rescan of every song was a good idea. Is that normal? Here’s what I see in the log (this is the start after I’d done a complete rebuild so I wouldn’t have expected so much activity):

...

2006-06-22 : Scanning iTunes id #457

2006-06-22 : scan_path: file://localhost//Horace/multimedia/iTunes/iTunes Music/Nine Inch Nails/Pretty Hate Machine/02 Terrible Lie.mp3

2006-06-22 : Executing: select * from songs where path='\horacemultimediaitunesitunes musicnine inch nailspretty hate machine2 terrible lie.mp3'

2006-06-22 : Executing: INSERT INTO updated VALUES (417)

2006-06-22 : Rows: 1

2006-06-22 : Executing: select count(*) from songs where path='\horacemultimediaitunesitunes musicnine inch nailspretty hate machine2 terrible lie.mp3'

2006-06-22 : Executing: UPDATE songs SET title='Terrible Lie',artist='Nine Inch Nails',album='Pretty Hate Machine',genre='Other',comment='',type='mp3',composer=

'Trent Reznor',orchestra='Nine Inch Nails',conductor='',grouping='',url='',

bitrate=128,samplerate=44100,song_length=282566,file_size=4528935,

year=1989,track=2,total_tracks=0,disc=0,total_discs=0,

time_modified=1105002673,db_timestamp=1150958979,bpm=0,

disabled=0,compilation=0,rating=0,sample_count=0,codectype='mpeg' WHERE path='\horacemultimediaitunesitunes musicnine inch nailspretty hate machine2 terrible lie.mp3'

2006-06-22 : Rows: 1

2006-06-22 : Executing: insert into updated (id) select id from songs where path

='\horacemultimediaitunesitunes musicnine inch nailspretty hate machine2 terrible lie.mp3'

2006-06-22 : Rows: 1

2006-06-22 : Scanning iTunes id #458

...

Thoughts?

M

I’ve figured out why I’m missing so many files. Firefly seems to be struggling to map files from the iTunes XML to the file system if the file name or path has an accented character. An accent in the iTunes song name or details isn’t a problem, just if it’s in the file path.

Here’s an example of something that doesn’t make it through (iTunes string then filename):

Locationfile://localhost//Horace/multimedia/iTunes/iTunes%20Music/Bj%C3%B6rk/Debut/10%20Violently%20Happy.m4a



\HoraceMultimediaiTunesiTunes MusicBjörkDebutViolently Happy.m4a

The same problem occurs whether the extended character is in the path or the filename.

I’m surprised this isn’t being flagged in the logfile, even with level 9 debug.

Cheers,
Mark