svn-1652_deb_lenny_armLE compiles but intern. chars problem

[mas]

Compiles well. Few warnings.

There was at first a problem with the plugin dir – I had via my mt-daapd.conf set a different dir then what the .deb seemed to install to so was loading old libs and barking. After a symlink from /usr/lib/mt-daapd/plugins to my conf’ed location /usr/share/mt-daapd/plugins and a reinstall of the .deb it all works fine.

What I did notice on doing a full rescan was however that it does not really like special chars in filenames. See log file:

2007-09-06 01:31:26 (40021600): Could not open /Media/Musik/Die Aerzte/Geraeusch/cd2/10. Ruhig angehn – [Ger?usch (disc 1 Schwarzes Ger?usch)].mp3 for reading: No such file or directory
2007-09-06 01:32:07 (40021600): Skipping /Media/Musik/Phantom Der Oper/06. Die Musik der Nacht – [Das Phantom der Oper Die H?hepunkte der Hamburger Auff?hrung].mp3 – scan failed
2007-09-06 01:32:20 (40021600): Skipping /Media/Musik/Alizee – Mes Courants Electricques/06. Temp?te [Mes courants ?lectriques].mp3 – scan failed
2007-09-06 01:33:20 (40021600): Could not open /Media/Musik/Alicia Keys/The Diary of Alicia Keys/07. Diary (feat. Tony! Toni! Ton?!) [The Diary of Alicia Keys].mp3 for reading: No such file or directory

It seems it cant handle certain characters in directory names and filenames.

e.g. ?!äüöÄÜÖß

Here an examples as it should look also to mt-daapd:

-rwxrwxr-x 1 mt-daapd users 6822948 2007-01-28 13:43 07. Diary (feat. Tony! Toni! Ton?!) [The Diary of Alicia Keys].mp3

[rpedde]

@mas wrote:

Compiles well. Few warnings.

Still some cleanups to do there. I noticed them as well.

There was at first a problem with the plugin dir – I had via my mt-daapd.conf set a different dir then what the .deb seemed to install to so was loading old libs and barking. After a symlink from /usr/lib/mt-daapd/plugins to my conf’ed location /usr/share/mt-daapd/plugins and a reinstall of the .deb it all works fine.

Ya, I didn’t change the scan order, and I’m not checking for module versions yet. I need to.

What I did notice on doing a full rescan was however that it does not really like special chars in filenames. See log file:

That sucks. I thought I had that fixed.

Plus I don’t have a ‘nix box that doesn’t have utf-8 filenames. Grr…

I’ll see if I can find a way to replicate this.

— Ron

[mas]

Uhm, I thought I had UTF-8.

schnecke:~# locale
LANG=de_DE.UTF-8
LC_CTYPE=”de_DE.UTF-8″
LC_NUMERIC=”de_DE.UTF-8″
LC_TIME=”de_DE.UTF-8″
LC_COLLATE=”de_DE.UTF-8″
LC_MONETARY=”de_DE.UTF-8″
LC_MESSAGES=”de_DE.UTF-8″
LC_PAPER=”de_DE.UTF-8″
LC_NAME=”de_DE.UTF-8″
LC_ADDRESS=”de_DE.UTF-8″
LC_TELEPHONE=”de_DE.UTF-8″
LC_MEASUREMENT=”de_DE.UTF-8″
LC_IDENTIFICATION=”de_DE.UTF-8″
LC_ALL=

[mas]

Grrrrr. Fu… Unicode.

Now I checked it all and discovered some nasty stuff:

1. I had that slug set to de_DE.UTF-8 a while ago, thats german unicode locale.

2. So far so good. Only I hadn’t noticed that the samba daemon wasnt correctly transcoding between windows (which insists to use UTF16 – stupid Bill Gates) and my box. So some chars were screwed up by this.

3. I now tried for 1h to find a way to get samba to correctly transcode that. Darn. Sure it works somehow but I will be damned if I find out how.

Ok, so I now pulled the plug on unicode and reverted to locale

“de_DE” – no unicode.

Because for this I at least know how to get samba to work correctly:

smb.conf:
[global]
unicode = yes
display charset = ISO-8859-15
dos charset = 850
unix charset = ISO-8859-15

---

Now unicode is nuked but at least the transcoding between windows samba debian_lenny_LE works again. One run of my tagging program and the files in my Musik folder look again normal.

============

Ok, rechecking the mt-daapd problem now WITHOUT unicode.

Heres all errors/warnings I get:

2007-09-06 21:30:43 (40021600): Rescanning database
2007-09-06 21:30:48 (40e654e0): Cannot resolve /usr/share/mt-daapd/admin-root//favicon.ico
2007-09-06 21:30:48 (40e654e0): Thread 18: Entering ws_returnerror (404: Not found)

–> Nothing to do with unicode issues but apparently your missing a file on the website code.

And that is the great thing. No more errors. The new svn can handle “de_DE” encoded filenames. It does “only” bark on messed up unicode filenames, which it also shouldnt but at least we can prevent that by not using unicode.

2007-09-06 22:48:15 (40021600): Starting playlist scan
2007-09-06 22:48:24 (40021600): Updating playlists
2007-09-06 22:48:30 (40021600): Scanned 4151 songs (was 4151) in 1469 seconds

---

Conclusion: The new svn can handle äöüß if the are coded in de_DE without unicode. If they are in de_DE.utf-8 and something gets messed around with windows/samba then it barks. Still a but but a much less critical one.

[rpedde]

@mas wrote:

Grrrrr. Fu… Unicode.

Now I checked it all and discovered some nasty stuff:

1. I had that slug set to de_DE.UTF-8 a while ago, thats german unicode locale.

2. So far so good. Only I hadn’t noticed that the samba daemon wasnt correctly transcoding between windows (which insists to use UTF16 – stupid Bill Gates) and my box. So some chars were screwed up by this.

3. I now tried for 1h to find a way to get samba to correctly transcode that. Darn. Sure it works somehow but I will be damned if I find out how.

Ok, so I now pulled the plug on unicode and reverted to locale

“de_DE” – no unicode.

Because for this I at least know how to get samba to work correctly:

smb.conf:
[global]
unicode = yes
display charset = ISO-8859-15
dos charset = 850
unix charset = ISO-8859-15

---

Now unicode is nuked but at least the transcoding between windows samba debian_lenny_LE works again. One run of my tagging program and the files in my Musik folder look again normal.

============

Ok, rechecking the mt-daapd problem now WITHOUT unicode.

Heres all errors/warnings I get:

2007-09-06 21:30:43 (40021600): Rescanning database
2007-09-06 21:30:48 (40e654e0): Cannot resolve /usr/share/mt-daapd/admin-root//favicon.ico
2007-09-06 21:30:48 (40e654e0): Thread 18: Entering ws_returnerror (404: Not found)

–> Nothing to do with unicode issues but apparently your missing a file on the website code.

And that is the great thing. No more errors. The new svn can handle “de_DE” encoded filenames. It does “only” bark on messed up unicode filenames, which it also shouldnt but at least we can prevent that by not using unicode.

2007-09-06 22:48:15 (40021600): Starting playlist scan
2007-09-06 22:48:24 (40021600): Updating playlists
2007-09-06 22:48:30 (40021600): Scanned 4151 songs (was 4151) in 1469 seconds

---

Conclusion: The new svn can handle äöüß if the are coded in de_DE without unicode. If they are in de_DE.utf-8 and something gets messed around with windows/samba then it barks. Still a but but a much less critical one.

Talk about a corner case.

Do you have any messed up filenames still? Could you grab a file with path and put it on a small filesystem and gzip and email it to me?

something like:

dd if=/dev/zero of=bad.img bs=1024 count=8192
mke2fs bad.img
mount bad.img /mnt -o loop
tar -cf – /full/path/to/bad/mp3/from/root.mp3 | tar -C /mnt -xf –
umount /mnt
gzip bad.img

that should keep full path filenames and what not, and it would give me a broken file system I could try.

my email is [email protected]

Apologies if that was obvious, but I just wanted to make sure I had the same not-quite-right filenames.

— Ron

[mas]

Well I did correct em all now.

But as I still now how they got corrupt I just replicated it by messing up one file again. Cant guarantee its the same as before but looks definitly also messed up to me now.

Sending you a DL link via email (too big for email).

[Author]

Posts