ogg problem.. whatever

[fizze]

hm, according to the log, firefly “digested” 3 of the mlarge ogg files in a flick

2006-09-02 11:17:18 (c4050bdf): Skipping file, not modified
2006-09-02 11:17:18 (c4050bdf): Found c:musicP-Z/Sunny Side Up FM4
2006-09-02 11:17:18 (c4050bdf): Found Sunny Side Up FM4.. recursing
2006-09-02 11:17:18 (c4050bdf): Found c:musicP-ZSunny Side Up FM4/sunny_side_up_20041121.ogg
2006-09-02 11:17:18 (c4050bdf): Executing: select * from songs where path=’c:musicP-ZSunny Side Up FM4sunny_side_up_20041121.ogg’ and idx=0
2006-09-02 11:17:18 (c4050bdf): Executing: INSERT INTO updated VALUES (2689)
2006-09-02 11:17:18 (c4050bdf): Rows: 1
2006-09-02 11:17:18 (c4050bdf): Skipping file, not modified
2006-09-02 11:17:18 (c4050bdf): Found c:musicP-ZSunny Side Up FM4/sunny_side_up_20040711.ogg
2006-09-02 11:17:18 (c4050bdf): Executing: select * from songs where path=’c:musicP-ZSunny Side Up FM4sunny_side_up_20040711.ogg’ and idx=0
2006-09-02 11:17:18 (c4050bdf): Executing: INSERT INTO updated VALUES (2690)
2006-09-02 11:17:18 (c4050bdf): Rows: 1
2006-09-02 11:17:18 (c4050bdf): Skipping file, not modified
2006-09-02 11:17:18 (c4050bdf): Found c:musicP-ZSunny Side Up FM4/sunny_side_up_20040801.ogg
2006-09-02 11:17:18 (c4050bdf): Executing: select * from songs where path=’c:musicP-ZSunny Side Up FM4sunny_side_up_20040801.ogg’ and idx=0
2006-09-02 11:17:18 (c4050bdf): Found music file: sunny_side_up_20040801.ogg
2006-09-02 11:17:18 (c4050bdf): Codec type: ogg

but its hogging the CPU since, and using about 8M RAM and 5M virtual memory, on my Windows PC.

it’s been a little more than 10 minutes like that, so far.
and, yes, Im runnning -d9.

I guess its some weird race /deadlock.

[fizze]

Alright, heres the update on this 😉

I just ran some tests with the svn-1463 on ubuntu. Same files, and ogginfo displayed no problems whatsoever.

Heres the log output:

2007-01-09 23:56:28 (b7cf56b0): Found /media/hda1/music/p-z/Sunny Side Up FM4

2007-01-09 23:56:28 (b7cf56b0): Found Sunny Side Up FM4.. recursing

2007-01-09 23:56:28 (b7cf56b0): Found /media/hda1/music/p-z/Sunny Side Up FM4/s

2007-01-09 23:56:28 (b7cf56b0): Executing: select * from songs where path='/med

2007-01-09 23:56:28 (b7cf56b0): Found music file: sunny_side_up_20041121.ogg

2007-01-09 23:56:28 (b7cf56b0): Codec type: ogg

2007-01-09 23:56:32 (b64aaba0): Thread 41: Entering ws_dispatcher (Connection f

2007-01-09 23:56:32 (b64aaba0): Thread 41: got request

2007-01-09 23:56:32 (b64aaba0): Request: GET /xml-rpc?method=stats HTTP/1.1M

2007-01-09 23:56:32 (b64aaba0): Thread 41: Read: Host: localhost:3689M

2007-01-09 23:56:32 (b64aaba0): Thread 41: Adding header *Host=localhost:3689*

2007-01-09 23:56:32 (b64aaba0): Added *Host=localhost:3689*

From then on its just stuck serving the XML-RPC method to my Firefox. Weird.
There is no apparent memory hole or other race, it just seams that scanning-pthread is deadlocked.

This is merely meant for me so I can document my debug endeavors here. 🙂

[rpedde]

@fizze wrote:

Alright, heres the update on this 😉

You sent me one once before, didn’t you?

[fizze]

Yup.
Anyhow, I want to debug this issue first, which I can do on my PC just as well as on the slug. (Well, a lil better I think, dont want to use gdb on the slug, networked if I dont have to)

Again, if I call ogginfo manually on this file it takes maybe 2-3secs to display the info. Havent tried to do this as the user mt-daapd runs as, but that shouldnt be an issue.
I had other ogg files which were smaller working fine.

edit:
Well, I just found out something.
ogginfo shows me that some of those files do have a “hole” in the stream.
They play just fine though. I read on some forums that ID3V2 tags do this to ogginfo, and ogg files. So I stripped my tags with id3v2, but it didnt help.

I found one “huge” file wihtout a hole and mt-daapd scanned it properly.
It seems the “hole” somehow kicks libvorbis’s tag reading out of sync 😉

update2:
Wow. actually I never tried to play those in Linux on a PC. Xmms chockes on the files with a hole. So does rhythmbox & beep-media-player.

They all play back fine in winamp5 and on my iRiver H320. both in rockbox and the normal iRiver FW. This is really weird.
On the slug the transcoding worked at some point, so I guess the tremor libs dont have this bug. That would explain why the iRiver DAP doesnt choke on them.

[xaviour]

@rpedde wrote:

I hadn’t, but I’m back on feature adding/bug fixing again, so I’ll have a look at it.

Ron, while you are at it, instead of writing your own parser for ogg maybe you should consider using more taglib than you do currently. I have been doing of lot of testing of taglib against the MP3 scanner of Firefly. It seams that the version of taglib in subversion with the proper patches works better than the parser of Firefly for MP3 file.

I also patched Firefly to use taglib for handling MP3 and ogg files. It is very simple. If you want, I can provide you with the files.

I have been spending a lot of time with MP3 and ID3 tags and I must recognize that it is a nightmare. Writing tag parsers might not be the best use of time and nerves…

[fizze]

w00t!

please provide those files to me too? I’d like to test this on my slug.

edit:
eek, just found out there is no taglib for optware atm.
too bad 🙁

[rpedde]

@xaviour wrote:

@rpedde wrote:

I hadn’t, but I’m back on feature adding/bug fixing again, so I’ll have a look at it.

Ron, while you are at it, instead of writing your own parser for ogg maybe you should consider using more taglib than you do currently. I have been doing of lot of testing of taglib against the MP3 scanner of Firefly. It seams that the version of taglib in subversion with the proper patches works better than the parser of Firefly for MP3 file.

I also patched Firefly to use taglib for handling MP3 and ogg files. It is very simple. If you want, I can provide you with the files.

I have been spending a lot of time with MP3 and ID3 tags and I must recognize that it is a nightmare. Writing tag parsers might not be the best use of time and nerves…

I wouldn’t mind seeing them. A single tag reader would be nice, but again, it’s the C++ nature of it that seems problematic. Wouldn’t mind playing with it though, to see if I could get it working on an embedded target.

Oh. You can email me at [email protected].

— Ron

[xaviour]

@rpedde wrote:

I wouldn’t mind seeing them.

See URL sent in personal email. Because I did that some times ago so some modified files might be missing. Maybe you should just read the code quickly and if you like what you see then I can work on making a proper patch so that you can test it at leisure. I am not familiar with autoconf/automake so I cannot patch the configuration files.

@rpedde wrote:

A single tag reader would be nice, but again, it’s the C++ nature of it that seems problematic.

I actually used the C bindings of taglib so the hairy binary compatibility problems of C++ have not influence here. Testing with my reference database shows that taglib is as fast or faster than Firefly mp3 parsing functions. This same testing shows that memory consumption is decent. There were a couple of memory leaks. I found them with valgrind and plugged them.

As far as I could see, the only shortcoming of the C bindings is that they do not expose the “reading style” (i.e scan effort in Firefly terminology). A change improving the C bindings should not be to hard to push upstream.

@rpedde wrote:

Wouldn’t mind playing with it though, to see if I could get it working on an embedded target.

I work exclusively on embedded targets (NSLU2 or Storcenter) so there is nothing to worry about. Someone damaged their configurations files but there is a workaround for it.

Finally I just would like to point out that with the code base in subversion, Firefly is better at reading metadata than taglib. However, with my series of patches taglib does a better job than Firefly. This statement is of course valid only for my reference database and your millage might vary.

The biggest problem I see with taglib is that the maintainer is currently very busy and has no time to integrate the patches. It is a pity that support for M4A and WMA is available in Amarok subversion repository but has not been merged yet into the official taglib repository.

As a side note, I have seen some activity on the forum regarding management of the tags from the web interface of Firefly. If this ever was to become a feature, know that taglib has write support and that the API is as simple as the read API.

[fizze]

Right!
Finally some progress with my “huge” ogg files, thingy.

2007-03-24 12:48:37 (00342405): Session 0: Streaming file 'world_wide_20050109.ogg' to 192.168.1.112 (offset 0)

2007-03-24 12:48:37 (00342405): Executing /opt/sbin/mt-daapd-ssc.sh "/share/hdd/data/public/music/new/world_wide_20050109.ogg" 0 7065.000 "ogg"

2007-03-24 12:50:20 (00349805): Session 0: Streaming file 'world_wide_20041219.mp3' to 192.168.1.112 (offset 0)

2007-03-24 12:50:20 (00339404): Write error: Broken pipe

2007-03-24 12:50:20 (00339404): Finished streaming file to remote: 2195456 bytes

2007-03-24 12:50:22 (0034a004): Session 0: Streaming file 'world_wide_20050109.ogg' to 192.168.1.112 (offset 0)

2007-03-24 12:50:22 (0034a004): Executing /opt/sbin/mt-daapd-ssc.sh "/share/hdd/data/public/music/new/world_wide_20050109.ogg" 0 7065.000 "ogg"

2007-03-24 12:50:28 (0034ac04): Session 0: Streaming file 'world_wide_20041219.mp3' to 192.168.1.112 (offset 0)

2007-03-24 12:50:29 (00349805): Write error: Broken pipe

now, the ogg file is some88MB in sice. its a 2h recording of a radio broadcast. Now decoding this as a complete piece on the NSLU2 would take up a considerable amount of time.
When exactly does mt-daapd or wavstreamer start to stream? Does this really happen on the fly?
I guess that oggdec chokes because the slug lacks memory.

I guess a workaround for this could be to save the wav file in a temporary directory, or even save it at all, as suggested by someone with a patch earlier….

edit:
Although the log sais, that bytes were streamed to he client, the soundbridge doesnt actually play anything. It immediatly aborts, telling me unable to play content.

[rpedde]

@fizze wrote:

Although the log sais, that bytes were streamed to he client, the soundbridge doesnt actually play anything. It immediatly aborts, telling me unable to play content.

It does indeed stream it out on-the-fly. As it transcodes, it streams what it has to the client in 1K blocks.

[Author]

Posts