From: "Timothy Motika" <###@###.###>
To: "JavaSound" <###@###.###>
Subject: [javasound] consumers and producers
Date: Tue, 29 Dec 1998 17:25:36 -0800
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3

Dear JavaSound,

I remember subscribing a year ago and wondering if Sound would
ever emerge. Finally it has, and it looks neat! How wonderful.

I have not resubscribed for long, and the archive appears to be down
(or at least when I checked twice), so if I offer duplicate questions,
please just point me towards the appropriate information resource.
I have looked through the documentation and and the API specs,
but things are sketchy.

I am still writing a sound manipulation and sequencing application,
with of course its own features. JavaSound should be able to serve
this by allowing me to be a generic consumer of its audio MediaFormat
and MIDI codecs, and then a producer for its audio output controllers
through a channel. Hopefully I could also be a producer for write
codecs, but I doubt that will happen, except it should happen with
MIDI due to file/device similarities. What I don't understand is, I have looked
at
the Buffer and the MediaStream, and while it appears that data is
probably transported through a byte[], what is the format of that byte[] data?
Since the example audio playing application sets the audio format
of the output channel as well, to match that of the source, is the information
in that byte[] decoded from ulaw/wav/MPEG/whatever into alternating
L/R stereo sample bytes (if stereo) at a given sampling rate? Or is it
not decoded? What is the Header on MediaStream's getHeader()?

It smells like transport is mostly opaque, or maybe it's just undocumented.
Header can be any object, though, pointing to opacity, and it sure smells
like example Capture's FilePushSink is just GIGO and writes preformatted
data (even though I couldn't get the example to record under NT4 even though
I can record sounds through the system, though it's an early release).
Is this where TransportEvent's help?

I assume I am not fully informed. If someone could direct me towards
information describing this it woud be definitely appreciated. Sadly,
but understandably there's wasn't source to clarify what was missing
with the documentation. Thanks!

Tim
###@###.###

-----

From: "Timothy Motika" <###@###.###>
To: "Kara Kytle" <Kara.Kytle@Eng>
Cc: "JavaSound" <###@###.###>
Subject: Re: [javasound] consumers and producers
Date: Tue, 5 Jan 1999 12:36:52 -0800
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3

Dear Kara,

Thank you very much.

Hmm. This is actually the conclusion I had come to,
barring the getHeader() details.

So the transport is the byte[], not unlike that I'd
get by reading the audio file with an InputStream
for any format? Including header information, or
is that consumed?

See, the problem is, even if transport isn't opaque,
because of the format information carried with the
stream/buffer, I still have to know how to decode the format from
the byte[]. Which means, despite JavaSound, I still have
to write my own codecs for each format and
subencoding/compression I want to support, right?

If so, I can understand, but still that's pretty frustrating.

I know that your codecs each cut the data down into a
bunch of samples at a given sampling rate
somewhere along the line, and feed them to the hardware
device driver, or intermediary controller, or something.
I also realize that java has to worry about platform
compatibilities, endianness, etc which complicates
things, as well as speediness and synchronization
to keep the audio flowing at a good rate without sucking
every processor cycle. However, all I want to do is
consume audio data at some predefined byte/short/float
sample rate per voice. I don't care what the data smells
like, as long as I can find out some determistic way to
read it per format, without having to worry about compression,
etc.

Do I have to look like a device to do this?

Is there any possible way to do this? I really shouldn't have to
rewrite the various nasty MPEG codecs, since you've already
done that.

How much parsing do the com.sun.media.sound.*StreamParser's
do? Hey, maybe what I want is some way to request use of the
com.sun.media.sound.FormatConverter. Then, if the file is some
nasty format, as I'm sure MP3 is (once implemented), I don't care,
because I can request conversion and reap 16-bit WAV or whatever.

Is there a way to do this? If not, I'm afraid I must sadly express, as
others have, that I have difficulty differentiating JavaSound and JMF.

Thanks for your patience.

Tim

PS. About capture failing on NT, removing the 1.2 JMF installation fixed
things.

-----

My apologies for not including a subject on this earlier, and for the subsequent
"fixed" duplicate repost, but this one will survive the mail filters and
file appropriately.
I knew I sent it too soon. Sorry.
------------------------------------------------------

Here's request #1:

From: Matthew Jones <###@###.###>
Date: Monday, January 04, 1999 7:40 PM
>Does it seem to anyone else that the JMF 2.0 and JavaSound API are in part
>redundant and conflicting?
>
>Why does both the JMF and JS need capture and they seem to get access in
>different and incompatible ways. Wouldn't it make more sense to have JMF
>get access to the devices for JS?
>
>It seems to me that an API of this type does three things
>
>1. Input
> Read from file
> Read from device (midi port, microphone, network....)
>
>2. Expose the data it some meaning full way so the content can be
>created/manipulated by a programmer.
>
>3. Output
> Write data to a file or device
>
>I think where the conflict is in 1) mostly and maybe a little 3).
>
>I think that 1) and maybe 3) should be coalesced into the JMF and let JS
>handle 2).
<...>


Here's my burning request #2:

From: Kara Kytle <###@###.###>
Date: Friday, January 08, 1999 10:54 AM
<..>
> <My earlier post>
>> I know that your codecs each cut the data down into a
>> bunch of samples at a given sampling rate
>> somewhere along the line, and feed them to the hardware
>> device driver, or intermediary controller, or something.
>> I also realize that java has to worry about platform
>> compatibilities, endianness, etc which complicates
>> things, as well as speediness and synchronization
>> to keep the audio flowing at a good rate without sucking
>> every processor cycle. However, all I want to do is
>> consume audio data at some predefined byte/short/float
>> sample rate per voice. I don't care what the data smells
>> like, as long as I can find out some determistic way to
>> read it per format, without having to worry about compression,
>> etc.
>>
>> Do I have to look like a device to do this?
>
>I'm a little confused. What is the source of the data you
>want to consume? Where are you trying to plug into Java
>Sound? I'm not sure I understand your goals....


And here's #3:

From: Steve Baker <###@###.###>
Date: Monday, January 11, 1999 3:28 PM
<...>
>If this is correct, I have the following questions:
>
>If the original file is compressed (eg, MP3) will the data be
>decompressed before being fed into Buffers? If not, where does the
>decompression occur in the chain?
>
>Is there an easier way to get audio data for any file format other than
>using the AudioFormat object to interpret the Buffer contents?
>
>If there was one feature I would really like to see in Javasound, it
>would be to make accessing audio data from any audio source trivial and
>transparent - that doesn't seem to be possible at the moment.

Thank you very much, Mr. Baker.

This is exactly what I want, too, more than anything else as well,
and, frankly, JavaSound is going to be pretty useless to me without it.

We don't have this facility in 0.8. Will it appear in 1.0, please?

And what I don't understand is that it exists already, hidden
in this class, which I used javap to dump:

class com.sun.media.sound.FormatConverter extends
com.sun.media.sound.HeadspaceEngine {
    static final int ULAW_TABH[];
    static final int ULAW_TABL[];
    private javax.media.format.audio.AudioFormat formatIn;
    private javax.media.format.audio.AudioFormat formatOut;
    private int frameSizeInInBits;
    private int frameSizeOutInBits;
    private boolean convertUlawToLinear;
    private boolean convertByteOrder;
    private boolean convertSignedToUnsigned;
    static java.lang.Class array$B;
    static {};
    com.sun.media.sound.FormatConverter(javax.media.format.audio.AudioFormat)
throws javax.media.format.UnsupportedFormatException;
    static java.lang.Class class$(java.lang.String);
    javax.media.Buffer convert(javax.media.Buffer);
    int getBytesIn(int);
    int getBytesOut(int);
    int getFramesIn(int);
    int getFramesOut(int);
    javax.media.format.audio.AudioFormat getInputFormat();
    javax.media.format.audio.AudioFormat getOutputFormat();
    void setInputFormat(javax.media.format.audio.AudioFormat) throws
javax.media.format.UnsupportedFormatException;
    void setupConversions() throws
javax.media.format.UnsupportedFormatException;
}

I want public access to something like this class, so Mr. Baker
and I can read in a file of any AudioFormat, from AU to MP3, and convert it to
audio buffer always of some simple, uncompressed format like
16-bit WAV or AIFF (WAV has the drawback of byte reversal
from Java) or SOMETHING SIMPLE. No, this converter can't
probably support converting to all formats, but I think some
generic converter should support converting to one, simple,
comprehensible format, even if it's not an external format,
but just a byte[] or short[] or or int[] or float[] at a given sample rate.
Is there any hard reason why this can't be granted?

I hate to blatant, but I'd like not be miunderstood again:
will JavaSound do this, or will it be absolutely useless for some of us?

Also, I thought the point of the prerelease was to get comments about
the API and its capabilities and appropriateness so that it can be
improved and filled out before release. Currently the API documentation
is sketchy in several places, making determining the weak points difficult.
I only found out about the discernable AudioFormat in the byte[] transport
from the info list. What other black holes are out there? Especially in
MIDI? I would like to try it and use it, but there isn't too much there to
go on outside of what's supported in 1.2 (playing a MIDI file), and
spending time thrashing around is a waste. Will there be another prerelease,
or will source be released in some form as well so we can find out
what exactly JavaSound is before we accidentally put our seal of
approval on it by not raising complaints regarding elements we couldn't
discover enough about to understand?

Thanks,

Tim
###@###.###