Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8211428

Unexplained latency on audio retrieved from microphone

XMLWordPrintable

      ADDITIONAL SYSTEM INFORMATION :
      We have recreated this problem on Windows 7 and Windows 10 with java 8, 9 and10. We have also replicated it on different sound cards and using both analog and USB headsets.

      A DESCRIPTION OF THE PROBLEM :
      I'm writing a audio chat application in Java and I'm experiencing some latency/delays that mostly shows up if the application is left running for a while.

      I recreated the problem in the below sample application. It simply loops sound from the microphone to the speaker. Initially it behaves as expected. When you press the button and speak into the microphone you hear yourself in the speaker with a tiny delay. However if the program is left running for a while (a week) then that delay is increased to several seconds.

      I tested with different headsets. I tested with Java 8, 9, 10 and it consistently displays the same behaviour. I also experimented with drain() and flush() and so on but the only thing that gets rid of the delay is to close and recreate the TargetDataLine. Recreating the line is however not an option for my application since it takes to long and audio is unavailable while you recreate the line.

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Run the attached code sample. You need a microphone. (Change the name of the headset to match something you have)

      Press the PTT button and speak into your microphone. You should then hear yourself with minimal latency in you headset.

      Leave the sample running for some days. 4 days usually does it.

      Press the PTT button and speak into the microphone again. On the computers where we have tested this the audio is played back with a significant delay.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      To still hear the audio into the headset with minimal delay
      ACTUAL -
      The audio is significantly delayed. By several seconds.

      ---------- BEGIN SOURCE ----------
      import javax.sound.sampled.*;
      import javax.swing.*;
      import java.awt.event.MouseAdapter;
      import java.awt.event.MouseEvent;

      /**
       * Created by asa on 2018-04-03.
       */
      public class Main {


         public static void main(String[] args) throws LineUnavailableException {
            String title = "";
            if (args.length > 0) {
               title = args[0];
            }
            String mixerName = "USB”; // Part of the name of my headset
            if (args.length > 1) {
               mixerName = args[1];
            }
            AudioFormat format = new AudioFormat(8000f,
                                                 16,
                                                 1,
                                                 true,
                                                 false);
            DataLine.Info targetInfo = new TargetDataLine.Info(TargetDataLine.class, format);
            TargetDataLine mic = null;
            for (Mixer.Info info : AudioSystem.getMixerInfo()) {
               if (info.getName().contains(mixerName) && !info.getName().contains("Port")) {
                  Mixer m = AudioSystem.getMixer(info);
                  if (m.isLineSupported(targetInfo)) {
                     mic = (TargetDataLine) m.getLine(targetInfo);
                     break;
                  }
               }
            }
            mic.open(format, 1280);
            mic.start();

            DataLine.Info sourceInfo = new DataLine.Info(SourceDataLine.class, format);
            SourceDataLine speaker = null;
            for (Mixer.Info info : AudioSystem.getMixerInfo()) {
               if (info.getName().contains(mixerName) && !info.getName().contains("Port")) {
                  Mixer m = AudioSystem.getMixer(info);
                  if (m.isLineSupported(sourceInfo)) {
                     speaker = (SourceDataLine) m.getLine(sourceInfo);
                     break;
                  }
               }
            }
            speaker.open(format, 8000);
            speaker.start();

            MicRunnable micRunnable = new MicRunnable(mic, speaker);
            new Thread(micRunnable).start();

            Frame.show(title, new Frame.PttListener() {
               @Override
               public void press() {
                  micRunnable.start();
               }

               @Override
               public void release() {
                  micRunnable.stop();
               }
            });
         }

         private static class MicRunnable implements Runnable {
            private final TargetDataLine _mic;
            private final SourceDataLine _speaker;

            private final Object runLock = new Object();
            private volatile boolean running = false;

            public MicRunnable(TargetDataLine mic, SourceDataLine speaker) {
               _mic = mic;
               _speaker = speaker;
            }

            public void start() {
               synchronized (runLock) {
                  running = true;
                  runLock.notify();
               }
            }

            public void stop() {
               synchronized (runLock) {
                  running = false;
               }
            }

            @Override
            public void run() {
               while (true) {
                  byte[] bytes = new byte[640];
                  _mic.read(bytes, 0, bytes.length);
                  if (running) {//tPeakGain(bytes) > 300) {
                     _speaker.write(bytes, 0, bytes.length);
                  }
               }
            }
         }

         private static class Frame extends JFrame {

            interface PttListener {
               void press();
               void release();
            }

            private Frame(String title, PttListener listener) {
               setTitle(title);
               JPanel content = new JPanel();

               JButton pttButton = new JButton("PTT");
               pttButton.addMouseListener(new MouseAdapter() {
                  @Override
                  public void mousePressed(MouseEvent e) {
                     listener.press();
                  }

                  @Override
                  public void mouseReleased(MouseEvent e) {
                     listener.release();
                  }
               });

               content.add(pttButton);

               setContentPane(content);
               setSize(300, 100);
            }

            public static void show(String title, Frame.PttListener pttListener) {
               new Frame(title, pttListener).setVisible(true);
            }
         }
      }
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      Closing and reallocating the TargetDataLine will get rid of the delay. But doing that while recording will introduce noticeable skips meaning it is not really an option for us.

      FREQUENCY : always


        1. AudioInput.java
          17 kB
          Pardeep Sharma
        2. Main.java
          4 kB
          Pardeep Sharma

            Unassigned Unassigned
            webbuggrp Webbug Group
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: