There are all kinds of delays that happen with sound and different effects on players and listeners:

– sound travels in air at 330meters/second, so there is a delay from the instrument to the ear or in case of electric music from the instrument to the microphone and from the speaker to the ear. the violin player hears his instrument with less than a millisecond (ms) latency, but the percussion player at the other side arrives up to 30ms late.

At an ordinary distance of 3m from a monitor the electric musician gets 10ms. These problems are old and mostly solved, but they are important as a reference to the newer ones.

– analog electronics can create some delays for some frequencies in filters, but since the first signal that comes through forms the attack, it is heard immediately.
– DSP machines treat sound by the sample, so the only latency happens in their AD/DA converters, usually less than 1ms.
– Computers treat sound in packages, so they have buffers at their inputs and outputs to write to and read from, while a complete package is in the processor. This delay is usually meant when we speak of latency.

It depends on how much we ask from the computer, because a shorter delay means smaller packages so the processor has to treat more of them so it stands less processing on each – you can use less channels or plugins.

A usual setting of 128 samples at input and output results in

2*128/44100 = 5,8 ms

– digital communication also happens in packages and may create a audible delay, depending on protocol. FW interfaces are very good at that, USB depends on mostly on the driver, but built in audio ports are often not sufficient (up to 250ms).
– even analog machines may take a while to analyse the sound before they know how to treat it, so the effect is late – which does not necessarily mean that the sound itself is delayed. Examples are Guitar2MIDI, pitch shifters, compressors (pumping or preview delay).

What is the problem for whom?

– For the listener in a big festival its strange to see the musician strike and hear the bang later. this could be fixed by putting speakers in the back, but (due to next point) usually those back speakers are delayed just as the sound travel would be, so we don’t fix this problem at all and nobody complains.
– Critical ears hear ugly filter effects if two different delays of the same signal are mixed. This happens when:

— the listener of a mono signal over a stereo system does not sit exactly in the center. Worse if the listener or the speaker move.

— the tech runs a signal into one mixer channel and parallel through a processor into another channel and mixes them at similar volume

— two microphones pick up the same sound source at similar volume and direction but different or even changing distances

– For musicians its hardly possible to play together sitting on opposite ends of a concert hall or playing over internet. it only works if they create very slow or very disorganized music or adapt the rhythm to the delay and play with it.

– For musicians using machines its complex and personal.

It can become hard to operate the equipment if there is a delay between switch press and its effect on the sound. This is not related to the latency of the audio tracks, but usually due to MIDI speed or computer drivers. A program change command delayed by 100ms is hardly a problem, but for a note trigger the same rules apply as for the audio latency – which is what the rest of this article is about:

Since the delay exists anyway in the air, the brain does have a mechanism to work with it, and we can improve it with training. So the answer to someone who says that he cannot play with a machines latency can be: “learn it”.

If the musician is used to the latency of the monitor distance and then changes to a recent computer with headphones, he should not have a problem since its in the same range.

If he goes on with the same monitor, he can deal with it by getting used if the latency is not too big.

How much is too much?

Up to 3 ms should never be a problem because we are used to it. Above, it depends:

If the sound has slow attack, even 100ms is no problem.

Slow spheric music is much less critical than quick complex rhythms.

A constant delay is less annoying than a changing one (in guitar2MIDI it depends on sound and note)

When a direct and a delayed sound is mixed, the effect can be beautiful or annoying. Its certainly easier for the musician to play because he hears his playing immediately. If the second sound is similar and only a few ms delayed, it creates a filter, usually ugly. If the second sound is delayed by 10…50ms, it seems ritcher. Above, it turns into a echo and then into a rhythmic element which has to fit into the composition.

As long as the computer is only used for recording or loops or reverb, its latency can be compensated – which has to be done carefully. And spontaneous operations will always be executed with a bigger latency than the one for the sound

Easily we get twice the computers latency because of inserting an (analog) effect box into the signal chain. This turns the switch to the computer especially hard for the musician:

To use a preamplifiers before the computer and effects between the computer and the speaker does not add delay, but a CryBaby after the distortion plugin does, because this requires another conversion into analog, and back again to digital. So the musician has to move all equipment at once into the computer.

Matthias Grob, 2006