The mac audio device manager was being "helpful" by restarting
playout and recording if the Default device was changed, assuming
the application wouldn't care.
However, we received an update of device change, and attempted to
stop and start playout anyway, causing a conflict.
The fix was simply to not deploy new devices when the device id didn't
change.
Changed iWebRTCUpdateDevices and iAudioDeviceMutex to gWebRTCUpdateDevicesand gAudioDeviceMutex to follow Firestorm coding standards.
They are global static variables.
As with the previous change, this is to address the issue FIRE-36022 where users who remove their USB headsets with microphones are having the viewer lock up.
This introduces a new inline mutex called iAudioDeviceMutex. This is stored in a shared header file located on llcommon. iAudioDeviceMutex is a timed_mutex which allows for a time limit to be placed on a wait for a lock. Once the time runs out or the lock is achiveved the code moves on. With a check after the lock, if the owner is the current thread, then the lock was successful, otherwise it's still being used by another thread and the function should exit to try again later on.
This logic is wrapping WebRTC, FMOD and Vinvox calls to the audio hardware by the Viewer. OpenAL does not currently support changing of audio hardware and always defaults to the default audio hardware.
Added exceptions handling for the new unique_lock with the timed_mutex as they can throw 2 exceptions if the thread is already locked by the current thread and if the mutex is invalid.
Further testing may reveal other areas which would need the timed_mutex added to protect from threads locking up.
This is created to address the issue of USB headsets being removed or Bluetooth headsets with bad connections causing the viewer to lock up.
First Part: WebRTC
WebRTC now has an added inline atomic bool used to track when the worker thread is accessing the audio hardware. If the coroutine that is always running detects 10 consecutive true flag values, it will set WebRTC to exit and exit itself.
This coroutine was what was locking up the viewer as it spammed the mMainQueue with messages every update, but the worker thread was locked up and unable to receive the messages. Once the queue reached 1024 messages, it would lock up.
Underlying issue is the Windows Core Audio driver system crashes if too many threads have requests to get audio updates upon hardware changes. And with a USB headset having 2 devices in 1, it multiples the updates by 2x. The WebRTC audio system itself locks a mutex as it calls the core audio system to get set the input, but if it is already being processed by another thread, it then gets an exception to try again. The audio driver is not handling the message and crashes and does not return leaving the WebRTC worker thread locked.
The mac audio device manager was being "helpful" by restarting
playout and recording if the Default device was changed, assuming
the application wouldn't care.
However, we received an update of device change, and attempted to
stop and start playout anyway, causing a conflict.
The fix was simply to not deploy new devices when the device id didn't
change.
Commented out a block of new code recently added which was also being called by one of the callback functions.
This had the side effect of having 2 LLWebRTCDevicesObserver listening to the hardware removal events and cause both to trigger events at the same time, which may be the source of the queue being locked up in a race condition.
Was able to reproduce issue 2x without code change by moving mouse around a lot while on rezing and then pulling USB headset out of the USB port. Direct motherboard USB port seemed to work the best as could not trigger while headset was in USB hub.
* Fix indexing problem with mac devices
This resulted in the wrong device being selected.
Also, fix a shutdown crash where recording was not being stopped, hence the recording
thread was still running on shutdown and crashed because it lost access to resources.
Fix an issue with p2p calls where they're coming up muted even though the button indicates
they are unmuted.
* Always refresh device list on notification of device changes
Even when the selected device doesn't change, we need to
re-deploy it as it might have had characteristics (sampling rate, etc.) changed.
Also, we need to redeploy when the Default device has changed
* [WebRTC] Rework device handling sequence so that we can handle unplugging/re-plugging devices
The device handling was not processing device updates in the proper sequence as
things like AEC use both input and output devices. Devices like headsets are both
so unplugging them resulted in various mute conditions and sometimes even a crash.
Now, we update both capture and render devices at once in the proper sequence.
Test Guidance:
* Bring two users in the same place in webrtc regions.
* The 'listening' one should have a headset or something set oas 'Default'
* Press 'talk' on one, and verify the other can hear.
* Unplug the headset from the listening one.
* Validate that audio changes from the headset to the speakers.
* Plug the headset back in.
* Validate that audio changes from speakers to headset.
* Do the same type of test with the headset viewer talking.
* The microphone used should switch from the headset to the computer (it should have one)
Do other various device tests, such as setting devices explicitly, messing with the device selector, etc.
* Fix race condition when multiple change device requests might come in at once
* Update to m137
The primary feature of this commit is to update libwebrtc from m114
to m137. This is needed to make webrtc buildable, as m114 is not buildable
by the current toolset.
m137 had some changes to the API, which required renaming or changing namespace
of some of the calls.
Additionally, this PR moves from a callback mechanism for gathering the energy
levels for tuning to a wrapper AudioDeviceModule, which gives us more control
over the audio stream.
Finally, the new m137-based webrtc has been updated to allow for 192khz audio
streams.
* Properly pass the observer setting into the inner audio device module
* Update to m137 and get rid of some noise
This change updates to m137 from m114, which required a few API changes.
Additionally, this fixes the hiss that happens shortly after someone unmutes: https://github.com/secondlife/server/issues/2094
There was also an issue with a slight amount of repeated after unmuting if there was audio right before unmuting. This is because
the audio processing and buffering still had audio from the previous speaking session. Now, we inject nearly a half second of
silence into the audio buffers/processor after unmuting to flush things.
* Install nsis on windows
* Use the newer digital AGC pipeline
m137 improved the AGC pipeline and the existing analog style is going away
so move to the new digital pipeline.
Also, some tweaking for audio levels so that we don't see inworld bars when tuning,
so one's own bars seem a reasonable size, etc.
* Install NSIS during windows sisgning and package build step
* Try pinning the packaging to windows 2022 to deal with missing nsis
* Adjust gain calculation and audio level calculations for tuning and peer connections
* Update with mac universal webrtc build
* Tuning of voice indicators for both tuning mode and inworld for self.
* Redo device deployment to handle cases where multiple deploy requests pile up
Also, mute when leaving webrtc-enabled regions or parcels,
and unmute when voice comes back.
* pre commit issue
* Fix indexing problem with mac devices
This resulted in the wrong device being selected.
Also, fix a shutdown crash where recording was not being stopped, hence the recording
thread was still running on shutdown and crashed because it lost access to resources.
Fix an issue with p2p calls where they're coming up muted even though the button indicates
they are unmuted.
* Always refresh device list on notification of device changes
Even when the selected device doesn't change, we need to
re-deploy it as it might have had characteristics (sampling rate, etc.) changed.
Also, we need to redeploy when the Default device has changed
* [WebRTC] Rework device handling sequence so that we can handle unplugging/re-plugging devices
The device handling was not processing device updates in the proper sequence as
things like AEC use both input and output devices. Devices like headsets are both
so unplugging them resulted in various mute conditions and sometimes even a crash.
Now, we update both capture and render devices at once in the proper sequence.
Test Guidance:
* Bring two users in the same place in webrtc regions.
* The 'listening' one should have a headset or something set oas 'Default'
* Press 'talk' on one, and verify the other can hear.
* Unplug the headset from the listening one.
* Validate that audio changes from the headset to the speakers.
* Plug the headset back in.
* Validate that audio changes from speakers to headset.
* Do the same type of test with the headset viewer talking.
* The microphone used should switch from the headset to the computer (it should have one)
Do other various device tests, such as setting devices explicitly, messing with the device selector, etc.
* Fix race condition when multiple change device requests might come in at once
* Update to m137
The primary feature of this commit is to update libwebrtc from m114
to m137. This is needed to make webrtc buildable, as m114 is not buildable
by the current toolset.
m137 had some changes to the API, which required renaming or changing namespace
of some of the calls.
Additionally, this PR moves from a callback mechanism for gathering the energy
levels for tuning to a wrapper AudioDeviceModule, which gives us more control
over the audio stream.
Finally, the new m137-based webrtc has been updated to allow for 192khz audio
streams.
* Properly pass the observer setting into the inner audio device module
* Update to m137 and get rid of some noise
This change updates to m137 from m114, which required a few API changes.
Additionally, this fixes the hiss that happens shortly after someone unmutes: https://github.com/secondlife/server/issues/2094
There was also an issue with a slight amount of repeated after unmuting if there was audio right before unmuting. This is because
the audio processing and buffering still had audio from the previous speaking session. Now, we inject nearly a half second of
silence into the audio buffers/processor after unmuting to flush things.
* Install nsis on windows
* Use the newer digital AGC pipeline
m137 improved the AGC pipeline and the existing analog style is going away
so move to the new digital pipeline.
Also, some tweaking for audio levels so that we don't see inworld bars when tuning,
so one's own bars seem a reasonable size, etc.
* Install NSIS during windows sisgning and package build step
* Try pinning the packaging to windows 2022 to deal with missing nsis
* Adjust gain calculation and audio level calculations for tuning and peer connections
* Update with mac universal webrtc build
* Tuning of voice indicators for both tuning mode and inworld for self.
* Redo device deployment to handle cases where multiple deploy requests pile up
Also, mute when leaving webrtc-enabled regions or parcels,
and unmute when voice comes back.
* pre commit issue
Muting was a bit random in the code, so it's now been straightened out and should
prevent echo.
Also, code was added to not attempt connection to non-webrtc regions in the webrtc code.
When transitioning from mic-on hands-free mode to mic off,
it's expected that the audio stream would return to stereo.
Inproper logic in the mac device code in webrtc was preventing
that.
The microphone issue was causing a short moment of sound, and was
causing bluetooth headsets to switch to hands-free/one channel mode
which is disruptive.
Also, update webrtc to deal with issue where airpods were garbled
after coming out of hands-free mode.
Fixes prevent attempting to start playout/recording before the devices
are set up, to prevent restarting playout/recording, to prevent
attempts to stop when not playing/recording, and so on...
This should address the case where audio device changes can cause
an assert. It should also address the case where audio was unnecessarily played
or transmitted when connecting.
And, when voice is disabled, the audio devices are not set up to play/record
so there should be no disruption of bluetooth music from other apps.
Ansariel