Repository navigation
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The FIFO prefilter can admit streams that exceed their allocated capacity and cause silent audio corruption, while rate-only refinement can accept unsupported sample rates.
2 open issues
What changed in this PR
Enables USB audio offload on Intel Lunar Lake and newer platforms, allowing the SOF DSP to stream through xHCI Audio Sideband while snd-usb-audio retains device ownership.
Changes:
- Adds xHCI sideband resource management and USB audio offload ownership APIs.
- Integrates SOF UAOL streams, feedback DMA, device constraints, suspend/resume handling, and per-stream route controls.
- Adds machine-driver integration, build configuration, documentation, and 384-kHz stream-format support.
| File | Description |
|---|---|
sound/usb/quirks.c |
Exports format quirks for offload use. |
sound/usb/pcm.h |
Declares substream claim/release APIs. |
sound/usb/pcm.c |
Adds exclusive substream claims and propagates endpoint errors. |
sound/usb/offload/offload.h |
Defines internal offload context and bridge interfaces. |
sound/usb/offload/Makefile |
Builds the offload library. |
sound/usb/offload/Kconfig |
Adds offload library configuration. |
sound/usb/offload/init.c |
Implements registration, notifications, and resource placement callbacks. |
sound/usb/Makefile |
Includes the offload build directory. |
sound/usb/Kconfig |
Includes offload configuration. |
sound/usb/endpoint.h |
Declares endpoint ownership APIs. |
sound/usb/endpoint.c |
Prevents host sharing of offloaded endpoints. |
sound/usb/card.h |
Adds ownership state and platform-release declaration. |
sound/usb/card.c |
Disconnects tracked devices before releasing platform operations. |
sound/soc/sof/topology.c |
Recognizes the UAOL DAI type. |
sound/soc/sof/sof-priv.h |
Adds machine-parameter finalization callback. |
sound/soc/sof/ipc4-topology.h |
Defines UAOL gateway configuration and stream mapping. |
sound/soc/sof/ipc4-topology.c |
Configures UAOL FIFOs and refreshes gateway data. |
sound/soc/sof/ipc4-priv.h |
Stores firmware UAOL support and capabilities. |
sound/soc/sof/ipc4-loader.c |
Parses firmware UAOL capabilities. |
sound/soc/sof/intel/Makefile |
Builds Intel UAOL integration. |
sound/soc/sof/intel/Kconfig |
Adds SOF USB offload configuration. |
sound/soc/sof/intel/hda.h |
Defines UAOL state, limits, and shared interfaces. |
sound/soc/sof/intel/hda.c |
Registers offload support and reports usable UAOL links. |
sound/soc/sof/intel/hda-uaol.c |
Implements UAOL routing, constraints, and stream lifecycle. |
sound/soc/sof/intel/hda-pcm.c |
Applies backend constraints and supports 384-kHz formats. |
sound/soc/sof/intel/hda-mlink.c |
Adds UAOL link lookup. |
sound/soc/sof/intel/hda-dai.c |
Adds UAOL DAIs and shares common callbacks. |
sound/soc/sof/intel/hda-dai-ops.c |
Adds UAOL DMA operations and feedback DMA allocation. |
sound/soc/sof/intel/hda-common-ops.c |
Wires UAOL-aware machine registration callbacks. |
sound/soc/sof/core.c |
Finalizes copied machine parameters before registration. |
sound/soc/soc-usb.c |
Adds non-DT matching and per-stream route controls. |
sound/soc/intel/common/sof-function-topology-lib.c |
Recognizes optional UAOL topology fragments. |
sound/soc/intel/boards/sof_uaol_common.h |
Shares UAOL backend naming and counting helpers. |
sound/soc/intel/boards/sof_sdw.c |
Creates UAOL backend links for SoundWire cards. |
sound/soc/intel/boards/sof_board_helpers.h |
Adds UAOL link type and board parameters. |
sound/soc/intel/boards/sof_board_helpers.c |
Creates UAOL links through shared board helpers. |
sound/soc/intel/boards/skl_hda_dsp_generic.c |
Includes UAOL links in generic HDA cards. |
sound/hda/core/device.c |
Adds 384-kHz HDA stream-format encoding. |
include/sound/usb-offload.h |
Defines the public USB audio offload interface. |
include/sound/sof/ipc4/header.h |
Defines firmware UAOL capability messages. |
include/sound/soc-usb.h |
Exposes route state, controls, and selection APIs. |
include/sound/soc-acpi.h |
Adds UAOL machine parameters. |
include/sound/hda-mlink.h |
Declares UAOL link lookup and its stub. |
include/linux/usb/xhci-sideband.h |
Exposes resource, endpoint-property, and restart APIs. |
drivers/usb/host/xhci.h |
Defines extended-property commands and resource tracking. |
drivers/usb/host/xhci.c |
Implements extended-property command execution. |
drivers/usb/host/xhci-sideband.c |
Implements audio endpoint handover and resource management. |
drivers/usb/host/xhci-ring.c |
Queues and handles extended-property commands. |
drivers/usb/host/xhci-mem.c |
Initializes and destroys sideband resource tracking. |
Documentation/sound/soc/usb.rst |
Documents routing APIs and platform-card controls. |
Documentation/sound/soc/index.rst |
Adds the UAOL documentation entry. |
Review effort: Balanced
5465b53 to
60134a0
Compare
The sideband structure caches a vdev pointer to the xhci_virt_device that the xhci driver may free and reallocate while the sideband is still registered, leaving the sideband with a dangling sb->vdev pointer. This can happen during resume. Either when xhci driver detects xHC power loss and resets xHC, freeing all virt devices without informing class drivers, or during reset-resume when usb core re-enumerates the device and xhci frees and allocates a new virt device. Store the usb_device instead. It stays valid across reset-resume and xHC reset. Look up the current virt device and ensure it still points to this sideband before using it in sideband API calls. Make sure xhci_sideband_unregister() works propoerly with an invalid vdev. Dont dereferece it, or ask xHC to stop non-existing endpoints, but make sure sideband is properly freed. Fail other relevant sideband API calls with -ENODEV Fixes: de66754 ("xhci: sideband: add initial api to register a secondary interrupter entity") Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Extended Property Get and Set commands are used to control and list extended properties. The Get/Set Extended Property Command references a specific Extended Capability using the Extended Capability Identifier (ECI). Some of these commands take a pointer to a Extended Property Context that xHC will fill. Get Extended Property with ECI set to 0 will list all supported extended capabilities Audio sideband control is currently the only supported extended capability. More details in xhci specification sections 6.4.3.16-17 and 7.9.1-2 Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Add easy and usable synchronous helpers for Get and Set Extended Property. These take care of the tedious tasks of allocating a command, dma mapping, queueing a command and waiting for completion. These helper are synchronous and will sleep. Note that the buffer *buf passed to get_ext_property() needs to be dma mappable for all buffers using capability IDs, except for PROP_ECI_ALL (ECI 0) which returns a bitmap of supported capabilities in the command completion event TRB Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Add support for registering standard xHCI Audio Sideband Capability, as described in xHCI specification section 7.9. This is the xHC hardware managed audio sideband that can be controlled by Set and Get Extended properties commands. Define the type of this xHC hardware controlled audio sideband as XHCI_SIDEBAND_AUDIO. Check that xHCI is at least version 1.2 with GSC support bit (Get/Set extended property capability) set, and that extended properties report Audio Sideband Extended Capability is supported. Check if the sideband type is XHCI_SIDEBAND_AUDIO or XHCI_SIDEBAND_VENDOR in codepaths where it matters Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
… IDs Endpoints controlled by the xHC hardware audio sideband need a unique resource ID, used when handing the endpoint over to the sideband with a Set Extended Property: Set Resource Assignment command (xHCI 7.9.2.1). Track the resource IDs of an xHC host in an xarray, and add xhci_sideband_alloc_resource_id() and xhci_sideband_free_resource_id() to the sideband API. An XHCI_SIDEBAND_AUDIO endpoint needs a reserved resource ID before it can be added with xhci_sideband_add_endpoint(). The caller can narrow the ID range within the xHC limits, as an offload engine may tie a function to specific resources. For example, the Intel DSP firmware selects a link FIFO stream by resource ID. Passing 0 for both bounds reserves any free ID. An endpoint holds only one reservation at a time. Co-developed-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com>
… audio sideband Support assigning endpoints over to xHC sideband control for the hardware controlled XHCI_SIDEBAND_AUDIO sideband type. These endpoints need to have a valid unique resource id allocated by calling xhci_sideband_alloc_resource_id() before assigning the endpoint over to sideband by calling xhci_sideband_add_endpoint() Endpoint control is handed over to sideband with a Set extended property command that includes the allocated unique resource id. See xhci specification 7.9.2.1 "Set Extended Property: Set Resource assignment Command" Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The max number of endpoints a xHC controller can offload for audio sideband can be fetched with a Get Extended Property: Get supported Resources Command. The limit is returned in the first 8 bit 'Resource count' field of the Extended Property Context structure Store and use this limit as the max xarray index. Intel xHC have a hard resource limit capped at 6 even if host reports otherwise. Many xarray functions use unsigned long types for index, but we can safely cast them to 'int' as the max index is a 8bit value. Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Prepare the xHCI Audio Sideband API for Intel UAOL consumers. Add xhci_sideband_get_ep_properties(), an endpoint property query returning all the xHC and endpoint properties needed to set up Intel UAOL Partially based on work by Peter Ujfalusi CC: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
… a hub Audio quality cannot be guaranteed with Intel xHC audio sideband offloading when the device is connected behind a hub. Only allow Intel hosts to register an audio sideband for devices connected directly to a root hub port. Other hosts are not affected. Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
A client driving endpoints over the audio sideband can stop them with xhci_sideband_stop_endpoint(), but has no way to start them again: a stopped endpoint only returns to the running state when its doorbell is rung, and nothing rings it for an endpoint the sideband owns. A stream which is stopped and then started again without its endpoints being reconfigured in between, like one recovering from an xrun, stays stalled. Add xhci_sideband_start_endpoint(), which rings the doorbell of an endpoint the sideband owns. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
snd_hdac_stream_format() silently falls back to the 48kHz encoding for any rate missing from rate_bits[]. USB audio devices offloaded over the HDaudio link can run at 384kHz, so add the encoding: base 48kHz, mult 8, div 1. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
hda_dsp_get_mult_div() falls back to the 48kHz encoding for a rate it does not recognise and only warns about it, so a 384kHz host stream had its SDnFMT MULT/DIV programmed for 48kHz. Add 384000 with the same 8x multiplier snd_hdac_stream_format() uses. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
snd_soc_find_usb_ctx() only resolves the backend context through the device tree, so on ACPI/PCI platforms snd_soc_usb_connect() and snd_soc_usb_disconnect() silently do nothing. Without an of_node, match the registered component's device against the caller's device instead. DT platforms are unaffected. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
A USB audio stream can be driven by the host or, on platforms with an audio DSP, by the DSP. Both configure the same endpoints of the same struct snd_usb_substream and nothing arbitrates between them: if user space opens the PCM while the DSP owns the endpoints, snd_usb_hw_params() either shares the endpoint or closes and reopens it, freeing the transfer ring behind the DSP's back. snd_usb_pcm_open() already refuses a second opener through subs->opened, so give the offload path a way to take the same reference. A claimed substream cannot be opened from user space and an opened substream cannot be claimed; whoever asks first gets the device, and the device stays fully usable by the host whenever nothing offloads it. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Claiming a substream for offloaded use keeps user space off that PCM, but not off its endpoints. With implicit feedback a playback stream syncs to the data endpoint of the capture interface: when the capture stream is offloaded, the host's playback still opens that endpoint as its sync source and submits URBs on it while the endpoint's ring belongs to the DSP, and both streams stall, whichever of them was started first. Let the offload path claim the endpoints the DSP drives with snd_usb_endpoint_claim_offload() and have snd_usb_endpoint_open() refuse to share a claimed endpoint. An endpoint which another stream has open already cannot be claimed, the offload path backs out then. snd_usb_endpoint_open() returns an ERR_PTR() for this, so that snd_usb_hw_params() reports the refusal as -EBUSY instead of the -EINVAL of a failed open. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
A platform that keeps state for the USB SND devices it is told of, an offload driver which holds their streams and xHCI sideband, has no safe way to let go of them when it unregisters. Once its operations are unregistered, a device disconnected in the meantime is not reported to it anymore and its chip, card and usb_device are freed while the platform still points to them. Unregistering after cleaning up instead leaves a window for a device connected in between. Add snd_usb_release_platform_ops(), the counterpart of snd_usb_rediscover_devices(): under register_mutex, which serialises it against probe and disconnect, it runs the disconnect callback for every connected device and then unregisters the operations. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Add a library that tracks offload capable USB audio devices and forwards their lifecycle to a host driver owning the DSP. Device ownership stays with snd-usb-audio. The library hooks the USB-SND platform-op callbacks, runs a UAC descriptor filter to decide whether a device can be offloaded, and reports hotplug events to the host through a callback table, struct usb_offload_ops. The host turns each callback into whatever its DSP expects; the library builds no IPC message, touches no register and owns no interrupt. The audio-control interface is looked up in the active configuration rather than assumed to be interface 0, so composite devices where the audio function sits behind other interfaces are candidates too. A device exposing more than one audio function is offloaded through the first one. The data path is built on the xHCI Audio Sideband, which routes an endpoint's traffic to the DSP. An endpoint only exists while the streaming altsetting is active, so on stream acquire the library first claims the USB substream and starts it, the same way a host driven stream is started, then reports the data endpoint. Claiming the substream also keeps user space off the device for as long as it is offloaded. A device can expose several PCMs of the same direction (e.g. separate speaker and line-out streams), so the acquisition takes the USB PCM device number the caller wants, or a negative index to take the first PCM matching the format. The acquired streams are kept per USB PCM: a PCM is offloaded once, the other PCMs of the device can be acquired alongside. The PCM number the stream landed on is reported back and names the stream to usb_offload_release_stream(). No card is created here and no stream is driven from here: the host's backend DAI acquires the endpoint with usb_offload_acquire_stream() when its own framework triggers it. A context stands for one offload engine, the link over which a DSP reaches a host controller; a host with several registers one context for each. The filter accepts UAC1, UAC2 and UAC3 audio functions and skips the interfaces of the function which are not audio streaming ones, such as MIDI streaming, which UAC1 lists in the same collection. usb_offload_connect() notifies the host, which can call back into the device record through its own framework before returning (e.g. a route control's busy check), so the device list lock is dropped around the connect and disconnect notifications. The record cannot go away meanwhile: the callbacks that create and free it are serialised by snd-usb-audio's register_mutex. A device being torn down is marked disconnected before the lock is dropped, so that a stream acquire racing with the unplug is refused instead of handing out a device which is already gone. Nothing in the interface is tied to a sound framework, but no consumer outside ALSA exists, so the header lives under include/sound/. Signed-off-by: Rene Sapiens <rene.sapiens@linux.intel.com> Co-developed-by: Pooja Katiyar <pooja.katiyar@intel.com> Signed-off-by: Pooja Katiyar <pooja.katiyar@intel.com> Co-developed-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The machine params are filled in when the machine is selected, before the firmware has booted, and the selected machine is not touched afterwards. The UAOL link mask is the first parameter that also depends on the booted firmware: the controller enumerates the links regardless of what the firmware can do with them, and a machine driver creating backends for them with a firmware that cannot serve them fails to load the topology. sof_machine_register() already hands the machine driver a copy of the selected machine. Add an optional finalize_mach_params() platform op and run it on that copy right before it is registered, so the platform can fold in what it only learned from the firmware while the selected machine stays as it was. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The USB audio offload library tracks offload capable USB audio devices and needs a host that owns the DSP. On Intel platforms that host is SOF, which reaches the xHCI controller over a USB Audio Offload Link (UAOL). Add the registration with the library, to run once the firmware has booted, and the unregistration on remove. One offload context in the library is one UAOL link, over which the DSP reaches an xHCI controller. The offload capabilities are part of the firmware configuration, which is only parsed after post_fw_run, so the registration is done from machine_register, the first hook that runs late enough. The hook is not installed yet: the library reports the devices it tracks to the driver, which needs the rest of the offload support in place to act on them. Implement finalize_mach_params() to tell the machine driver what the booted firmware can serve: a link with T output and I input streams carries T playback and I - T capture streams, the remaining T input streams being the clock feedback of the playback streams. Report the two counts in two new mach_params fields, with the static DAI count as the ceiling, and withdraw the link mask when the firmware does not support UAOL or the link has no usable stream. The firmware lays the UAOL link FIFOs out by stream, and the xHCI resource an endpoint gets is what selects the stream (stream = resource - 1), so a capture endpoint has to stay on the streams the firmware reads audio from and a clock feedback endpoint on the ones it reads feedback from. Implement the library's get_resource_range() op from the link's stream capabilities: playback on the output streams, capture on the input streams before the feedback ones, feedback on the last T input streams. With two output and four input streams this is resources 1-2, 3-4 and 5-6, which is the layout the Windows driver programs for the same firmware. SOF's pipeline only has S24_LE as 24-bit representation, while a USB device commonly packs 24-bit audio into 3 bytes. Set s24_3le_as_s24_le so the library treats the two as interchangeable during negotiation. The firmware applies the clock feedback of an asynchronous playback stream by spreading the samples linearly over the service interval, which is audible on the gear that uses asynchronous playback in the first place. Set async_playback_on_host so the library leaves such streams to the host. A failing registration only means that offload is unavailable, it does not fail the probe of the audio card. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The UAOL alternate link is already enumerated and its link type is reported as HDA_BUS_ML_LINK_UAOL, but there is no way for a client to retrieve the hdac_ext_link for it. Add hdac_bus_eml_uaol_get_hlink() following the existing SSP, DMIC and SoundWire helpers. It is needed by the UAOL DAI to assign a link DMA channel. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
SOF_DAI_INTEL_UAOL is defined in the IPC ABI but it is not reachable from topology: find_dai() has no matching name, so a topology using it resolves to SOF_DAI_INTEL_NONE, and the IPC4 dai_config op rejects the type as unsupported. Add the "UAOL" name to the DAI type table and accept the type in sof_ipc4_dai_config(). Unlike HDA, the node index of the copier gateway is not the link DMA stream tag but the UAOL link stream the endpoint is routed to, which is only known once the USB endpoint has been claimed; the DAI sets it from its own hw_params path. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Provide the CPU DAIs and the widget DMA ops needed to run a stream over a UAOL link. The UAOL link is an ACE2+ alternate link using a link DMA, so the generic HDA link DMA handling applies unchanged and the ops are identical to the SSP and DMIC ones, only the hlink lookup and the stream format calculation differ. The UAOL DAI code gets a file of its own, hda-uaol.c, built into the HDA common module when SND_SOC_SOF_INTEL_USB_OFFLOAD is enabled; the DAI callbacks and helpers of hda-dai.c it builds on are made visible within the module for that. A UAOL link runs several streams at a time, two in each direction on the current hardware, and each stream the link can carry concurrently needs a backend of its own, in the way a SoundWire link gets a backend per PDI. Give the component a playback and a capture DAI for each stream, "UAOL0 Playback Pin0", "UAOL0 Playback Pin1", "UAOL0 Capture Pin0" and "UAOL0 Capture Pin1". The DAIs are unidirectional: the link's streams are, and the backend hw_params fixup is not told the direction, so a duplex DAI would gain nothing. The pin number only tells the DAIs of a direction apart, it is not the UAOL stream the DAI runs on: that is decided by the xHCI sideband resource the endpoint gets when the USB device is acquired. The DAIs are kept last in skl_dai[], as the number of DAIs the component registers is a prefix of the array. The link DMA stream allocator already knows about HDA_BUS_ML_LINK_UAOL and applies the ACE2+ allocation constraints to it. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The offload library reports offload capable USB audio devices to the driver, but they still have to be presented to the ASoC USB framework so that a card can route to them. Allocate the ASoC USB port when the first UAOL DAI is bound into a card and release it when the DAI goes away. It cannot be done when the offload library is started: the firmware boots before the audio component is registered, so there is no component to hand over at that point. Devices which enumerated before the card existed are rediscovered by the ASoC USB core when the port is added. The connected devices are kept on a list owned by the controller, which is where the DAI looks up the device to offload. The record has its own list node, struct snd_soc_usb_device's list_head belongs to the ASoC USB core. A device can expose several PCMs (e.g. separate speaker and line-out streams) and the link runs several streams per direction, each on a backend of its own. Which device and PCM a backend plays to is decided by the ASoC USB route controls: give the port as many routes per direction as the link has streams, create the controls with the port and report each PCM's name for the controls to show. The selected PCM's sideband resources are acquired only once a stream on it is actually started. The UAOL PCMs exist whether or not a device is connected, so report the availability on a SND_JACK_USB jack named "USB Offload Jack", the name the Qualcomm offload path uses. The jack follows the device list, so removing one of several connected devices leaves it reported. It is created before the port is added, as that is what reports the already connected devices. The UAOL PCMs also exist if the driver could not register with the offload library, so the jack is created in that case too and reported off. The port can go away with the card while a device is being reported, so the port check and the record are handled under the device list lock and a device the port refused is dropped again rather than kept as known. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The UAOL backend DAI only sets up the link DMA, the DSP side of the link. Nothing claims the USB side, so the endpoint the DSP is supposed to read from is still owned by the host and its transfer ring does not even exist: a ring is only allocated once the streaming altsetting is selected. Acquire the stream from the offload library in hw_params and release it in hw_free. This starts the device on the USB side, hands its endpoint ring to the DSP and takes the device away from user space for as long as the DAI holds it. Every UAOL CPU DAI claims a struct hda_uaol_stream of its own when it probes, found from the DAI on every callback: that is where the acquired device and its USB PCM, the backend substream and the suspended flag of the backend live, so the backends of a direction do not step on each other. The ASoC USB route a backend plays to is the pin number of its DAI name, "UAOL0 Playback Pin1" playing to playback route 1. DPCM runs a backend's hw_params once for every frontend that reaches its own hw_params while the backend is still in the OPEN, HW_PARAMS or HW_FREE state, so with a playback PCM and its deep buffer opened at once the backend sees two calls. The stream is acquired by the first one only, and released from hw_free, which DPCM runs once the last frontend is gone. A system suspend resets the pipeline and frees the DAI copier while the stream stays open, taking the device's streaming altsetting with it. DPCM brings such a stream back through the backend's prepare, not hw_params, so acquire the stream again from prepare when the backend saw a suspend trigger. The trigger runs under the PCM stream lock and only records the fact. Which device and which of its PCMs a backend offloads is the ASoC USB route control's decision: the DAI resolves the selection of its route when acquiring the stream and reports the route as busy while it holds one, so the selection cannot change under a running stream. Unplugging the device a stream plays to would otherwise leave that stream running: the library gives the USB side back and the host drops the device from the port, but nothing reaches the frontend, so the link keeps running into a dead endpoint and the application keeps feeding it. The USB audio card ends its users' streams when it goes away, by stopping them into the DISCONNECTED state; do the same for the frontends routed to the backends whose stream belongs to the unplugged device, so that the application sees the device disappear the way it does when playing to the card directly. The backend substream is remembered along with the device for that, and the frontends are walked under the card's DPCM mutex, which the backend's hw_params and hw_free also hold. The backend then forgets the device: the library has given the USB side back already and the usb_device goes away with the disconnect, so the hw_free of the frontends' teardown must not hand it to the library again. SOF only hands this DAI S24_LE, while a USB device commonly packs 24-bit audio into 3 bytes. Resolve the negotiated format to the container the selected device has before asking for the stream, and set the link DMA up from the resolved params, so that it runs the same 24-bit container and the DSP packs the samples on their way to the link. DPCM hands hw_params a copy of the params, while prepare programs the link DMA again after a stop from the backend's own copy, so prepare resolves that one as well. A second frontend, the deep buffer, can share the backend with the one which has it open. They share the one endpoint and the one mixer, so the second one can only run what the first one opened the backend with: the rate, the channel count and the format. Remember those when the stream is acquired and refuse a joining frontend which asks for anything else, before the core prepares anything for it. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The UAOL gateway needs a configuration blob the topology cannot provide: it describes the link and the USB endpoint behind it, neither of which is known before the stream is opened. The firmware reads it as a sequence of TLVs: the xHCI bus address, the link configuration, the FIFO start address offsets, the data and feedback endpoint information, the ART divider and the endpoint's service interval. Define the blob and build it when the DAI widget is set up, since its size is part of the copier IPC. The link index, the ART divider and the FIFO layout are known at that point: the link FIFO is divided evenly between the streams of a direction, with the T output streams carrying playback and the I input streams carrying the capture and the clock feedback. The fields which describe the USB endpoint are left for the platform code to fill in once the stream has been acquired. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The UAOL gateway configuration blob describes the USB endpoint behind the link, which is only known once the stream has been acquired. Complete the blob in hw_params with the xHCI address, the endpoint information, the sideband resource the endpoint was given and the endpoint's service interval. The resource number is also the UAOL stream, which selects the gateway node index. The service interval lets the firmware size the FIFO time slot to the endpoint's real cadence (1000 us for a standard isochronous endpoint, 125-500 us for high bandwidth ones) instead of assuming one. The feedback fields of the configuration are cleared when the acquired stream has no feedback endpoint, as they can be left over from a previously offloaded device. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
…setup A widget freed on system suspend stays prepared, so on resume its IPC payload is not rebuilt and sof_ipc4_widget_setup() refreshes the parts which may have changed since: the copier data with the node ID, and the DMA config TLVs. The gateway configuration blob in between is sent as it was copied at prepare time. The UAOL DAI acquires its USB stream again on resume, and the xHCI audio sideband resource, and with it the UAOL stream index the blob carries, need not come back the same; neither need the slot of the device. The DAI rewrites copier_config accordingly, as it does the node ID, but the firmware is sent the old blob with the new node ID. Refresh the gateway configuration from copier_config as well. For the other DAI types it has not changed since prepare and the copy is the same. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The share of the link FIFO a UAOL stream is given is decided by the FIFO layout computed for the gateway configuration. The platform code needs the same number to tell before a stream starts whether a format fits in it, as an overrun is not reported anywhere, the audio just comes out corrupted. Add a helper returning the bytes a stream of a direction can use on a link. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
…FIFO The share of the link FIFO a UAOL stream is given is decided when the gateway is configured, and the rate, the channel count and the format together decide whether the stream fits in it. An overrun is not reported anywhere, the audio just comes out corrupted. The limit is a single byte rate, so it cuts across all three parameters at once: 192kHz stereo S32_LE needs exactly as much room as 48kHz 8 channel S32_LE. Add hardware rules for the three parameters to the frontend so that only the combinations which fit are offered. The real service interval is not known before a device's altsetting is resolved, so the rule assumes the shortest possible one (125 us) as a coarse pre-filter rather than capping high-bandwidth devices needlessly. Once the USB stream is acquired, the actual service interval is known: check the payload of two such intervals, rounded to whole sample frames, against the FIFO share and refuse the stream if it is larger. A payload which is exactly as large as the share fits. The frontend's backends are resolved before it is opened, and everything the core does to runtime->hw afterwards only narrows it, so the rules are added from the platform PCM open. The hook is generic, so it can be extended to other backends whose capabilities are only known at runtime, like HDMI. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The capabilities of the connected device cannot be coded into the topology file, the user can connect a wide range of devices. The topology provides a broad set of formats, rates and channels for the UAOL PCM, which is then narrowed down to what the actual USB device supports. For example: tplg: S16_LE/S24_LE/S32_LE, 48K/96K, stereo, headset: S16_LE only, 8-192K, stereo/2ch result on PCM: S16_LE (S24_LE/S32_LE dropped), 48K/96K, stereo The device is brought in as hardware rules and not as plain ranges, because what one parameter can be depends on the other two: a device can have a multichannel stream at one rate and a stereo one at another, and the ranges alone would offer the combination of the two that it does not have. The rules resolve the selected device on every call, so a device connected, removed or selected while a PCM sits open is picked up on the next negotiation. Each rule also depends on the parameter it refines, so that a request which changes only that parameter is checked against the device as well. A frontend which shares the backend with one that has it open, the deep buffer, can only run what the first one opened the backend with. Add rules which clone those parameters as the only ones allowed to the second frontend, so that userspace negotiates against the running stream. They read what the first frontend opened the backend with when they are evaluated, so they allow everything while the backend is idle. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The firmware reads a USB device's clock feedback endpoint through a link DMA of its own, which it expects as a second DMA config TLV in the UAOL gateway configuration (host_dma_config[1]). The driver only ever supplied the audio link DMA, so the firmware ran every playback stream without rate feedback. When the acquired USB stream has a feedback endpoint, allocate a second link DMA stream on the UAOL link, in the capture direction, and describe it in dma_config_tlv[1] the same way the audio one is described in dma_config_tlv[0]. It belongs to the playback substream but is neither the DAI's DMA data nor a stream of its own, so it is kept with the backend's stream state, is not registered as the link_substream, and is run and stopped from the UAOL trigger alongside the audio stream. It is released, and its TLV cleared, together with the USB stream, so that a following device without feedback does not inherit it. hda_link_stream_assign() gets the stream direction as a parameter for that, and the release of a link stream is factored out so that it can be undone without a DAI DMA data pointer. Co-developed-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com> Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The offloaded stream is released from the host's hw_free, which runs only after the DSP has stopped and freed its side of the link. Until then the xHCI keeps servicing the offloaded endpoints. An isochronous IN endpoint, the feedback endpoint of a playback stream in particular, then has nowhere to deliver its packets, which can stall the controller's periodic schedule and lock it up. Add usb_offload_quiesce_stream() so that the host can stop the endpoints on the controller once its own side of the link has stopped, before it is torn down. The feedback endpoint is stopped first, then the data endpoint. The sideband resources stay claimed and usb_offload_release_stream() still has to follow. The stream is named the way release_stream() names it, by its direction and USB PCM device number. Co-developed-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com> Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
usb_offload_quiesce_stream() stops the endpoints of an offloaded stream when the host stops it. A stream can be started again without being released in between, which is how an xrun is recovered from, and its endpoints then stay stopped: the stream stalls. Add usb_offload_restart_stream() for the host to run before it starts its own side of the link again. The data endpoint is restarted first, the feedback endpoint, which the quiesce stops first, last. The stream lookup is shared with the quiesce. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The library claims the USB substream it offloads, but the endpoints the DSP drives can still be shared with a host stream: with implicit feedback the capture data endpoint is also the sync source of the device's playback. An offloaded capture stream and the host's playback then both run that endpoint, the host submitting URBs on a ring the sideband owns, and both stall. Claim the data endpoint, and an explicit feedback endpoint, with snd_usb_endpoint_claim_offload() right after their hw_params, before anything is set up on them, and release them before the stream's hw_free. An endpoint which another stream has open already fails the acquisition with -EBUSY, and the host cannot open one while the stream is held. The test whether a stream has an explicit feedback endpoint is shared with the feedback reservation. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
snd_usb_pcm_prepare() runs snd_usb_set_format_quirk() whenever it sets up the data endpoint afresh. The library prepares the endpoints itself and skipped it, leaving a device which takes its sample rate through vendor requests at whatever rate it ran last. Export the quirk and run it the same way. Some quirks only set up the host's URB path: the MacroSilicon MS2100 and MS2109 capture cards need the first two bytes of the stream dropped, or the channels come out swapped and out of phase, and the E-Mu devices need the start of each packet skipped at 176.4 and 192kHz. The DSP gets the data as the device sends it, so refuse a stream that is left with such an offset with -EOPNOTSUPP. The E-Mu devices are vendor class and never pass the candidate filter, the MacroSilicon cards do. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
…s UAOL The USB stream is released from hw_free, after the DAI copier has been freed and the UAOL link powered off. Until then the xHCI keeps servicing the offloaded endpoints, and the feedback IN endpoint has nowhere to deliver its packets, which can lock up the controller. On TRIGGER_STOP and TRIGGER_SUSPEND, stop the xHCI endpoints with usb_offload_quiesce_stream() right after hda_dai_trigger() has stopped the DAI pipeline. They cannot be stopped any earlier: a UAOL stream cannot go idle once the xHCI no longer services an endpoint still bound to it, so the DSP would time out stopping both the audio and the feedback stream and fall back to resetting them. A stopped endpoint stays stopped until it is restarted, and only a hw_free followed by hw_params, or the reacquire after a system suspend, brings up fresh ones. A stream started again right after a stop, which is how an xrun is recovered from (STOP, prepare, START), would run the DSP against stopped endpoints: the link makes no progress and the application's next write fails with -EIO. Remember that the endpoints were stopped and restart them with usb_offload_restart_stream() on TRIGGER_START, before the DSP starts its side of the link, so that they run then as they do on the first start. Releasing the stream, at hw_free or before a reacquire, forgets it. Co-developed-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com> Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
…meanwhile The route control refuses to change a route while its backend holds a stream, but the backend only reads the selection before acquiring the device and marks the route busy once it has it. A change that goes through in between leaves the stream playing to the device the route selected before, while the control shows the new one. Read the selection back once the stream is set. The control decides whether the route is busy under the lock the selection is read with, so a change made during the acquisition is seen there and a later one is refused. Release the stream and fail with -EBUSY when the selection moved, the stream has not started yet. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
…quired prepare acquires a stream suspended by the SUSPEND trigger again, but the flag is cleared when the old acquisition is released, before the new one is attempted. A prepare retried after a failed one, a route change in between or a device gone meanwhile, finds the flag cleared and no device, and runs the common prepare: the DSP is started on a gateway whose xHCI sideband resources were released and may belong to another stream by then. The same happens on a prepare after the device was unplugged. Clear the flag only once the stream is acquired again, or the stream is closed, and fail prepare when the backend has no device. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The pieces of the USB audio offload are in place: the device notifications reach the ASoC USB port, the UAOL DAIs acquire, configure, quiesce and restart the USB stream, and the frontends are constrained to what the link and the device can carry. Install hda_machine_register() as the machine_register op, so that the driver registers with the USB audio offload library once the firmware has booted and the library starts reporting the offloadable devices. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Create the UAOL backend DAI links from the link mask and the stream counts reported in mach_params, following the BT offload link, with their own function topology so that the PCMs they add do not collide with the other functions of the card. Each stream of each direction gets a link of its own: the backend hw_params fixup is called without the stream direction, so a duplex link could not tell which of the two DAI copiers describes the stream being configured and would have to constrain both directions to the same format. With two streams per direction the backends are "UAOL0-Play0", "UAOL0-Play1", "UAOL0-Cap0" and "UAOL0-Cap1", on the CPU DAIs "UAOL0 Playback Pin0" and so on. The link and CPU DAI names are kept in sof_uaol_common.h for the other boards to share. The backends are created ordinal first, playback before capture within an ordinal, so the first playback and capture backend keep two consecutive IDs whatever the stream count, and a topology describing one stream per direction stays valid. The ordinal only tells the backends of a direction apart: it is not the UAOL stream the backend runs on, which is decided by the xHCI sideband resource the endpoint gets when the USB device is acquired. The backend names are abbreviated on purpose: the topology names its DAI copier widget "dai-copier.UAOL.<backend>.<direction>", and alsa-lib keeps element ids in SNDRV_CTL_ELEM_ID_NAME_MAXLEN bytes, so from 38 characters on the ids of the widget's "dai.word" and "dai.string" tuple sections collide and one set is silently dropped from the topology. The topology is named sof-uaol-id<BE id>.tplg after the id of the first playback link. The function name carries no link index, like dmic and hdmi, as the host driver reports a single link. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Add a link type creating the UAOL backend DAI links from a link mask and the stream counts per direction, so that boards using the helpers can place them in their link order like any other backend. Each stream of each direction gets a link of its own, named like sof_sdw does it, see sof_uaol_common.h. The backends are created ordinal first, playback before capture within an ordinal, so the IDs of the first playback and capture backend do not depend on how many streams the link has. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Create the UAOL backend DAI links from the link mask and the stream counts reported in mach_params, following the BT offload link. The board assigns fixed backend ids, so the UAOL links can take the ids after the BT offload one regardless of which other functions are present. A topology describing the UAOL therefore does not need a variant per board configuration. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Add Documentation/sound/soc/sof-uaol.rst, describing the USB audio offload on Intel platforms with SOF: how the xHCI Audio Sideband, the USB audio offload library, the ASoC USB framework, the SOF HDA driver and the machine drivers fit together, what the user visible interface is (the route controls and the jack), how a stream is acquired and released, how the UAOL gateway is configured and how the firmware's FIFO layout decides where an endpoint may be placed, the asynchronous playback policy, and the current limits. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
…s behind a hub" This reverts commit 0e544eef569fbdb90db410b3457e37224788a6e2. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
|
Changes since v1 Folded into the existing UAOL patches in
Tested on PTL (SoundWire and HDA): 384 kHz S16 on a high-speed adapter, two |
There was a problem hiding this comment.
🟡 Changes recommended
Critical shutdown-ordering and sample-width accounting errors can stop USB endpoint service while the DSP remains active and underestimate FIFO requirements, risking stalled streams and corrupted audio.
3 open issues
2 resolved since last review
Review effort: Balanced
| bytes = frames * params_channels(params) * | ||
| uaol_sample_bytes(params_format(params)) * SOF_UAOL_FIFO_INTERVALS; |
| if ((cmd == SNDRV_PCM_TRIGGER_STOP || cmd == SNDRV_PCM_TRIGGER_SUSPEND) && | ||
| stream->udev) { | ||
| usb_offload_quiesce_stream(hdev->uaol, stream->udev, direction, | ||
| stream->pcm_index); | ||
| stream->quiesced = true; |
| case XHCI_SIDEBAND_AUDIO: | ||
| if (!xhci_sideband_hw_capable(xhci)) { | ||
| dev_info(&udev->dev, "%s: Host not sideband capable\n", | ||
| __func__); | ||
| return NULL; | ||
| } | ||
| break; |


kconfig patch: thesofproject/kconfig#111
sof-test patch for sof_remove.sh: thesofproject/sof-test#1366
USB Audio Offload Link (UAOL) support. The series lets an Intel audio DSP
source and sink isochronous USB audio directly through the xHCI Audio Sideband,
bypassing the host URB path, while
snd-usb-audiokeeps full ownership of thedevice (descriptors, alternate settings, mixer, PM).
Base:
topic/sof-dev. 54 patches (+1 test-only revert at the tip, see below),53 files, +7248/-141.
Design in brief
sideband registration, resource ID reservation, endpoint hand-over and
property query.
xhci_sideband_start_endpoint()restarts an endpoint stoppedon the sideband.
hands to the DSP; the host cannot open them meanwhile (
-EBUSY).snd_usb_release_platform_ops()lets a platform let go of all devices whenit unregisters.
sound/usb/offload): a host registers oncewith
usb_offload_register(). The library:explicit feedback only);
sideband resources and runs the device's format quirk;
USB Offload <n> Playback/Capture Route, enumerating<card>,<pcm>pairs,and a
USB Offload Jack.constraints on the frontends;
recovery works;
refreshed for the resources it gets;
hda-uaol.cholds the Intel glue.the UAOL backend links. A missing
sof-uaolfunction topology only leavesUSB offload out.
USB Offload <n>frontend; the kernel never migrates a stream.
snd_usb_offload.allow_feedback=1to test the explicit feedback path).
to stay on the host. Devices with nothing to offload are not reported.
Layout
sof-uaol.rst,usb.rst)Testing
-fanalyzer: clean on the CI directories.checkpatch.pl --strict: clean apart from MAINTAINERS notes on the newfiles (xhci patches as provided).
capture.
explicit-feedback playback with
allow_feedback=1.plays there while other devices are offloaded.
capture.
384 kHz on the second PCM, and capture on both.
deep-buffer frontends, and with host playback on another device;
different sideband resource.
Known limitations
is not reacquired: ALSA only sends
TRIGGER_SUSPENDto running streams.This applies to all PCMs, not UAOL specific
test-only revert at the tip lifts that restriction.