[Enhancement]Expose subscribed audio track to CallParticipant#1136
Conversation
📝 WalkthroughWalkthroughAdds remote audio track exposure: Changes
Sequence DiagramsequenceDiagram
participant Peer as StreamRTC PeerConnection
participant Adapter as RemoteAudioMediaAdapter
participant Subject as Track Subject
participant State as WebRTC StateAdapter
participant Model as CallParticipant
Peer->>Adapter: AddedReceiverEvent(audio)
activate Adapter
Adapter->>Adapter: extract RTCAudioTrack, receiverId, trackId
Adapter->>Subject: send .added(id, trackId, .audio, track)
deactivate Adapter
Subject->>State: receive .added TrackEvent
activate State
State->>Model: assign audioTrack to participant
deactivate State
Peer->>Adapter: RemovedReceiverEvent(receiverId)
activate Adapter
Adapter->>Subject: send .removed(id, .audio)
deactivate Adapter
Subject->>State: receive .removed TrackEvent
activate State
State->>Model: clear audioTrack for participant
deactivate State
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (3)
Sources/StreamVideo/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter.swift (1)
17-35: Nit:AudioTrack.trackIdnaming is slightly misleading.The doc comment describes
trackIdas "the participant lookup id it belongs to", but the value assigned isstream.trackId(the stream/track lookup prefix). Consider renaming tolookupId(ortrackLookupId) to avoid confusion withRTCAudioTrack.trackId/event.receiver.receiverId, both of which are also in scope here.♻️ Optional rename for clarity
- private struct AudioTrack { - var receiverId: String - var trackId: String - var track: RTCAudioTrack + private struct AudioTrack { + var receiverId: String + /// Lookup id derived from the audio stream, used as the participant + /// track lookup key (not to be confused with `RTCAudioTrack.trackId`). + var lookupId: String + var track: RTCAudioTrack @@ - self.receiverId = event.receiver.receiverId - self.trackId = stream.trackId - self.track = audioTrack + self.receiverId = event.receiver.receiverId + self.lookupId = stream.trackId + self.track = audioTrackThen update
processAddedTrack/processRemovedTrackto emitid: trackEntry.lookupId.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@Sources/StreamVideo/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter.swift` around lines 17 - 35, Rename the AudioTrack property trackId to a clearer name (e.g., lookupId or trackLookupId) throughout the RemoteAudioMediaAdapter: update the private struct AudioTrack to declare lookupId and assign stream.trackId in its initializer, and then update all usages (notably processAddedTrack and processRemovedTrack) to emit id: trackEntry.lookupId; ensure any references to AudioTrack.trackId are replaced so the receiverId and RTCAudioTrack identifiers remain unambiguous.StreamVideoTests/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter_Tests.swift (1)
126-128: Optional:TrackEventRecorderwrite isn't fully atomic.
@Atomic var eventsguards individual get/set of the array reference, butrecorder.events.append($0)is read-modify-write and not atomic. In practice this is fine here because only a single Combine subscription writes to it, but if this recorder is ever reused from multiple sinks/threads it could race. If you want to keep it defensive:♻️ Optional tightening
private final class TrackEventRecorder: `@unchecked` Sendable { - `@Atomic` var events: [TrackEvent] = [] + `@Atomic` var events: [TrackEvent] = [] + + func record(_ event: TrackEvent) { + _events.mutate { $0.append(event) } + } }(Only relevant if
@Atomicin this codebase exposes amutateclosure; otherwise ignore.)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@StreamVideoTests/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter_Tests.swift` around lines 126 - 128, The TrackEventRecorder's `@Atomic` var events only makes the getter/setter atomic but recorder.events.append(...) is a non-atomic read-modify-write; update TrackEventRecorder to perform mutations via a thread-safe API (e.g., use the `@Atomic-provided` mutate { } closure if available on events, or guard accesses with a dedicated serial DispatchQueue or NSLock) so all appends and reads happen inside the atomic/mutating block; update any call sites that do recorder.events.append(...) to use the chosen mutate/locked access pattern to avoid races.Sources/StreamVideo/WebRTC/v2/WebRTCStateAdapter.swift (1)
798-834: Combined audio/video presence logging looks fine.The four-way switch covers all combinations and produces a readable debug line. Minor nit (optional): in the
(false, false)case, the two names lists are appended independently, so a participant with both tracks appears in both clauses — acceptable for debug telemetry.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@Sources/StreamVideo/WebRTC/v2/WebRTCStateAdapter.swift` around lines 798 - 834, The logging in set(participants:) can list the same participant twice when they have both audio and video; update the generation of participantsWithAudioTracks and participantsWithVideoTracks (and/or the combined message in the (false, false) branch) to deduplicate names (e.g., use a Set or union of names) so each participant appears only once in the debug output; reference the participantsWithAudioTracks and participantsWithVideoTracks symbols and ensure the final logMessage still reports counts correctly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In
`@Sources/StreamVideo/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter.swift`:
- Around line 17-35: Rename the AudioTrack property trackId to a clearer name
(e.g., lookupId or trackLookupId) throughout the RemoteAudioMediaAdapter: update
the private struct AudioTrack to declare lookupId and assign stream.trackId in
its initializer, and then update all usages (notably processAddedTrack and
processRemovedTrack) to emit id: trackEntry.lookupId; ensure any references to
AudioTrack.trackId are replaced so the receiverId and RTCAudioTrack identifiers
remain unambiguous.
In `@Sources/StreamVideo/WebRTC/v2/WebRTCStateAdapter.swift`:
- Around line 798-834: The logging in set(participants:) can list the same
participant twice when they have both audio and video; update the generation of
participantsWithAudioTracks and participantsWithVideoTracks (and/or the combined
message in the (false, false) branch) to deduplicate names (e.g., use a Set or
union of names) so each participant appears only once in the debug output;
reference the participantsWithAudioTracks and participantsWithVideoTracks
symbols and ensure the final logMessage still reports counts correctly.
In
`@StreamVideoTests/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter_Tests.swift`:
- Around line 126-128: The TrackEventRecorder's `@Atomic` var events only makes
the getter/setter atomic but recorder.events.append(...) is a non-atomic
read-modify-write; update TrackEventRecorder to perform mutations via a
thread-safe API (e.g., use the `@Atomic-provided` mutate { } closure if available
on events, or guard accesses with a dedicated serial DispatchQueue or NSLock) so
all appends and reads happen inside the atomic/mutating block; update any call
sites that do recorder.events.append(...) to use the chosen mutate/locked access
pattern to avoid races.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ef2f14a7-868a-485a-91ea-475dbb57f811
📒 Files selected for processing (9)
CHANGELOG.mdSources/StreamVideo/Models/CallParticipant.swiftSources/StreamVideo/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter.swiftSources/StreamVideo/WebRTC/v2/PeerConnection/MediaAdapters/MediaAdapter.swiftSources/StreamVideo/WebRTC/v2/PeerConnection/RTCPeerConnectionCoordinator.swiftSources/StreamVideo/WebRTC/v2/WebRTCStateAdapter.swiftStreamVideoTests/Mock/CallParticipant_Mock.swiftStreamVideoTests/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter_Tests.swiftStreamVideoTests/WebRTC/v2/WebRTCStateAdapter_Tests.swift
# Conflicts: # CHANGELOG.md
1427247 to
e76e67b
Compare
Public Interface public struct CallParticipant: Identifiable, Sendable, Hashable
- public var track: RTCVideoTrack?
+ public var audioTrack: RTCAudioTrack?
- public var trackSize: CGSize
+ public var track: RTCVideoTrack?
- public var screenshareTrack: RTCVideoTrack?
+ public var trackSize: CGSize
- public var showTrack: Bool
+ public var screenshareTrack: RTCVideoTrack?
- public var isSpeaking: Bool
+ public var showTrack: Bool
- public var isDominantSpeaker: Bool
+ public var isSpeaking: Bool
- public var sessionId: String
+ public var isDominantSpeaker: Bool
- public var connectionQuality: ConnectionQuality
+ public var sessionId: String
- public var joinedAt: Date
+ public var connectionQuality: ConnectionQuality
- public var audioLevel: Float
+ public var joinedAt: Date
- public var audioLevels: [Float]
+ public var audioLevel: Float
- public var pin: PinInfo?
+ public var audioLevels: [Float]
- public var pausedTracks: Set<TrackType>
+ public var pin: PinInfo?
- public var source: ParticipantSource
+ public var pausedTracks: Set<TrackType>
- public var userId: String
+ public var source: ParticipantSource
- public var name: String
+ public var userId: String
- public var profileImageURL: URL?
+ public var name: String
- public var isPinned: Bool
+ public var profileImageURL: URL?
- public var isPinnedRemotely: Bool
+ public var isPinned: Bool
- public var shouldDisplayTrack: Bool
+ public var isPinnedRemotely: Bool
-
+ public var shouldDisplayTrack: Bool
-
+
- public init(id: String,userId: String,roles: [String],name: String,profileImageURL: URL?,trackLookupPrefix: String?,hasVideo: Bool,hasAudio: Bool,isScreenSharing: Bool,showTrack: Bool,track: RTCVideoTrack? = nil,trackSize: CGSize = CGSize(width: 1024, height: 720),screenshareTrack: RTCVideoTrack? = nil,isSpeaking: Bool = false,isDominantSpeaker: Bool,sessionId: String,connectionQuality: ConnectionQuality,joinedAt: Date,audioLevel: Float,audioLevels: [Float],pin: PinInfo?,pausedTracks: Set<TrackType>,source: ParticipantSource = .webRTCUnspecified)
+
-
+ public init(id: String,userId: String,roles: [String],name: String,profileImageURL: URL?,trackLookupPrefix: String?,hasVideo: Bool,hasAudio: Bool,isScreenSharing: Bool,showTrack: Bool,audioTrack: RTCAudioTrack? = nil,track: RTCVideoTrack? = nil,trackSize: CGSize = CGSize(width: 1024, height: 720),screenshareTrack: RTCVideoTrack? = nil,isSpeaking: Bool = false,isDominantSpeaker: Bool,sessionId: String,connectionQuality: ConnectionQuality,joinedAt: Date,audioLevel: Float,audioLevels: [Float],pin: PinInfo?,pausedTracks: Set<TrackType>,source: ParticipantSource = .webRTCUnspecified)
-
+
- public static func ==(lhs: CallParticipant,rhs: CallParticipant)-> Bool
+
- public func withUpdated(trackSize: CGSize)-> CallParticipant
+ public static func ==(lhs: CallParticipant,rhs: CallParticipant)-> Bool
- public func withUpdated(track: RTCVideoTrack?)-> CallParticipant
+ public func withUpdated(trackSize: CGSize)-> CallParticipant
- public func withUpdated(screensharingTrack: RTCVideoTrack?)-> CallParticipant
+ public func withUpdated(audioTrack: RTCAudioTrack?)-> CallParticipant
- public func withUpdated(audio: Bool)-> CallParticipant
+ public func withUpdated(track: RTCVideoTrack?)-> CallParticipant
- public func withUpdated(video: Bool)-> CallParticipant
+ public func withUpdated(screensharingTrack: RTCVideoTrack?)-> CallParticipant
- public func withUpdated(screensharing: Bool)-> CallParticipant
+ public func withUpdated(audio: Bool)-> CallParticipant
- public func withUpdated(showTrack: Bool)-> CallParticipant
+ public func withUpdated(video: Bool)-> CallParticipant
- public func withUpdated(trackLookupPrefix: String)-> CallParticipant
+ public func withUpdated(screensharing: Bool)-> CallParticipant
- public func withUpdated(isSpeaking: Bool,audioLevel: Float)-> CallParticipant
+ public func withUpdated(showTrack: Bool)-> CallParticipant
- public func withUpdated(dominantSpeaker: Bool)-> CallParticipant
+ public func withUpdated(trackLookupPrefix: String)-> CallParticipant
- public func withUpdated(connectionQuality: ConnectionQuality)-> CallParticipant
+ public func withUpdated(isSpeaking: Bool,audioLevel: Float)-> CallParticipant
- public func withUpdated(pin: PinInfo?)-> CallParticipant
+ public func withUpdated(dominantSpeaker: Bool)-> CallParticipant
- public func withPausedTrack(_ trackType: TrackType)-> CallParticipant
+ public func withUpdated(connectionQuality: ConnectionQuality)-> CallParticipant
- public func withUnpausedTrack(_ trackType: TrackType)-> CallParticipant
+ public func withUpdated(pin: PinInfo?)-> CallParticipant
+ public func withPausedTrack(_ trackType: TrackType)-> CallParticipant
+ public func withUnpausedTrack(_ trackType: TrackType)-> CallParticipant |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
Sources/StreamVideo/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter.swift (1)
59-71: Consider symmetric scheduling of the two subscriptions.The added-receiver chain runs
compactMap(AudioTrack.init)beforereceive(on: processingQueue), while the removed-receiver chain runscompactMapafter. The added branch's mapping therefore executes on whichever thread the peer connection publisher emits from (dereferencingevent.receiver.track). Movingreceive(on:)earlier would make both flows serialize on the same queue and keep WebRTC track inspection on a predictable executor:♻️ Proposed reorder
peerConnection .publisher(eventType: StreamRTCPeerConnection.AddedReceiverEvent.self) - .compactMap(AudioTrack.init) .receive(on: processingQueue) + .compactMap(AudioTrack.init) .sink { [weak self] in self?.processAddedTrack($0) } .store(in: disposableBag)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@Sources/StreamVideo/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter.swift` around lines 59 - 71, The two peerConnection publisher subscriptions should schedule onto processingQueue before mapping to keep execution symmetric: in the added-receiver chain that uses peerConnection.publisher(eventType: StreamRTCPeerConnection.AddedReceiverEvent.self) and compactMap(AudioTrack.init) then .receive(on: processingQueue) (which currently runs the compactMap on the publisher thread), move .receive(on: processingQueue) earlier so the chain becomes publisher(...).receive(on: processingQueue).compactMap(AudioTrack.init).sink { [weak self] in self?.processAddedTrack($0) } .store(in: disposableBag) so both added and removed flows serialize on processingQueue and track inspection happens on the predictable executor; keep processAddedTrack, processRemovedTrack, audioReceivers and disposableBag references intact.StreamVideoTests/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter_Tests.swift (1)
33-85: Consider adding negative-path coverage.Both existing tests exercise the happy path. The adapter's
AudioTrack.init?silently drops receivers whose first matching stream isn't audio (or whose track isn't anRTCAudioTrack), and the removed pipeline'scompactMapdrops unknownreceiverIds. A small test each would lock down those filters against future regressions (e.g., if someone changes the stream-selection predicate inAudioTrack.init?).Suggested additions:
test_addedReceiver_nonAudioStream_doesNotPublishTrackEvent()— send anAddedReceiverEventwhose streamtrackTypeis.video, assert recorder stays empty.test_removedReceiver_unknownReceiver_doesNotPublishTrackEvent()— sendRemovedReceiverEventwith a receiver that was never added, assert recorder stays empty.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@StreamVideoTests/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter_Tests.swift` around lines 33 - 85, Add two negative-path tests in RemoteAudioMediaAdapter_Tests.swift: implement test_addedReceiver_nonAudioStream_doesNotPublishTrackEvent() that creates a TrackEventRecorder, a receiver via makeAudioReceiver(), and a stream whose trackType is .video (use makeAudioStream but adjust to video or create a stream with video track type), call observeEvents(with: recorder), send StreamRTCPeerConnection.AddedReceiverEvent(receiver: receiver, streams: [stream]) via mockPeerConnection.subject.send and then assert the recorder has no events; and implement test_removedReceiver_unknownReceiver_doesNotPublishTrackEvent() that creates a TrackEventRecorder, a receiver that was never added, calls observeEvents(with: recorder), sends StreamRTCPeerConnection.RemovedReceiverEvent(receiver: receiver) and then asserts the recorder remains empty. Use the same helpers used in other tests (TrackEventRecorder, observeEvents(with:), mockPeerConnection.subject.send, waitForEvents/wait helpers) and name the tests exactly as suggested.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@Sources/StreamVideo/WebRTC/v2/WebRTCStateAdapter.swift`:
- Around line 801-833: The current logic computes participantsWithAudioTracks
and participantsWithVideoTracks as joined String values and then uses isEmpty on
those strings, which falsely treats a single participant with an empty name as
"no track"; change to first compute filtered arrays (e.g., audioParticipants =
participants.filter { $0.value.audioTrack != nil }.map(\.value.name) and
videoParticipants = participants.filter { $0.value.track != nil
}.map(\.value.name)), branch on audioParticipants.isEmpty and
videoParticipants.isEmpty in the switch, and only call joined(separator: ",")
when building logMessage (e.g., join audioParticipants/videoParticipants for the
message), updating references to
participantsWithAudioTracks/participantsWithVideoTracks accordingly and
preserving the logMessage construction and log.debug call.
---
Nitpick comments:
In
`@Sources/StreamVideo/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter.swift`:
- Around line 59-71: The two peerConnection publisher subscriptions should
schedule onto processingQueue before mapping to keep execution symmetric: in the
added-receiver chain that uses peerConnection.publisher(eventType:
StreamRTCPeerConnection.AddedReceiverEvent.self) and compactMap(AudioTrack.init)
then .receive(on: processingQueue) (which currently runs the compactMap on the
publisher thread), move .receive(on: processingQueue) earlier so the chain
becomes publisher(...).receive(on:
processingQueue).compactMap(AudioTrack.init).sink { [weak self] in
self?.processAddedTrack($0) } .store(in: disposableBag) so both added and
removed flows serialize on processingQueue and track inspection happens on the
predictable executor; keep processAddedTrack, processRemovedTrack,
audioReceivers and disposableBag references intact.
In
`@StreamVideoTests/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter_Tests.swift`:
- Around line 33-85: Add two negative-path tests in
RemoteAudioMediaAdapter_Tests.swift: implement
test_addedReceiver_nonAudioStream_doesNotPublishTrackEvent() that creates a
TrackEventRecorder, a receiver via makeAudioReceiver(), and a stream whose
trackType is .video (use makeAudioStream but adjust to video or create a stream
with video track type), call observeEvents(with: recorder), send
StreamRTCPeerConnection.AddedReceiverEvent(receiver: receiver, streams:
[stream]) via mockPeerConnection.subject.send and then assert the recorder has
no events; and implement
test_removedReceiver_unknownReceiver_doesNotPublishTrackEvent() that creates a
TrackEventRecorder, a receiver that was never added, calls observeEvents(with:
recorder), sends StreamRTCPeerConnection.RemovedReceiverEvent(receiver:
receiver) and then asserts the recorder remains empty. Use the same helpers used
in other tests (TrackEventRecorder, observeEvents(with:),
mockPeerConnection.subject.send, waitForEvents/wait helpers) and name the tests
exactly as suggested.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ba534fc3-4726-46e0-859b-71103c0d22d5
📒 Files selected for processing (9)
CHANGELOG.mdSources/StreamVideo/Models/CallParticipant.swiftSources/StreamVideo/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter.swiftSources/StreamVideo/WebRTC/v2/PeerConnection/MediaAdapters/MediaAdapter.swiftSources/StreamVideo/WebRTC/v2/PeerConnection/RTCPeerConnectionCoordinator.swiftSources/StreamVideo/WebRTC/v2/WebRTCStateAdapter.swiftStreamVideoTests/Mock/CallParticipant_Mock.swiftStreamVideoTests/WebRTC/v2/PeerConnection/MediaAdapters/LocalMediaAdapters/RemoteAudioMediaAdapter_Tests.swiftStreamVideoTests/WebRTC/v2/WebRTCStateAdapter_Tests.swift
✅ Files skipped from review due to trivial changes (4)
- CHANGELOG.md
- StreamVideoTests/Mock/CallParticipant_Mock.swift
- Sources/StreamVideo/WebRTC/v2/PeerConnection/RTCPeerConnectionCoordinator.swift
- Sources/StreamVideo/Models/CallParticipant.swift
🚧 Files skipped from review as they are similar to previous changes (2)
- Sources/StreamVideo/WebRTC/v2/PeerConnection/MediaAdapters/MediaAdapter.swift
- StreamVideoTests/WebRTC/v2/WebRTCStateAdapter_Tests.swift
SDK Size
|
StreamVideo XCSize
|
StreamVideoSwiftUI XCSize
Show 11 more objects
|
|
martinmitrevski
left a comment
There was a problem hiding this comment.
nice stuff, let's test this next week



🔗 Issue Links
Resolves https://linear.app/stream/issue/IOS-1639/enhancementaudiotrack-on-callparticipant
🎯 Goal
Expose each participant's subscribed remote
RTCAudioTrackso consumers can access remote audio tracks directly fromCallParticipant.📝 Summary
CallParticipant.audioTrack.🛠 Implementation
Remote audio is surfaced by WebRTC through
RTCRtpReceivercallbacks rather than media stream add/remove callbacks. This PR adds a remote audio adapter that observes audio receiver additions and removals, maps each receiver to the participant track lookup id, and emits the existing track add/remove flow.WebRTCStateAdapternow assigns stored audio tracks to matching participants alongside video and screenshare tracks.☑️ Contributor Checklist
Summary by CodeRabbit
New Features
Bug Fixes
Tests