Meeting: Fuzzy video for the first 10 seconds or so

I did some research and this semi fuzzy video feed for the first few seconds is normal in realtime video applications, it happens on Google Meet too, and depending on how good your internet connection is it will be more or less noticeable.

Let’s add this as a low priority task to Linear though, here is some context Grok came up with about this issue:

The fuzzy video feed you’re experiencing during the initial setup of a conversation in your LiveKit-based video meeting service is a common behavior in real-time communication systems like those built on WebRTC, which LiveKit uses under the hood. Let’s break down why this happens and what you can do to improve it.

Why the Video Feed Starts Fuzzy

When a video call begins, LiveKit (and WebRTC in general) establishes a peer-to-peer connection between participants using a process called ICE (Interactive Connectivity Establishment). During this initial phase, several factors can contribute to the video appearing fuzzy before improving:

1. Keyframe Delay: Video streams are encoded with keyframes (full images) and subsequent delta frames (changes between keyframes). At the start of a stream, the receiver might not have a keyframe yet, leading to incomplete or low-quality rendering until the first keyframe arrives. Depending on the codec and settings, keyframes might be sent every few seconds (e.g., 3 seconds in some default configurations), causing a delay before the video stabilizes.

2. Bandwidth Probing: LiveKit uses adaptive streaming to optimize video quality based on available bandwidth. At the start, it might send a lower bitrate or resolution to quickly establish the connection, then ramps up as it probes and confirms the network’s capacity. This gradual adjustment can make the video appear fuzzy initially.

3. Connection Setup: During the ICE negotiation and signaling phase, the connection might not be fully optimized yet. The system is still figuring out the best path (e.g., UDP, TCP, or TURN relay) and establishing a stable media flow, which can temporarily degrade quality.

4. Codec Initialization: The video codec (e.g., VP8, H.264, or VP9) might start with conservative settings to ensure compatibility and low latency, then adjust as the stream progresses and more data is exchanged.

In short, the fuzziness is a natural part of the system prioritizing a fast connection over instant high quality, with improvements kicking in once the connection stabilizes and more data (like keyframes) is available.

How to Improve This in Your App

While some of this behavior is inherent to real-time video systems, there are steps you can take within your LiveKit-based app to minimize the fuzzy period or make it less noticeable:

1. Increase Keyframe Frequency:

   • Adjust the video encoding settings to send keyframes more often at the start of the stream. This reduces the wait time for a full, clear image. In LiveKit, you can tweak this via the server or client SDK by setting a lower keyframe interval (e.g., every 1 second instead of 3). However, this increases bandwidth usage slightly, so test the trade-off.

2. Start with a Higher Initial Bitrate:

   • Configure LiveKit to begin with a slightly higher bitrate or resolution, assuming decent network conditions. You can set this in the publishTrack options when a participant starts sharing their video. For example, in the JavaScript SDK:
     

     room.localParticipant.publishTrack(track, {  videoQuality: 'high', // or a specific bitrate  simulcast: true, });

   • Be cautious: if the network can’t handle it, this could delay the connection further.

3. Enable Simulcast:

   • LiveKit supports simulcast, where multiple quality layers (e.g., low, medium, high) are sent simultaneously. The receiver can start with the low-quality layer and switch to a higher one as soon as bandwidth allows. Ensure simulcast is enabled when publishing tracks, which can smooth the transition from fuzzy to clear.

4. Preload a Static Image:

   • Display a placeholder (like a user’s profile picture or a blurred background) until the video stream stabilizes. This masks the fuzziness with a cleaner UI element. You’d implement this in your app’s front-end logic, switching to the live feed once quality improves (e.g., after a few seconds or a quality threshold).

5. Optimize Network Configuration:

   • Ensure your LiveKit server is deployed in a region close to your users to reduce latency during ICE negotiation. If using LiveKit Cloud, leverage its multi-region setup. For self-hosted setups, consider edge nodes or a CDN-like distribution.

   • Use UDP over TCP where possible (LiveKit prefers UDP by default), and minimize reliance on TURN servers, which add latency.

6. Fine-Tune Adaptive Streaming:

   • LiveKit’s Selective Forwarding Unit (SFU) and client SDKs automatically adjust quality based on bandwidth. You can customize this behavior via the client SDK to prioritize faster quality ramp-up. For instance, set a higher initial target bitrate in the RoomOptions or TrackPublishOptions.

7. Client-Side Feedback:

   • Add a subtle loading indicator (e.g., “Connecting…” or a spinner) during the initial fuzzy phase to set user expectations. This doesn’t fix the video but improves the perceived experience.

8. Test and Profile:

   • Use LiveKit’s debug tools (e.g., enable trace logging in the client SDK) to monitor connection stats like bitrate, packet loss, and keyframe arrival times. This can help you pinpoint whether the issue is codec-related, network-related, or something else.

Trade-Offs to Consider

• Bandwidth vs. Speed: Higher initial quality or frequent keyframes use more data, which might slow things down for users on poor networks.

• Latency vs. Quality: Pushing for instant clarity might increase buffering or jitter if the network isn’t ready.

• Complexity: Some fixes (like simulcast or keyframe tweaks) require deeper integration with LiveKit’s APIs, so weigh this against your development timeline.

Quick Recommendation

Start by enabling simulcast and setting a reasonable initial bitrate (e.g., 500kbps) in your track publishing options. Pair this with a UI placeholder to cover the first few seconds. Test with your target audience’s typical network conditions, and adjust from there based on feedback and metrics.

If you’re still seeing issues, feel free to share more details about your setup (e.g., codec, server config, or network conditions), and I can refine the advice further!