Streaming Flow

Streaming Flow in VaahShare

All methods are in root.js store which is in renderer process.

• The toggleStream method toggles the streaming state.
• If streaming is currently active (this.is_streaming is true), it calls the stopStream method to stop streaming.
• Otherwise, it calls the startStream method to begin streaming.

startStream() :

• The startStream method initiates the video streaming process.
• It checks if a source ID is selected (this.selected_source_id). If no source is selected, it alerts the user to select a source.
• Upon successful validation:
  • Sets the is_streaming flag to true.
  • Calls setupMediaRecorder to configure the media recorder.
  • Emits a start-streaming event via WebSocket with relevant streaming details.

setupMediaRecorder() :

• The setupMediaRecorder method configures the MediaRecorder to record the video stream, learn more about MediaRecorder.
• Sets the MIME type to video/webm; codecs="vp8, opus".
• The ondataavailable event handler:
  • Sends recorded data chunks to the server via WebSocket if data is available and the recorder is in the recording state.
• The media recorder is started with an interval of 2000 milliseconds (2 seconds) to send video frames periodically.

stopMediaRecorder() : The stopMediaRecorder method stops the media recorder if it is active (this.media_recorder is not null).

Streaming Flow in Server (Fastify)

When a client starts streaming, the server logs the event and updates the client's status. Video frames are received, buffered, and broadcasted. When streaming stops, the server logs the event, processes and saves the buffered video data, and updates the client's status. The video is saved to a specified folder path.

1. start-streaming Event:

• Logs the start of the streaming session.
• Sets sendBufferStatus to true.
• Calls handleStartStreaming to update the client's streaming status and notify other clients.

2. stop-streaming Event

• Logs the stop of the streaming session.
• Calls handleStopStreaming to update the client's streaming status, notify other clients, and process video buffers.

3. video-frame Event

• Checks if a buffer exists for the client (data.socket_id); if not, creates one.
• Appends the received video frame (data.buffer) to the client's buffer.
• Logs the received video frame.
• Calls handleVideoFrame to broadcast the frame to other clients.

Handler Functions

1. handleStartStreaming:

• Updates the client's streaming status to true.
• Broadcasts a message to other clients that streaming has started.

2. handleStopStreaming :

• Updates the client's status to is_streaming: false.
• Broadcasts a message to other clients that streaming has stopped.
• Calls the handleVideoBuffers callback to process video data.

3. handleVideoFrame :

• Broadcasts the received video frame to other clients.

4. handleVideoBuffers :

• Combines video frames from the buffer into a single video file.
• Calls saveVideoFromArrayBuffer to save the video data.

5. saveVideoFromArrayBuffer :

• Converts the ArrayBuffer to a Buffer.
• Generates the folder path getCompleteVideoPath for saving the video.
• Creates the folder if it doesn't exist.
• Saves the video data as a temporary file.
• Processes the temporary video file for final storage (further processing steps in next section).
• Logs any errors that occur

6. getCompleteVideoPath :

• Generates the path for saving the video based on the client's company and user information.

Handling Streaming in Backend (VaahBackend)

When streaming starts, the client initializes media sources and updates the streaming status. Video frames are received, converted, and appended to the media source buffer. When streaming stops, the client's status is updated, and resources are cleaned up, ensuring seamless video streaming and proper management of video data. the following functions and events are used to manage streaming in the backend which is located in the VaahCMS/store/root.js file.

1. start-streaming Event :

• Logs the start of the streaming session.
• Sets streaming_client_id to the ID of the client that started streaming.
• Initializes statistics for the streaming client.
• Updates the client's streaming status in the list. Calls initMediaSources to initialize media sources with the client's socket ID

2. video-frame Event :

• Logs the received video frame.
• Converts the buffer to a Uint8Array.
• Appends the buffer to the source buffer if the media source is in an open state.

3. stop-streaming Event :

• Updates the client's streaming status in the list.
• Calls cleanUp to clear intervals and perform cleanup actions

Functions:

1. initMediaSources :

• Initializes a new MediaSource for the specified socket ID.
• Sets the video element's source to the media source object URL.
• Adds event listeners for sourceopen, sourceclose, and sourceended events.
• Adds a SourceBuffer to the MediaSource and handles updateend and error events.
• Learns more about MediaSource.

2. getRemoteVideoElement :

• Retrieves the video element with the specified socket ID, with retry logic to handle asynchronous loading.
• Resolves the promise when the video element is found, or rejects after a specified number of attempts.