Без кейворда

One of the most arousing things about the current web development landscape is the capability to utilize ‘hot off the press’ browser APIs to create native web apps with functionalities that once required tinkering with Flash plugins and/or force users to upgrade their version of Java. Here at Forio, we choose browser native features and open sourced technologies over what is proprietary whenever possible.

One of our current projects is a training simulation in which users work together to improve their counseling abilities. In this project, users will role-play counselor/clients over the web, the movie of which will be uploaded to a server so users can provide constructive feedback to each other after. Inspired by this project, today I will present a quick tutorial that uses Firefox’s MediaDevices, MediaRecorder, and WebRTC APIs, along with a Knot server on the backend and the movie processing instrument FFmpeg to create a movie conference recorder app, showcasing how far browsers have come since early days of the web. While presently only Firefox supports all three web APIs, Google Chrome supports two and will unveil support for the MediaRecorder API early next year, and with the age of auto-updating browsers upon us, it is my hope that in the near future all major browsers will run the demo sleekly.

Before we get embarked, you can find the repo with all the code used for the tutorial here.

Create a Single-Page Movie Recorder

Very first, let’s create a plain site that records users’ movies and plays them back. You can find the tutorial code in the singe-video directory of the repo.

One of the coolest things about the modern browser is its support for hardware device access, and getting a stream of user audio/movie data is as plain as the following snippet:

Once we have the media stream in place, we can create a fresh MediaRecorder object that takes the media stream and an optional parameter object that contains output parameters.

Next, set up the recorder’s ondataavaiable callback to treat the movie data (blob typed) that it comes back, begin the recorder, and we are in business.

Now all we have to do is stop the recorder when we are done recording, create an object URL from the binary data we get back from the recording, and play it back to the user. We have ourselves a movie recorder!

Upload Your Movie to a Server

Once we have the movie recorder working, we instantaneously run into the issue that Firefox becomes unstable while recording movies longer than a few minutes (understandably, most programs not designed to do so tend to run into a few issues when treating hundreds of megs of data). Just as importantly, movie files this large take at least a few minutes to upload to the server, which isn’t acceptable because we want the users to see their movies right away.

Since most of us do not have the good fortune of attending movie conferences less than a few minutes long, we need to find a way to split up and save the movie data. Fortunately, the MediaRecorder’s embark method takes an optional timeslice parameter that controls how often movie data is generated (and the ondataavailable method called).

But what is the best way to send these movie blobs over to our server? We can attempt and do ajax requests, but that will could run into concurrency issues, and what format should these blobs of binary data be? We have a few ways to manipulate the blob data with FileReader (convert it to binary string, base sixty four string encoding, to just name a few), but sending binary strings over ajax is inefficient. And with base sixty four movie data strings, we’ll have corrupt ending characters, rendering the movies not playable. After much trial and error, it turns out the best way to upload the movie is to use WebSocket’s binary data capabilities and send the movie data as ArrayBuffer.

Simply create a elementary Knot Express server with WebSocket capabilities and set the client’s connection binaryType to ‘arrayBuffer’:

Then when we get the blobs of movie data, convert them into array buffers via FileReaders and send them over the socket. On the other side of the connection, our Knot server receives these buffers, stores them as .webm files on the server, and serves them back as static content.

Record a Movie Conference

“Neat, but I thought this is a movie conference tutorial?” I hear you thinking. Fear not impatient movie conference fans, now we use some webRTC magic to connect participants (even however they will look more or less like me still).

The creation of an RTC connection consists of three separate phases: the creation of the peer connection object and the attachment of media flows, the session description suggest/reaction handshake, and lastly the ICE candidate exchange and the resulting remote media stream exchange.

Creating of PeerConnection

To create a movie conference, we’ll very first grab the user media stream as before, then create a mozRTCPeerConnection object with Overwhelm/TURN server parameters and fasten our local stream to the peer connection.

The next step of the RTC connection consists of the creation of an suggest session description from one peer connection and the corresponding creation of an reaction session description from the other browser. Note that these two session descriptions must be sent via a third party channel, which in our case will be the Knot socket server that we created in the earlier part of this post.

The last thing we need to do to create an RTC connection is to set the onicecandidate handler, which takes care of the exchange of the ICE candidate information inbetween two parties, the completion of which triggers the onaddstream event, carrying the media stream from the other party.

Now, just like before we’ll have each party send chunks of movie data to our Knot server to be saved on disk. We will use the excellent FF library to very first disrobe out the audio portion of the movies and re-encode them to include timestamps, then concatenate them side by side to create one movie with both parties. With that, we’ve got ourselves a movie conference recording site!

Related video: