Home

An experiment with WebAssembly

Here’s a look into my experiment running ffmpeg via WebAssembly directly in the browser. This will take a video input and convert it to a GIF. On the top left is an input field to drop a video file. Once added, the conversion of the file begins immediately and the resulting file will appear in the output container at the bottom.

🎥 Video to GIF

Is it really better to run your processing server-side over using client-side resources? Our devices, mobile or not, are quite powerful but bottlenecked by limitations placed at runtime; namely the lack of native code execution and our JS runtime being locked to a single thread. WASM is potentially an incredible solution to this problem. It’s been around for a while now but I still don’t think enough research has been done to prove what it can be capable of. This experiment is a way for me to become more familiar with the technology and build a simple tool I’m frustrated with not existing.

Why ffmpeg?

ffmpeg’s a swiss army knife for video processing and I’ve been comfortable using the CLI tooling for simple file conversions literally since I was 12. That being said, there are some annoying limitations to being locked to interacting with ffmpeg strictly via CLI:

  • I constantly forget which codec flags are compatible with my target format
  • I want to convert files on the go and not rely on a server

Knowing it’s popularity, I figured a prepackaged WASM build of ffmpeg already existed so a browser based tool to convert files could work to solve my problems. Thanks to Jerome Wu and the team building ffmpeg.wasm, there was!

Wacking it out

Just as a proof of concept, my goal for my first iteration is to get ffmpeg running in the browser, support multithreading and convert any video to a GIF. The implementation was quite straightforward according to the documentation provided by the ffmpeg.wasm team though not without a few hurdles.

To get it embeddable into this website itself, I had to ensure the library was loaded client-side only and any interactions with ffmpeg.wasm were done so explicitly by the browser.

Thankfully this is simple with the <ClientOnly> Nuxt component.

<ClientOnly>
  <Convert />
</ClientOnly>

I created a Pinia store to manage the interactions with the ffmpeg.wasm module. This houses all of the state and methods for initialization and performing a transcode on an input file.

The initialization of the module with support for multithreading was a pain due to ffmpeg.wasm’s poor support for Vite based builds.

My attempts had no luck in loading the module, providing no errors in the console and a hung promise. Being that the promise was hung and that this implementation was to support multithreading, in which Web Workers are a prerequisite, my assumption was that an issue within the worker.js file in the ffmpeg.wasm library was causing this.

I was correct in this assumption and found the point at which createFFmpegCore was missing at runtime.

self.createFFmpegCore = (await import(
  /* webpackIgnore: true */ /* @vite-ignore */ _coreURL)).default;

This assignment only ran in the event of a failed module import which was not the case here. This needed to be assigned in the event of a successful import as well.

The future

Et voilà, a working implementation of ffmpeg in the browser. Unless my curiosity urges me in another direction, I’ll flesh this project out by referencing the ffmpeg manual for a fully fleshed out in-browser interface for transcoding files without restriction.

Doing this would allow me to isolate submodules of the entire ffmpeg library to load in as needed. The full library is 31MB which on a suboptimal connection is a bit much to load in reliably.

- Pumposh

Back to top