Encoding Video For The Nintendo 3DS

With a screen sporting the resolution & pixel densisity a fraction that of your phone while being thicker, weighing more, and having worse battery life, your 3DS is the perfect candidate for the consumption of your favorite media!

Before getting to the actual video encoding and playback, first you need to make sure you have a hacked 3DS running CFW (Custom FirmWare). If you don't already have that, then you should take a look at 3ds.hacks.guide. It works even if your 3DS is fully updated and lets you do lots of cool and fun things. Once you have that set up, we can move onto the meat and potaotes of the post.

All of this works off the back of one homebrew project: Core 2 Extreme's creatively titled 'Video player for 3DS'. (and of course ffmpeg, but thats pretty much a given when converting video) You can download it through Universal Updater if you dont want to deal with installing things manually.

I could start a tour of the video player and how to use it, but we don't even have any videos to play yet so let's do that first. This'll require a little bit of familiarty with a command line interface, but don't worry too much. Its a lot more ffmpeg jargon than CLI jargon. Speaking of, make sure you have ffmpeg installed. On your preferred flavor of Linux, this should just be as simple as installing the ffmpeg package. The ffmpeg website links to some pre-built versions for Windows and MacOS. I run Linux, so all the commands will be formatted as if you are using the same. You might need to alter them slightly depending on your OS.

The 3DS is not a very powerful system, even by the standards of the early to mid 2010s, so we need to be a bit picky on what video we feed it if we hope to get smooth playback. The original 2011 3DS in particular only has 2 ARM11 cpu cores clocked at 268MHz and no hardware acceleration for video playback whatsoever. The New 3DS doubles the core count, triples the clock speed, and adds HW accelerated video playback (at least for H.264 encoded video)[1] which makes our lives a lot easier. I only have a New 3DS XL, and thus can only test for that. Any recommendations I make about the original are based on the benchmarks Core 2 Extreme provides on the Video player page.

Because of the original's anemic decode performance you are pretty much stuck using the ancient mpeg1/mpeg2 codecs, losing out on the quality of both x264 & x265 plus the filesize savings of x265. On the New 3DS, you get a lot more options. Mpeg becomes totally irrelevant basically instantly. x265 becomes the best choice for file size, but if you cant stand the slower encoding or if the 3DS is struggling to play your video, then x264 is the better option with its HW decoding support.

Codec Comparison

mpeg2

x264

x265

The above samples were encoded with default settings other than the resolution (800x240, then converted as losslessly into webp as possible for this page) and with the pixel format set to yuv420p (performance reasons for the 3ds). Mpeg1&2 look nearly identically bad (I've only included mpeg2 here because of this), and take up a surprisingly large amount of space for their quality. x264 looks MUCH clearer, but takes up about 130% the space mpeg does. x265 looks at least as good as x264 but is also only 85% the size of the mpeg file, and only 65% the size of x264.

At its simplest, you can get a video that will play on the 3DS with one of the following commands:

mpeg2 - ffmpeg -⁠i INPUT.file -⁠s 400x240 -⁠c:v mpeg2video -⁠pix_fmt yuv420p OUTPUT.mp4

I've changed the resolution to 400x240 here since you would really only use this with the original 3DS and it would likely struggle with 800x240. Might as well save the filesize.

x264 - ffmpeg -⁠i INPUT.file -⁠s 800x240 -⁠c:v libx264 -⁠pix_fmt yuv420p OUTPUT.mp4

x265 - ffmpeg -⁠i INPUT.file -⁠s 800x240 -⁠c:v libx265 -⁠pix_fmt yuv420p OUTPUT.mp4

Going forward, I'll be using x265 at 800x240. You can change it to another codec or resolution by changing the values following -⁠s and -⁠c:v

The Video Player

Once it's finished encoding we can put it on the 3DS and finally get to the video player. Take the SD card out of the 3DS and put it in your computer. You can theoretically put the videos anywhere but to stay organized make a videos folder at the root of the SD card and drop it in there. You can make as many subfolders you want to keep things like shows organized. As an alternative to messing with the SD card, you can use FTPD [Universal Updater link] on your 3DS and an FTP client like Filezilla to transfer files over the network.

Depending on how you installed it (cia vs 3dsx) you can either launch the video player from the home screen or in the homebrew launcher.

Its a little barebones, but lets touch the big movie button in the center of the bottom screen.

Lets press x to open up the file selector.

Now lets navigate down to our video folder and choose what we encoded earlier.

ignore the fact its not the same file as what was in the last image

The video starts in "Fullscreen" mode, which you can toggle with select or just exit by touching the bottom screen. Here we can press Y to open up the options menu. You can touch and drag to scroll and hit the bottom bars to swap tabs. The rightmost tab contains some debug info so you can check when you are dropping frames or just if the buffer is being eaten into during high motion scenes.

Everything's all well and good so far, but we really only did the bare minimum. Theres still plenty of things we can do to make it a bit better overall.

Doing A Bit Better

Selecting Streams

First up, lets actually make sure we're only using the parts of the original file we really want. A video file is made up from several 'streams'. There's one for the video data itself, along with one for every audio and subtitle track. A player like VLC will let you choose which video track, audio track, and subtitle track to display. Since we are making files that are being played with a specific setting in mind (and that space and transfer speed may be limited in that setting) we should really just pick and choose the ones we want and get rid of the rest.

Let's check what streams make up our input file by running ffmpeg -⁠i INPUT.file

We should see an Input #0 with some metadata about the original file followed by a list of Stream #0:X each with their own metadata. The first number represents the input file, and the second represents which stream in that file, both starting from 0. Stream #1:2 for example would be the second input file's third stream. We won't be dealing with more than one input file though, so it'll always be #0:X

In the case of an anime, there might be Stream #0:0: Video containing the main video, Stream #0:1(jpn): Audio containing the original japanese audio, Stream #0:2(eng): Audio containing the english dubbed audio, and Stream #0:3(eng): Subtitle containing the english subtitles.

Since I prefer subs over dubs and know I'm not going use one of the audio tracks, I might as well just only include the one I will. We can use the -map flag to tell ffmpeg to only include certain streams.

ffmpeg -⁠i INPUT.file -⁠map 0:v -⁠map 0:a:0 -map 0:s -⁠s 800x240 -⁠c:v libx265 -⁠pix_fmt yuv420p OUTPUT.mp4

Instead of mapping to the absolute stream, we can just tell it to use any video stream with -⁠map 0:v, the first audio stream with -⁠map 0:a:0, and any subtitle stream with -⁠map 0:s. You'll need to change these to fit the original videos that you're using. If you've been getting an error so far about mp4 not supporting subtitles, then you can just omit the -⁠map 0:s entirely for now and I'll get back to subs in a little bit.

Speed and Quality

The next tweak we can make is the preset value. This controls how much time and effort the computer spends encoding. It comes in the flavors ultrafast superfast veryfast faster fast medium slow slower veryslow. The default is medium. Setting it to ultrafast cuts the encoding time in half, but sacrifices quality and filesize. veryslow takes over twice as long as the default, but can increase quality and reduce filesize by a few percent.[2] The gains in quality and size aren't massive, but the only downside to using a slower preset is the time it takes to encode. Since we'll probably be reencoding things multiple times to find the right settings, lets set it to fast for now, then change it later when we have everything set up the way we want.

ffmpeg -⁠i INPUT.file -⁠map 0:v -⁠map 0:a:0 -map 0:s -⁠s 800x240 -⁠c:v libx265 -⁠pix_fmt yuv420p -⁠preset fast OUTPUT.mp4

For an even greater impact on quality and filesize, we can start using a Constant Rate Factor. Its basically a quality setting. We can set -⁠crf to a value between 0 and 51 with lower values being larger and better visual quality and higher values being smaller and lower quality. x264 defaults to 23, and x265 defaults to 28, with both looking roughly equal at those values but with x265 being roughly half the size.[3] Not only is this a way to fine tune the quality and filesize, but also to avoid framedrops on the 3DS. If it's struggling to play a video then you can increase the crf by a couple, thus dropping the bitrate and making it easier to play. Setting it to 30 is a good place to start, you can always fine tune it from there.

ffmpeg -⁠i INPUT.file -⁠map 0:v -⁠map 0:a:0 -map 0:s -⁠s 800x240 -⁠c:v libx265 -⁠pix_fmt yuv420p -⁠preset fast -⁠crf 30 OUTPUT.mp4

Making Subtitles Actually Work

Up next is a big one: subtitles. The softsubs don't really cut it with this player. In fact, depending on your original video, they might not have even been FUNCTIONING yet. mp4 files don't support all the formats of subs there are, so if your original has unsupported subs then ffmpeg will just error out. We can instead burn the subtitles into the video as hardsubs. This is usually a complete taboo since its a non-reversible change that ruins the video, but we're already doing that by mangling it for the 3DS so its fine.

ffmpeg -⁠i INPUT.file -⁠map 0:v -⁠map 0:a:0 -⁠s 800x240 -⁠c:v libx265 -⁠pix_fmt yuv420p -⁠preset fast -⁠crf 30 -⁠vf "subtitles=INPUT.file:si=0:force_style='Fontsize=48'" OUTPUT.mp4

Here, we're applying a video filter -⁠vf called subtitles, which takes the original video file, selects the first subtitle track si=0, and sets the font size to 48 force_style='Fontsize=48'. If you have a seperate file containing subtitles (like a .srt file) then you can replace INPUT.file:si=0with your subtitle file.

There are some subtitle formats, like PGS, that won't work at all even though we are essentially throwing away the subtitle track entirely and merging it into the video track. This is because its not an issue with the mp4 container not supporting the subtitles like before, but because the subtitles themselves make a bizarre decision. Instead of being a bunch of strings of text with timestamp and maybe format data attached, they're actually a bunch of *images* of text with timestamps attached. The subtitles filter has no idea what to do with images and will just error out. My brain has no idea what to do with the images and just errors out. I kind of got a more complicated solution working that treats it as a second video stream that gets overlayed on the first, but it doesnt scale them which makes it hard to read on the 3DS. It might be possible to make it work, but honestly just find a different source with more normal subtitles.

Finishing Touches

Ffmpeg provides a few options for a -⁠tune parameter for x264 and x265. It makes some subtle changes to the encoding to better support certain cases. In my case, I set it to animation. It might also be worth using fastdecode for some more headroom for the 3DS, but I haven't done much testing. You can find documentation here.

ffmpeg -⁠i INPUT.file -⁠map 0:v -⁠map 0:a:0 -⁠s 800x240 -⁠c:v libx265 -⁠pix_fmt yuv420p -⁠preset fast -⁠crf 30 -⁠vf "subtitles=INPUT.file:si=0:force_style='Fontsize=48'" -⁠tune animation OUTPUT.mp4

If you are making a bunch of changes to get the quality or font size just right, then waiting for a whole episode - or worse, a whole movie - to encode just to check it for 3 seconds and encode again is agonizing. You can use the -⁠ss and -⁠t to set (in seconds) the start time and length of the ouput respectively. [...] -⁠ss 142 -⁠t 180 starts the output at one hundred and 42 seconds into the original and then ends 3 minutes later. It's also possible to use just one or the other to lets say only encode the first 30 seconds of video or to render the whole video starting from a certain timestamp.

All Done

Once you've twiddled all the settings to your liking and did all the fast 30 second encodes to get the font size right you're ready to do your finalized encode. For real this time. Using this command, I've been bringing individual anime episodes down to about 40MB each.

ffmpeg -⁠i INPUT.file -⁠map 0:v -⁠map 0:a:0 -⁠s 800x240 -⁠c:v libx265 -⁠pix_fmt yuv420p -⁠preset veryslow -⁠crf 30 -⁠vf "subtitles=INPUT.file:si=0:force_style='Fontsize=48'" -⁠tune animation OUTPUT.mp4