Tuesday, July 31, 2007

I was wrong about Transport Stream Audio on PS3

I'd recently written that sound playback is screwed up on PS3 for MPEG2-TS files. It turns out I was wrong...its a "feature" not a bug. I got the following from Sony:

The PLAYSTATION(R)3 computer entertainment system will only support video files saved under the the
following formats.

  • - Memory Stick Video Format (MPEG-4 SP (AAC LC), H.264/MPEG-4 AVC Main Profile AAC LC)
  • - MPEG-1 (MPEG Audio Layer 2)
  • - MPEG-2 PS (MPEG2 Audio Layer 2, AAC LC, AC3(Dolby Digital), LPCM)
  • - MPEG-2 TS (MPEG2 Audio Layer 2)
  • - AVI (Motion JPEG (Linear PCM), Motion JPEG (u-law))
  • - AVCHD (.m2ts)

Notice that Dolby Digital audio is supported in program streams, but unsupported in transport streams. So that is why my TS files are missing audio occasionally. This STILL doesn't explain why some valid MPEG-2 PS files are choppy and not working. I have posted a clip to Sony...hopefully they will take a look at it and find the bug that causes the playback to suffer. I'll keep updates posted here.

Thursday, July 26, 2007

Backing Up files from your DVR (and why its not pirating if you do)

Since yesterday's post, I've gotten some readers who seem to think that copying content off of your DVR is illegal, and somehow piracy and/or copyright violation. The FCC, in late 2003, ruled that all cable customers with HD set top boxes should have functioning Firewire ports on them (see paragraph 24 on page 12, and paragraph 36 on page 17). They say this is so that "Digital VCRs" will interoperate properly with HD set top boxes. That's right, the FCC, a government agency, ruled that we were allowed to connect digital VCRs to our Hi-Def cable boxes. Clearly they're tricking us into thinking that it is legal, so they can bust down our doors! Engadget has written about this, so I'll spare you even more details.

What is DVB?
Digital broadcasts over cable, called DVB-C, come down the pipes as an MPEG2 Transport Stream. This is a stream of packets not unlike those sent over IP networks. Each packet is part of the video stream, the audio stream, or a part of some other meta-data stream (sub-titles, program info, etc). These packets are interleaved in time, and the receiving end identifies each one by a PID number, and demultiplexes the data using these PID numbers. The receiver then decodes the streams into signals your TV understands, and sends them on their merry way via HDMI, component, or whatever you use.

Capturing DVB
So we want to capture the transport stream. The essential idea is to capture a raw transport stream over Firewire as it plays by making your computer emulate a D-VHS recorder. This is not unlike a PCAP dump for you networking folks, essentially a packet trace, a log of all the packets that are coming fromt he cable company. I stress again that this constitutes fair use and is explicitly allowable BY THE FCC according to the link above. Wikibooks has a good collection of resources, so read on there if you want to do this. In short, this overcomes drive size limits on your DVR. I have a Motorola box from Comcast, and it has a 160GB drive. I DVR things onto the box, then play them back while recording them onto a 500GB external USB drive using my MacBook. Then, delete off my DVR box as necessary, and lo and behold, I effectively have 760GB hard drive for my DVR. These transport streams aren't playable by many players, but I know VLC can demux and play them. Connect my Macbook to the TV via DVI/HDMI, and there we go.

One last note, sometimes VLC will choke on the TS files. In this case, I suggest ProjectX, a free/open-source MPEG tool. I'll do a tutorial about using the latest version in the future. Or, you can use VLC's transcoding options to go from TS to PS, which is what you want. I was hoping my new PS3 would play these files correctly, but my last post was about my tribulations with making that work. More on this later. Any way, happy DVRing!

Why the PS3 is NOT a HD media Hub

I just bought a Playstation 3, my rationale being that it is "three revolutionary devices all in one" (sorry Steve): a Linux PC, a gaming device, and a Blu-Ray player. And after the latest price drop, it was more attractive than ever, who could resist? I certainly couldn't, I wanted something to really show off what my new 1080p TV can do. It turns our the Playstation isn't the box to do the things that I want when it comes to HD.

1. Linux is nerfed.
So I get the box home, get Ubuntu 7.04 on there, open up Synaptic, and download VLC. I try some 1080i MPEG2 content from a USB external HFS+ formatted drive I have. These recordings are direct, uncompressed digital rips of DVB-C MPEG2 recorded from my Cable Box via Firewire. When the picture looks grainy and the framerate is about 10fps, I remember that the Linux kernel is running on top of a hypervisor that restricts access to certain parts of the hardware, namely the badass mother of a GPU. Without access to the graphics hardware (and without a custom kernel and apps designed for the additional six availiable SIMD cores), there is no hope of being able to decode a 1.5MB/s MPEG2 stream. So I give up on Linux as my solution.

2. The Playstation3 OS supports no modern filesystem on external media.
As they might say on Law and Order, Linux is OUT. So I try the next logical step, booting up the PS3 OS and seeing if it will support my files. I connect my HFS+ drive...and nothing happens. As a sanity check, I connect a FAT32 drive I have, and it shows up as a USB device. After some google-ing, I read that the PS3 OS will only support FAT on external drives. This KILLS any hope of using it as a HD media hub. Why? Because FAT32 only supports files up to 4GiB in size. While this is okay for 1080i content under about 45 minutes, it is USELESS for movies. I have several 9GB and 10GB rips of 1080i broadcast movies, and I know there are plenty of people out there who are starting to rip 20GB and 30GB copies of their HD-DVD and BD-ROM movies. No dice for you if these are on drives that support large file sizes and you want to use them with the PS3. Hell, the Xbox360 supports HFS+ so that it can play music off of your Mac formatted Ipod. The PS3 needs support for SOMETHING modern that PCs use, either NTFS (Windows), EXT2/3 (most Linuxes), or HFS (MacOS).

3. Even when all else is right, playback can suck!
There is a small piece of good news: the Playstation3 OS can play raw MPEG2 TS, something which was unexpected. This should mean that if you are recording from a cable box via firewire, you don't have to perform a remux to a program stream. Of course it doesn't mean this, because of a bug in the TS playback. It for some reason cannot identify the audio packets in streams that VLC on a computer has no problem with, and doesn't demux them properly for playback. The end result: no sound for some TS files. Okay, so I remux to PS, and playback. Bear in mind these are all 30min TV show files under the 4GiB limit, transferred using a FAT32 drive. Hey, I have sound, I have 1080i video!...I have jerky motion, frames skipped...and this is the best, frames filled in from about two seconds back! Thats right, the scene has cut to the next shot, and I'm seeing randomly interspersed frames from about two seconds ago. Looks like the decoder hiccups and goes crazy when the scene changes drastically but before an I-frame comes up. This problem only happens with content recorded from certain channels, and not with others. Either way, it is a bug in the video playback, with no convenient bug reporting system. Give me bugzilla, and I would post a clip to Sony devs proving that the MPEG-2 playback can choke on regular files that play fine on a computer with VLC. They are no help on the phone, Sony Entertainment pawning me of on Sony Corporate, and vice versa, with no one listening to my complaint. I finally sent an e-mail...I wish I could talk to a developer or engineer.

4. Video File Format Support?
Its nice that the system supports MPEG-4 AVC and SP, somewhat cool for people who have things that were already converted for their iPods. I tried some of my iPod compatible videos both 320*240 AVC, and 640*480 SP, and they worked...but what joy is there in watching these on a 1920*1080 display? The advanced simple profile, however, is not supported, meaning that any DVDs you might have encoded with DivX or Xvid won't play, as these are MPEG4 ASP. I tried a few CDs with 700MB Divx/Xvid files, with the PS3 saying "unsupported format". And anyway these would look like crap on my display.

In conclusion, they shouldn't try to bill it as a HD media hub. It plays games well, and it plays Blu-Ray...but not much else useful.