Recently, I’ve been working on streaming live video from IP cameras to a Flash player on a website. It sounds simple, but not so much, (if you’re in a hurry, skip to the solution).
The problem is that most IP cameras are not made for streaming live, full-motion, events to the web. They’re made for surveillance at 11 FPS in Motion JPEG format, (that’s just a bunch of JPEG images coming one after another). This is obviously not ideal from a bandwidth perspective at all. The cameras that our project uses are Axis 207 and 210 cameras. These cameras are capable of streaming MPEG-4 video, and when you look at the video in a web browser it looks pretty good. When I first saw that, I was excited – I could just stream into a Flash media server, (We’re using Wowza Media Server Pro at the moment), and that would send everything off to the player, right? Wrong.
It turns out that most, if not all, Axis products stream in MPEG4-ES, which flash cannot understand, and therefore our server rejects. I had to find a way to change MP4V-ES to h.264.
The obvious solution to transcoding the video stream from MP4V-ES to h.264 is VLC. While it looks like a media player that can handle a lot of formats, under the surface lies a powerful, command-line based, transcoding and streaming program. I discovered that, in theory, I should be able to issue one command to VLC and have it receive the MPEG4-ES stream from the camera, transcode it to h.264, and stream it to the Wowza, which would handle the rest.
I started by opening a file, transcoding it to h.264, and streaming it to Wowza:
vlc -vvv /path/to/file/Extremists.m4v --sout "#transcode{venc=x264,vcodec=x264,vb=500,scale=1,acodec=mp4a,ab=32,channels=2,samplerate=22100}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///path/to/wowza/content/myStream.sdp}"
It was a little rough, as the testing server doesn’t have a lot of processing power, but it worked. Awesome. Now I just have to hook it up to the stream from the camera, right?
I added the camera as the source and removed the sound:
vlc -vvv rtsp://camera-ip-address:554/mpeg4/1/media.amp --no-sout-audio --sout "#transcode{venc=x264,vcodec=x264,vb=200,scale=1}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///absolute/path/to/wowza/content/myStream.sdp}"
With Wowza already running on the server, I typed that in to terminal, and it started to look good. Then PAF! I get this error:
[00000385] access_output_udp private debug: mmh, hole (147841635484807 > 2s) -> drop
VLC seems to think that a frame, or some piece of information has been delayed 147841635484807 seconds. I highly doubt that, but VLC is convinced. Try as I might, I was not able to get VLC to realize that the frame, (or whatever bit of info), was simply missing a timestamp or something.
So, I figured I would debug just the connection to the camera. I opened a VNC session, and ran this:
vlc -vvv rtsp://camera-ip-address:554/mpeg4/1/media.amp
To see if I could view the video. I could, so I tried saving it to a file:
vlc -vvv rtsp://CAMERA-IP-ADDRESS:554/mpeg4/media.amp --no-drop-late-frames --no-sout-audio --sout "#std{mux=ts,access=file,dst=/tmp/camstream.m4v}"
This worked also. It appears that the problem only occurs when I am trying to open, transcode, and send out the stream all at once.
I realized, if I can transcode from a file and stream, and if I can capture a stream and save it to a file, I should be able to do both at the same time. It works! The steps are, make the pipe:
mkpipe /tmp/vpipe
Then capture the stream and save it to the pipe:
vlc -vvv rtsp://CAMERA-IP-ADDRESS:554/mpeg4/media.amp --no-drop-late-frames --no-sout-audio --sout "#std{mux=ts,access=file,dst=/tmp/vpipe}"
And finally read from the pipe, transcode, and stream to the flash server:
vlc -vvv /tmp/vpipe --no-sout-audio --sout "#transcode{venc=x264,vcodec=x264,vb=500,scale=1}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///path/to/wowza/content/myStream.sdp}"
Shazam!
There are a couple of caveats, though:
- This sucks major processing horsepower. Make sure you’ve got enough
- To make a stream work this way you have to have 2 copies of VLC running, plus your flash server. That’s a lot of parts that could have a problem
Because of these caveats, I am still looking for an alternate solution, and may stream MPEG-4 to browsers until Axis has its h.264 products ready later this year.
Yesterday, I did find a second possible solution. Instead of using the first instance of VLC to capture the stream, it is sometimes possible to use Darwin Streaming Server to capture the stream, then use 1 instance of VLC to transcode it. This seems to use much less processing power, but is not 100% reliable either.