Using VLC to transcode an Axis Camera’s video stream, and stream it out again
Recently, I’ve been working on streaming live video from IP cameras to a Flash player on a website. It sounds simple, but not so much, (if you’re in a hurry, skip to the solution).
The problem is that most IP cameras are not made for streaming live, full-motion, events to the web. They’re made for surveillance at 11 FPS in Motion JPEG format, (that’s just a bunch of JPEG images coming one after another). This is obviously not ideal from a bandwidth perspective at all. The cameras that our project uses are Axis 207 and 210 cameras. These cameras are capable of streaming MPEG-4 video, and when you look at the video in a web browser it looks pretty good. When I first saw that, I was excited - I could just stream into a Flash media server, (We’re using Wowza Media Server Pro at the moment), and that would send everything off to the player, right? Wrong.
It turns out that most, if not all, Axis products stream in MPEG4-ES, which flash cannot understand, and therefore our server rejects. I had to find a way to change MP4V-ES to h.264.
The obvious solution to transcoding the video stream from MP4V-ES to h.264 is VLC. While it looks like a media player that can handle a lot of formats, under the surface lies a powerful, command-line based, transcoding and streaming program. I discovered that, in theory, I should be able to issue one command to VLC and have it receive the MPEG4-ES stream from the camera, transcode it to h.264, and stream it to the Wowza, which would handle the rest.
I started by opening a file, transcoding it to h.264, and streaming it to Wowza:
vlc -vvv /path/to/file/Extremists.m4v --sout "#transcode{venc=x264,vcodec=x264,vb=500,scale=1,acodec=mp4a,ab=32,channels=2,samplerate=22100}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///path/to/wowza/content/myStream.sdp}"
It was a little rough, as the testing server doesn’t have a lot of processing power, but it worked. Awesome. Now I just have to hook it up to the stream from the camera, right?
I added the camera as the source and removed the sound:
vlc -vvv rtsp://camera-ip-address:554/mpeg4/1/media.amp --no-sout-audio --sout "#transcode{venc=x264,vcodec=x264,vb=200,scale=1}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///absolute/path/to/wowza/content/myStream.sdp}"
With Wowza already running on the server, I typed that in to terminal, and it started to look good. Then PAF! I get this error:
[00000385] access_output_udp private debug: mmh, hole (147841635484807 > 2s) -> drop
VLC seems to think that a frame, or some piece of information has been delayed 147841635484807 seconds. I highly doubt that, but VLC is convinced. Try as I might, I was not able to get VLC to realize that the frame, (or whatever bit of info), was simply missing a timestamp or something.
So, I figured I would debug just the connection to the camera. I opened a VNC session, and ran this:
vlc -vvv rtsp://camera-ip-address:554/mpeg4/1/media.amp
To see if I could view the video. I could, so I tried saving it to a file:
vlc -vvv rtsp://CAMERA-IP-ADDRESS:554/mpeg4/media.amp --no-drop-late-frames --no-sout-audio --sout "#std{mux=ts,access=file,dst=/tmp/camstream.m4v}"
This worked also. It appears that the problem only occurs when I am trying to open, transcode, and send out the stream all at once.
I realized, if I can transcode from a file and stream, and if I can capture a stream and save it to a file, I should be able to do both at the same time. It works! The steps are, make the pipe:
mkpipe /tmp/vpipe
Then capture the stream and save it to the pipe:
vlc -vvv rtsp://CAMERA-IP-ADDRESS:554/mpeg4/media.amp --no-drop-late-frames --no-sout-audio --sout "#std{mux=ts,access=file,dst=/tmp/vpipe}"
And finally read from the pipe, transcode, and stream to the flash server:
vlc -vvv /tmp/vpipe --no-sout-audio --sout "#transcode{venc=x264,vcodec=x264,vb=500,scale=1}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///path/to/wowza/content/myStream.sdp}"
Shazam!
There are a couple of caveats, though:
- This sucks major processing horsepower. Make sure you’ve got enough
- To make a stream work this way you have to have 2 copies of VLC running, plus your flash server. That’s a lot of parts that could have a problem
Because of these caveats, I am still looking for an alternate solution, and may stream MPEG-4 to browsers until Axis has its h.264 products ready later this year.
Yesterday, I did find a second possible solution. Instead of using the first instance of VLC to capture the stream, it is sometimes possible to use Darwin Streaming Server to capture the stream, then use 1 instance of VLC to transcode it. This seems to use much less processing power, but is not 100% reliable either.
4:24 pm
Whew, I am tired just reading your process. We are trying to take a linksys camera and push it through to Mogulus. Mogulus can not take the IP and I was wondering if I could use that IP camera routed through Adobe Flash encoder 2.5.
Here is the link to the camera.
http://XXX.XXX.XXX/img/main.cgi?next_file=main.htm
Here is an example of Mogulus on our site streaming live from Beijing.
http://www.kare11.com/news/olympics/beijing/default.aspx
Jeff Kraker
kare11.com
5:13 pm
That could be rough. When we were first exploring our options we played around with Mogulus a bunch, but we couldn’t find a way to stream in from an IP camera, (I actually E-mailed them and they said that they didn’t support it at the time).
From what I’ve seen, and I tried both, it’s not possible to hook Adobe Flash Encoder, or the On2 encoder, up to an IP camera. It would be great if that was possible.
If you can get your camera’s stream to play in Quicktime or something, you might try, (and this is pretty hack-ish), is play the video in Quicktime on a computer, and then capture the screen of that computer and send it to Mogulus. I’m not sure about audio, or quality, but it’ll get a picture there.
Another option would be to transcode the video somehow, then stream it to Mogulus, but I think you have to stream to Mogulus in Adobe’s proprietary RTMP format, and I don’t think VLC can stream out in RTMP, so you’d need something else in the middle, like Wowza that I’m using.
5:23 pm
Thanks, we are looking at an I-frame of the live video and moving on to fight again another day.
Jeff
5:45 pm
I know this is a little off topic specifically to flash, but I stream live broadcasts beautifully using the Axis camera -> Windows Media Encoder -> WMS publish point. While the output obviously isn’t flash, this configuration is highly supported, easy, well tested. Is there a specific reason your outputs need to be flash video?
10:40 am
The reason we wanted flash was so we could use a custom player with some stats in it and some advertising overlays on the video. We had given some thought to using Darwin Streaming Server for a while and abandoning flash altogether but in the end we bought new cameras.
3:38 pm
Hello,
I am trying to capture the steam from an Axis M211 camera and simply save it as a file. The codecs and encapsulation that I have been attempting work for a brief time, then the video starts to break up, then smear and be unviewable.
Any help would be appreciated. If I could just save the stream in a format that could be played back using VLC, it would solve our issue.
Thanks!
Bob
7:38 pm
Robert: You may be able to make the camera save the stream as an MP4 and upload it somewhere by http://FTP. The 207s, 210s, and P3301s that I have worked with all have this ability - would that work for you?
9:10 am
John,
Thanks for the reply. The requirements for the project require that the files are saved in hour long chunks (no motion detect).
I thought that this might be easier accomplished using VLC, as I can get the stream for an hour, exit, then start again.
8:11 pm
It may be easier to do with VLC. If you’re having problems with smear see if there’s a setting for keyframe interval - it’ll increase your bandwidth slightly, but it should reduce the smear.
Are you able to view the stream without saving it?
5:17 pm
Is there a H264 Ip camera avaible so we can directly read the stream in Flash player without having to transcode ?
and what is the difference between MPEG4-ES and H264 ?
Regards
11:01 pm
There are h.264 IP cameras available. I’ve used the Axis P3301, and Axis is coming out with some cheaper ones soon as well, if they haven’t already. You will still want a streaming server of some sort, I’ve tested successfully with Wowza Media Server Pro.
I’m not sure exactly the difference between MPEG4-ES and h.264, but I know that Flash can play h.264, but it can not play MPEG4-ES
8:45 am
Ok… But are there RTMP protocol builtin cameras ?
The purpose is to have view real time stream inside Flash player WITHUT having a PC (server) switched on: Flash is connected directlty to the webcam
Regards
9:43 am
Sorry, there isn’t RTMP built into any camera that I know of - only RTSP, which flash won’t play directly. This means that somewhere there has to be a computer switched on.
The good news is that it doesn’t have to be your computer. In fact, it will work better if it’s a webserver somewhere. If you don’t have a webserver you want to use, (and $1k to spend on the media server software), there’s an easier solution: SimpleCDN offers decently-priced pay-as-you-go streaming, and they should be able to handle streaming from a camera with h.264 built in out to a flash player. They’ve been doing a lot of testing recently, and I believe bought some cameras for testing. Their service is probably the cheapest way to get started that I’ve found.
2:18 am
The quality of CDN’s streaming looks pretty good.
I too would like to try and stream images from an IP-cam. I didn’t know Flash was an option, but the problem comes from the h.264 conversion.
I’ve got access to quite a few different cams but all of them are still on the MPEG-4 technology. I was hoping that MJPEG would solve some of these issues (and works with Safari as well) but I guess not.
Good info anyway and I’ll try and keep in touch with what you guys are talking about. cheers!
7:48 am
Yeah, live transcoding is very tricky - and processor intensive if you don’t have a hardware-based transcoder.
You could stream as a non-h.264 MPEG-4 stream, people would still be able to watch the stream using Quicktime or maybe Windows Media Player. MJPEG sounds good, except it uses a massive amount of bandwidth compared to true video formats.
4:57 am
We’re working with similar (vlc 0.8.6d and wowza 1.5.3) scenario and we experimented the same situation, only in some cases. With your soloution (fifo) the problem continues appearing. Only if we save to a file the stream and transcoding after this file with VLC the issue doesn’t appear, but it’s unviable.
We tested new 0.9.6 version of VLC and it seems to work greatfully, but with one of our input (hw encoder accessed via rtsp), this version of VLC lost the connection after 30 seconds.
Can you describe with detail how to set Darwin between the rtsp source and the VLC transcoding instance? It will solve the ‘drop’ issue with old version of VLC, and this new issue with the new version.
Regards
12:21 pm
If you only need to use Darwin for one stream it’s pretty easy to set up. It’s been a while since I did it, but I think if you only need to run a single stream through DSS it’s pretty easy. Check out this blog post about it.