Using VLC to transcode an Axis Camera’s video stream, and stream it out again
Recently, I’ve been working on streaming live video from IP cameras to a Flash player on a website. It sounds simple, but not so much, (if you’re in a hurry, skip to the solution).
The problem is that most IP cameras are not made for streaming live, full-motion, events to the web. They’re made for surveillance at 11 FPS in Motion JPEG format, (that’s just a bunch of JPEG images coming one after another). This is obviously not ideal from a bandwidth perspective at all. The cameras that our project uses are Axis 207 and 210 cameras. These cameras are capable of streaming MPEG-4 video, and when you look at the video in a web browser it looks pretty good. When I first saw that, I was excited – I could just stream into a Flash media server, (We’re using Wowza Media Server Pro at the moment), and that would send everything off to the player, right? Wrong.
It turns out that most, if not all, Axis products stream in MPEG4-ES, which flash cannot understand, and therefore our server rejects. I had to find a way to change MP4V-ES to h.264.
The obvious solution to transcoding the video stream from MP4V-ES to h.264 is VLC. While it looks like a media player that can handle a lot of formats, under the surface lies a powerful, command-line based, transcoding and streaming program. I discovered that, in theory, I should be able to issue one command to VLC and have it receive the MPEG4-ES stream from the camera, transcode it to h.264, and stream it to the Wowza, which would handle the rest.
I started by opening a file, transcoding it to h.264, and streaming it to Wowza:
vlc -vvv /path/to/file/Extremists.m4v --sout "#transcode{venc=x264,vcodec=x264,vb=500,scale=1,acodec=mp4a,ab=32,channels=2,samplerate=22100}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///path/to/wowza/content/myStream.sdp}"
It was a little rough, as the testing server doesn’t have a lot of processing power, but it worked. Awesome. Now I just have to hook it up to the stream from the camera, right?
I added the camera as the source and removed the sound:
vlc -vvv rtsp://camera-ip-address:554/mpeg4/1/media.amp --no-sout-audio --sout "#transcode{venc=x264,vcodec=x264,vb=200,scale=1}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///absolute/path/to/wowza/content/myStream.sdp}"
With Wowza already running on the server, I typed that in to terminal, and it started to look good. Then PAF! I get this error:
[00000385] access_output_udp private debug: mmh, hole (147841635484807 > 2s) -> drop
VLC seems to think that a frame, or some piece of information has been delayed 147841635484807 seconds. I highly doubt that, but VLC is convinced. Try as I might, I was not able to get VLC to realize that the frame, (or whatever bit of info), was simply missing a timestamp or something.
So, I figured I would debug just the connection to the camera. I opened a VNC session, and ran this:
vlc -vvv rtsp://camera-ip-address:554/mpeg4/1/media.amp
To see if I could view the video. I could, so I tried saving it to a file:
vlc -vvv rtsp://CAMERA-IP-ADDRESS:554/mpeg4/media.amp --no-drop-late-frames --no-sout-audio --sout "#std{mux=ts,access=file,dst=/tmp/camstream.m4v}"
This worked also. It appears that the problem only occurs when I am trying to open, transcode, and send out the stream all at once.
I realized, if I can transcode from a file and stream, and if I can capture a stream and save it to a file, I should be able to do both at the same time. It works! The steps are, make the pipe:
mkpipe /tmp/vpipe
Then capture the stream and save it to the pipe:
vlc -vvv rtsp://CAMERA-IP-ADDRESS:554/mpeg4/media.amp --no-drop-late-frames --no-sout-audio --sout "#std{mux=ts,access=file,dst=/tmp/vpipe}"
And finally read from the pipe, transcode, and stream to the flash server:
vlc -vvv /tmp/vpipe --no-sout-audio --sout "#transcode{venc=x264,vcodec=x264,vb=500,scale=1}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///path/to/wowza/content/myStream.sdp}"
Shazam!
There are a couple of caveats, though:
- This sucks major processing horsepower. Make sure you’ve got enough
- To make a stream work this way you have to have 2 copies of VLC running, plus your flash server. That’s a lot of parts that could have a problem
Because of these caveats, I am still looking for an alternate solution, and may stream MPEG-4 to browsers until Axis has its h.264 products ready later this year.
Yesterday, I did find a second possible solution. Instead of using the first instance of VLC to capture the stream, it is sometimes possible to use Darwin Streaming Server to capture the stream, then use 1 instance of VLC to transcode it. This seems to use much less processing power, but is not 100% reliable either.
4:24 pm
Whew, I am tired just reading your process. We are trying to take a linksys camera and push it through to Mogulus. Mogulus can not take the IP and I was wondering if I could use that IP camera routed through Adobe Flash encoder 2.5.
Here is the link to the camera.
http://XXX.XXX.XXX/img/main.cgi?next_file=main.htm
Here is an example of Mogulus on our site streaming live from Beijing.
http://www.kare11.com/news/olympics/beijing/default.aspx
Jeff Kraker
kare11.com
5:13 pm
That could be rough. When we were first exploring our options we played around with Mogulus a bunch, but we couldn’t find a way to stream in from an IP camera, (I actually E-mailed them and they said that they didn’t support it at the time).
From what I’ve seen, and I tried both, it’s not possible to hook Adobe Flash Encoder, or the On2 encoder, up to an IP camera. It would be great if that was possible.
If you can get your camera’s stream to play in Quicktime or something, you might try, (and this is pretty hack-ish), is play the video in Quicktime on a computer, and then capture the screen of that computer and send it to Mogulus. I’m not sure about audio, or quality, but it’ll get a picture there.
Another option would be to transcode the video somehow, then stream it to Mogulus, but I think you have to stream to Mogulus in Adobe’s proprietary RTMP format, and I don’t think VLC can stream out in RTMP, so you’d need something else in the middle, like Wowza that I’m using.
5:23 pm
Thanks, we are looking at an I-frame of the live video and moving on to fight again another day.
Jeff
5:45 pm
I know this is a little off topic specifically to flash, but I stream live broadcasts beautifully using the Axis camera -> Windows Media Encoder -> WMS publish point. While the output obviously isn’t flash, this configuration is highly supported, easy, well tested. Is there a specific reason your outputs need to be flash video?
10:40 am
The reason we wanted flash was so we could use a custom player with some stats in it and some advertising overlays on the video. We had given some thought to using Darwin Streaming Server for a while and abandoning flash altogether but in the end we bought new cameras.
3:38 pm
Hello,
I am trying to capture the steam from an Axis M211 camera and simply save it as a file. The codecs and encapsulation that I have been attempting work for a brief time, then the video starts to break up, then smear and be unviewable.
Any help would be appreciated. If I could just save the stream in a format that could be played back using VLC, it would solve our issue.
Thanks!
Bob
7:38 pm
Robert: You may be able to make the camera save the stream as an MP4 and upload it somewhere by FTP. The 207s, 210s, and P3301s that I have worked with all have this ability – would that work for you?
9:10 am
John,
Thanks for the reply. The requirements for the project require that the files are saved in hour long chunks (no motion detect).
I thought that this might be easier accomplished using VLC, as I can get the stream for an hour, exit, then start again.
8:11 pm
It may be easier to do with VLC. If you’re having problems with smear see if there’s a setting for keyframe interval – it’ll increase your bandwidth slightly, but it should reduce the smear.
Are you able to view the stream without saving it?
5:17 pm
Is there a H264 Ip camera avaible so we can directly read the stream in Flash player without having to transcode ?
and what is the difference between MPEG4-ES and H264 ?
Regards
11:01 pm
There are h.264 IP cameras available. I’ve used the Axis P3301, and Axis is coming out with some cheaper ones soon as well, if they haven’t already. You will still want a streaming server of some sort, I’ve tested successfully with Wowza Media Server Pro.
I’m not sure exactly the difference between MPEG4-ES and h.264, but I know that Flash can play h.264, but it can not play MPEG4-ES
8:45 am
Ok… But are there RTMP protocol builtin cameras ?
The purpose is to have view real time stream inside Flash player WITHUT having a PC (server) switched on: Flash is connected directlty to the webcam
Regards
9:43 am
Sorry, there isn’t RTMP built into any camera that I know of – only RTSP, which flash won’t play directly. This means that somewhere there has to be a computer switched on.
The good news is that it doesn’t have to be your computer. In fact, it will work better if it’s a webserver somewhere. If you don’t have a webserver you want to use, (and $1k to spend on the media server software), there’s an easier solution: SimpleCDN offers decently-priced pay-as-you-go streaming, and they should be able to handle streaming from a camera with h.264 built in out to a flash player. They’ve been doing a lot of testing recently, and I believe bought some cameras for testing. Their service is probably the cheapest way to get started that I’ve found.
2:18 am
The quality of CDN’s streaming looks pretty good.
I too would like to try and stream images from an IP-cam. I didn’t know Flash was an option, but the problem comes from the h.264 conversion.
I’ve got access to quite a few different cams but all of them are still on the MPEG-4 technology. I was hoping that MJPEG would solve some of these issues (and works with Safari as well) but I guess not.
Good info anyway and I’ll try and keep in touch with what you guys are talking about. cheers!
7:48 am
Yeah, live transcoding is very tricky – and processor intensive if you don’t have a hardware-based transcoder.
You could stream as a non-h.264 MPEG-4 stream, people would still be able to watch the stream using Quicktime or maybe Windows Media Player. MJPEG sounds good, except it uses a massive amount of bandwidth compared to true video formats.
4:57 am
We’re working with similar (vlc 0.8.6d and wowza 1.5.3) scenario and we experimented the same situation, only in some cases. With your soloution (fifo) the problem continues appearing. Only if we save to a file the stream and transcoding after this file with VLC the issue doesn’t appear, but it’s unviable.
We tested new 0.9.6 version of VLC and it seems to work greatfully, but with one of our input (hw encoder accessed via rtsp), this version of VLC lost the connection after 30 seconds.
Can you describe with detail how to set Darwin between the rtsp source and the VLC transcoding instance? It will solve the ‘drop’ issue with old version of VLC, and this new issue with the new version.
Regards
12:21 pm
If you only need to use Darwin for one stream it’s pretty easy to set up. It’s been a while since I did it, but I think if you only need to run a single stream through DSS it’s pretty easy. Check out this blog post about it.
6:04 pm
[...] John Beales: Using VLC to transcode an Axis Camera’s video stream, and stream it out again Recently, I’ve been working on streaming live video from IP cameras to a Flash player on a website. It sounds simple, but not so much, (if you’re in a hurry, skip to the solution). (tags: vlc camera transcode streaming) [...]
8:50 pm
[...] public links >> h264 Using VLC to transcode an Axis Camera’s video stream, and stream … First saved by gregp01 | 20 days ago [BSS] Natsume Yuujinchou – 08 (H.264) First saved by [...]
7:58 pm
Has anyone seen a way to serve a flash player directly from a h.264 camera? I think this would be the best solution so you can run just the IP camera and not a server.
Ideally serving the h.264 streams directly from the camera would require some sort of RTMP server on the camera. RTMP over HTTP would be ideal.
I have a H.264 camera running linux that I’m trying to hack this on.
9:32 pm
You’re right – the camera would have to have a built-in RTMP server. I’m not aware of any camera that has one at the moment, but I don’t know all of the cameras on the market. The problem with working this way is that many cameras don’t have a high-bandwidth connection to the internet, but in situations where they do it may work well. If you’re able to get it working let me know for sure!
12:07 am
I was thinking about the RTMP support so you can use a flash viewer, not necessarily for high bandwidth. Alternatives are to use Active-x (IE only) or require quicktime. I think flash is more universal. I agree that for large scale viewing a fat server would be needed, but for one or two viewers something small scale would be all that is needed.
I’m going to look into it to see if I can hack something up, I haven’t quite grasped all the encapsulation of H.264 inside RTMP yet, and ultimately I’d like to use RTMPT as running multiple ports on a device make it hard to share via port forwarding or advanced technologies like Yoics in a flexible manner.
6:43 am
Hi John.
Now that AXIS are selling h264 cameras, has all of this topic been resolved – ie an AXIS camera can generate the right type of stream for Flash Media Server to handle without any modification?
Thanks for your article.
Paul C.
8:07 am
An h.264 capable camera from Axis can generate a stream that Wowza Media Server can handle without any modification, and I would assume that the same goes for FMS, I just haven’t tried it.
At this point, if buying cameras for web streaming, I recommend buying an h.264-capable camera. The cost involved in transcoding will cost more than the price difference in the cameras.
If there’s a large inventory of older cameras, especially if the replacement cost is high, this may still apply, but it is probably in the buyer’s best interest to try to find an economical hardware-based transcoder. It will most likely have a lower total cost of ownership and have fewer problems.
4:47 am
I just follwed what you did upto the part where you got the error about “147841635484807 seconds”, but didn’t get an error and every thing worked perfectly.
I am running a Axis 241S video server, this video server doesn’t have any support for h.264, but thanks to your info it’s working through Wowza.
6:10 am
That’s excellent news. Maybe the Axis 241S makes a better stream than the old 207/210 cameras. What version of VLC are you using?
9:46 am
I have logitech quickcam notebook pro camera and i use vlc-1.0.0-rc3 in windows XP. I can stream and transcode it using H.264. But I can do it using Dirac codec. How do you think about it??!!Thank for ur suggestion
10:04 am
I am not familiar with the Dirac codec, I’ll look into it, thanks.
Most of the problems seem to stem from the fact that the camera is remote, and we have to use an RTSP stream as input, but if a codec change can help things, and if that codec is still flash-compatible, that would be great news.
9:47 am
John,
What did you do to get the axis stream to be captured by the Darwin Streaming Server? Will Darwin stream it directly and if so will it support multiple cameras?
Any help is appreciated,
Thanks,
Robert.
2:42 pm
It’s been a while, but I think that I copied the .sdp file from the camera, edited a couple of lines, and gave that to Darwin. Check out this thread in the Wowza forums, and the web page that is linked in the thread. They deal with getting Darwin to stream from some Axis cameras.
11:14 am
Hello John,
I use a different approach on feeding Wowza with the camera stream. With FFMPEG, you can feed Wowza just like with VLC, but it’s a lot less CPU intensive, you have a lot more control and reliability, and you just need to use one command line.
This is the current command line I use for this:
ffmpeg -er 4 -r 15 -s 352×240 -f mjpeg -an -i ‘http://192.168.xx.xx/axis-cgi/mjpg/video.cgi?resolution=352×240&compression=50&fps=15′ -vcodec libx264 -vpre fastfirstpass -vpre baseline -b 24k -bt 32k -threads 0 -f rtp ‘rtp://192.168.xx.xx:5000/’
Even though, it still needs a lot of testing, it works way better than VLC in our preliminary testing phases.
Kudos from Mexico,
José Antonio
3:41 pm
That sounds great. When I did this I tried with ffmpeg, but wasn’t able to get it working, however, it seems like you’re able to, which is awesome.
Hopefully this will mean that more people can transcode affordably instead of having to upgrade their cameras.
7:31 am
I wonder if the ffmpeg solution posted above also works for mpeg 4 streams. It seems that the only way the cameras will send mpeg4 is through rtsp and ffmpegs rtsp implementation has some problems.
Anyone got ffmpeg live transcoding to work with mpeg4 and not just mjpeg?
Özgür
1:50 pm
You should also check out this solution. Is basically running crtmpserver on the camera itself. No 3rd party software needed
http://groups.google.com/group/c-rtmp-server/browse_thread/thread/f5de18f4e8b5ea5f
1:23 pm
Interesting – I’m not 100% clear, would I have to buy a special camera to make this work, or is it software that I can install on an existing camera?
5:53 am
You can compile it for axis camera. But if you don’t want to do that, you can also pull the RTSP stream from the camera with crtmpserver. And serve it via RTMP or even publish that stream further to another RTMP server
Cheers,
Andrei
1:30 pm
Cool, that sounds like another good option – and much cheaper than some of the commercial options out there