Category: How To

Getting MacPorts to work Snow Leopard

Impatient? Skip to the process and go right to the solution.

I’m writing this using my shiny new OS X 10.6 Snow Leopard installation. So far it’s pretty good. I haven’t tried any heavy lifting so I don’t know if it’s really that much faster.

This is the first in a series of posts on how to get things working in Snow Leopard that got eaten during the installation. First up: MacPorts.

I ran into some issues with MySQL, (more on that later), and since my MySQL install is a MacPort I thought I would do an update to make sure things were working properly. However, this is what happened when I typed sudo port list installed:

dlopen(/opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib, 10): no suitable image found. Did find:
/opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib: no matching architecture in universal wrapper
while executing
"load /opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib"
("package ifneeded Pextlib 1.0" script)
invoked from within
"package require Pextlib 1.0"
(file "/opt/local/bin/port" line 40)

So, I tried sudo port selfupdate and ran into the same problem. Whatever port command I tried I got the same response, clearly I needed to take some sort of action. After a quick look at the MacPorts website I figured I should update my Xcode Tools to the Snow Leopard version, so I did that, (there’s an installer in the Optional Installs folder on the Snow Leopard disk) and tried sudo port list installed again, and got the same response as before.

Some quick Googling & reading resulted in no quick answer, so I figured that since I don’t really have anything irreplaceable in my MySQL databases I would download the Snow Leopard DMG from the MacPorts download page, install, and see what happened. That’s the solution! Just download the Snow Leopard DMG and re-install. All of your ports will still be there, and mine seem to be working so far.

Using Microsoft’s VPC Images with VMWare Fusion

We all know the sinking feeling when we have to test a website in Internet Explorer. You built the site, it looks beautiful in Firefox and maybe Safari too, but now you have to open up Internet Explorer 6, 7, and 8, and make sure it plays nice with all three of them. Even better, Microsoft has made it so that you can only have one version of IE installed on a computer at a time. True, you can use things like Multiple IEs or other similar products, but they never play quite right.

Fortunately, Microsoft has supplied us with Virtual PC images of Windows with Internet Explorer installed. Unfortunately, Virtual PC is a Windows-only program so you need a PC to run them on. Or do you?

You don’t! If you’re using OS X you can use VMWare Fusion to run those Microsoft VPC images, after a little tweaking. It is much easier if you have a copy of Windows available to you during the install process, (that’s how I did it), but I don’t believe this is an absolute necessity. Here’s how it works:

Basically, you need to download the VPC images, extract them, and convert them into VMWare Fusion virtual machines. It sounds trickier than it is.

First, download and extract the VPC images. If you can use Windows to do this it’s easy, (the images have self-extractors), if not try p7zip, (see instructions in this forum thread).

[edit: July 15, 2009]: Then, somehow, you have to convert your VHD files into VMC files. The easiest way to do this is to use Microsoft’s VPC to make a new virtual machine from the VHD files, but you do need windows to do that. You will be converting these .VMC files into VMWare native virtual machines.
[/edit]

To convert the VPC images to something else, use VMWare vCenter Converter. It’s a stand-alone program for Windows or Linux that easily converts VPC images to VMWare Fusion virtual machines, as well as several other formats. You can even choose between Fusion 1.x and Fusion 2.x. It will even install the VMWare tools pagkage for you. I did the conversion under Windows, but there’s probably a way to get the linux version to run under OS X, at least hopefully.

Once the conversion is complete, fire up OS X & VMWare Fusion and open your new Virtual PC image. There are some things that run on the first startup of each machine, give it a few minutes then hit cancel on all of the “Please insert the XP SP3 CD” messages that remain, it doesn’t seem to hurt Windows. I think it’s looking for a battery driver in my case, (maybe I should try to install the Bootcamp battery driver?).

That’s it, enjoy testing. I am able to run, slowly, all 3 IE versions with the Windows XP images, and my computer isn’t as slow as when I run only my Bootcamp Vista install under VMWare Fusion. I’m thrilled to have these 3 new debugging tools at my disposal.

Using CSS attribute selectors to simulate legacy HTML layout

Have you ever created new clean XHTML template, applied it to a CMS with years of back content, and discovered that the years of legacy HTML looks terrible?  Because you’re using a new XHTML doctype many, if not all, of the presentational attributes in the old HTML no longer work, however, CSS can be used to fix this.

You may say that the whole point of building a new template with XHTML & CSS is so that we won’t have any of the old <p align="right"> in our code, and you are right, but in situations when there can be thousands of pages of content it is often not practical to re-code them all using CSS, that’s where the attribute selectors come in.

Here is a very basic stylesheet that will help get you started:

img[align="right"], table[align="right"] {float:right;}
img[align="left"], table[align="left"] {float:left;}
img[align="center"], table[align="center"] {display:block;margin-left:auto;margin-right:auto;}

p[align="center"] {text-align:center;}
p[align="left"] {text-align:left;}
p[align="right"] {text-align:right;}

table[border="2"], table[border="2"] td {border:1px solid #000;}
table[border="1"], table[border="1"] td {border:1px solid #000;}
table[border="0"], table[border="0"] td {border:0;}

Although it would be nice to have one master stylesheet that we could drop in to a document and automatically have the old HTML look great, I believe that stylesheet would have to be extremely large, so I recommend using a base set of styles, (like the one above), and adding any extra styles to it that may be needed, (for example, img[border="5"] {border:5px solid #000} would not be needed every time).

What about browser compatibility? I’ve tested in FireFox 3, Safari 3, Internet Explorer 7, and Internet Explorer 6, and, if I remember correctly, Opera 9.  The only browser that didn’t render the styles as I would have liked to see them was IE6, which is no surprise, but also has a rapidly diminishing market share.

If you are looking for a way to keep your legacy HTML from looking terrible that doesn’t involve recoding it all, this may be the way for you to go.

Using VLC to transcode an Axis Camera’s video stream, and stream it out again

Recently, I’ve been working on streaming live video from IP cameras to a Flash player on a website.  It sounds simple, but not so much, (if you’re in a hurry, skip to the solution).

The problem is that most IP cameras are not made for streaming live, full-motion, events to the web.  They’re made for surveillance at 11 FPS in Motion JPEG format, (that’s just a bunch of JPEG images coming one after another).  This is obviously not ideal from a bandwidth perspective at all.  The cameras that our project uses are Axis 207 and 210 cameras.  These cameras are capable of streaming MPEG-4 video, and when you look at the video in a web browser it looks pretty good.  When I first saw that, I was excited – I could just stream into a Flash media server, (We’re using Wowza Media Server Pro at the moment), and that would send everything off to the player, right?  Wrong.

It turns out that most, if not all, Axis products stream in MPEG4-ES, which flash cannot understand, and therefore our server rejects.  I had to find a way to change MP4V-ES to h.264.

The obvious solution to transcoding the video stream from MP4V-ES to h.264 is VLC.  While it looks like a media player that can handle a lot of formats, under the surface lies a powerful, command-line based, transcoding and streaming program.  I discovered that, in theory, I should be able to issue one command to VLC and have it receive the MPEG4-ES stream from the camera, transcode it to h.264, and stream it to the Wowza, which would handle the rest.

I started by opening a file, transcoding it to h.264, and streaming it to Wowza:

vlc -vvv /path/to/file/Extremists.m4v --sout "#transcode{venc=x264,vcodec=x264,vb=500,scale=1,acodec=mp4a,ab=32,channels=2,samplerate=22100}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///path/to/wowza/content/myStream.sdp}"

It was a little rough, as the testing server doesn’t have a lot of processing power, but it worked.  Awesome.  Now I just have to hook it up to the stream from the camera, right?

I added the camera as the source and removed the sound:

vlc -vvv rtsp://camera-ip-address:554/mpeg4/1/media.amp --no-sout-audio --sout "#transcode{venc=x264,vcodec=x264,vb=200,scale=1}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///absolute/path/to/wowza/content/myStream.sdp}"

With Wowza already running on the server, I typed that in to terminal, and it started to look good.  Then PAF! I get this error:

[00000385] access_output_udp private debug: mmh, hole (147841635484807 > 2s) -> drop

VLC seems to think that a frame, or some piece of information has been delayed 147841635484807 seconds.  I highly doubt that, but VLC is convinced.  Try as I might, I was not able to get VLC to realize that the frame, (or whatever bit of info), was simply missing a timestamp or something.

So, I figured I would debug just the connection to the camera.  I opened a VNC session, and ran this:

vlc -vvv rtsp://camera-ip-address:554/mpeg4/1/media.amp

To see if I could view the video.  I could, so I tried saving it to a file:

vlc -vvv rtsp://CAMERA-IP-ADDRESS:554/mpeg4/media.amp --no-drop-late-frames --no-sout-audio --sout "#std{mux=ts,access=file,dst=/tmp/camstream.m4v}"

This worked also.  It appears that the problem only occurs when I am trying to open, transcode, and send out the stream all at once.

I realized, if I can transcode from a file and stream, and if I can capture a stream and save it to a file, I should be able to do both at the same time.  It works!  The steps are, make the pipe:

mkpipe /tmp/vpipe

Then capture the stream and save it to the pipe:

vlc -vvv rtsp://CAMERA-IP-ADDRESS:554/mpeg4/media.amp --no-drop-late-frames --no-sout-audio --sout "#std{mux=ts,access=file,dst=/tmp/vpipe}"

And finally read from the pipe, transcode, and stream to the flash server:

vlc -vvv /tmp/vpipe --no-sout-audio --sout "#transcode{venc=x264,vcodec=x264,vb=500,scale=1}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///path/to/wowza/content/myStream.sdp}"

Shazam!

There are a couple of caveats, though:

  • This sucks major processing horsepower.  Make sure you’ve got enough
  • To make a stream work this way you have to have 2 copies of VLC running, plus your flash server.  That’s a lot of parts that could have a problem

Because of these caveats, I am still looking for an alternate solution, and may stream MPEG-4 to browsers until Axis has its h.264 products ready later this year.

Yesterday, I did find a second possible solution.  Instead of using the first instance of VLC to capture the stream, it is sometimes possible to use Darwin Streaming Server to capture the stream, then use 1 instance of VLC to transcode it. This seems to use much less processing power, but is not 100% reliable either.

Coding in a Vacuum: Subversion Hooks

A month ago I wrote about setting up an in-house subversion server on the cheap. After a month of using my setup it’s been pretty successful, and I’ve learned even more about subversion.

One of the things I had to do was set up a post-commit hook to publish changes to a webserver.  The site I did this for is still in development, and every commit needed to be pushed to the webserver for testing.  I did this using rsync, (or, since the SVN server is in Windows, cwRsync), and it’s a pretty basic rsync/openSSH setup, but that’s for another day.

What I am talking about today is actually creating the hook script.  What a pain in the ass.  It’s not actually hard to create the script and run it.  On Windows you just write a batch file, debug it, call it post_commit.bat, and drop it in the hooks folder of the repository you want to use.  Sounds simple, doesn’t it?  Why then, when you commit a revision, does Subversion hang, and the script not execute?  Because subversion runs hooks in a near-null environment. This means that nothing is set.  In Windows, not PATH variable.  In everything, pretty much no environment variables of any kind.  So, when you tested your script yourself it ran great because it had lots of support, but now Subversion is running it entirely alone.  Poor, lonely script.

It gets even more frustrating.  Since Subversion is running the script, there’s nothing being printed to the screen to tell you what’s going on, so debugging is like shooting bugs in the dark.

There are a couple of things you can do, however, to make your script work.  First, use an absolute, full, path for everything.  For Windows this means start with the drive letter.  For *nix this means start with a slash.  If that doesn’t work, there’s a trick that allows you to see what’s going on.  You can log the hook’s output to a file.  This way you can find out what’s going on, read the errors, and fix them.  Thanks to Dan Switzer for posting about it. His blog showed up in a Google search and ended my frustration.  In my case, it was some more environment that had to be set up for cwrsync to work properly.