Category: Hints, Techniques & More

Getting MacPorts to work Snow Leopard

Impatient? Skip to the process and go right to the solution.

I’m writing this using my shiny new OS X 10.6 Snow Leopard installation. So far it’s pretty good. I haven’t tried any heavy lifting so I don’t know if it’s really that much faster.

This is the first in a series of posts on how to get things working in Snow Leopard that got eaten during the installation. First up: MacPorts.

I ran into some issues with MySQL, (more on that later), and since my MySQL install is a MacPort I thought I would do an update to make sure things were working properly. However, this is what happened when I typed sudo port list installed:

dlopen(/opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib, 10): no suitable image found. Did find:
/opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib: no matching architecture in universal wrapper
while executing
"load /opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib"
("package ifneeded Pextlib 1.0" script)
invoked from within
"package require Pextlib 1.0"
(file "/opt/local/bin/port" line 40)

So, I tried sudo port selfupdate and ran into the same problem. Whatever port command I tried I got the same response, clearly I needed to take some sort of action. After a quick look at the MacPorts website I figured I should update my Xcode Tools to the Snow Leopard version, so I did that, (there’s an installer in the Optional Installs folder on the Snow Leopard disk) and tried sudo port list installed again, and got the same response as before.

Some quick Googling & reading resulted in no quick answer, so I figured that since I don’t really have anything irreplaceable in my MySQL databases I would download the Snow Leopard DMG from the MacPorts download page, install, and see what happened. That’s the solution! Just download the Snow Leopard DMG and re-install. All of your ports will still be there, and mine seem to be working so far.

PHP’s mysql_connect() Reuses Connections by Default

As I mentioned yesterday, I’m doing some work in WordPress right now, and a few minutes ago I tweeted that my custom code is messing with WP’s wp_get_archives() and wp_list_categories() functions, well, I found the problem.

I am including the 4RoadService.com header & utilities files in my WordPress theme, and I am using the same user here on my test server for both the main 4RoadService.com database and the WordPress database. It turns out that when the 4RoadService.com database connection was initialized, since it uses the same connection info as the WordPress database, the existing connection was just reused, (this behaviour, by the way, is well described in the PHP documentation), then when the 4RoadService.com connection was told to use the main 4RoadService database it did, thus switching our one and only connection away from the WordPress database, and making WordPress think that there were no posts on the blog.

Fortunately, there is a quick workaround, just add one more attribute to the mysql_connect() function so it looks like this:

$dblink = mysql_connect($host, $user, $pass, true);

This way a new connection is established, and the WordPress connection is left alone.

I am left wondering why, in the loop, WordPress was able to see my posts, perhaps it establishes a second database connection in there. However, I’m not going to spend the afternoon poking through the guts of WordPress.

Endpoints: A little secret for URL manipulation in WordPress

Today I’ve been setting up WordPress as the News section of a website which loads its pages via AJAX requests whenever possible, but falls back on normal HTTP requests when the AJAX loads are not possible.

The when the AJAX requests are initiated from Javascript, /outputxml/ is added to the end of the URL. This gets translated, with some mod_rewrite magic, to a $_GET parameter called output. /outputxhtml is also possible but since that’s the default it doesn’t get used very much.

After, (mostly), building the WordPress theme I started testing, and as I expected I ran into some problems when /outputxml/ was added to the end of the WordPress URLs. I got 404 errors, which makes total sense. I thought I could get around this by simply doing a little extra mod_rewrite magic, however, it seems there’s not way to simply replace /outputxml somewhere in a url with an empty string using mod_rewrite alone. After some time, I stumbled upon an underdocumented WordPress function: WP_Rewrite::add_endpoint and its friend, add_rewrite_endpoint. These functions make it so that WordPress recognizes /category/post-name/trackback, and /category/post-name/outputxml. Excellent!

I just had to create a plugin, make sure that WordPress wouldn’t kill my $_GET[‘output’] variable, add 1 line to my .htaccess and I was good to go.

References:

And this is what my plugin looks like, (for educational purposes only. I am not distributing it):

function fourRS_outputxml_activate() {
    global $wp_rewrite;
    add_rewrite_endpoint('outputxml',array(EP_PERMALINK, EP_PAGES));
    add_rewrite_endpoint('outputxml',EP_ALL);       

    $wp_rewrite->flush_rules();
}
register_activation_hook( __FILE__, 'fourRS_outputxml_activate');


function fourRS_outputxml_deactivate() {
    global $wp_rewrite;
    $wp_rewrite->flush_rules();
}

register_deactivation_hook( __FILE__, 'fourRS_outputxml_deactivate');


/* Makes it so WP doesn't eat my nice $_GET variable */
function fourRS_parameter_queryvars( $qvars )
{
    $qvars[] = 'output';
    return $qvars;
}
add_filter('query_vars', 'fourRS_parameter_queryvars' );

Edit (August 25, 2009): Changed the attrbutes in the add_rewrite_endpoint() function.

Using Microsoft’s VPC Images with VMWare Fusion

We all know the sinking feeling when we have to test a website in Internet Explorer. You built the site, it looks beautiful in Firefox and maybe Safari too, but now you have to open up Internet Explorer 6, 7, and 8, and make sure it plays nice with all three of them. Even better, Microsoft has made it so that you can only have one version of IE installed on a computer at a time. True, you can use things like Multiple IEs or other similar products, but they never play quite right.

Fortunately, Microsoft has supplied us with Virtual PC images of Windows with Internet Explorer installed. Unfortunately, Virtual PC is a Windows-only program so you need a PC to run them on. Or do you?

You don’t! If you’re using OS X you can use VMWare Fusion to run those Microsoft VPC images, after a little tweaking. It is much easier if you have a copy of Windows available to you during the install process, (that’s how I did it), but I don’t believe this is an absolute necessity. Here’s how it works:

Basically, you need to download the VPC images, extract them, and convert them into VMWare Fusion virtual machines. It sounds trickier than it is.

First, download and extract the VPC images. If you can use Windows to do this it’s easy, (the images have self-extractors), if not try p7zip, (see instructions in this forum thread).

[edit: July 15, 2009]: Then, somehow, you have to convert your VHD files into VMC files. The easiest way to do this is to use Microsoft’s VPC to make a new virtual machine from the VHD files, but you do need windows to do that. You will be converting these .VMC files into VMWare native virtual machines.
[/edit]

To convert the VPC images to something else, use VMWare vCenter Converter. It’s a stand-alone program for Windows or Linux that easily converts VPC images to VMWare Fusion virtual machines, as well as several other formats. You can even choose between Fusion 1.x and Fusion 2.x. It will even install the VMWare tools pagkage for you. I did the conversion under Windows, but there’s probably a way to get the linux version to run under OS X, at least hopefully.

Once the conversion is complete, fire up OS X & VMWare Fusion and open your new Virtual PC image. There are some things that run on the first startup of each machine, give it a few minutes then hit cancel on all of the “Please insert the XP SP3 CD” messages that remain, it doesn’t seem to hurt Windows. I think it’s looking for a battery driver in my case, (maybe I should try to install the Bootcamp battery driver?).

That’s it, enjoy testing. I am able to run, slowly, all 3 IE versions with the Windows XP images, and my computer isn’t as slow as when I run only my Bootcamp Vista install under VMWare Fusion. I’m thrilled to have these 3 new debugging tools at my disposal.

Using VLC to transcode an Axis Camera’s video stream, and stream it out again

Recently, I’ve been working on streaming live video from IP cameras to a Flash player on a website.  It sounds simple, but not so much, (if you’re in a hurry, skip to the solution).

The problem is that most IP cameras are not made for streaming live, full-motion, events to the web.  They’re made for surveillance at 11 FPS in Motion JPEG format, (that’s just a bunch of JPEG images coming one after another).  This is obviously not ideal from a bandwidth perspective at all.  The cameras that our project uses are Axis 207 and 210 cameras.  These cameras are capable of streaming MPEG-4 video, and when you look at the video in a web browser it looks pretty good.  When I first saw that, I was excited – I could just stream into a Flash media server, (We’re using Wowza Media Server Pro at the moment), and that would send everything off to the player, right?  Wrong.

It turns out that most, if not all, Axis products stream in MPEG4-ES, which flash cannot understand, and therefore our server rejects.  I had to find a way to change MP4V-ES to h.264.

The obvious solution to transcoding the video stream from MP4V-ES to h.264 is VLC.  While it looks like a media player that can handle a lot of formats, under the surface lies a powerful, command-line based, transcoding and streaming program.  I discovered that, in theory, I should be able to issue one command to VLC and have it receive the MPEG4-ES stream from the camera, transcode it to h.264, and stream it to the Wowza, which would handle the rest.

I started by opening a file, transcoding it to h.264, and streaming it to Wowza:

vlc -vvv /path/to/file/Extremists.m4v --sout "#transcode{venc=x264,vcodec=x264,vb=500,scale=1,acodec=mp4a,ab=32,channels=2,samplerate=22100}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///path/to/wowza/content/myStream.sdp}"

It was a little rough, as the testing server doesn’t have a lot of processing power, but it worked.  Awesome.  Now I just have to hook it up to the stream from the camera, right?

I added the camera as the source and removed the sound:

vlc -vvv rtsp://camera-ip-address:554/mpeg4/1/media.amp --no-sout-audio --sout "#transcode{venc=x264,vcodec=x264,vb=200,scale=1}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///absolute/path/to/wowza/content/myStream.sdp}"

With Wowza already running on the server, I typed that in to terminal, and it started to look good.  Then PAF! I get this error:

[00000385] access_output_udp private debug: mmh, hole (147841635484807 > 2s) -> drop

VLC seems to think that a frame, or some piece of information has been delayed 147841635484807 seconds.  I highly doubt that, but VLC is convinced.  Try as I might, I was not able to get VLC to realize that the frame, (or whatever bit of info), was simply missing a timestamp or something.

So, I figured I would debug just the connection to the camera.  I opened a VNC session, and ran this:

vlc -vvv rtsp://camera-ip-address:554/mpeg4/1/media.amp

To see if I could view the video.  I could, so I tried saving it to a file:

vlc -vvv rtsp://CAMERA-IP-ADDRESS:554/mpeg4/media.amp --no-drop-late-frames --no-sout-audio --sout "#std{mux=ts,access=file,dst=/tmp/camstream.m4v}"

This worked also.  It appears that the problem only occurs when I am trying to open, transcode, and send out the stream all at once.

I realized, if I can transcode from a file and stream, and if I can capture a stream and save it to a file, I should be able to do both at the same time.  It works!  The steps are, make the pipe:

mkpipe /tmp/vpipe

Then capture the stream and save it to the pipe:

vlc -vvv rtsp://CAMERA-IP-ADDRESS:554/mpeg4/media.amp --no-drop-late-frames --no-sout-audio --sout "#std{mux=ts,access=file,dst=/tmp/vpipe}"

And finally read from the pipe, transcode, and stream to the flash server:

vlc -vvv /tmp/vpipe --no-sout-audio --sout "#transcode{venc=x264,vcodec=x264,vb=500,scale=1}:rtp{dst=SERVER-IP-ADDRESS,sdp=file:///path/to/wowza/content/myStream.sdp}"

Shazam!

There are a couple of caveats, though:

  • This sucks major processing horsepower.  Make sure you’ve got enough
  • To make a stream work this way you have to have 2 copies of VLC running, plus your flash server.  That’s a lot of parts that could have a problem

Because of these caveats, I am still looking for an alternate solution, and may stream MPEG-4 to browsers until Axis has its h.264 products ready later this year.

Yesterday, I did find a second possible solution.  Instead of using the first instance of VLC to capture the stream, it is sometimes possible to use Darwin Streaming Server to capture the stream, then use 1 instance of VLC to transcode it. This seems to use much less processing power, but is not 100% reliable either.