Using the AIR 2.0 Native Process API to control MPlayer

Adobe AIR has been a big step forward in creating cross-platform applications. Unfortunately, there are still some big gotchas. For us at cloud.tv our biggest issue are the lack of media playback options. Our AIR-based media center, cloudskipper, can play any media Flash supports, which has gotten better since Adobe added H.264 support in Flash 9.0.124, but still leaves out a lot (XVID, AAC audio supporrt, etc). This is particularly true with desktop media where there dozens of different codecs.

Luckily, a few weeks ago Adobe released the beta of AIR 2.0 and with that the NativeProcess API. The NativeProcess API essentially allows you to invoke any native process similar to how you would via the command line but in your AIR app. You then communicate with that process via STDIN and STDOUT. Jeff Swartz over at the Adobe Developer Connection wrote nice introduction to this, which I highly recommend.

Can we use this new capability to do something a little more interesting than “echotest”, like say play a video HD video encoded with XVID? I’m happy to say that with some caveats the answer is yes =)

Media Player Choice

The first thing we need to play media is well… a media player. Not just any media player, a media player with a few specific requirements:

  1. Plays many different media types (this probably means it has various media codecs packaged with it)
  2. Cross platform (mac, linux, windows)
  3. Can be invoked via the command line
  4. Can receive input and send out via STDIN/STDOUT
  5. Open source (distribution limitations and the inability to make modifications are show stoppers for commercial software)

From my research the only two media players I could find that meet both these requirements are VLC and MPlayer. Both MPlayer and VLC can be invoked via the command line. They also both have modes where they work with STDIN/STDOUT, “remote control” mode (use the argument -Irc) in VLC and “slave” mode (use the argument -slave) in MPlayer. You can find about VLC’s remote control mode here and MPlayer’s slave mode here.

VLC seems to actually be able to play a wider array of media, but the choice for me came down to what will work with AIR. Unfortunately, for some reason I just couldn’t get AIR to play nice with VLC in remote control mode. AIR seems to ignore the -Irc argument and just open up VLC. Maybe, I’m missing something with VLC, but I was able to get MPlayer to open up just fine in slave mode so that made my choice moving forward pretty easy.

MPlayer Installation

The easiest way to install MPlayer on OS X is via macports. Just open up the terminal and type the command “sudo port install mplayer-devel”. Now sit back make yourself a cup of coffee, maybe have lunch or go for a run around the block while macports compiles and installs mplayer and all of it’s dependencies. Note we’re using mplayer-devel, which I assume is the “latest” development release. This has some fixes to make MPlayer work on Snow Leopard and generally seems pretty stable (you can still install “mplayer” if you’re on Leopard or below). When MPlayer is done compiling/installing do a mplayer -v to make sure everything installed correctly. Then execute “which mplayer” and not the location (it will most likely be /opt/local/bin/mplayer). Note MPlayer’s location because we’re going to need this later.

Windows seems to be a bit tricker. MPlayer HQ seems to have both windows soure and binaries though so I’d recommend heading over there. I’d love to hear some first hand reports on this.

If your on linux I think app-get should work similiarly to macports, but I haven’t tested this.

AIR 2 SDK Installaton

You’re going to need to install the AIR 2 SDK to take advantage of the NativeProcess API. Luckily, I just posted a guide on how to do this =) You can check that out here: Installing the AIR 2 SDK on OS X (this is be pretty similiar for linux and windows).

The Code

Ok, finally, to the code! I’m not going to go through everything line by line here,  but I will go over the key parts.

Update Application Descriptor
Extended desktop mode needs to be enabled for NativeProccess support.

<supportedProfiles>extendedDesktop</supportedProfiles>

Set the location of mplayer

protected const MAC_MPLAYER_PATH:String = '/opt/local/bin/mplayer';
protected const WIN_MPLAYER_PATH:String = 'c:\\path\\to\\mplayer';

(Sorry, didn’t add linux, but it should be fairly trivial)

Create the arguments

The three arguments we’re passing to MPlayer are:

  1. slave – switches MPlayer to use STDIN/STDOUT, making communication with AIR possible. You can find more information about slave mode here.
  2. quiet – makes output less verbose. this speeds things up a bit and make STDOUT parasing more reliable. Read more on MPlayer’s man page.
  3. idle – supposed to keep mplayer open when a file finishes playing, but doesn’t seem to work though.

Note, file here is the platform specific mplayer executable.

var nativeProcessStartupInfo:NativeProcessStartupInfo =  new NativeProcessStartupInfo();
nativeProcessStartupInfo.executable = file;
var args:Vector.<String> = new Vector.<String>();
args.push('-slave');
args.push('-quiet');
args.push('-idle');
args.push(mediaFile.nativePath);
nativeProcessStartupInfo.arguments = args;

Send commands to STDIN

This is pretty straight forward. I wrote a little wrapper called “cmd” to handle error checking, but really all it does is write to the process’s standard input.

public function cmd(cmdStr:String):void
{
  if(process && process.running){
    process.standardInput.writeUTFBytes(cmdStr + "\n");
  }
}

Receive commands from STDOUT and parse

MPlayer sends back information in a standard format. This means we just have to parse out certain strings to get the information we want. In this case we’re looking for the video’s position and length to update the scrubber.

process.addEventListener(ProgressEvent.STANDARD_OUTPUT_DATA, onOutputData);
protected function onOutputData(event:ProgressEvent):void
{
  var output:String = process.standardOutput.readUTFBytes(process.standardOutput.bytesAvailable);
  outputText += output;
  textReceived.verticalScrollPosition = textReceived.maxVerticalScrollPosition;
 
  var arr:Array;
  if(output.indexOf('ANS_TIME_POSITION') > -1){
    arr = output.split('ANS_TIME_POSITION=');
    vidPosition = parseFloat(arr[1]);
  }
 
  if(output.indexOf('ANS_LENGTH') > -1){
    arr = output.split('ANS_LENGTH=');
    vidLength = parseFloat(arr[1]);
  }
}

Scrubbing
Scrubbing is a little tricker because we have to worry about states (paused or playing). I also added the volume hack you see people because issuing all commands unpauses MPlayer and causes an audible pop.

protected function onScrub(event:SliderEvent):void
{
  if(!playing) {
    // cmd('volume 0 1');
    cmd('mute 1');
    cmd('pause');
  }
  cmd('seek '+ (event.currentTarget as HSlider).value + ' 2');
  cmd('get_time_pos');
  if(!playing) {
    cmd('pause');
  }
}

Download source:

AIRmplayer.mxml and AIRmplayer-app.mxml

Playing Audio

From my tests MPlayer actually performs very well with audio. Not only does it seem to play mp3, m4a, ogg, flac, it also handles large files MUCH better than flash and even iTunes. There also aren’t the same windowing issues you have with video (except for m4a’s with visual content).

Issues and Final Thoughts

This is by no means a complete solution. There are several issues that need to be addressed before I would consider it viable in production environment:

  1. windowing – communicating between the MPlayer window and the AIR app is challenge if you want to integrate video playback
  2. separate app – as stands MPlayer opens up it’s dock icon (on os x) when it opens up a new movie window
  3. distribution – it will be tricky to compile mplayer with all dependencies localized in order to distribute it as native installer

Challenges aside, I think this still serves as an interesting demo of the possibilities of what we can now do with AIR 2 and the NativeProcess API.

Resources

http://blogs.adobe.com/cantrell/archives/2009/11/air_2_public_beta_resources.html
http://www.adobe.com/devnet/air/flex/quickstart/interacting_with_native_process_02.html

  • Share/Bookmark

Installing the Adobe AIR 2 Beta SDK on OS X

I’m sure I’m not the only one who was super stoked when Adobe released the beta of AIR 2.0. For those not familiar,  Adobe added a laundry list of new toys, like support for mass storage devices, UDP sockets, and my personal favorite native code integration. You can check out the full list at Adobe Labs.

Of course, as these things always seem to go with beta stuff it wasn’t completely trivial to get things up and a running so I decided to write this post up. Note these instructions assume you’re using Flex 3 and Flex Builder 3, but there’s no reason this shouldn’t work with Flex 4 and Flash Builder 4. I haven’t tried this yet because I haven’t made the jump myself, but would love to hear from anyone who can verify this. Without further ado…

1. Download and run the AIR 2.0 runtime and SDKhttp://labs.adobe.com/downloads/air2.html

2. Make a copy of your Flex SDK – On OS X this should located in ~/Applications/Adobe\ Flex\ Builder\ 3/sdks. I called mine “3.4andAir2″. Alternatively if you haven’t updated your flex SDK in a while you can get the latest version of Flex 3 from here.

3. Overlay the AIR 2 SDK over the copied SDK – Since the AIR SDK is interwoven with  the Flex SDK this is actually a bit tricky. I first tried to do it manually by copying over each relevant file and folder. I’m not sure if I was careless, but flex builder didn’t recognize any of the AIR 2.0 API calls. Luckily, I did a bit of search and found the “ditto” command, which will do exactly what we need.  Once you’ve mounted the AIR 2 dmg, open up your terminal and you can use ditto as follows to overlay AIR 2.0:

ditto -V /Volumes/AIR\ SDK/ /Applications/Adobe\ Flex\ Builder\ 3/sdks/3.4andAir2/

This should ovewrite the AIR 1.5 SDK with AIR 2.0 SDK.

4. Add the new SDK to Flex Builder – Go to Flex Builder > Preferences > Installed Flex SDKs > Add… Point it to your 3.4andAir2 sdk folder you made.

5. Start a new project

6. Change compiler settings to point to the new SDK – Project > Properties > Flex Compiler > Use a Specific SDK

7. Change the application descriptior file - Change the line 2 to: <application xmlns=”http://ns.adobe.com/air/application/2.0beta”>

8. Verify – Add the following line of code and make sure you don’t get any build errors:

private var process:NativeProcess

That’s it! Stay tuned for my next post where I’ll dig into the NativeProcess API!

Update: In response to my blog post Christian Cantrell, AIR product manger, pointed out the key to getting the overlaying to work is not using the finder. Other methods besides ditto also work like cp -r and tar -xfj

  • Share/Bookmark

Extending Flex 3 Components

If you’ve more than dabbled with Flex I’m sure you’ve come to a point where your excitement for all the whiz-bang awesomeness you can do so easily is replaced by dread when you realize a component is doing about 90% of what you want. Sometimes 90% just aint good enough though, right? Exactly, that’s why I figured I’d write a little tutorial on how to extend a Flex component based on real-life problem I had with the Flex 3.x Tree control. So without further ado…

The Flex 3 tree control has an issue with icons. The issue is you cannot define icons via the XML dataprovider for branch nodes (a branch is defined as non-leaf node). That sentence is a bit dense so let me explain with an example.

The following xml code is the dataprovider of a Flex tree:

<menu>
  <node label="Home" icon="appIcon"/>
  <node label="Channels" icon="channelsIcon">
    <channel label="Vimeo HD Channel" thumb="http://images.vimeo.com/channelbadge-113006734.jpg"/>
    <channel label="Vimeo Staff Picks" thumb="http://images.vimeo.com/channelbadge-112957409.jpg"/>
  </node>
</menu>

The flex team was nice enough to create an iconField attribute, which allows you to define an icon for any node in your tree. In our case we’ll use “@icon” like so:

  <mx:Tree dataProvider="myXMLListCollection" iconField="@icon"/>

You’d expect the icon attribute to work on every node right? Wrong:

The second node (label=”Channels”) renders with the default icon instead of the defined channelsIcon (this is defined as an embedded image which is not shown).

I’m actually not sure if it’s an issue, a design flaw, or if there’s another way to do what I want that I’m just not getting. Either way, I did the first thing I always do when a Flex component doesn’t behave the way I want… open the hood and start hacking.

If you’ve never taken a look at the source of Flex component it’s actually pretty easy. Just navigate to whichever Flex SDK your using and you’ll find Tree.as in [your-flex-directory]/frameworks/project/framework/src/mx/controls/Tree.as. The easiest way to get to this file if you’re using Flex Builder is to cmd+click (ctrl-click on windows I believe) the component in mxml or actionscript. If everythings linked correctly Tree.as

So now you’ve got the source, what the heck do you do with this 3452 line monster? Well, this changes a bit each time, but in our case with the Tree component I decided to just look through the methods of Tree and see if i could find a promising one. Low and behold on line 1169 there’s a public method named itemToIcon. So noq f we track back a bit we can see that itemToIcon is used in initListData (line 2659). Specifically we see that on line 2666 we have this:

treeListData.icon = itemToIcon(item);

Bingo! It look like itemToIcon might just be what we’re looking for. Unfortunately, there’s no documentation for this method, but if we just take a second to think about the method name, it’s inputs and outputs, we can quickly see that itemToIcon converts an item object into an icon class. Scanning a little further down we can start to see series of if / else if / else statements that define the priority in which an icon is returned:

if (iconClass)
{
    return iconClass;
}
else if (iconFunction != null)
{
    return iconFunction(item)
}
else if (branch)
{
    return getStyle(open ? "folderOpenIcon" : "folderClosedIcon");
}
else
//let's check the item itself
{
    if (item is XML)
    {
        try
        {
            if (item[iconField].length() != 0)
               icon = String(item[iconField]);
        }
        catch(e:Error)
        {
        }
    }
    else if (item is Object)
    {
        try
        {
            if (iconField && item[iconField])
                icon = item[iconField];
            else if (item.icon)
                icon = item.icon;
        }
        catch(e:Error)
        {
        }
    }
}

Basically, itemToIcon prioritizes what icon the item will use by the following conditionals:

  1. does iconClass exist? (this has to do with the itemIcons parameter existing)
  2. is an IconFunction defined?
  3. is it a branch?
  4. does iconField exist?

It seem like case 1 should be able to handle custom branch icons, but I could not figure out how to get that to work with XML (any enlightenment on this would be appreciated). It was easy enough to figure out the logic and rearrange it though.

If you take a look at the priority of operations you should see the problem at case 3. If it finds a node that’s a branch it uses the default folder icon (either folderOpenIcon or folderClosedIcon). If you remember our example tree this is exactly our problem. Rather than the tree item rendering the default icon renders. Now if we looks at the 4th case, we see that the logic is there to use the iconField (@icon in our case), but it never gets to it on branch nodes! It just uses the default icons because of case 3.

Luckily we should be able to just create a new component that extends the Tree component and override the itemToIcon method with our new logic. Essentially all we have to do is take case 4, and make it the top priority, like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
public class IconTree extends Tree
{
override public function itemToIcon(item:Object):Class
{
    if (item == null)
    {
        return null;
    }

    var icon:*;
    var open:Boolean = isItemOpen(item);
    var branch:Boolean = isBranch(item);
    var uid:String = itemToUID(item);

    //first lets check the component
    var iconClass:Class =
            itemIcons && itemIcons[uid] ?
            itemIcons[uid][open ? "iconID2" : "iconID"] :
            null;
   
    // put precident of object and xml over
    if (item is XML){
      try{
         if(item[iconField].length() != 0){
           icon = String(item[iconField]);
          }
      } catch(e:Error) {}
    }else if (item is Object){
      try{
        if(iconField && item[iconField]){
          icon = item[iconField];
        }
        else if(item.icon){
          icon = item.icon;
        }
      }  catch(e:Error) {}
    }
   
    if(icon == null){
      if (iconClass){
        return iconClass;
      }else if (iconFunction != null){
        return iconFunction(item)
      }else if (branch){
        return getStyle(open ? "folderOpenIcon" : "folderClosedIcon");
      }
      else { /* xml object check used to bed here */ }
    }
   

    //set default leaf icon if nothing else was found
    if (icon == null)
      icon = getStyle("defaultLeafIcon");

    //convert to the correct type and class
    if (icon is Class){
        return icon;
    }
    else if (icon is String){
        iconClass = Class(systemManager.getDefinitionByName(String(icon)));
        if (iconClass)
            return iconClass;

        return document[icon];
    }
    else{
        return Class(icon);
    }

}
}

If you look at line 21 you’ll see that now we’re putting the logic to use the iconField as priority 1. All should be gravy right? Not so fast…

Attempted access of inaccessible property isBranch through a reference with static type cloudtv.components:IconTree

Unfortunately, isBranch (used on line 12) is private function in Tree, therefore we cannot use it in our custom component. We could if isBranch was protected (or public). In fact I don’t really see a reason for any method to ever be private, but dont get me started on that…

Anyway, there’s any easy, although not so elegant solution to this problem: copy & paste. If we just copy the method isBranch to our new component everything works just fine and we have exactly what we want by using our custom component (I call it IconTree). Take a look see:

That wasn’t that hard was it? Well it was kind of a pain, but at least we got the result we wanted =)

In case you find this particular case useful here is link to the entire componenent: IconTree.as

  • Share/Bookmark

Bikes, Code, and a Little More Love

It’s been a while, but I’m drawing a line in the sand. This blog has a lot updating coming to it. But before I switch to some code for a while let me leave you with a treat from the physical realm that deserves its own post in the near or maybe not so near future.

Custom Touring Bicycle BeforeCustom Touring Bicycle

What you see here is before and after photos of my custom touring bicycle. 1.5 years in the making, it’s custom all the way down to the the frame built by my favorite newlywed couple Ryan and Marie at Folk Engineered.

  • Share/Bookmark

A Change is Gonna Come

It’s been quite a bit of time since I last posted, but I just wanted to take a moment to extend a belated thanks to all of those who donated to my campaign to aid the victims of cyclone Nargis. I didn’t quite reach my goal, but we managed to raise $1,498 for Doctor’s Without Borders. Thanks again so much everyone!

As the title says though, a change is gonna come. Over the next few months I’ll be making some changes to the blog and hopefully posting more. Till then, I’ll let Lauren explain.

  • Share/Bookmark

Plight Worse for Survivors Worse Than Thought

The plight of those devastated by Cyclone Nargis seems worse that ever.  The AP released some stunning facts about the current situation in a recent article:

  • A survey of families in 291 villages showed that 55 percent have less than one day of food left and no stocks to fall back on. Some 924,000 people will need food assistance until the November rice harvest, while around 300,000 will need relief until April 2009.
  • The fishing industry, the delta’s second-most-important source of income and food, remains devastated. More than 40 percent of fishing boats and 70 percent of fishing gear were destroyed and very little has been replaced.
  • More than 360,000 children will not be able to go to elementary school in coming months because at least 2,000 schools were so badly damaged they cannot reopen anytime soon.

Another quote from the same article:

We lost everything — our house, our rice, our clothes. We were given just a little rice by a private aid group from Yangon. I don’t know where the government or foreign organizations are helping people, but not here

-Khin Maung Kyi, a 60-year-old farmer who lost six children to the killer storm (via AP)

In addition, the Daily Times reports that the UN is pleading for another $51 million to help restore the destroyed rice paddies.

Lastly, ABC News has released a stunning slideshow of the aftermath of the cyclone. Here are a few samples:

An aerial photo shows damage to rooftops of residential buildings in Yangon Devastated Rice Paddies

As the presidential politics and the Olympics dominate the headlines, there continues to be a critical need for aid in Myanmar. The lack of media coverage means organizations like MSF (Doctor’s Without Borders) who are making great progress in Myanmar have a much harder time raising money they need to continue operations. We’ve made great progress so far with nearly $1500 raised, help me make it $2000 with a donation today.

  • Share/Bookmark

MSF Delivers Results in Myanmar

Doctor’s Without Borders (MSF), the non-profit that I’m supporting with this blog series, posted their latest update on the great work they’ve done in Myanmar a few weeks ago (sorry for my lateness on this one). In addition to an in-depth report on the help MSF is providing with mental health and other medical issues in the devastated Irrawaddy Delta, MSF also cites some very impressive stats:

MSF Assistance to Cyclone Survivors To Date
Food Aid

* 5,826,156 lbs. of rice
* 1,362,320 lbs.of beans
* 202,390 ltrs. of oil
* 151,302 packages of Plumpy’nut (a nutrient-rich ready-to-eat food for malnourished children)
* 216,679 lbs.of salt
* 64,111 packets of high-energy biscuits (for patients who are malnourished or at risk of malnourishment)
* 50,159 lbs. of fish

Relief Items

* 186,611 rolls of plastic sheeting
* 183,196 mosquito nets
* 131,125 water containers
* 63,560 wool blankets
* 19,717 hygiene kits
* 10,865 bars of soap
* 2 water treatment units

Yes, you’re reading correctly, that’s over 7,000,000 pounds of rice and beans delivered to date. In comparison, the US Department of Defense delivered 10,000,000 pounds of “food and supplies” during the 2005 Tsunamai according to whitehouse.gov.

All of the work MSF has done is truly amazing, but with an estimated 2.4 million people in need of food and shelter in the wake of the disaster, there is still a great need for more resources. Thanks to everyone who’s taken the time to help. While I’ve set the bar high with a $5,000 goal, I’m proud to report that you’ve almost doubled my initial donation of $500, which is absolutely fantastic. For those who haven’t donated yet, please take the time and donate today. As little as $25 can make a real difference.

  • Share/Bookmark

60,000 Children at Risk of Malnurition says the UN

The UN just released a statement saying that it’s seeking $300 million more to support the relief efforts in Myanmar.  “60,000 children were at risk of acute malnutrition,” according to U.N. humanitarian affairs chief John Holmes. Furthermore:

.. the cyclone had destroyed 42 percent of Myanmar’s food stocks and 55 percent of families had stocks for one day or less, meaning food aid was needed in the affected areas for many months to come.

via Reuters

This is precisely why I’m supporting Doctor’s Without Borders (MSF). They are an extremely capital efficient organization, who are not only providing medical care, but also much needed food aid to Myanmar. With a donation of as little as $25 MSF can deliver 2 meals a day to 150 children. Please take the time to donate.

  • Share/Bookmark

Funds Earmarked

A few people have asked over the past couple of weeks if the funds donated to Doctors Without Borders will be earmarked specifically for the relief effort in Myanmar. I’m happy to report that I received confirmation before the holiday this weekend that all funds donated via my firstgiving page will be properly allocated for Myanmar.

Thanks so much to everyone who’ve donated so far. We’ve managed to raise $1,250. For those of you haven’t had a chance yet, now is a fansastic time. For as little as $25, Doctors Without Borders can deliver two meals a day to 150 children. Please take a minute to donate.

  • Share/Bookmark

Eyewitness Report

[...] a great number more suffered far worse conditions and had nothing left but the clothes on their backs. Some did not even have that: the storm blew them off while they clung to trees for survival as the cyclone tore away their homes and their families.

via eFoodAlert

  • Share/Bookmark

Next Page »