erlang, adservers and the iPhone

Recently viewed a great Erlang video from the Jonathan Rentzsch organised C4 conference. Bob Ippolito, who is well known for his work helping to marry the joys of Mac with the joys of Python, gives the talk in which he provides an overview of Erlang and discusses the use of Erlang in powering his companies ad server and traffic monitoring products. The ad server is MochiAds which focuses on the Flash causal games market, embedding preroll and inter level ads into Flash content. If you’ve been to the Auckland Web Meetup in the last year then you may be familiar with it via Shaun Lee’s presentation on his Flash games portal shaunsflights.com or the presentation from Stephen Harris on Ninja Kiwi which inspired Shaun to become a Flash game developer.

The slides and code from Bob’s presentation are also available in addition to the hour long video.

I also dialed into the HD version of the Steve Jobs iPhone SDK keynote video. Having recently swapped out my old PowerBook for a bright shiny new MacBook Pro I was blown away by the ability to watch beautiful crisp HD video streamed. The content of the keynote was pretty impressive as well, though I’m not sure that all other competitors should pack up and go home quite yet;)

"In the space of an hour announcement, Apple has destroyed most of the big mobile players, Nokia, Microsoft, and RIM. I'd expect Google to abandon Android development at this point, it's now just irrelevant (if it weren't already)" - Polar Bear Farm

12 herbs & spices :: django & capistrano

Web operations have been described as the secret sauce of 2.0. It’s not an area that I deal with at work but I keep up to date with architectural decisions that impact web ops - especially around caching. I found Artur Bergman’s talk on the LiveJournal approach to architecture at Kiwi Foo both entertaining and informative. The LJ approach is being leveraged by many in both the Rails and Django community, with memcached, perbal or nginx being used as building blocks for scaling 2.0 operations. For more background on web operations/architecture check out the O’Reilly Radar Operations blog for more of Artur’s and others wisdom. O’Reilly are running a conference on the subject next year. High Scalability is another source of information on scalable web architecture.

With this in mind, when I was asked this weekend for some advice from a friend looking to put together a web play together I stressed the need for them to get the combo of a decent architecture, an ops machine in place before going crazy with feature development. Once this is in place they can get a minimal release up, then push out new functionality quickly and reliably - well at least from the web ops side of the equation. The less effort they have to focus on deployment and admin the more they can spend on developing the functionality that matters for their users.

So after dispensing this advice I thought that I should walk the talk myself and implement some of my own advice for this blog’s system. This blog was the typical ‘my first django project’. It was rapidly constructed around Christmas last year, porting existing content from the Rails based Typo system I had previously used.

I don’t make many changes to the code, arguably I don’t make that many changes/additions to the content either. However I’ve now placed the code under Subversion control and have installed Capistrano to automate admin tasks, specifically the deployment of new code onto my production server. Version control is mandatory for projects with more than one person working on them and highly desirable even if there is just one. For more information/explanation - should it be required - check out this visual guide to version control.

Simply put, Capistrano is a tool for automating tasks on one or more remote servers. It executes commands in parallel on all targeted machines, and provides a mechanism for rolling back changes across multiple machines. It is ideal for anyone doing any kind of system administration, either professionally or incidentally.

Capistrano is a Rails related tool. It’s primarily used for deployment but can be used to automate any set of tasks. And whilst it has come out of the Rails community and is written in Ruby it can be used in association with any language/web framework. Tasks to be automated are described in its own domain specific language (DSL) so there’s no requirement to learn Ruby. Capistrano is now at 2.1 and whilst there is a chapter on version 1.0 in the online Rails book there is little on 2.0, though improved 2.0 documentation can be expected. So here’s the sources I used to grok it for for deploying a non-Rails web application.

Prerequisites:

1. ssh and client certificates setup to provide password-less access to your production server from your workstation
2. knowledge of how your production environment works
3. a version control system, such as subversion setup and accessible from your production server

Timothee Peignier has written up the best description on how one can utilise Capistrano with Django. However I would qualify that by saying that you will get more, or get it quicker, if you first read up on Capistrano generally. Start with the minimal getting started doco, then read through the slides by Jamis Buck from the RailsConf. Test things out with some of the small examples. Understand how Capistrano is typically used with Rails, via the capify command that produces a Capfile and an associated config file. Find and read through the source (deploy.rb) for the Capistrano default tasks. Then read through Timothee’s post. You’ll almost certainly have to tweak the tasks to get things working for your own environment.

Unless you have a compelling reason to do otherwise I would go with the flow and use Capistrano’s method for organising releases. This approach lets you get the most from what Capistrano provides for free, with you only having to refine the tasks specific to your environment.

update:
Django added auto escaping of HTML in the version that I installed when putting Capistrano in place. I missed addressing that in the feed's templates.
Fixed now.

barcamp/creative commons

I have recently become an expert on starting and not completing posts. So I’ll bang out this mix of memes double quick.

Auckland Does Community
Firstly there is a BarCamp happening in Auckland, well Botany Downs, on December 15. Get involved.
Some of those attending can be seen at the Auckland Web Meetup - which is a fantastic forum, much kudos to John and Karl for orchestrating such positive, productive, fun evenings. Next one is Feb.

Creative Commons
I got myself down to the launch of Kiwi Creative Commons licence(s) a few weeks ago. It was interesting, the speaker talks are available online for viewing. The workshop I attended was great - meet a bunch of people doing intriguing things and we had some robust discussion around how CC would affect our collecting institutions. In some small gesture I have licensed the contents of syntho.org as NY-NC (NZ). This means you can do anything you like with the content, unless you’re going to turn a profit off it, and if you do use it - then give me credit for what you do use. As I say it’s a gesture. There hasn’t been much real CC NZ action. I believe Otago Poly are CC licensing some of their course materials. Are there other examples to cite?

The licences are are available on the main CC site at http://creativecommons.org/license/. Select NZ from the ‘Jurisdiction’ menu. I’m not sure why the Creative Commons NZ site appears to contradict the availability of the NZ licences (as of Nov 20 2007).

BTW: CC is on a fund raising drive, why not donate a monthly utility bill worth towards CC?

libarylookup:: auckland region public libraries

I’ve used Jon Udell’s LibraryLookup bookmarklet for doing lookups against the University of Auckland Library for quite some time. It’s a great tool that provides you with a quick way of seeing whether a book of interest is available in your local library. Say you’re looking at an interesting read on Amazon, click on LibraryLookup in your bookmark toolbar and a new browser window opens up, with search results from the local library, showing whether the book is available. If it is you can log in to the library and request the book, quickly mitigating the temptation of a one click purchase.

LibraryLookup works by finding an ISBN number within a web page and using that to search the library’s collection. Which is great however as I’m not enrolled at university this year I have been using Auckland City Public library to quell my urges to purchase books. After a cursory g search returned naught for the Public Library I peeked and poked the necessary url gubbins into the librarylookup code. So here are the results, bookmarklets for the Auckland region libraries. Drag the relevant one into your bookmark toolbar. If you’re having problems with this check out this screencast from Jon.

Auckland ISBN Lookup

Manakau ISBN Lookup

North Shore ISBN Lookup

Rodney ISBN Lookup

Waitakere ISBN Lookup

For those that want details

As the Auckland, Manakau, Waitakere and North Shore city libraries all use ELGAR (e Libraries for a Greater Auckland Region) there is a single base url for all libraries. The parameter searchscope defines which library’s collection is searched and the associated skin in which the results are returned within. Here are the codes for the specific libraries.

1 = ELGAR overview
2 = Auckland
3 = Auckland Kids
4 = Manakau
5 = Manakau Kids
6 = North Shore
7 = North Shore Kids
8 = Rodney
9 = Rodney Kids
10 = Waitakere
11 = Waitakere Kids

ibc conference audio freely available

Back a week, from Amsterdam & London. I didn’t attend any of conference sessions at IBC as I was fully booked with media management, video editing, video storage and traffic/scheduling vendors. However due to some forward thinking from the IBC organisers I can listen to all the sessions freely. So can you.

The pages with mp3 links can be found here. I couldn’t be asked to right-click, save and import all those files into iTunes, so I russled up a feed to do it automatically. Point your iTunes/NetNewsWire or similar at http://syntho.org/site_media/ibc07.xml Apparently its the same deal as last year, not that I knew that last year’s were available freely, so here’s a feed for 2006.

phones, video and synthetic programming

A quick Pythonic burst before I depart for IBC later this week.

Video
In my spare time I’ve been tinkering around with Final Cut Pro’s XML capabilities and FCP’s interesting Apple Event support with the goal of automating the production of some channel elements. Python’s great libraries and the community behind their production is one of its strengths. For this work I have employed Appscript from HAS to allow me to send Apple Events to FCP. For the XML processing Fredrik Lundh’s elementtree was the natural choice. I used the Python Imaging Library (PIL) for a similar project several years ago for the automated production of id boards.
Once I’ve cleaned things up I’ll publish my FCP Apple Event class as an example of using Appscript.

Phones
Python for s60 mobile platform went gold last month. I need to dust off my PC to install some Windows only software that’s required to create a developer certificate. The dev cert allows you to get at the tastier capabilities of the phone.

Synthetic Programming
Corepy is a Python package for creating applications targeted at the IBM’s PowerPC and Cell architectures.

“Synthetic programming was developed to provide a new approach to high-performance and multi-core computing using scripting languages. The design of the synthetic programming environment encourages developers to experiment with different approaches for mapping algorithms to processing resources and generating optimized code sequences.

Synthetic programming itself is a methodology for building applications that combines high-level language code with user generated machine code. The synthetic programming environment exposes the underlying processor instructions as first-class functions in the host language and provides components for building and executing instruction sequences built using the instruction interfaces. The instruction sequences, called synthetic programs, can contain any instruction available for the processor, allowing developers to create highly optimized kernels for high-performance tasks. An application can contain many synthetic programs, all of which can be executed an arbitrary number of times synchronously or asynchronously, allowing the application to make full use of data and processing resources.” - Mueller C., Synthetic Programming: User-directed Run-time Code Synthesis for High Performance Computing

So the ease of use and elegance of Python is employed to create high performance machine code, making “assembly fun again”.

This is one of those times when having an older machine has its advantages as copepy will only run on PPC Macintoshes, so time available I’ll be able to load it up on the trusty PowerBook. Probably of more interest to those in scientific programming, such bioinformatics.

mac :: printing in a windows world

Unless you use print from your Mac in a Windows environment which has a forced password change policy this post won’t be much interest. Its probably old news for most who do… but if you’re not aware of the lpadmin command and hate every time you have to change your Windows (AD) password this could be seriously useful.

I use my PowerBook in at work, which is a largely Windows based environment. In order to print I need to connect the Mac to one of the Samba (SMB) based printer queues. This can be achieved by using the web based admin tool (http://localhost:631/) and following the convention for creating the device string as follows:

smb://myaccount:mypassword@windowsserver/windowsprinter

So in a environment where:

myaccount = chris
mypassword = goldstar
windowsserver = nz_printer01
windowsprinter = nz_akl_moa

you would use the following string:

smb://chris:goldstar@nz_printer01/nz_akl_moa

Then you need to need to tweak the config to your printer driver and set up all the defaults for that printer. However when the inevitable password change occurs in the Windows environment you have to tweak the device settings in the web based system to take account that your password has changed. But then you have to redo all the associated driver and default settings. Now there is probably a way to avoid this but I’m now aware of it. The web based admin is now become a royal pain for me, due to factors I won’t go into.

I have always known of the command line …err.. commands that lie behind the web based admin. But until now have not had the suitable motivation to dig into their effective use. Typically the solution is simple and only took a ten minute scan of the associated man pages.

So if you already have a SMB printer setup and your password has changed all you need to do is issue the following command within the shell (Terminal):

lpadmin -p yourprintername -v smb://myaccount:mypassword@windowsserver/windowsprinter

So if using the example above the password was changed to lucky and you had named the printer MOA the command would be:

lpadmin -p MOA -v smb://chris:lucky@nz_printer01/nz_akl_moa

If you don’t know what printer names you have installed you can use lpstat -p or simply view them through the Print & Fax in System Preferences.... For a look at adding SMB printers using AppleScript check out the Samba Assistant script over at MacEnterprise.org

Apols for the justification on the lpadmin commands - the command is a one liner and those huge gaps just represent a single space each. Must tweak the css style for code!

nab roundup & disney explains copyright

Couple of quick media link outs on a Monday morning…

media tech geekery
Alex Lindsay from TWIT/Pixel Corps provides a great walk through of NAB. About twenty minutes long.
The rest of Gear Media Tech is worth subscribing to if you find this interesting.

disney characters explain copyright
Personally I find the edits a little clunky but as a concept it’s 10 out of 10. Professor Eric Faden of Bucknell University has created an overview of (US) copyright using excerpts from Disney animations.
Stream version
YouTube version

“they’re applauding because i’m a pissant”

Freeview

Freeview has been around for a month on my non-Freeview tuner. I don’t get to use the MHEG-5 based EPG but contrary to some press reports I still have an EPG. Given that’s how I select what to record on the Topfield PVR I would not be terribly happy if it disappeared.
I am happy to have TV3 and C4 available to record. The Daily Show now gets selected in my weekly time-shifting trawl through the EPG, where I select the programmes that week that will get recorded and which we will watch…whenever and without commercials. After a hard days work a bit of Jon Stewart with dinner is nice way to start the evening. The quote above is from last night’s show, or rather the episode that I watched last night, but was probably recorded the night before.

Final Cut Pro & MXF

Talking of work I never really concluded the discussion of my investigation into mixing FCP and MXF within a workflow. FCP 6.0 does not provide any MXF capabilities period. Compressor 3, bundled as part of FCP Studio, can use an optional Episode Pro plugin from Telestream that provides the ability to rewrap QuickTime <=> MXF. I believe that Omneon’s ProExchange also provides this functionality.
In the scenario that I was looking into we’ll probably stick with QuickTime from ingest through production and play-out but will rewrap into MXF for archiving.

Python and the Nokia E65

Don’t have much time for happy hacking the Nokia E65. I am a bit disappointed that the 3rd edition S60 security model is so restrictive that I couldn’t get some of the API to produce any results in my initial peeking. location.gsm_location() produces nada without ReadDeviceData, ReadUserData, and Location privileges. It seems that in order to get these privileges on the phone you own you need to acquire a developer certificate. Hmmmmmm.
I’m not a big user/fan of Twitter but I was planning to do a proof of concept script that would update Twitter as one moved between cell locations.

Lastly

Jim Barksdale argues for the need to more effectively archive our bits.
And given Microsoft’s recent patent threats this video of Columbia Law Professor, Eben Moglen, is worth five minutes of your time.

Post NAB Catchup

Despite ambitious plans to blog everyday about NAB its now a week after my return from Vegas. So here’s a quick wrap….

I spent way too much time in meetings at NAB. To do justice to show like NAB you really need at least four people, allowing each to attend the meetings relevant to them and allowing each to have time to free range the floor. It was only on the last day that I managed to get a limited walk of the floor. Even then it was highly directed as I paced the halls looking up the stands others had recommended to me.

The world of broadcast management applications isn’t booming. Its a small and increasingly smaller market so that’s no surprise. I was surprised to also see a lack of movement and excitement around media asset management (MAM). Obviously Proximity reborn as Final Cut Server provided some noise but otherwise it seemed very much business as usual with the existing players and no interesting new entrants.

The exception to this was the MXF Mastering Format demo given by the Advanced Media Workflow Association. Unfortunately I can’t link to anything substantial on it as the materials are available to members only. But it represents a new approach to the way in which media is managed, a more distributed method, and a concrete example of how MXF can be used to solve broadcasters media management problems. It is result of two years work by Omneon, EMC, Marquis Broadcast, Metaglue, OpenCube, Pro-Bel, Quantum, Snell & Wilcox, Softel and TMD , who focused their efforts on the multilingual audio and captioning requirements of Turner Broadcasting in London.

On the new distribution front Microsoft and Adobe battled it out with their respective announcements on Silverlight and Apollo. In terms of impact Silverlight is available as a cross platform, cross browser plug in capable of playing DRM wrapped media. In theory this means that DRM video services should now be able to be made available to Macintosh users. Practically and pragmatically this is of course not guaranteed. This week at Microsoft’s MIX further announcements on Silverlight have been made around programming. Silverlight contains a new dynamically focused version (DLR) of the .NET runtime (CLR) to provide an environment in which dynamic languages such as Javascript, Python and Ruby can better be run within. The creator of the .NET/CLR version of Python, Jim Hugunin, who now works for Microsoft provides some more details on the Dynamic Language Runtime (DLR). Whilst the full presentation by Ray Ozzie is available as a Windows video stream. I’ve already tried some of the Python samples running within Firefox on a Mac (PPC) and they’re taking down Firefox every time. Will update if I can get these working.

[Back at NAB] One night I excused myself from the industry cocktail circuit and paid cash money to attend a massive FCP Users Group meeting. Walter Murch gave a great talk on the production process of the new film, Youth Without Youth from Francis Ford Coppola. Murch is the editor on YWY and described how he approached the edit in FCP. Similar stuff to his book In the Blink of an Eye. The film was shot on the digital Sony CineAlta F900 cameras, 2 used statically, and when they needed a third an Arri film camera was employed. It was slightly unclear but they seemed to record vi HDSDI onto an external deck and record onto internal cassettes as well. The material was digitised as DV for the offline editing, effects done in Shake and then conformed against the HD material. Sound was handled in Pro Tools.

Apple’s announcements on FCP 6, Color and Final Cut Server made the largest noise at NAB. I asked everyone I met at NAB what were the big things for them and just about everyone polled listed Apple’s products in their top three. I was posed the question what will the impact of Apple’s “bang for buck” announcements be for mainstream media. Apple have aggressively priced FCP and the introduction of Color - effectively bundled for free and FCP Server continue this. So for traditional broadcasters and decent sized production houses they can look to continue to see cost savings, either by switching to FCP or to some extent by waiting for Avid to react and continue their ongoing price cutting.

More importantly it continues to drop the barrier of entry to video production. Be it for traditional programmes or the new net based forms of video delivery. For a great source of ongoing discussion on this topic I recommend checking out Twit.tv and the Pixel Corps - who are closely related, as is Video Grunt, who has a great series of videocasts on aspect ratio which a few people should try watching…

For me FCP/MXF and possibly FCP Server will be the focus of some ongoing exploration in the coming months. In particular around integration, using FCP’s XML project format, which I hope I’ll have some time to comment about. Be

nab2007 14 april

Arrived in Las Vegas yesterday, the same day we left, April the 13th. Finding suitable food to eat was the order of the day followed by the time sync exercise ‘how long can I stay awake’…

Today we hit the shopping opportunities just past the airport including Fry’s Electronics. Generally remarkably well restrained as we wandered past HD-DVDs, Blue Ray discs, and a massive section of ordinary old DVDs. Was mightily tempted to pick at least one of the three volume box sets of the Tomorrow People.

This afternoon we participated in an Omneon PR event where TVNZ announced the purchase of a MediaGrid and MediaDecks as part of the infrastructure being put in place for the new digital channels.

Just a minutes until Sunday, when there is the Apple event, which I’m looking forward to…

nab, fcp and spam

NAB FCP
Today I’m off to NAB, the massive US broadcasting show held in Vegas. Should be fun but manic. Every hour of my first three days is already booked. Hope to be able to get away from my vendor relationship duties and prowl the floor for the small funky stuff. Apparently Apple have an event on Sunday.

Will there be a new version of Final Cut Pro? Probably.
Will FCP support MXF? I hope so.
How will the acquisition of Proximity by Apple be expressed in product terms at NAB? No bloody idea, but we’ll find out soon enough.

SPAM
The judicious use of postfix’s configuration in combination with Gmail has created a good solution for my email. I had almost given up on email prior to the new approach. Configuring postfix was reasonably straight forward as there are good online resources available. However at one stage I did manage to generate an email storm. Over the years I have collected a number of domains and the new email system handles them all. With my first attempt at configuring postfix I generated an email for each of the domains when the system received an email for any of the domains. Email stopped getting through to Gmail. Looked in the postfix logs to see Gmail telling me ‘slow down you move too fast’…

Apple TV plays DivX/Xvid

Barely out on the US street the Apple TV is winning friends and being happily hacked.

There are obligatory unmake disassembly photo-sets, with the best being the dissection and commentary at anandtech.com.

There are walkthroughs on how to supersize the hard disk inside of the Apple TV.

And then there are the approaches to getting Divx/Xdvid content playing on an Apple TV. There two basic angles, either transcode the xvid into a codec that the Apple TV can playback or arm the Apple TV with the codecs necessary to play xvid.
The hacks are coming thick and fast because the Apple TV is running a version of Mac OS X that naturally uses QuickTime as the media foundation. At the moment all but the transcode require disassembling the Apple TV - however all that’s needed is for someone to discover how to connect (ssh) into Apple TV for there to be a reasonably friendly method of watching divx/xvid on your TV.

Odds on… by the time Apple TV gets into NZ shops this will have happened.

Updated: Awkward TV is a wiki pulling together a great collection Apple TV resources.

too busy to be coherent...

…but i’m due an entry so type i must

web.2.0 talks
Interested in the web dev world, web 2.0 and that? Then point yr browser/player at the Future of Web Apps for most of the presentations as audio, some even with the slides. That’s a nice attitude from Carson Systems who make their money from putting these events together.

freakin spam
Spam and/or spam prevention measures crippled my email and during last week email just stopped for me. Not sure if the issue was with my ISP’s anti-spam system or the freebie mail service provided by the domain provider. I decided to get the middlemen out of the way and over the weekend set up a small VPS at rimuhosting as a mail service. This is not really what I want to do with my life and I’m involving gmail in the equation to help me out. I’m almost certainly going to deploy a SMTP proxy such as Anti-Spam-SMPT-Proxy or qpsmtpd to help reduce wasted bandwidth by just zeroing those spam asap. I’m averaging just over 10K of spam per day. That’s for a email/domain that’s 12 years in the wild.