As part of my accessibility work for Mozilla and Xiph, it is necessary to define how time-aligned text such as subtitles, captions, or annotations, are encapsulated into Ogg. In the fansubber community this is called “hard subtitles” as opposed to “soft subtitles” which are subtitles that stay in a text file and are loaded separately to the video file into a media player and synchronised with the video by the media player. (as per comment below, all text annotations are “soft” - or also “closed”.)

I can hear you ask: so how do I do subtitles/captions with Ogg now? Well, it would have been possible to simply choose one subtitling format and map that into Ogg, then ask everyone to just use that one format and be done. But which one to choose? And why prefer a simpler one over a more complex one? And why just do subtitles and not any other time-aligned text?

So, instead, I analysed what types of time-aligned text “codecs” I have come across. Each one would have a multitude of text formats to capture the text data, because it is easy to invent a new format and standardisation hasn’t really happened in this space yet.

I have come up with the following list of typical time-aligned text codecs:

• CC: closed captions (for the deaf)
• SUB: subtitles
• TAD: textual audio descriptions (for the blind - to be transferred to braille or TTS)
• KTV: karaoke
• TIK: ticker text
• AR: active regions
• NB: metadata & semantic annotations
• TRX: transcripts / scripts
• LRC: lyrics
• LIN: linguistic markup
• CUE: cue points, DVD style chapter markers and similar navigational landmarks

Let me know if you can think of any other classes of video/audio-related time-aligned text.

All of these texts can be represented in text files with some kind of time marker, and possibly some header information to set up the interpretation environment. So, the simplest way of creating a representation of these inside Ogg was to define a generic mapping for time-aligned text into Ogg.

The Xiph wiki holds the current draft specification for mapping text codecs into Ogg. For anyone wanting to map a text codec into Ogg, this should provide the framework. The idea is to separate the text codec’s data into header data and into timed text segments (which can have all sorts of styling and other information with it). Then, the mapping is simple. An example for srt is described on the wiki page.

The specification is still in draft status, because we’re still expecting feedback. In fact, what we now need is people trying an implementation and providing fixes to the specification.

To map your text codec of choice into Ogg, you will probably requrie further mapping specifications. Dependent on how complex your text codec of choice is, these additional mapping specifications may be rather simple or quite complicated. In the case of srt, it should be trivial. Considering the massive amount of srt already freely available online, the srt mapping may well have a really large impact. Enough hits. Let me know if you’re coding up something!

My next duty is to look for a representation that is generic enough to provide representations for any of the above listed text codecs. This representation is what will need to be available to a Web Browser when working with a Web video that has related text. Current contenders are OggKate and W3C TimedText, but I am not sure if either are too restrictive. I am indeed looking for the next generation of captioning technology that will be able to provide any type of time-aligned text that relates to audio/video.

Recently, I noticed an increasing number of videos on YouTube were no longer available - even if they had just been shared through a blog post by friends or even if they were the main video on a producer’s YouTube page, such as QuantumOfSolace.

I was suspicious for a while that there was something wrong with my browser, but when my colleague was able to play the video from the same network and I wasn’t, something had to be done.

I am running Firefox 3.0.4 on OS X 10.5.5 with the Flash 10.0 d26 plugin. First we thought it might be blocked for au.youtube.com and not for www.youtube.com, but there was no difference. Still the same “Sorry, this video is no longer available”.

Finally, an installation of the latest Flash plugin 10.0 r12 fixed the issue. So, if a large number of videos on YouTube isn’t available to you for no apparent reason, you might want to upgrade your Flash plugin.

Today, there were so many news that I can only summarise them in a short post.

The guys from Collabora have announced that they are going to support the development of PiTiVi - one of the best open source video editors around. They are even looking to hire people to help Christian Schaller, the author of PiTiVi. The plan is to have a feature-rich video editor ready by April next year that is comparable in quality to basic proprietary video editors.

The BBC Dirac team have today announced a ffmpeg2dirac software package, which is built along the same lines as the commonly used ffmpeg2theora and of course transcodes any media stream to Ogg Dirac/Vorbis. With Ogg Dirac/Vorbis playback already available in vlc and mplayer, this covers the much needed creation side of Ogg Dirac/Vorbis files. Dirac is an open source, non-patent-encumbered video codec developed by the BBC. It creates higher quality video than Theora at comparable bitrates.

The FOMS - Foundations of Open Media Software hacker workshop for open media software announced today the current list of confirmed participants for the January Workshop. It seems that this year we have a big focus on open video codecs, on browser support of media, on open Flash software, and on media frameworks. It is still possible to take part in the workshop - check out the CFP page.

Finally an important security message: Mozilla has decided to put a security measure around the HTML5 audio and video elements that will stop them from being exploited by cross-site scripting exploits. Chris Double explains the changes that are necessary to your setup to enable your published audio or video to be displayed on domains that are different to the domain on which these files are hosted.

In the media fragment working group at the W3C, we are introducing a standard means to address fragments of media resources through URIs. The idea is to define URIs such as http://example.com/video.ogv#t=24m16s-30m12s, which would only retrieve the subpart of video.ogv that is of interest to the user and thus save bandwidth. This is particularly important for mobile devices, but also for pointing out highlights in videos on the Web, bookmarking, and other use cases.

I’d like to give a brief look into the state of discussion from a technical viewpoint here.

Let’s start by considering the protocols for which such a scheme could be defined. We are currently focusing on HTTP and RTSP, since they are open protocols for media delivery. P2P protocols are also under consideration, however, most of them are proprietary. Also, p2p protocols are mostly used to transfer complete large files, so fragment addressing may not be desired. RSTP already has a mechanism to address temporal fragments of media resources through a range parameter of the play request as part of the protocol parameters. Yet, there is no URI addressing scheme for this. Our key focus however is HTTP, since most video content nowadays is transferred over HTTP, e.g. YouTube.

Another topic that needs discussion are the types of fragmentation for which we will specify addressing schemes. At the moment, we are considering temporal fragmentation, spatial fragmentation, and fragmentation by tracks. In temporal fragmentation, a request asks for a time interval that is a subpart of the media resource (e.g. audio or video). In spatial fragmentation, the request is for an image region (e.g. in an image or a video). Track fragmentation addresses the issue where, e.g. a blind person would not require to receive the actual video data for a video and thus a user agent could request only the data tracks from the resource that are really required for the user.

Another concern is the syntax of URI addressing. URI fragments (”#”) have been invented to created URIs that point at so-called “secondary” resources. Per definition, a secondary resource may be some portion or subset of the primary resource, some view on representations of the primary resource, or some other resource defined or described by those representations. It is therefore the perfect syntax for media fragment URIs.

The only issue is that URI fragments (”#”) are not expected to be transferred from the client to the server (e.g. Apache strips it off the URI if it receives it). Therefore, in the temporal URI specification of Annodex we decided to use the query (”?”) parameter instead. This is however not necessary. The W3C working group is proposing to have the user agent strip off the URI fragment specification and transform it into a protocol parameter. For HTTP, the idea is to introduce new range units for the types of fragmentation that we will define. Then, the Range and Content-Range headers can be used to request and deliver the information about the fragmentation.

The most complicated issue that we are dealing with is the issue of caching in Web proxies. Existing Web proxies will not be able to understand new range units and will therefore not cache such requests. This is unfortunate and we are trying to devise two schemes - one for existing Web proxies and one for future more intelligent Web proxies - to enable proxy caching. This discussion has many dimensions - such as e.g. the ability to uniquely map time to bytes for any codec format, the ability to recompose new fragment requests from existing combined fragment requests, or the need and abilities for partial re-encoding. Mostly we are dealing with the complexities and restrictions of different codecs and encapsulation formats. Possibly, the idea of recomposition of ranges in Web proxies is too complex to realise and caching is best done by regarding each fragment as its own cacheable resource, but this hasn’t been decided yet.

We now have experts from the squid community, from YouTube/Google, HTTP experts, Web accessibility experts, SMIL experts, me from Annodex/Xiph, and a more people with diverse media backgrounds in the team. It’s a great group and we are covering the issues from all aspects. The brief update above is given from my perspective, and only lists the key issues superficially, while the discussions that we’re having on the mailing list and in meetings are much more in-depth.

I am not quite expecting us to meet the deadline of having a first working draft before the end of this month, but certainly before Christmas.

To quote Justaan:

This is what we call the Melbourne Cup Network Effect

It seems it really is the race that stops the nation. This is a graph of Bulletproof’s outbound web traffic for today. That’s a 37% drop in outbound traffic just after 3pm.

Make sure you take note of my l33t gimp skills!

While the open source codec “Theora” has been available since 2004 in a stable format, the open source community is very careful about giving any piece of software the “1.0″ stamp of quality and libtheora has been put under scrutiny for years.

Today, libtheora 1.0 was finally released - rejoice and go ahead using it in production!

More hard-core improvements to libtheora are also in the pipeline under a version nick-named “Thusnelda”, improving mostly on quality and bit-rate.

At Vquence we do a lot of crawling of various video hosting sites and where possible we like to use APIs or RSS feeds instead of page scraping. A semi-recent features of firefox is that when you click on an RSS link you get a “Subscribe to this feed in your favourite reader” header and then the formatted contents of the feed.

This is really annoying if what you really want to see is the raw XML. Sure I could hit CTRL-U to see the source but thats an extra step and a whole other window I now have open. I couldn’t find any way to disable this functionality so I ended up writing a greasemonkey script called disable_subscribe_feed.js.

The meat of the script looks like

// Pick three element ids that appear in the "Subscribe to page" and probably
var tag1 = document.getElementById('feedHeaderContainer');
var tag2 = document.getElementById('feedSubscriptionInfo2');
var tag3 = document.getElementById('feedSubscribeLine');

// Show the source
if (tag1 && tag2 && tag3) {
    location.href = 'view-source:' + document.location.href;
}

Basically it tries to detect the “subscribe to feed” page based on a couple of tag ids that exist on it and then performs a redirect to view-source: for that page. Which gives us nicely formatted XML.

I’m currently going through the process of becoming a Debian Maintainer so that I can upload Annodex packages without bugging one of the DDs I know. Thanks to horms and jaq for their help thus far.

As part of this process you need to file a bug against the debian-maintainers package to get your key added. You need to do this using a piece of software called jetring. jetring allows you to create changesets for a gpg keyring, a binary format, to make it easy for the maintainers to add and remove keys and know exactly whats being added and removed. I couldn’t find very much information on how you actually do this and hence the reason for this post.

To start with you need to grab the latest copy of the debian-maintainers keyring and extract the actual keyring from it. You can find the link to the latest version at debian-maintainers, just click on all to download it.

Here is the process I followed with comments along the way

# Download the latest debian-maintainers keyring
wget http://http.us.debian.org/debian/pool/main/d/\
debian-maintainers/debian-maintainers_1.38_all.deb
dpkg-deb -x *.deb keyring
mv keyring/usr/share/keyrings/debian-maintainers.gpg .
rm -rf keyring *.deb

# Create a copy of it and add your key to it
cp debian-maintainers.gpg debian-maintainers.gpg.orig
gpg --export 
 This e-mail address is being protected from spambots. You need JavaScript enabled to view it
  | \
    gpg --import --no-default-keyring --keyring `pwd`/debian-maintainers.gpg

# Create the changset with jetring
jetring-gen debian-maintainers.gpg.orig debian-maintainers.gpg \
    "Add John Ferlito <johnf @inodes.org> as a Debian Maintainer"

# Check the changeset
jetring-review -d debian-maintainers.gpg.orig add-*

Once you have completed the above you should have a file with something like the following contents

Comment: Add John Ferlito <johnf @inodes.org> as a Debian Maintainer
Date: Sat, 05 Jul 2008 14:26:31 +1000
Action: import
Data:
  -----BEGIN PGP PUBLIC KEY BLOCK-----
  Version: GnuPG v1.4.6 (GNU/Linux)

  mQGiBEd6MmQRBADF+BLVChN/AqKVXkrJFU2LtJoiCdYJ
  <snip>
  =SSNk
  -----END PGP PUBLIC KEY BLOCK-----

You should now add something along the lines of the below to the top of the file.

Recommended-By:
  Simon Horman <horms @verge,net.au>,
  Jamie Wilkinson <jaq @spacepants.org>
Agreement: http://lists.debian.org/debian-newmaint/2008/07/msg00010.html
Advocates:
  http://lists.debian.org/debian-newmaint/2008/07/msg00011.html,
  http://lists.debian.org/debian-newmaint/2008/07/msg00012.html

The agreement line should be a URL to your signed email applying to become a DM and the advocates should be the URLs for the signed emails from your advocates.

Once you’ve done that, submit a bug with the file attached and hopefully sometime later you will have become a DM.

Wouldn’t it make more sense for firefox to allow popups based on the destination site rather than on the source?

For example most popups I click on are for YouTube. Now some on these are on random blogging sites. Which means that to jump to the YouTube page for that video I have to allow popups for some random blog, which can now popup as many ads as it wants.

Wouldn’t it make more sense to allow YouTube as a popup destination. It really comes down to the fact that I trust YouTube more than some random blog embedding YouTube videos.

I haven’t thought about this very much so maybe there is a good reason why you wouldn’t want this. If a few other people agree with me I’ll go file a bug. Hmm I wonder if you could write an extension to do it.

Update: Peter raises a good point as to why this is a bad idea.

I was updating an svn branch today using bzr, thanks to bzr-svn, and I wanted to know what svn revision I was at.

You can easily see the bzr revision by running

johnf@zoot:~/trunk$ bzr revno
34

But it gives no indication of where you are in SVN land. After a bit of rummaging around I discovered the following

johnf@zoot:~/trunk$ bzr version-info
revision-id: svn-v3-trunk0:90e61fa5-4541-0410-a685-e5b9dba3c764:trunk:74
date: 2008-05-29 19:24:44 +0000
build-date: 2008-06-14 19:10:59 +1000
revno: 34
branch-nick: trunk

The revision-id field seems to be the key and seems to indicate I’m using SVN revision 74. Checking the branch via the web confirmed that.

What a mess is being made of copyright in the online world.

People steal works from others. People borrow stuff. Some people rely on what they think is the "fair use" provisions of the copyright act. People in Australia think that because they have read an article online about how Google appears to be fighting the Viacom lawsuit that they can use similar arguments in Australia, where copyright law is different to the US.... and on it goes.

And then you see this kind of story appear and you realize that the whole thing is a rats nest of infinite proportions. (And thanks to Laurel Papworth and her blog for pointing to this story).

Seems like Viacom has muscled YouTube into pulling down a piece of content that was part of a program on VH1. Only problem is that the program on VH1 used the material that they have forced YT to pull without permission in the first place. Talk about the pot calling the kettle black!

We have to realize that copyright in the digital age doesn't work in the way that pre-digital legislators and rights owners contemplated (and the rights owners were the people who lobbied for the laws in the first place, and essentially wrote th first drafts of the statutes anyway).

It is a really interesting time for this discussion too because we are moving rapidly to a point where the amount of time spent consuming media online will be greater than the amount of time spent consuming traditional electronic media. I read the following in a report from Strategic News Service that came out last month:


So you can see with this sort of growth infrastructure will grow apace. And content will then have the air to also grow at a phenomenal rate. And at the same time the dirty little secret of the long tail will be there biting us on the bum. That secret of course is the corollary of the millions of markets scenario. This is the fact that content in the digital realm never goes away and continues to propagate at an ever increasing rate.

Which means that copyright owners (copywrong, I prefer to call them now) are going to find that the instances of their works are going to keep on appearing somewhere...

And unless they start by measuring the number of uses or number of occurrences they will never know how much money is leaking out of the system, how much revenue they are foregoing.

And of course this applies perhaps more strongly to new and emerging copyright creators and authors than it does to the incumbents. The incumbents already tend to be paid more than their due, because of the existing policies of the various societies. The rules of the societies are to pay the black box monies on a follow the dollar basis (of course they always deny the existence of a 'black box' but it is there under a variety of names).

So where does all this lead?

You have to have measurement, metrics, analytics. Not just so that advertisers can figure where the best spot is to place an ad, but so that a proper system of reward can be instituted for the creatives in the community.

This is really an imperative for collections societies, I believe. Whether we move to a Larry Lessig vision for the adoption of Creative Commons licenses or some other model, you need to start first with measuring the market. (Full disclosure: This is the vision for Vquence now, which I am a director of, to measure every aspect of social media online).

Well, after a long haul, yesterday we finally got our first 'real' deal done at Vquence.

We are supplying a quite complex set of video solutions to an about to be launched web site that is focusing on the forthcoming federal election. More on this once we are green lit to be able to talk about it.

But we think that the set of solutions that we are providing are going to be very cool for people who are interested in following all the video activity of both the pollies and the public.

There is now no doubt that we are in the early part of a massive shift in the culture of Australia.

We are about to become a fully integrated YouTube society.

For a lot of people this will not be apparent for possibly several years. By that time the momentum will be sufficient that many marketers and PR people will not be able to get their clients and employers onto the bandwagon.

The amazing thing is that this revolution is being led by the mainstream political parties in Australia. First the Liberals with 4 major policy statements being announced by our Prime Minister, John Howard, on YouTube.

Then, the launch yesterday of the Kevin07 website. Laden with video - that is also on YouTube - it, together with the PM, is setting the pace for communications this spring in Australia as we head into a Federal Election.

At the same time there are new websites in the process of coming onstream like www.federalelection.com.au. The goal of this one is to provide the total forum for debate in the coming months. A site where all parties and candidates can be presented side by side. Great idea. Not sure if they are going to be able to make it the commercially viable success they would like as rapidly as they would like. We shall see.

What I find amazing is that this sea change to our culture. Because this is as big as when TV became the debating platform in the US. And Richard Nixon became 'Tricky Dicky' because under the powerful TV lights his 12 0'clock shadow made it look like he wasn't clean. Or prior to that when Eisenhower used TV to advertise in his presidential campaign. These were firsts.

Now in Australia we will have the first true YouTube election anywhere in the world. It stands to be a bell weather that campaigners in the US and UK will watch and use to avoid mistakes. And it should also set some examples for how we can expect corporations to get their marketing messages through to us in coming years.

(By the way, at Vquence we are starting to realize that the tools that we have been developing that enable us to rapidly crawl video hosting sites and monitor changes in near real time also provide us with a huge amount of raw data that can be massively valuable to companies that want to play in this space.)

Advertising

Got game

Jun 7th 2007
From The Economist print edition

Inserting advertisements into video games holds much promise

THEY are known to television executives as the “Lost Boys”—the generation of video-gaming young men who are watching less television and, thanks to ad-skipping technologies such as TiVo, even fewer advertisements. The obvious response is to start putting advertisements into games instead, by incorporating billboards into the game environment, for example. But incorporating static advertisements into games is unsatisfactory. Now that most PCs and a growing number of games consoles are connected to the internet, however, it is possible to update advertisements when required. As a result, static in-game advertisements are now giving way to dynamic adverts, which accounted for $26m of the $76m spent on in-game advertising last year, and will account for 55% of the $182m spent this year, says the Yankee Group, a consultancy.

Vertical search-engines

Know your subject

Jul 12th 2007
From The Economist print edition

Topic-specific search-engines hope to challenge Google, at least in some areas

ARE you a generalist or a specialist? The question can be asked of people, but it is increasingly being asked about internet search-engines, as specialist or “vertical” sites take on generalists such as Yahoo! and Google. Some are already prospering: GlobalSpec.com, for example, a profitable search-engine for engineers, has 3.5m registered users and signs up another 20,000 each week. “They own that market,” says Charlene Li of Forrester, a consultancy.

This is due in large part to GlobalSpec's definable customer base. Its knowledge about the needs of its users sets it apart from the generalist search-engines, says Angela Hribar of GlobalSpec. Vertical sites, which serve up search results from a carefully selected group of topic-specific websites, can also target advertising at particular audiences more precisely...