Friday, September 14, 2007

Linux Kernel-Userland Interface Design, Testing, and Documentation: An Update from LinuxConf EU, and the 2007 Linux Kernel Summit

| More

Google Code - Updates: Linux Kernel-Userland Interface Design, Testing, and Documentation: An Update from LinuxConf EU, and the 2007 Linux Kernel Summit

As you may know,Google allows its engineers to spend 20% of their time on projects independent of their regular day to day job. For my 20% time, I chose to continue and expand my work on maintaining the Linux man-pages.

Since April, we've managed to ship 21 new releases, with a dozen or so new pages, and around 400 major and minor improvements to existing pages.

My work on the Linux Man-pages project man-pages led me to talk about kernel-userland interface design, testing, and documentation at the recent LinuxConf Europe, where my Zurich colleague Roman Marxer also spoke about Google's recently open-sourced Ganeti virtual server management software.

I was lucky enough to be invited to the immediately following USENIX Linux Kernel Developers Summit, where I joined Google colleagues Andrew Morton, Paul Menage, and Martin Bligh to participate in the discussion of current topics related to kernel development, including the topic of kernel-userland API design, testing, and documentation.

You can read my talk, and in-depth coverage of the Kernel Developer Summit at It's available to subscribers only until the 20th of September, but you can already see the obligatory group photo.

Googlers Andrew Morton and Paul Menage relaxing at the end of the Linux Kernel Summit, Cambridge, England

(photo credit: Michael Kerrisk)

Ed. note: Post updated to fix typo.


Google welcomes ISO decision on OOXML

| More

Google Code - Updates: Google welcomes ISO decision on OOXML

Google welcomes the ISO decision to not approve the fast track of Office Open XML (OOXML) proposed standard DIS 29500 (ECMA 376).

Our engineers conducted an independent analysis of the OOXML specification and found several areas of concern, which we communicated both to the ISO and to the public. These include and are not limited to the following:
  • for a specification of this size it was not given enough time for review;
  • the undocumented features of OOXML prevents its implementation by other vendors;
  • dependencies on other Microsoft proprietary formats and their technical defects makes it difficult to fully implement; and
  • the overall cost for vendors of implementing multiple standards (hence the lack of OOXML implementations in the marketplace).
It is also incompatible with the existing ISO standard ISO 26300:2006, the Open Document Format (ODF), which already offers a high degree of interoperability, wide support, and offers the level playing field the world needs. Google is a supporter of ODF and has successfully integrated this open format into Google Docs and Spreadsheets. ODF also enjoys implementation in over twelve other products.

The ISO approval required at least 2/3 (i.e., 66.66%) of the votes cast by participating (P) members to be positive, and no more than 1/4 (i.e., 25%) of the total number of national body votes cast negative. Neither of these criteria were met by the proposed standard.

The concerns from many technical experts around the world were submitted as comments by the voting bodies to ISO on September 2, 2007. These must now be resolved at a Ballot Resolution Meeting (BRM) on February 25-29, 2008. In contrast, ODF was approved unanimously (23-0 among P members, 31-0 overall) as an international standard by ISO in May 2006.

As we represented our position in many countries, we were encouraged by the process observed in some places that truly evaluated the proposed standard on its technical merits as well as the feasibility of implementing the standard for the people of the country. These countries successfully declined or abstained due to insufficient information about the standard or the lack of time to evaluate the specification. In addition, many irregularities have been reported in the voting process (see here, here and here).

Technical standards should be arrived at transparently, openly, and based on technical merit. Google is committed to helping the standards community remain true to this ideal and maintain their independence from any commercial pressure.

Google supports one open document format and calls on industry participants to collaboratively work on ODF. With multiple implementations of one open standard for documents, users, businesses and governments around the world can have both choice and freedom to access their own documents, share with others and pass onto future generations.


Thursday, September 13, 2007

UW and Google: Teaching in Parallel

| More

Google Code - Updates: UW and Google: Teaching in Parallel

Earlier this year, the University of Washington partnered with Google to develop and implement a course to teach large-scale distributed computing based on MapReduce and the Google File System (GFS). The goal of developing the course was to expose students to the methods needed to address the problems associated with hundreds (or thousands) of computers processing huge datasets ranging into terabytes. I was excited to take the first version of the class, and stoked to serve as a TA in the second round.

But you can't program air, so Google provided a cluster computing environment to get us started. And since computers can't program themselves (yet?), UW provided the most essential component: students with sweet ideas for a huge cluster. After learning the ropes with these new tools, students finished the course by producing an impressive array of final projects, including an n-body simulator, a bot to perform Bayesian analysis on Wikipedia edits to search for spam, and an RSS aggregator that clustered news articles by geographic location and displayed them using the Google Maps API. Check out Geozette.

We are looking at ways to encourage other universities to get similar classes going, so we've also published the course material that was used at the University of Washington on Google Code for Educators. You're more than welcome to check out the Google Summer Intern video lectures on MapReduce, GFS, and parallelizing algorithms for large scale data processing. This summer I've been working on exposing these educational resources and other tools so that anyone can work on and think about cool distributed computing problems without the overhead of installing his or her own cluster. In that vein, we've released a virtual machine containing a pre-configured single node instance of Hadoop that has the same interface as a full cluster without any of the overhead. Feel free to give it a whirl.

We're happy to be able to expose students and researchers to the tools Googlers use everyday to tackle enormous computing challenges, and we hope that this work will encourage others to take advantage of the incredible potential of modern, highly parallel computing. Virtually all of this material is Creative Commons licensed, and we encourage educators to remix it, build upon it, and discuss it in the Google Code for Educators Forum.

Lastly, a quick shout out to the other interns who helped out on our team this summer: Aaron Kimball, Christophe Taton, Kuang Chen, and Kat Townsend. I'll miss you guys!


Monday, September 10, 2007

[Gd] ExtMapTypeControl 1.2: Now Supporting Custom Map Types

| More

Official Google Maps API Blog: ExtMapTypeControl 1.2: Now Supporting Custom Map Types

First off, I'd like to thank Pamela Fox and the whole Google Maps team for creating such an awesome API, and especially for getting the Open Source Utility Library going.

Next, a little background. In my quest to implement custom controls, I discovered the ExtMapTypeControl code in the Google Maps API Open Source Utility Library. Much to my delight, the code was dazzlingly obvious and the control which I created fit perfectly within the GMaps UI. It was super-easy to extend the control with my own custom button(s) in lieu of the Traffic view, and I had hacked the original source to bits in no time. However, there was one part of the code that struck me as a bit inelegant: the assumption that map type controls always displayed the default map-types: G_NORMAL_MAP, G_SATELLITE_MAP and G_HYBRID_MAP. I wanted a control that could display custom map types as well.

For those of you who don't know, you can create your own custom map types, cut your own tiles, or even remove default map types. The GMapTypeControl respects whatever map types are defined, and behaves as expected. I fixed up a version of the ExtMapTypeControl class to work the same way, signed up with the GMaps API Utility Library, and ported those changes into the existing library.

Check out the sample code and example below. You can grab the new javascript from the 1.2/src folder, or wait two weeks for it to be pushed into the release directory.

  var map = new GMap2(document.getElementById("map")); map.setCenter(new GLatLng(37.441944, -122.141944), 13); map.addControl(new ExtMapTypeControl());  map.removeMapType(G_NORMAL_MAP); var copyright = new GCopyright(1, new GLatLngBounds(new GLatLng(-90, -180), new GLatLng(90, 180)), 0, 'owned by NASA'); var copyrights = new GCopyrightCollection('The Blue Marble Imagery'); copyrights.addCopyright(copyright); var tilelayer = new GTileLayer(copyrights, 0, 17); tilelayer.getTileUrl = function(tile, zoom) {    return "";  }; var CUSTOM_MAP = new GMapType( [tilelayer], new GMercatorProjection(20), "NASA" ); map.addMapType(CUSTOM_MAP); 
<br><a href="">Link to ExtMapTypeControl with Custom Map Types example</a><br>


[Gdev] PassPack

| More

Google Code - Featured Projects: PassPack

Passpack Screenshot
Google APIs used:
PassPack is a web based password and data tracking tool that now works offline.


Sunday, September 9, 2007

[Gd] Documentation Updates to the Maps API!

| More

Official Google Maps API Blog: Documentation Updates to the Maps API!

Some of you may have noticed that we've given the Maps API documentation a makeover. If you haven't looked at the documentation in a while, check it out now; you might learn something new! (I did.)

In addition to rewriting several sections and adding new content, we've also tried to make it easier to read, navigate, and find the information you're looking for. Some of this reorganization has come at a price. Originally, when the Maps API was small, we tried our best to give you all the information in one page. However, as the Google Maps API has grown, we've been forced to admit that structure helps.

The original "Concepts and Examples" file has been split up into 6 sections:

Examples are also now listed in one file for easy reference.

Some of the new information discusses a common question: how we organize and serve map tiles for the API. We hope this encourages more development of your own custom map types.

We've also added tab navigation to the top of each page to ease navigation between different concepts. We hope you find the redesign easier to follow. In the coming months, I'll be working with Pamela Fox to add more in-depth tutorials, tips and tricks, and conceptual information. If you have requests for specific documentation, let us know on the Google Maps API Developer Forum. (Use a subject line of Documentation Request: so we don't miss it ... :)