Friday, September 18, 2009

[Gd] Dev Channel Update

| More

Google Chrome Releases: Dev Channel Update

This week's changes for the dev channel,, is being released to all platforms.

  • All
  • Win
    • Ensure that tips change when the Chrome language changes. [r25834]
  • Mac
    • Import from Firefox no longer hangs. [r25174]
    • Added SSL icons. [r26307]
    • Implement the search engine manager. [r26078]
    • Allow windows with a single tab to be merged into other windows with drag and drop.
    • Allow Snow Leopard systems to connect to certain IPv6-enabled web sites when only IPv4 is available. [r26051]
    • Prevent a sad tab when loading certain images on Snow Leopard. [r26089]
    • Don't show "Google Chrome did not shut down properly" when quit from the Dock, logout, restart, or shut down. [r26269]

  • Linux:
    • Make the bookmark toolbar folders act like a menu bar. [r25677]
    • Bookmark bar shows a menu on too many bookmarks. [r25200]
    • Implement external protocol handler dialog (e.g. for aim: URLs).[r25373]
    • Extensions can register page actions. [r25934]
    • Fix a crash when closing tabs that have open login prompts. [r26066]
    • Work around a Flash crash that mostly affects Gentoo users. [r26265]

  • Extensions
    • Enable/disable extension button on chrome://extensions
    • Update extensions now button to force autoupdate check on chrome://extensions
    • chrome.window and APIs can now reference relative URLs inside an extension 

Known issues:
  • All
    • Large files do not download completely - bug 406
  • Linux:
    • Cannot be set as the default browser in GNOME (Already fixed by [r26314r26316]).

More details about additional changes are available in the svn log of all revisions.

You can find out about getting on the Dev channel here:

If you find new issues, please let us know by filing a bug at

Anthony Laforge
Google Chrome Program Manage


[Gd] Reinforcements in the war on slow

| More

iGoogle Developer Blog: Reinforcements in the war on slow

On the iGoogle team we're always working to balance the needs of our users with the needs of our developers, to make sure we're creating an environment where everyone benefits. We want users to have access to the very best gadgets, hence we want to make sure we provide our developers with all the tools and information they need to create those gadgets.

Recently, we announced plans to mark gadgets in the directory that were especially slow to load. We have some new tools on the way that will help make it easier for developers to streamline their gadgets. So we've decided to hold off on labeling gadgets until we've released these new tools and give developers a chance to use them to improve their gadgets.

In the meantime, there are still plenty of things that can be done to fight gadget latency — be sure to check out our latency tips on Google Code, and our Latency Combat Field Manual!

Posted by Tyler Odean, Product Manager, iGoogle Team

Thursday, September 17, 2009

[Gd] Japan's mixi has launched its OpenSocial Container for all users!

| More

OpenSocial API Blog: Japan's mixi has launched its OpenSocial Container for all users!

Hello! My name is Yoichiro Tanaka, and I belong to the Platform Team of mixi, Inc. "mixi" is currently the most popular social networking service (SNS) in Japan and has more than 17 million registered users. I am happy to announce that we have released "mixi apps" which is based on OpenSocial to all of our users. As of September 4th, more than 220 apps have already been registered and launched!


mixi was originally launched in February 2004 as one of the first social networking services in Japan. It lets users create profiles, make friends with other users, post diaries, discuss in communities, share pictures and music with friends, etc.

mixi apps is a new service which lets social application providers develop applications that use social graphs formed within mixi, and provide these applications to mixi users. To achieve this, we chose OpenSocial as the best solution.

In order to be ready with a quality set of applications for our public launch, we have been running a beta version of mixi apps for our developers since last December.


Since our launch of mixi apps, we are finding some applications that are substantially more popular than others. Here's two of the more popular applications.

The first application is "Kanji test" produced by Drecom. It challenges users, who compete against each other, to write down how to read Kanji characters (also known as to "furiganize"). It also shows the ranking of the number of Kanji sets that the user successfully furiganized. In the relatively short period since it has been available to users, it has skyrocketed in popularity and already has over three hundred thousand users!

The next application is "Recollect sketch" produced by REAL. Users are given a theme, and are challenged to draw it entirely from memory! What results is usually a very unique and funny image which users can share and enjoy with friends. This application is visually entertaining and social, and it already has about one hundred thousand users.

The preceding applications are just a sample of the continually growing application directory of mixi apps, which averages several new applications per day!

Supported Features
  • OpenSocial 0.8.1 core APIs (People, Activities, Persistence, and Invite).
  • The gadgets "core" and "feature-Specific" APIs (mostly supported).
  • Partly supports Albums API in OpenSocial 0.9.
  • A mixi specific "Community API," which allows application access to mixi community (or group) information along with its members.

Upcoming Supported Features
  • Sending a notification between users using a requestSendMessage function.
  • Sending a event of app's life-cycle for developers.

mixi Ad Program

In addition to mixi apps, we have also recently announced the "mixi Ad Program". This service is one of the business support programs from mixi for developers. If you decide to join this service, ads will be shown within your application, and you will make profits through revenue sharing based on your application's page view. This service has already started from September 3rd, 2009.

The features of "mixi Ad Program" are the following:
  • Registry and usage fee are free!
  • Both corporate and individual developers can join this program.
  • Ordering and confirming your reports is easy through a single management page.
  • Assured profit of a minimum of 0.01 yen per page view.

More information on this service can be found at: (Note: this site is in Japanese)

We believe this production release of "mixi apps" and overall support of the OpenSocial standards will enable us to easily continue innovating on compelling future social services for our users. In addition, we hope that our business support programs will greatly contribute to developing the market of mixi apps.

Posted by Yoichiro Tanaka, mixi Platform Team.

[Gd] Introducing Quick Search Box for Android

| More

Android Developers Blog: Introducing Quick Search Box for Android

One of the new features we're really proud of in the Android 1.6 release is Quick Search Box for Android. This is our new system-wide search framework, which makes it possible for users to quickly and easily find what they're looking for, both on their devices and on the web. It suggests content on your device as you type, like apps, contacts, browser history, and music. It also offers results from the web search suggestions, local business listings, and other info from Google, such as stock quotes, weather, and flight status. All of this is available right from the home screen, by tapping on Quick Search Box (QSB).

What we're most excited about with this new feature is the ability for you, the developers, to leverage the QSB framework to provide quicker and easier access to the content inside your apps. Your apps can provide search suggestions that will surface to users in QSB alongside other search results and suggestions. This makes it possible for users to access your application's content from outside your application—for example, from the home screen.

The code fragments below are related to a new demo app for Android 1.6 called Searchable Dictionary.

The story before now: searching within your app

In previous releases, we already provided a mechanism for you to expose search and search suggestions in your app as described in the docs for SearchManager. This mechanism has not changed and requires the following two things in your AndroidManifest.xml:

1) In your <activity>, an intent filter, and a reference to a searchable.xml file (described below):

<action android:name="android.intent.action.SEARCH" />
<category android:name="android.intent.category.DEFAULT" />

<meta-data android:name=""
android:resource="@xml/searchable" />

2) A content provider that can provide search suggestions according to the URIs and column formats specified by the Search Suggestions section of the SearchManager docs:

<!-- Provides search suggestions for words and their definitions. -->
<provider android:name="DictionaryProvider"
android:syncable="false" />

In the searchable.xml file, you specify a few things about how you want the search system to present search for your app, including the authority of the content provider that provides suggestions for the user as they type. Here's an example of the searchable.xml of an Android app that provides search suggestions within its own activities:

<searchable xmlns:android=""

Note that the android:searchSuggestAuthority attribute refers to the authority of the content provider we declared in AndroidManifest.xml.

For more details on this, see the Searchability Metadata section of the SearchManager docs.

Including your app in Quick Search Box

In Android 1.6, we added a new attribute to the metadata for searchables: android:includeInGlobalSearch. By specifying this as "true" in your searchable.xml, you allow QSB to pick up your search suggestion content provider and include its suggestions along with the rest (if the user enables your suggestions from the system search settings).

You should also specify a string value for android:searchSettingsDescription, which describes to users what sorts of suggestions your app provides in the system settings for search.

<searchable xmlns:android=""

These new attributes are supported only in Android 1.6 and later.

What to expect

The first and most important thing to note is that when a user installs an app with a suggestion provider that participates in QSB, this new app will not be enabled for QSB by default. The user can choose to enable particular suggestion sources from the system settings for search (by going to "Search" > "Searchable items" in settings).

You should consider how to handle this in your app. Perhaps show a notice that instructs the user to visit system settings and enable your app's suggestions.

Once the user enables your searchable item, the app's suggestions will have a chance to show up in QSB, most likely under the "more results" section to begin with. As your app's suggestions are chosen more frequently, they can move up in the list.


One of our objectives with QSB is to make it faster for users to access the things they access most often. One way we've done this is by 'shortcutting' some of the previously chosen search suggestions, so they will be shown immediately as the user starts typing, instead of waiting to query the content providers. Suggestions from your app may be chosen as shortcuts when the user clicks on them.

For dynamic suggestions that may wish to change their content (or become invalid) in the future, you can provide a 'shortcut id'. This tells QSB to query your suggestion provider for up-to-date content for a suggestion after it has been displayed. For more details on how to manage shortcuts, see the Shortcuts section within the SearchManager docs.

QSB provides a really cool way to make your app's content quicker to access by users. To help you get your app started with it, we've created a demo app which simply provides access to a small dictionary of words in QSB—it's called Searchable Dictionary, and we encourage you to check it out.


[Gd] Spanish Site Clinic now live

| More

Official Google Webmaster Central Blog: Spanish Site Clinic now live

The Google Webmaster Central blog in Spanish has launched a Site Clinic especially for the Spanish-speaking market. We're offering to analyze a series of websites in order to share some best practices with our community using real web sites. The plan is to offer constructive advice on accessibility and improvements that can lead to better visibility in Google's search results.
During this month, we will be receiving submissions from any legitimate website, but it must be primarily in Spanish. So before you submit your site, please visit the original post and if you want to participate fill out the form as soon as possible, because we will only be selecting 3-5 websites from the first 200 submitted for this Site Clinic, so don't miss out!

Posted by Esperanza, Search Quality Team

Wednesday, September 16, 2009

[Gd] New Reporting Dashboard for Developers

| More

YouTube API Blog: New Reporting Dashboard for Developers

Getting your YouTube web or client application up and running is only half the battle. We know that you'll also want to monitor your application to see how it's performing. Our new Developer Dashboard shows you at a glance the number of API requests, playbacks, uploads and errors that your app is generating. You can visualize the data using our interactive chart or download a file to process the data offline. To use the dashboard, you'll need to provide a developer key in your API requests and then pass the media URLs from the API responses to the standard embedded player or the chromeless player.

You can log into the new Developer Dashboard at

Posted by Kuan Yong, YouTube APIs and Tools Team

[Gd] Checked exceptions I love you, but you have to go

| More

Google Testing Blog: Checked exceptions I love you, but you have to go

Once upon a time Java created an experiment called checked-exceptions, you know, you have to declare exceptions or catch them. Since that time, no other language (I know of) has decided to copy this idea, but somehow the Java developers are in love with checked exceptions. Here, I am going to "try" to convince you that checked-exceptions, even though look like a good idea at first glance, are actually not a good idea at all:

Empirical Evidence

Let's start with an observation of your code base. Look through your code and tell me what percentage of catch blocks do rethrow or print error? My guess is that it is in high 90s. I would go as far as 98% of catch blocks are meaningless, since they just print an error or rethrow the exception which will later be printed as an error. The reason for this is very simple. Most exceptions such as FileNotFoundException, IOException, and so on are sign that we as developers have missed a corner case. The exceptions are used as away of informing us that we, as developers, have messed up. So if we did not have checked exceptions, the exception would be throw and the main method would print it and we would be done with it (optionally we would catch all exceptions in the main log them if we are a server).

Checked exceptions force me to write catch blocks which are meaningless: more code, harder to read, and higher chance that I will mess up the rethrow logic and eat the exception.

Lost in Noise

Now lets look at the 2-5% of the catch blocks which are not rethrow and real interesting logic happens there. Those interesting bits of useful and important information is lost in the noise, since my eye has been trained to skim over the catch blocks. I would much rather have code where a catch would indicate: "pay, attention! here, something interesting is happening!", rather than, "it is just a rethrow." Now, if we did not have checked exceptions, you would write your code without catch blocks, test your code (you do test right?) and realize that under some circumstances an exception is throw and deal with it by writing the catch block. In such a case forgetting to write a catch block is no different than forgetting to write an else block of the if statement. We don't have checked ifs and yet no one misses them, so why do we need to tell developers that FileNotFound can happen. What if the developer knows for a fact that it can not happen since he has just placed the file there, and so such an exception would mean that your filesystem has just disappeared! (and your application is not place to handle that.)

Checked exception make me skim the catch blocks as most are just rethrows, making it likely that you will miss something important.

Unreachable Code

I love to write tests first and implement as a consequence of tests. In such a situation you should always have 100% coverage since you are only writing what the tests are asking for. But you don't! It is less than 100% because checked exceptions force you to write catch blocks which are impossible to execute. Check this code out:
bytesToString(byte[] bytes) {
ByteArrayOutputStream out = new ByteArrayOutputStream();
try {
return out.toSring();
} catch (IOException e) {
// This can never happen!
// Should I rethrow? Eat it? Print Error?

ByteArrayOutputStream will never throw IOException! You can look through its implementation and see that it is true! So why are you making me catch a phantom exception which can never happen and which I can not write a test for? As a result I cannot claim 100% coverage because of things outside my control.

Checked exceptions create dead code which will never execute.

Closures Don't Like You

Java does not have closures but it has visitor pattern. Let me explain with concrete example. I was creating a custom class loader and need to override load() method on MyClassLoader which throws ClassNotFoundException under some circumstances. I use ASM library which allows me to inspect Java bytecodes. The way ASM works is that it is a visitor pattern, I write visitors and as ASM parses the bytecodes it calls specific methods on my visitor implementation. One of my visitors, as it is examining bytecodes, decides that things are not right and needs to throw a ClassNotFondException which the class loader contract says it should throw. But now we have a problem. What we have on a stack is MyClassLoader -> ASMLibrary -> MyVisitor. MyVisitor wants to throw an exception which MyClassLoader expects but it can not since ClassNotFoundException is checked and ASMLibrary does not declare it (nor should it). So I have to throw RuntimeClassNotFoundException from MyVisitor which can pass through ASMLibrary which MyClassLoader can then catch and rethrow as ClassNotFoundException.

Checked exception get in the way of functional programing.

Lost Fidelity

Suppose java.sql package would be implemented with useful exception such as SqlDuplicateKeyExceptions and SqlForeignKeyViolationException and so on (we can wish) and suppose these exceptions are checked (which they are). We say that the SQL package has high fidelity of exception since each exception is to a very specific problem. Now lets say we have the same set up as before where there is some other layer between us and the SQL package, that layer can either redeclare all of the exceptions, or more likely throw its own. Let's look at an example, Hibernate is object-relational-database-mapper, which means it converts your SQL rows into java objects. So on the stack you have MyApplication -> Hibernate -> SQL. Here Hibernate is trying hard to hide the fact that you are talking to SQL so it throws HibernateExceptions instead of SQLExceptions. And here lies the problem. Your code knows that there is SQL under Hibernate and so it could have handled SqlDuplicateKeyException in some useful way, such as showing an error to the user, but Hibernate was forced to catch the exception and rethrow it as generic HibernateException. We have gone from high fidelitySqlDuplicateKeyException to low fidelity HibernateException. An so MyApplication can not do anything. Now Hibernate could have throw HibernateDuplicateKeyException but that means that Hibernate now has the same exception hierarchy as SQL and we are duplicating effort and repeating ourselves.

Rethrowing checked exceptions causes you to lose fidelity and hence makes it less likely that you could do something useful with the exception later on.

You can't do Anything Anyway

In most cases when exception is throw there is no recovery. We show a generic error to the user and log an exception so that we con file a bug and make sure that that exception will not happen again. Since 90+% of the exception are bugs in our code and all we do is log, why are we forced to rethrow it over and over again.

It is rare that anything useful can be done when checked exception happens, in most case we die with error! Therefor I want that to be the default behavior of my code with no additional typing.

How I deal with the code

Here is my strategy to deal with checked exceptions in java:

  • Always catch all checked exceptions at source and rethrow them as LogRuntimeException.

    • LogRuntimeException is my runtime un-checked exception which says I don't care just log it.

    • Here I have lost Exception fidelity.

  • All of my methods do not declare any exceptions

  • As I discover that I need to deal with a specific exception I go back to the source where LogRuntimeException was thrown and I change it to <Specific>RuntimeException (This is rarer than you think)

    • I am restoring the exception fidelity only where needed.

  • Net effect is that when you come across a try-catch clause you better pay attention as interesting things are happening there.

    • Very few try-catch calluses, code is much easier to read.

    • Very close to 100% test coverage as there is no dead code in my catch blocks.


Tuesday, September 15, 2009

[Gd] Duplicate content and multiple site issues

| More

Official Google Webmaster Central Blog: Duplicate content and multiple site issues

Webmaster Level: All

Last month, I gave a talk at the Search Engine Strategies San Jose conference on Duplicate Content and Multiple Site Issues. For those who couldn't make it to the conference or would like a recap, we've reproduced the talk on the Google Webmaster Central YouTube Channel. Below you can see the short video reproduced from the content at SES:

You can view the slides here:

Posted by Greg Grothaus, Search Quality Team

[Gd] Android 1.6 SDK is here

| More

Android Developers Blog: Android 1.6 SDK is here

I am happy to let you know that Android 1.6 SDK is available for download. Android 1.6, which is based on the donut branch from the Android Open Source Project, introduces a number of new features and technologies. With support for CDMA and additional screen sizes, your apps can be deployed on even more mobile networks and devices. You will have access to new technologies, including framework-level support for additional screen resolutions, like QVGA and WVGA, new telephony APIs to support CDMA, gesture APIs, a text-to-speech engine, and the ability to integrate with Quick Search Box. What's new in Android 1.6 provides a more complete overview of this platform update.

The Android 1.6 SDK requires a new version of Android Development Tools (ADT). The SDK also includes a new tool that enables you to download updates and additional components, such as new add-ons or platforms.

You can expect to see devices running Android 1.6 as early as October. As with previous platform updates, applications written for older versions of Android will continue to run on devices with Android 1.6. Please test your existing apps on the Android 1.6 SDK to make sure they run as expected.

Over the next several weeks, we will publish a series of blog posts to help you get ready for the new developer technologies in Android 1.6. The following topics, and more, will be covered: how to adapt your applications to support different screen sizes, integrating with Quick Search Box, building gestures into your apps, and using the text-to-speech engine.

If you are interested to see some highlights of Android 1.6, check out the video below.

Happy coding!


[Gd] Registration Opens for fall 2009 Google Developer Days

| More

Google Code Blog: Registration Opens for fall 2009 Google Developer Days

We are excited to open registration for our second round of Google Developer Day events this year:
Both events will offer opportunities to learn the latest about our APIs and developer tools, including Android, Google Chrome, Google Wave, App Engine, AJAX APIs and more. There will also be time for developers to socialize - whether at "office hours" or the "after hours".  You'll be able to chat about your latest project or discuss that brain-busting question with fellow developers and Google engineers.

Similar to our Google I/O event in the US, the Developer Days will host a developer sandbox area, highlighting partners who have used Google developer products to build their own unique applications.  You can also take the opportunity to share a project or coding experience during our lightning talks. We look forward to seeing what developers bring to the table this year!

Space is limited so register soon!

By Alyssa England Sachs, Google Developer Programs

[Gd] [Language][Update] Nine New Virtual Keyboard Layouts

| More

Google AJAX API Alerts: [Language][Update] Nine New Virtual Keyboard Layouts

Added keyboards for Bulgarian, Czech, Greek, Hebrew, Hungarian 101 layout, Slovak, Slovenian, Turkish Q layout, and the Ukrainian 101 layout.

[Gd] [Language][Update] Ten New Translate Languages

| More

Google AJAX API Alerts: [Language][Update] Ten New Translate Languages

Added Afrikaans, Belarusian, Icelandic, Irish, Macedonian, Malay, Persian, Swahili, Welsh, and Yiddish.

[Gd] Stable Channel Update

| More

Google Chrome Releases: Stable Channel Update has graduated from Beta to the Stable channel today.

This release includes themes support, a brand new New Tab page, an updated omnibox, support for audio and video tags, and a higher performing V8 engine.

You can read more about it here.

Anthony Laforge
Google Chrome Program Manager

Security Fixes:

We would like to extend special thanks to Will Dormann of CERT for working with us to improve the security of the new audio and video codecs in this release.

CVE-2009-XXXX  Content-Type: application/rss+xml being rendered as active content

Previously, we rendered RSS and Atom feeds as XML.  Because most other browsers render these documents with dedicated feed previewers, some web sites do not sanitize their feeds for active content, such as
JavaScript.  In these cases, an attacker might be able to inject JavaScript into a target web site.

More info:
(This issue will be made public once a majority of users are up to date with the fix.)

Severity: Medium.  Most web sites are not affected because they do not include untrusted content in RSS or Atom feeds.

Credit: Inferno of


  • A victim would need to visit a page under an attacker's control.
  • The target web site would need to let the attacker inject JavaScript into an RSS or an Atom feed.

CVE-2009-XXXX  Same Origin Policy Bypass via getSVGDocument() method

The getSVGDocument method was lacking an access check, resulting in a cross-origin JavaScript capability leak.  A malicious web site operator could use the leaked capability to inject JavaScript into a target web site hosting an SVG document, bypassing the same-origin policy.

More info:
(This issue will be made public once a majority of users are up to date with the fix.)

Severity: High

Credit: Isaac Dawson


  • A victim would need to visit a page under an attacker's control.
  • The target web site would need to host an SVG document.

[Gd] More Languages, More Keyboards

| More

Google AJAX APIs Blog: More Languages, More Keyboards

The language APIs keep right on trucking, released recently are a handful of new translation languages, pairs, and keyboard layouts.

We've added the ability to use machine translation to or from the following languages:

  • Afrikaans
  • Belarusian
  • Icelandic
  • Irish
  • Macedonian
  • Malay
  • Persian
  • Swahili
  • Welsh
  • Yiddish

With the addition of the above the total count for language pair combination comes to a mind boggling 2550 pairs. In addition, we find the above additions exciting because, for the first time, African languages are available through the API and we now support all 23 Official European Union languages.

A few months ago we announce our virtual keyboard API and this month we've added nine new keyboard layouts:

  • Bulgarian
  • Czech
  • Greek
  • Hebrew
  • Hungarian - 101 layout
  • Slovak
  • Slovenian
  • Turkish - Q layout
  • Ukrainian - 101 layout

Here's a simple example of using the Slovak keyboard layout.


[Gd] Business of Social Applications

| More

OpenSocial API Blog: Business of Social Applications

In just over two years, social applications have greatly enhanced the user experience on social networks.  The number of app installs among social networks has climbed into the billions, creating an industry with hundreds of millions of dollars in annual revenue.  In this series of blog posts we are going to cover a wide range of topics such as business models, characteristics of successful apps, best practices, and metrics/stats.

Historic Overview

The industry started out with the introduction of the Facebook Application Platform but has since expanded to many other social networks all over the world thanks to the OpenSocial standard. Social networks that benefited from adoption of OpenSocial include MySpace, orkut, hi5, Yahoo, and more than 20 other social networks.  You can visit the OpenSocial
wiki for technical details, tutorials and samples, and check out the Containers page for a comprehensive listing of all the OpenSocial networks along with traffic and demographics data.

In a short period of time, social applications have gone through two distinct phases in terms of feature richness and monetization methods.  The first phase was dominated by apps providing simple social interactions such as self-expression or giving gifts.  The second phase exhibits increasingly sophisticated characteristics in apps such as game design patterns (levels, points, rankings, etc.) and user generated content.

Business Models Overview

The business models of social apps have closely followed the development of social apps.  In the first phase, the most revenue was generated from traditional display ads, supplemented by affiliate/referral offers and app installs.  This worked well for simple interactive apps with relatively low engagement because users quickly moved on to next thing, be it an ad, a link or another app install.  In this phase, less engagement by users in apps kept page view inventory in balance with available ads, resulting in high eCPM and profits for developers.

As apps matured in sophistication and richness, users became much more engaged with apps, less inclined to click on ads, links, or another app installs.  Higher engagement also generated much higher demand for page view inventory, outweighing the supply from ad networks and therefore reducing eCPM rates.  Out of necessity, app developers learned to take advantage of high user engagement by selling virtual goods, and in doing so, creating virtual economy business models that have provided the second boost to the ecosystems of social apps.

Stay tuned as we explore the business of social apps in upcoming posts on the OpenSocial blog.

Posted by Shawn Shen, Google Developer Relations

Monday, September 14, 2009

[Gd] Recommendations for webmaster friendly freehosts.

| More

Official Google Webmaster Central Blog: Recommendations for webmaster friendly freehosts.

Most of the recommendations we've made in the past are for individual webmasters running their own websites.  We thought we'd offer up some best practices for websites that allow users to create their own websites or host users' data, like Blogger or Google Sites.  This class of websites is often referred to as freehosts, although these recommendations apply to certain "non-free" providers as well.
  • Make sure your users can verify their website in website management suites such as Google's Webmaster Tools.

    Webmaster Tools provides your users with detailed reports about their website's visibility in Google.  Before we can grant your users access, we need to verify that they own their particular websites.  Verifying ownership of a site in Webmaster Tools can be done using a custom HTML file, a meta tag, or seamless integration in your system via Google Services for Websites.  Other website management suites such as Yahoo! Site Explorer and Bing Webmaster Tools may use similar verification methods; we recommend making sure your users can access each of these suites.

  • Choose a unique directory or hostname for each user.

    Webmaster Tools verifies websites based on a single URL, but assumes that users should be able to see data for all URLs 'beneath' this URL in the site URL hierarchy.  See our article on verifying subdomains and subdirectories for more information.  Beyond Webmaster Tools, many automated systems on the web--such as search engines or aggregators--expect websites to be structured in this way, and by doing so you'll be making it easier for those systems to find and organize your content.

  • Set useful and descriptive page titles.

    Let users set their own titles, or automatically set the pages on your users' websites to be descriptive of the content on that page.  For example, all of the user page titles should not be "Blogger: Create your free blog".  Similarly, if a user's website has more than one page with different content, they should not all have the same title: "User XYZ's Homepage".

  • Allow the addition of tags to a page.

    Certain meta tags are reasonably useful for search engines and users may want to control them.  These include tags with the name attribute of "robots""description""googlebot", "slurp", or "msnbot".  Click on the specific name attributes to learn more about what these tags do.

  • Allow your users to use third-party analytics packages such as Google Analytics.

    Google Analytics is free enterprise-class analytics software that can run on a website by just adding a snippet of JavaScript to the page.  If you don't want to allow users to add arbitrary JavaScript for security reasons, the Google Analytics code only changes by one simple ID.  If your let your users tell you their Google Analytics ID, you can set up the rest for them. Users get more value out of your service if they can understand their traffic better. For example, see Weebly's support page on adding Google Analytics. We recommend considering similar methods you can use for enabling access to other third-party applications.

  • Help your users move around.

    Tastes change.  Someone on your service might want to change their account name or even move to another site altogether.  Help them by allowing them to access their own data and by letting them tell search engines when they move part or all of their site via the use of 301 redirect destinations. Similarly, if users want to remove a page/site instead of moving it, please return a 404 HTTP response code so that search engines will know that the page/site is no longer around.  This allows users to use the urgent URL removal tool (if necessary), and makes sure that these pages drop out of search results as soon as possible.

  • Help search engines find the good content from your users.

    Search engines continue to crawl more and more of the web.  Help our crawlers find the best content across your site. Allow us to crawl users' content, including media like user-uploaded images.  Help us find users' content using XML Sitemaps.  Help us to steer clear of duplicate versions of the same content so we can find more of the good stuff your users are creating by creating only one URL for each piece of content when possible, and by specifying your canonical URLs when not.  If you're hosting blogs, create RSS feeds that we can discover in Google Blog Search.  If your site is down or showing errors, please return 5xx response codes.  This helps us avoid indexing lots of "We'll be right back" pages by letting crawlers know that the content is temporarily unavailable.

Can you think of any other best practices that you would recommend for sites that host users' data or pages?

Posted by Greg Grothaus, Staff Software Engineer, Search Quality Team

[Gd] Legacy gadget API deprecation

| More

Google Code Blog: Legacy gadget API deprecation

If you're a gadget developer, you've probably used the gadgets.* API, a re-namespaced and improved version of the original legacy, or _IG_*, gadgets API. The gadgets.* API has gained wide acceptance, both on Google and non-Google gadget containers, and is the standard API for gadget development.

However, there remains a number of gadgets using the legacy API, primarily gadgets developed for iGoogle, and the time to upgrade those gadgets is now. As of today, the legacy gadgets API is officially deprecated. For a period of one year, gadgets using the legacy API will continue to be supported, and function. After that, the legacy API will be turned off for the majority of Google containers (such as iGoogle, orkut, Gmail, and Calendar).

For more specifics on how the deprecation affects iGoogle developers, and details on coming resources to help in the API transition, check out this post on the iGoogle developer blog.

By Dan Holevoet, Developer Programs

[Gd] Migration to a Better Datastore

| More

Google App Engine Blog: Migration to a Better Datastore

At Google, we've learned through experience to treat everything with healthy skepticism. We expect that servers, racks, shared GFS cells, and even entire datacenters will occasionally go down, sometimes with little or no warning. This has led us to try as hard as possible to design our products to run on multiple servers, multiple cells, and even multiple datacenters simultaneously, so that they keep running even if any one (or more) redundant underlying parts go down. We call this multihoming. It's a term that usually applies narrowly, to networking alone, but we use it much more broadly in our internal language.

Multihoming is straightforward for read-only products like web search, but it's more difficult for products that allow users to read and write data in real time, like GMail, Google Calendar, and App Engine. I've personally spent a while thinking about how multihoming applies to the App Engine datastore. I even gave a talk about it at this year's Google I/O.

While I've got you captive, I'll describe how multihoming currently works in App Engine, and how we're going to improve it with a release next week. I'll wrap things up with more detail about App Engine's maintenance schedule.

Bigtable replication and planned datacenter moves

When we launched App Engine, the datastore served each application's data out of one datacenter at a time. Data was replicated to other datacenters in the background, using Bigtable's built-in replication facility. For the most part, this was a big win. It gave us mature, robust, real time replication for all datastore data and metadata.

For example, if the datastore was serving data for some apps from datacenter A, and we needed to switch to serving their data from datacenter B, we simply flipped the datastore to read only mode, waited for Bigtable replication to flush any remaining writes from A to B, then flipped the switch back and started serving in read/write mode from B. This generally works well, but it depends on the Bigtable cells in both A and B to be healthy. Of course, we wouldn't want to move to B if it was unhealthy, but we definitely would if B was healthy but A wasn't.

Planning for trouble

Google continuously monitors the overall health of App Engine's underlying services, like GFS and Bigtable, in all of our datacenters. However, unexpected problems can crop up from time to time. When that happens, having backup options available is crucial.

You may remember the unplanned outage we had a few months ago. We published a detailed postmortem; in a nutshell, the shared GFS cell we use went down hard, which took us down as well, and it took a while to get the GFS cell back up. The GFS cell is just one example of the extent to which we use shared infrastructure at Google. It's one of our greatest strengths, in my opinion, but it has its drawbacks. One of the most noticeable drawback is loss of isolation. When a piece of shared infrastructure has problems or goes down, it affects everything that uses it.

In the example above, if the Bigtable cell in A is unhealthy, we're in trouble. Bigtable replication is fast, but it runs in the background, so it's usually at least a little behind, which is why we wait for that final flush before switching to B. If A is unhealthy, some of its data may be unavailable for extended periods of time. We can't get to it, so we can't flush it, we can't switch to B, and we're stuck in A until its Bigtable cell recovers enough to let us finish the flush. In extreme cases like this, we might not know how soon the data in A will become available. Rather than waiting indefinitely for A to recover, we'd like to have the option to cut our losses and serve out of B instead of A, even if it means a small, bounded amount of disruption to application data. Following our example, that extreme recovery scenario would go something like this:

We give up on flushing the most recent writes in A that haven't replicated to B, and switch to serving the data that is in B. Thankfully, there isn't much data in A that hasn't replicated to B, because replication is usually quite fast. It depends on the nature of the failure, but the window of unreplicated data usually only includes a small fraction of apps, and is often as small as a few thousand recent puts, deletes, and transaction commits, across all affected apps.

Naturally, when A comes back online, we can recover that unreplicated data, but if we've already started serving from B, we can't automatically copy it over from A, since there may have been conflicting writes in B to the same entities. If your app had unreplicated writes, we can at least provide you with a full dump of those writes from A, so that your data isn't lost forever. We can also provide you with tools to relatively easily apply those unreplicated writes to your current datastore serving out of B.

Unfortunately, Bigtable replication on its own isn't quite enough for us to implement the extreme recovery scenario above. We use Bigtable single-row transactions, which let us do read/modify/write operations on multiple columns in a row, to make our datastore writes transactional and consistent. Unfortunately, Bigtable replication operates at the column value level, not the row level. This means that after a Bigtable transaction in A that updates two columns, one of the new column values could be replicated to B but not the other.

If this happened, and we switched to B without flushing the other column value, the datastore would be internally inconsistent and difficult to recover to a consistent state without the data in A. In our July 2nd outage, it was partly this expectation of internal inconsistency that prevented us from switching to datacenter B when A became unhealthy.

Megastore replication saves the day!

Thankfully, there's a solution to our consistency problem: Megastore replication. Megastore is an internal library on top of Bigtable that supports declarative schemas, multi-row transactions, secondary indices, and recently, consistent replication across datacenters. The App Engine datastore uses Megastore liberally. We don't need all of its features - declarative schemas, for example - but we've been following the consistent replication feature closely during its development.

Megastore replication is similar to Bigtable replication in that it replicates data across multiple datacenters, but it replicates at the level of entire entity group transactions, not individual Bigtable column values. Furthermore, transactions on a given entity group are always replicated in order. This means that if Bigtable in datacenter A becomes unhealthy, and we must take the extreme option to switch to B before all of the data in A has flushed, B will be consistent and usable. Some writes may be stuck in A and unavailable in B, but B will always be a consistent recent snapshot of the data in A. Some scattered entity groups may be stale, ie they may not reflect the most recent updates, but we'd at least be able to start serving from B immediately, as opposed waiting for A to recover.

To Paxos or not to Paxos

Megastore replication was originally intended to replicate across multiple datacenters synchronously and atomically, using Paxos. Unfortunately, as I described in my Google I/O talk, the latency of Paxos across datacenters is simply too high for a low-level, developer facing storage system like the App Engine datastore.

Due to that, we've been working with the Megastore team on an alternative: asynchronous, background replication similar to Bigtable's. This system maintains the write latency our developers expect, since it doesn't replicate synchronously (with Paxos or otherwise), but it's still consistent and fast enough that we can switch datacenters at a moment's notice with a minimum of unreplicated data.

Onward and upward

We've had a fully functional version of asynchronous Megastore replication for a while. We've been testing it heavily, working out the kinks, and stressing it to make sure it's robust as possible. We've also been using it in our internal version of App Engine for a couple months. I'm excited to announce that we'll be migrating the public App Engine datastore to use it in a couple weeks, on September 22nd.

This migration does require some datastore downtime. First, we'll switch the datastore to read only mode for a short period, probably around 20-30 minutes, while we do our normal data replication flush, and roll forward any transactions that have been committed but not fully applied. Then, since Megastore replication uses a new transaction log format, we need to take the entire datastore down while we drop and recreate our transaction log columns in Bigtable. We expect this to only take a few minutes. After that, we'll be back up and running on Megastore replication!

As described, Megastore replication will make App Engine much more resilient to hiccoughs and outages in individual datacenters and significantly reduce the likelihood of extended outages. It also opens the door to two new options which will give developers more control over how their data is read and written. First, we're exploring allowing reads from the non-primary datastore if the primary datastore is taking too long to respond, which could decrease the likelihood of timeouts on read operations. Second, we're exploring full Paxos for write operations on an opt-in basis, guaranteeing data is always synchronously replicated across datacenters, which would increase availability at the cost of additional write latency.

Both of these features are speculative right now, but we're looking forward to allowing developers to make the decisions that fit their applications best!

Planning for scheduled maintenance

Finally, a word about our maintenance schedule. App Engine's scheduled maintenance periods usually correspond to shifts in primary application serving between datacenters. Our maintenance periods usually last for about an hour, during which application serving is continuous, but access to the Datastore and memcache may be read-only or completely unavailable.

We've recently developed better visibility into when we expect to shift datacenters. This information isn't perfect, but we've heard from many developers that they'd like more advance notice from App Engine about when these maintenance periods will occur. Therefore, we're happy to announce below the preliminary maintenance schedule for the rest of 2009.

  • Tuesday, September 22nd, 5:00 PM Pacific Time (migration to Megastore)
  • Tuesday, November 3rd, 5:00 PM Pacific Time
  • Tuesday, December 1st, 5:00 PM Pacific Time

We don't expect this information to change, but if it does, we'll notify you (via the App Engine Downtime Notify Google Group) as soon as possible. The App Engine team members are personally dedicated to keeping your applications serving without interruption, and we realize that weekday maintenance periods aren't ideal for many. However, we've selected the day of the week and time of day for maintenance to balance disruption to App Engine developers with availability of the full engineering teams of the services App Engine relies upon, like GFS and Bigtable. In the coming months, we expect features like Megastore replication to help reduce the length of our maintenance periods.

Posted by Ryan Barrett, App Engine Team


[Gd] Beta Channel Update

| More

Google Chrome Releases: Beta Channel Update

The Windows Beta channel has been updated to

This release includes some minor fixes:
  • A fix for issue 3380 which caused the browser to lose focus in certain conditions after installing a theme.
  • Fix About box truncation in some locales when a new version is available.

You can install the current Beta channel release from

Anthony Laforge
Google Chrome Program Manager

[Gd] The more things change, the more they stay the same

| More

iGoogle Developer Blog: The more things change, the more they stay the same

The legacy gadgets API has had a storied life, as both the first version of the gadgets API that drove iGoogle, and the direct predecessor of the current gadgets.* API. As with many APIs there comes a time when we must say goodbye to the past, and embrace the present. The gadgets.* API has gained wide acceptance, both on Google and non-Google gadget containers, and is the standard API for gadget development. And so, as of today, the legacy gadgets API is officially deprecated.

I'll give you all a moment to wipe away the tears of sadness (or joy as the case may be). Now, here are the details:
  • The legacy API is officially deprecated as of today, September 14th.
  • For three months, the legacy API will continue in its current state.
  • On or around December 14th, any new gadget submissions to the iGoogle directory must be using the gadgets.*, in order to be accepted, but existing gadgets may continue to use the legacy API.
  • On the same date, the remaining inlined gadgets will be disabled.
  • Finally, one year after deprecation, September 14th, 2010, gadgets using the legacy API will cease to function on iGoogle, and the majority of other Google-owned gadget containers (such as orkut, Gmail, and Calendar).
  • Reminders will be posted when these important dates approach.
We're also working on some tools to aid you in the transition: a gadget migration tool that will parse your existing gadget and convert legacy calls to gadgets.*, and a migration guide for developers who wish to migrate their gadgets by hand. Watch for announcements on these tools in the next few weeks.

For most gadgets, the changes should be simple to implement. For each _IG_* method, there is usually a direct equivalent gadgets.* method. For instance, _IG_AdjustIFrameHeight maps directly to gadgets.window.adjustHeight, and performing a find and replace is sufficient. In a small subset of cases, multiple _IG_* methods map to a single gadgets.* method. For instance, _IG_FetchContent and _IG_FetchXmlContent both map to with different parameters. Developers should refer to the relevant section of the developer's guide to find gadgets.* equivalents.

If you have any questions, as always, feel free to inquire in the iGoogle Developer Forum.

Posted by Dan Holevoet, Developer Programs