(Cross-posted on the Inside Search blog)
Webmaster level: All
For many months, we’ve been focused on trying to return high-quality sites to users. Earlier this year, we rolled out our “Panda” change for searches in English around the world. Today we’re continuing that effort by rolling out our algorithmic search improvements in different languages. Our scientific evaluation data show that this change improves our search quality across the board and the response to Panda from users has been very positive.
For most languages, this change impacts typically 6-9% of queries to a degree that a user might notice. This is distinctly lower than the initial launch of Panda, which affected almost 12% of English queries to a noticeable amount. We are launching this change for all languages except Chinese, Japanese, and Korean, where we continue to test improvements.
For sites that are affected by this algorithmic change, we have a post providing guidance on how Google searches for high-quality sites. We also have webmaster forums in many languages for publishers who wish to give additional feedback and get advice. We’ll continue working to do the right thing for our users and serve them the best results we can.
Posted by Amit Singhal, Google Fellow
Feeds
5540 items (5015 unread) in 3 feeds
Google
(325 unread)
-
Official Google Webmaster Central Blog (325 unread)
SEO
(4690 unread)
-
Emily Chang (347 unread)
-
Search Engine Roundtable
(4343 unread)
Items by A Googler
Official Google Webmaster Central Blog
-
High-quality sites algorithm launched in additional languages
Posted: August 12th, 2011, 6:00am CEST by A Googler
-
Introducing the Google Webmaster Team
Posted: May 11th, 2011, 11:09am CEST by A Googler
We’re pleased to introduce the Google Webmaster Team as contributors to the Webmaster Central Blog. As the team responsible for tens of thousands of Google’s informational web pages, they’re here to offer tips and advice based on their experiences as hands-on webmasters.
Back in the 1990s, anyone who maintained a website called themselves a “webmaster” regardless of whether they were a designer, developer, author, system administrator, or someone who had just stumbled across GeoCities and created their first web page. As the technologies changed over the years, so did the roles and skills of those managing websites.
Around 20 years after the word was first used, we still refer to ourselves as the Google Webmaster Team because it’s the only term that really covers the wide variety of roles that we have on our team. Although most of us have solid knowledge of HTML, CSS, JavaScript and other web technologies, we also have specialists in design, development, user experience, information architecture, system administration, and project management.
Part of the Google Webmaster Team, Mountain ViewIn contrast to the Google Webmaster Central Team—which mainly focuses on helping webmasters outside of Google understand web search and how things like crawling and indexing affect their sites—our team is responsible for designing, implementing, optimizing and maintaining Google’s corporate pages, informational product pages, landing pages for marketing campaigns, and our error page. Our team also develops internal tools to increase our productivity and help to maintain the thousands of HTML pages that we own.
We’re working hard to follow, challenge and evolve best practices and web standards to ensure that all our new pages are produced to the highest quality and provide the best user experience, and we’re constantly evaluating and updating our legacy pages to ensure their deprecated HTML isn’t just left to rot.
We want to share our work and experiences with other webmasters, so we recently launched our @GoogleWebTeam account on Twitter to keep our followers updated on the latest news about our projects, web standards, and anything else which may be of interest to other webmasters, web designers and web developers. We’ll be posting here on the Webmaster Central Blog when we want to share anything longer than 140 characters.
Before we share more details about our processes and experiences, please let us know if there’s anything you’d like us to specifically cover by leaving a comment here or by tweeting @GoogleWebTeam.
Posted by Tony Ruscoe, Google Webmaster Team
-
An update on Google Video: Finding an easier way to migrate Google Video content to YouTube
Posted: April 22nd, 2011, 5:33pm CEST by A Googler
[Cross-posted on the YouTube Blog]
Last week we sent an email to Google Video users letting them know we would be ending playbacks of Google Videos on April 29 and providing instructions on how to download videos currently hosted on the platform. Since then we’ve received feedback from you about making the migration off of Google Video easier. We work every day to make sure you have a great user experience and should have done better. Based on your feedback, here’s what we’re doing to fix things.
Google Video users can rest assured that they won't be losing any of their content and we are eliminating the April 29 deadline. We will be working to automatically migrate your Google Videos to YouTube. In the meantime, your videos hosted on Google Video will remain accessible on the web and existing links to Google Videos will remain accessible. If you want to migrate to YouTube now, here’s how you do it:- We’ve created an “Upload Videos to YouTube” option on the Google Video status page. To do this, you’ll need to have a YouTube account associated with your Google Video account (you can create one here). Before doing this you should read YouTube’s Terms of Use and Copyright Policies. If you choose this option, we’ll do our best to ensure your existing Google Video links continue to function.
If you’d prefer to download your videos from Google Video, that option is still available.
As we said nearly two years ago, the team is now focused on tackling the tough challenge of video search. We want to thank the millions of people around the world who have taken the time to create and share videos on Google Video. We hope today's improvements will help ease your transition to another video hosting service.
Posted by Mark Dochtermann, Engineering Manager -
High-quality sites algorithm goes global, incorporates user feedback
Posted: April 11th, 2011, 2:00pm CEST by A Googler
Over a month ago we introduced an algorithmic improvement designed to help people
find more high-quality sites in search. Since then we’ve gotten a lot of positive responses about the change: searchers are finding better results, and many great publishers are getting more traffic.
Today we’ve rolled out this improvement globally to all English-language Google users, and we’ve also incorporated new user feedback signals to help people find better search results. In some high-confidence situations, we are beginning to incorporate data about the sites that users block into our algorithms. In addition, this change also goes deeper into the “long tail” of low-quality websites to return higher-quality results where the algorithm might not have been able to make an assessment before. The impact of these new signals is smaller in scope than the original change: about 2% of U.S. queries are affected by a reasonable amount, compared with almost 12% of U.S. queries for the original change.
Based on our testing, we’ve found the algorithm is very accurate at detecting site quality. If you believe your site is high-quality and has been impacted by this change, we encourage you to evaluate the different aspects of your site extensively. Google's quality guidelines provide helpful information about how to improve your site. As sites change, our algorithmic rankings will update to reflect that. In addition, you’re welcome to post in our Webmaster Help Forums. While we aren’t making any manual exceptions, we will consider this feedback as we continue to refine our algorithms.
We will continue testing and refining the change before expanding to additional languages, and we’ll be sure to post an update when we have more to share.
Posted by Amit Singhal, Google Fellow -
Using named anchors to identify sections on your pages
Posted: September 25th, 2009, 2:29pm CEST by A Googler
We just announced a couple of new features on the Official Google Blog that enable users to get to the information they want faster. Both features provide additional links in the result block, which allow users to jump directly to parts of a larger page. This is useful when a user has a specific interest in mind that is almost entirely covered in a single section of a page. Now they can navigate directly to the relevant section instead of scrolling through the page looking for their information.
We generate these deep links completely algorithmically, based on page structure, so they could be displayed for any site (and of course money isn't involved in any way, so you can't pay to get these links). There are a few things you can do to increase the chances that they might appear on your pages. First, ensure that long, multi-topic pages on your site are well-structured and broken into distinct logical sections. Second, ensure that each section has an associated anchor with a descriptive name (i.e., not just "Section 2.1"), and that your page includes a "table of contents" which links to the individual anchors. The new in-snippet links only appear for relevant queries, so you won't see it on the results all the time — only when we think that a link to a section would be highly useful for a particular query.
Posted by Raj Krishnan, Snippets Team -
Improved Flash indexing
Posted: June 30th, 2008, 11:31pm CEST by A Googler
Tags: crawling and indexing
Written by Ron Adler, Janis Stipins, and Maile Ohye
We've received numerous requests to improve our indexing of Adobe Flash files. Today, Ron Adler and Janis Stipins—software engineers on our indexing team—will provide us with more in-depth information about our recent announcement that we've greatly improved our ability to index Flash.
Q: Which Flash files can Google better index now?
We've improved our ability to index textual content in SWF files of all kinds. This includes Flash "gadgets" such as buttons or menus, self-contained Flash websites, and everything in between.
Q: What content can Google better index from these Flash files?
All of the text that users can see as they interact with your Flash file. If your website contains Flash, the textual content in your Flash files can be used when Google generates a snippet for your website. Also, the words that appear in your Flash files can be used to match query terms in Google searches.
In addition to finding and indexing the textual content in Flash files, we're also discovering URLs that appear in Flash files, and feeding them into our crawling pipeline—just like we do with URLs that appear in non-Flash webpages. For example, if your Flash application contains links to pages inside your website, Google may now be better able to discover and crawl more of your website.
Q: What about non-textual content, such as images?
At present, we are only discovering and indexing textual content in Flash files. If your Flash files only include images, we will not recognize or index any text that may appear in those images. Similarly, we do not generate any anchor text for Flash buttons which target some URL, but which have no associated text.
Also note that we do not index FLV files, such as the videos that play on YouTube, because these files contain no text elements.
Q: How does Google "see" the contents of a Flash file?
We've developed an algorithm that explores Flash files in the same way that a person would, by clicking buttons, entering input, and so on. Our algorithm remembers all of the text that it encounters along the way, and that content is then available to be indexed. We can't tell you all of the proprietary details, but we can tell you that the algorithm's effectiveness was improved by utilizing Adobe's new Searchable SWF library.
Q: What do I need to do to get Google to index the text in my Flash files?
Basically, you don't need to do anything. The improvements that we have made do not require any special action on the part of web designers or webmasters. If you have Flash content on your website, we will automatically begin to index it, up to the limits of our current technical ability (see next question).
That said, you should be aware that Google is now able to see the text that appears to visitors of your website. If you prefer Google to ignore your less informative content, such as a "copyright" or "loading" message, consider replacing the text within an image, which will make it effectively invisible to us.
Q: What are the current technical limitations of Google's ability to index Flash?
There are three main limitations at present, and we are already working on resolving them:
1. Googlebot does not execute some types of JavaScript. So if your web page loads a Flash file via JavaScript, Google may not be aware of that Flash file, in which case it will not be indexed.
2. We currently do not attach content from external resources that are loaded by your Flash files. If your Flash file loads an HTML file, an XML file, another SWF file, etc., Google will separately index that resource, but it will not yet be considered to be part of the content in your Flash file.
3. While we are able to index Flash in almost all of the languages found on the web, currently there are difficulties with Flash content written in bidirectional languages. Until this is fixed, we will be unable to index Hebrew language or Arabic language content from Flash files.
We're already making progress on these issues, so stay tuned!
Update: Everyone, thanks for your great questions and feedback. Our focus is to improve search quality for all users, and with better Flash indexing we create more meaningful search results. Listed below, we’ve also answered some of the most prevalent questions. Thanks again!
Flash site in search results before improvements
Flash site after improved indexing, querying [nasa deep impact animation]
Helping us access and index your Flash files@fintan: We verified with Adobe that the textual content from legacy sites, such as those scripted with AS1 and AS2, can be indexed by our new algorithm.
@andrew, jonny m, erichazann, mike, ledge, stu, rex, blog, dis: For our July 1st launch, we didn't enable Flash indexing for Flash files embedded via SWFObject. We're now rolling out an update that enables support for common JavaScript techniques for embedding Flash, including SWFObject and SWFObject2.
@mike: At this time, content loaded dynamically from resource files is not indexed. We’ve noted this feature request from several webmasters -- look for this in a near future update.
Interaction of HTML pages and Flash@captain cuisine: The text found in Flash files is treated similarly to text found in other files, such as HTML, PDFs, etc. If the Flash file is embedded in HTML (as many of the Flash files we find are), its content is associated with the parent URL and indexed as single entity.
@jeroen: Serving the same content in Flash and an alternate HTML version could cause us to find duplicate content. This won't cause a penalty -- we don’t lower a site in ranking because of duplicate content. Be aware, though, that search results will most likely only show one version, not both.
@All: We’re trying to serve users the most relevant results possible regardless of the file type. This means that standalone Flash, HTML with embedded Flash, HTML only, PDFs, etc., can all have the potential to be returned in search results.
Indexing large Flash files@dsfdgsg: We’ve heard requests for deep linking (linking to specific content inside file) not just for Flash results, but also for other large documents and presentations. In the case of Flash, the ability to deep link will require additional functionality in Flash with which we integrate.
@All: The majority of the existing Flash files on the web are fine in regard to filesize. It shouldn’t be too much of a concern.
More details about our Flash indexing algorithm@brian, marcos, bharath: Regarding ActionScript, we’re able to find new links loaded through ActionScript. We explore Flash like a website visitor does, we do not decompile the SWF file. Unless you're making ActionScript visible to users, Google will not expose ActionScript code.
@dlocks: We respect rel="nofollow" wherever we encounter it in HTML. -
Becoming Social
Posted: May 13th, 2008, 1:09am CEST by A Googler
Posted by Mussie Shore, Product Manager
Wondering how to make your site more social? We'd like to make it easier for you, which is why earlier tonight at Campfire One at the Googleplex, we announced a preview release of Google Friend Connect.
Google Friend Connect is a service that that helps you grow traffic by enabling you to easily provide social features for your visitors. Just add a snippet of code, and, voilà, you can add social functionality -- picking and choosing from built-in functionality like user registration, invitations, members gallery, message posting, and reviews, as well as third-party applications built by the OpenSocial developer community.
Social features can generate buzz and traffic to your pages. Using Google Friend Connect on your site, your visitors will be able to see, invite, and interact with their friends from existing sources of friends, including Facebook, Google Talk, hi5, LinkedIn, orkut, Plaxo, and others. And you'll be able to more actively engage your visitors by adding social features from a growing gallery of social applications.
We've heard from many site owners that even though their sites aren't social networks, they'd still like them to be social. Whether your site sells car parts or dishes out great guacamole recipes -- like the sample site in the YouTube video above -- you can visit [www.google.com] or read more on the Official Google Blog to learn about Google Friend Connect. Right now, the preview is available for only a few sites, but soon we'll give the green light to even more. Sign up now to be on the wait list. -
Cross-submissions via robots.txt on Sitemaps.org
Posted: February 27th, 2008, 7:00pm CET by A Googler
Posted by Prashanth Koppula, Product Manager
Last spring, the Sitemaps protocol was expanded to include the autodiscovery of Sitemaps using robots.txt to let us and other search engines supporting the protocol know about your Sitemaps. We subsequently also announced support for Sitemap cross-submissions using Google Webmaster Tools, making it possible to submit Sitemaps for multiple hosts on a single dedicated host. So it was only time before we took the next logical step of marrying the two and allowing Sitemap cross-submissions using robots.txt. And today we're doing just that.
We're making it easier for webmasters to place Sitemaps for multiple hosts on a single host and then letting us know by including the location of these Sitemaps in the appropriate robots.txt.
How would this work? Say for example you want to submit a Sitemap for each of the two hosts you own, www.example.com and host2.google.com. For simplicity's sake, you may want to host the Sitemaps on one of the hosts, www.example.com. For example, if you have a Content Management System (CMS), it might be easier for you to change your robots.txt files than to change content in a directory.
You can now exercise the cross-submission support via robots.txt (by letting us know the location of the Sitemaps):
a) The robots.txt for www.example.com would include:
Sitemap: http://www.example.com/sitemap-www-example.xml
b) And similarly, the robots.txt for host2.google.com would include:
Sitemap: http://www.example.com/sitemap-host2-google.xml
By indicating in each individual host's robots.txt file where that host's Sitemap lives you are in essence proving that you own the host for which you are specifying the Sitemap. And by choosing to host all of the Sitemaps on a single host, it becomes simpler to manage your Sitemaps.
We are making this announcement today on Sitemaps.org as a joint effort. To see what our colleagues have to say, you can also check out the blog posts published by Yahoo! and Microsoft.
