Google Image Classification and Landmarks

No Comments

Sharing is caring!

Picture Classification up to now

Again in 2008, I used to be writing about how a search engine would possibly study from picture databases like Flickr, and the way individuals label photos there in a publish I wrote referred to as, Group Tagging and Rating in Pictures of Landmarks

In one other publish that covers the Flickr picture classification Landmark work, Faces and Landmarks: Two Steps In direction of Smarter Picture Searches, I discussed a part of what the Yahoo research uncovered:

Utilizing robotically generated location information, and software program that may cluster collectively comparable photos to study photos once more goes past simply wanting on the phrases related to photos to study what they’re about.

That’s utilizing metadata from photos in a picture assortment, which could be very totally different from what Google is doing on this publish about figuring out landmarks within the publish, How Google Could Interpret Queries Primarily based on Areas and Entities (Examined), the place it’d determine landmarks primarily based upon a data of their precise location.

Extra Current Picture Classification of Landmarks

I point out these earlier posts as a result of I wished to share what I had written about landmarks, earlier than pointing to newer research from Google about how they could acknowledge landmarks, a yr other than one another, with one being a followup to the opposite.

The primary of those papers, Google-Landmarks: A New Dataset and Problem for Landmark Recognition, begins out by telling us about an issue that wants fixing:

Picture classification know-how has proven outstanding enchancment over the previous few years, exemplified partially by the Imagenet classification problem, the place error charges proceed to drop considerably yearly. As a way to proceed advancing the state-of-the-art in pc imaginative and prescient, many researchers at the moment are placing extra concentrate on fine-grained and instance-level recognition issues – as a substitute of recognizing basic entities akin to buildings, mountains and (after all) cats, many are designing machine studying algorithms able to figuring out the Eiffel Tower, Mount Fuji or Persian cats. Nevertheless, a big impediment for analysis on this space has been the dearth of huge annotated datasets.

A yr later, Google labored to enhance the dataset that was getting used for picture classification when figuring out landmarks, and up to date the dataset that they’d created the yr earlier than, as they inform us in,Asserting Google-Landmarks-v2: An Improved Dataset for Landmark Recognition & Retrieval A part of the trouble behind that work got here from getting a whole lot of assist as described within the weblog publish asserting it:

A specific downside in making ready Google-Landmarks-v2 was the technology of occasion labels for the landmarks represented since it’s just about unattainable for annotators to acknowledge all the a whole lot of 1000’s of landmarks that might doubtlessly be current in a given picture. Our resolution to this downside was to crowdsource the landmark labeling via the efforts of a world-spanning neighborhood of passion photographers, every conversant in the landmarks of their area.

Google Patent for Picture Classification when Figuring out Landmarks in Picture Collections

Google was not too long ago granted a patent that focuses on figuring out in style landmarks in giant digital picture collections. Contemplating Google operates Google images, that makes a whole lot of sense. The landmark identification efforts at Flickr sound slightly much like this effort on Google’s half. The patent does goal a selected downside which it tells us is:

Nevertheless, there isn’t a recognized system that may robotically extract data akin to the most well-liked vacationer locations from these giant collections. As quite a few new pictures are added to those digital picture collections, it might not be possible for customers to manually label the images in an entire and constant method that can improve the usefulness of these digital picture collections. What is required, subsequently, are methods and strategies that may robotically determine and label in style landmarks in giant digital picture collections.

A few of it does sound much like the Flickr efforts the place it talks about working to populate and replace “a database of photos of landmarks together with geo-clustering geo-tagged photos in keeping with geographic proximity to generate a number of geo-clusters, and visual-clustering the a number of geo-clusters in keeping with picture similarity to generate a number of visible clusters.”

How would possibly this play into picture classification and search involving landmarks?

The patent describes the way it may match into searches, with the next steps:

  • Enhancing consumer queries to retrieve photos of landmarks, together with the phases of receiving a consumer question
  • Figuring out a number of set off phrases within the consumer question
  • Choosing a number of corresponding tags from a landmark database similar to the a number of set off phrases
  • Supplementing the consumer question with the a number of corresponding tags, producing a supplemented consumer question

Set off phrases showing in queries is fascinating.

The patent additionally tells us that it may additionally contain a technique of robotically tagging a brand new digital picture, which might additionally cowl:

  • Evaluating the brand new digital picture to photographs in a landmark picture database, whereby the landmark picture database includes visible clusters of photos of a number of landmarks
  • tagging the brand new digital picture with a minimum of one tag primarily based on a minimum of one among mentioned visible clusters

The patent is:

Computerized discovery of in style landmarks
Inventors: Fernando A. Brucher, Ulrich Buddemeier, Hartwig Adam and Hartmut Neven
Assignee: Google LLC
US Patent: 10,289,643
Granted: Could 14, 2019
Filed: October 3, 2016


In a single embodiment the current invention is a technique for populating and updating a database of photos of landmarks together with geo-clustering geo-tagged photos in keeping with geographic proximity to generate a number of geo-clusters, and visual-clustering the a number of geo-clusters in keeping with picture similarity to generate a number of visible clusters. In one other embodiment, the current invention is a system for figuring out landmarks from digital photos, together with the next parts: a database of geo-tagged photos; a landmark database; a geo-clustering module; and a visible clustering module. In different embodiments, the current invention could also be a technique of enhancing consumer queries to retrieve photos of landmarks or a technique of robotically tagging a brand new digital picture with textual content labels.

Even Smarter Picture Classification of Landmarks

This method seems to be able to find extremely popular landmarks in picture collections throughout the online and storing these in a landmark database, the place it’d geocluster these. It’s fascinating occupied with this effort. If Google May use these landmark photos in Picture Search Outcomes, it could not cease picture classification at that time

I not too long ago wrote about Google Picture Search Labels Turning into Extra Semantic? the place we had been instructed in an up to date Google Patent that photos had been being labeled primarily based upon an ontology associated to the subjects of these photos. A Google picture seek for a landmark like The Washington Monument exhibits plenty of picture classification labels on the prime of the outcomes that may be clicked on if you wish to slim down the outcomes to particular points of these monuments.

So, picture classification could embody particular monuments, after which much more slim classifications, like having the next labels utilized to the Washington Monument:

Reflecting Pool
Lincoln Memorial
Washington DC
Remark deck
Nationwide Mall

So, Google could have smarter picture classification on the subject of landmarks, however it’s labeling them in order that they’re extra significant, too.

Sharing is caring!

About us and this blog

We are a digital marketing company with a focus on helping our customers achieve great results across several key areas.

Request a free quote

We offer professional SEO services that help websites increase their organic search score drastically in order to compete for the highest rankings even when it comes to highly competitive keywords.

More from our blog

See all posts

Leave a Comment