Google Co-op — An Intro & Some Insider Hacks

http://www.google.com/coop

So what is it? It’s called Google Co-op, a platform which enables users to build their own vertical search engines and make money off the advertisements. It provides a clean, easy interface for simple site restrictions (like what Yahoo! Search Builder and Live Macros offer) plus a number of power user features for tweaking the search results. The user has control over the look and feel (to embed the search box on their own site), can rank results, and even (multi) tag sites to let viewers filter out results by category.

But talk is cheap. So let me show you some examples of what you can do with Co-op:

http://vik.singh.googlepages.com/techstuff

This is a technology specific search engine, which lets users refine results based off Google Topics (global labels which anyone can annotate with). Basically, I was lazy here. I didn’t feel like multi-tagging sites/domains individually, so instead I just collected a laundry list of popular technology site domains in a flat file and pasted it into Google Co-op’s Custom Search Engine control panel/sites page. In addition, something I think is really useful, Google Co-op allows users to bulk upload links from OPML files. So, to make my life easier when building this, I uploaded Scoble’s and Matt Cutt’s OPML’s. Tons of great links there (and close to 1000 total). Then I clicked on the ‘filter results to just the sites I listed’ option (which I recommend you use since if you muddle your results with normal Google web search’s you typically won’t see your results popping up on the first page of results despite the higher priority level promise for hand chosen sites). To enable the filters you see on the results page (Reviews, Forums, Shopping, Blogs, etc.), I did an intersection with the background label of my search engine and the Google Topics labels. How do you that? The XML context configuration exposes a <BackgroundLabels> tag. Any labels listed in the BackgroundLabels block will be AND’ed (how cool is that). So I added the label of my search engine (each search engine has a unique background label – it can be found bolded on the Advanced Tab page) and a Google Topic label (News, Reviews, Stores, Shopping_Comparison, Blogs, Forums, etc.) in the BackgroundLabels XML block. I made a separate XML context file for each Google Topic intersection. By doing this, I didn’t have to tag any of my results and was still able to provide search filters. Google Topics does most of the hardwork and gives me search refinements for free!

But say you’re not lazy. Here’s an example of what you can do with multi-tagging and refinements.

http://vik.singh.googlepages.com/machinelearningsearch2

This one is more of a power user example – notice the refinements onebox on the search results page, and the labels with “>>” at the end. These labels redirect to another label hierarchy (a hack, I used the label redirect XML option to link to other custom search engine contexts – basically I’m nesting search engines here)

Now, say you want to get fancy with the search results presentation. Here’s a way to do it with Google’s Ajax Search API:

http://www.google.com/uds/samples/cse/index.html

Thanks to Mark Lucovsky and Matt Wytock for developing that great example.
For more information about how to use the Ajax Search API with Custom Search, please take a look at this informative post: http://googleajaxsearchapi.blogspot.com/2006/10/custom-search-engine-support.html

While writing this blog post, I realized it would take me forever to go over the number of tricks one can pull with Co-op. Instead, I’ll summarize some of the big selling point features to encourage everyone to start hacking away. Also, to help jump start power users, I’ve linked the XML files I used to make my featured search examples at the bottom of this post.

Key Feature Summary (in no particular order):

and much much more (especially for power users).

If you need a search engine for your site, and your content has been indexed by Google, then seriously consider using this rather than building your own index – or worse, using the crappy full-text functions available in relational databases.

Here are my XML files:

ml-context.xml

ml-pop-context.xml

ml-complx-context.xml

ml-source-context.xml

tech-stuff-context.xml

techreviews.xml

techforums.xml

techshopping.xml

techblogs.xml

technews.xml

tech-stuff-scoble-annotations.xml

tech-stuff-matcutts-annotations.xml

Happy Coop hacking!

About these ads

55 Comments

Filed under Blog Stuff, Databases, Google, Tagging, Tutorial

55 responses to “Google Co-op — An Intro & Some Insider Hacks

  1. ok, looks wonderful, but the limited page width of googlepages does not allow the ads to appear… (same on my trial, http://gtziralis.googlepages.com/search)
    any ideas on that?
    thanx in advance

  2. DeSantes

    In your ajax example, where in the code would you put your adsense info?

  3. Pingback: Google’s new personalized search engine talk of town « Scobleizer - Tech Geek Blogger

  4. Pingback: Google Co-op: A Big Deal « Midspot

  5. Pingback: Google Custom Search Engines « //engtech

  6. Pingback: help.net : Google Co-op — An Intro & Some Insider Hacks

  7. Hi George – Google is working on fixing that issue. Currently the size of the iframe is hardcoded but they plan on making it resize dynamically to adjust for different page layouts. For now, try going to the Code tab and selecting Top and Bottom for the ads presentation option.

  8. Vik,

    great work. I was reading your post on Techcrunch and I thought you came to the table with some real value. I gave your AI search engine a spin, very impressive. Regards

  9. looks some good and interesting work here

  10. Pingback: WWWScope » Google Custom Search - more features than you think

  11. Pingback: inventive title : Search MSDN with Google Co-op

  12. Pingback: Jimmy Daniels

  13. Pingback: Naik’s News » Google’s new personalized search engine talk of town

  14. Pingback: Google Custom Search Engine « Venture Skills Blog

  15. defleshed

    Seems really cool. But how far can you edit? What does Google profit on this`?

  16. ventureskills

    The Custom search engine pages on Google allow some limited customizing colour etc, but for more advanced customising try using the Google AJAX search combined with Custom search engine I have put a quick how to on our blog to help people get started.

    http://ventureskills.wordpress.com/2006/10/26/google-custom-search-engine/

    Google have also put up a couple of examples on there site, to customise the output you need to download the css which I don’t go into on that post.

    Tim

  17. In The Stromboli Project we´ve prepaired an erotic stories search engine!

    That´s in http://searcheroticstories.thestromboliproject.com

    Enjoy it!
    ;)

  18. How does one embed the search results in a worpdress site?

  19. Vik,
    Thanks for sharing and explaing some of the features.

  20. sid

    i am trying to re-design the refine div,
    is that possible like in the techstuff page?

  21. ventureskills

    Allyn – if your looking to embed in wordpress.com site, you will find it very difficult if not impossible but on your own instance of wordpress, you can easily add it as a widget or directly into the template.

    Sid – what do you mean refine the div? if your using the ajax search to intergrate it into your own site copy googles css from there site onto your own and make changes to that.

  22. Guillermo

    Great Article! I just have one question: It is possible to refine the search before the results appears? I mean if the search box can be tunning to perform the search in one (or several, but not all) refinement categories?
    I am looking the answer of this, but just cant find it…
    Thank you!

  23. First
    Thakns ventureskills for responding to the previous two questions

    Guillermo – Yes you can refine from the get go. A couple of ways of doing this:
    (1) Treat each refinement as a separate search context. Then just put their background labels in the block in the xml context for your primary search.
    (2) Append “more:” operators to the user’s search query.
    (3) Actually, just check the advanced documentation under Background Labels. You can place refinement labels in the block directly without having to abstract each one as a separate search.

  24. Guillermo

    Thank you so much, zooie!!! For the great answer and how fast you reply. I will look for what you sugest. Thanks!!!!

  25. Guillermo

    Ok, the first two methods definitely will work. The third method sounds very nice, but I only find information to use the background labels in the xml to configure the site, not to perform the search… Is that correct?. Actually I think that the question should be: It is possibly to specify the refine labels in the code of the search box? Is that can be done with the background labels in the javascript code, i didnt find the way… Sorry for bother you…

  26. http://www.google.com/coop/profile?user=015417630718833783855

    I would like to combine all my search engines into one “meta” search engine for stained glass.

    How can I do that?

  27. Guillermo – I’m not sure what you’re referring to by the ‘javascript code’. Refer to the advanced documentation that describes backgrounds labels (the XML context). If for some reason that’s not working for you, then try the first two options (the second being the easiest, just append ‘more:” to the search query).

    Stellula – Hmm … there are a couple of ways. First is to redo your annotations and add them to just one search engine context (you just need to download the annotations in XML format from the Advanced tab and the change the context string so that it references the primary search context). Another way is to provide all these search engines as refinements (like I did in techsearch). You can create refinements for each meta search engine which label redirects (the label here should be the background label string specified in the advanced tab for that particular search engine).

  28. My Google Co-op example:
    You Search for unprotected live webcam streams found through a variety of clever search techniques done with the Google Co-op custom search engine tool.

    http://www.camhacker.com

  29. Pingback: Using Google Co-Op for Current Awareness? « alt+tab+lib

  30. Hi everyone.
    We created a free directory for your cse at http://www.coopdir.com
    Only cse created with google co-op will be accepted.
    Thanks

    ps Nice tips.

  31. Pingback: Tag You Google Custom Search! - SEO News Blog

  32. Luke

    Vik/Zooie, your post is an absolute treasure trove of great CSE info….

    I don’t have the tech skills to pull 95% of this XML or Ajax stuff off…. but if I get the time to try, the ideas, examples, and files you’ve posted here will be critical….

    Many thanks…. Your search engines and website rock.

    Here’s my CSE Attempt, by the way:

    Film & Performing Arts

    http://google.com/coop/cse?cx=012633772325726105531%3Avkzo3156diy

    (I’m also planning on adding nested refinement labels, eventually, as well as related Google Base, Blog, and News integration, although it’s likely way beyond my technical skill…)

    cheers,

    Luke
    Writer/Editor, NYC

  33. Svyatoslav Lavryk

    Guys, what are the possibilities to tweak Google CSE programmatically ? Can I create CSE using any API ? Can I programmatically manage list of sites, indexed by specific CSE?
    I know that all this stuff can be done on Google site using UI, but I need that to be done on my site programmatically.

  34. Svyatoslav – Yes. Refer to the advanced documentation. You can specify your refinements and annotations in XML (just use a program to write that) and upload them accordingly in the Advanced tab. Also, stay tuned for more support.

  35. Pingback: AI3:::Adaptive Information » Blog Archive » Authoritative SweetSearch Semantic Web and Web 2.0 Custom Search Engine

  36. Hi Vik,

    Thanks for this fabulous information. I tried to incorporate Google co-op, it worked fine for couple of days on my website but just stopped working after 5 days. I mean its there on my website, configured to display results from my website but it wont display the results anymore but just ads.

    It used to give me results for the same query from within my website but it does does not anymore. At most what it returns is the home page URL. Pretty strange isnt it. Need your advice please.

    Just to let you know, I had made a lot many changes in just 3-4 days within my website. In title tag and even in the body just to practice some SEO stuff. Could you please help me out.

    Many thanks,

    Chandan

  37. Hi Chandan – My advice is to post your issue on the Google Co-op discussion forum (they are _very_ good at responding quickly to matters like these):

    http://groups.google.com/group/google-co-op/topics

    Tell me if they help. If not, email me and I’ll work with you through the steps.

  38. Hi. I just set up a Google custom search at http://www.boatr.org.uk for the boating community. It took far less time to set up the website and incorporate Google co-op, adsense and analytics than it did to do the research into which 200+ websites to incorporate. I’m getting some good feedback from the users and hope to play around more soon. Thanks for the ideas!

  39. Is there a way to customize COOP to display results on your site without showing Google Ads?

  40. I think if you’re a special non-profit group you can avoid displaying advertisements. You can check out the TOS or better just email the Co-op discussion forum to see if you can get leeway for this.

  41. Nice articles you have on your site. Just wondering about the strength of google COOP and decided to create a web search site with it and wanted to share this with you and your users about how interesting and powerful the google COOP is.

  42. New Google CSE searchengine lauched! SearchHacker, Quick searches for many different file types like:
    mp3. mid, wav, au, wmv, avi, mpg, mov, mpeg, doc, xls, pds, txt, jpg, rar, torrent… you name it, can come in very handy! Have a look: http://www.searchhacker.com

  43. Check this out http://www.googlepowersearch.com.
    I created GooglePowerSearch so you can power search for Video, News, Maps, Images and more…
    Google Power Search helps to unleash the built in power of Googles special features.
    Using Google Power Search you are able to get better-targeted results.
    Check out Google Power Search and let me know what you think.
    Thanks
    Steve

  44. Sam

    Checkout http://www.unitedtorrents.com – The power of google for all your torrent searches

    Searching 1200+ torrent websites with over 6,000,000 torrents indexed. The ultimate torrent search engine using google co-op.

  45. Looking for ebooks?

    http://ebooksearchr.googlepages.com/ searches more than 100 ebook sites with direct download links available.

  46. Man, let me tell you! I was looking for a great and easy way to have a search on my site. I went to GOOGLE after tons of messing around with lamer scripts. So, used the co-op thingy…see what it could do but wanted more options. In seeing what you did like removing some if not all of the links placed by google. See, I don’t want to remove any google copyrights but was pondering over the thought if I could in fact add my own links to my custom search? That’d suit me fine. Thanks again for all this great reading and examples.

  47. >>If you need a search engine for your site, and your content has been indexed by Google, then seriously consider using this rather than building your own index

    We seem to have a problem getting indexed and re-indexed by Google fast enough to get a good use of Co-Op as our own site search.
    Anybody knows some secrets how to get indexed fast enough so we could use Co-Op on the site with user generated content?

    Thanks.

  48. Unfortunately there’s not much you can do here. You can fiddle with Google Sitemaps to hint to Google when the site needs to be crawled again, but there are no guarantees. Co-op, just like normal web search, doesn’t do real-time indexing. You may want to look into server side search solutions (such as those crappy database full-text plug-ins). The rule of thumb is if you’re searching through lots of different sites, already indexed by Google, that don’t typically change often, then you should opt for Co-op. If you’re requirements are different, then you should look into server side search.

  49. Vik,

    I really like your CSE. How I can I fix the search to have it automatically return a keyword search, when someone clicks on a link to a site after doing a search on the net?

    Example: You search for golf clubs, ABC Golf is in your search results. You click the listing and the CSE generates the listing according to the keyword.

    Any suggestions?

    Thanks,
    Rod

  50. Pingback: The Most Up To Date Health News

  51. Josh

    Vik,

    I love your implementation. But I’m trying to figure out — are these your Adsense ads displayed above and below the results? I don’t see what in the code links to your AdSense? And if not, then I’m assuming these are just google’s own Adsense ads…. is there then a way to modify this to act like Google’s Site Search for Adsense, where you get a cut of the ad clicks in the search results, but with the flexibility of your CSE implementation? I know that one can do this with the standard CSE implementation, but that involves using Google’s own code.

  52. Pingback: Google blog» Архив блога » Here’s some love for our Custom Search friends

  53. thank you vik for the hacks and the codes. Very usefull. I am running a coop by myself since 2006 ( http://www.lukas119.de ) and i am very excited that coop rocks. Thanks again for the CSE xml examples above

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s