February 18, 2006

deliSearch - Searching the Contents of a List of URLs

Update (12/03/06): - when a deliSearch direct search is run, the Yahoo results page is now loaded into a results frame on the deliSearch page.

Update (12/03/06): - deliSearch now supports page-based or domain based searches/search URL generation. So for example, you can search a list of pages linked to from delicious, or the domains those pages live on.

Update (04/3/06): deliSearchLink creator is deliSearch - err - oops - it wasn't - I think it's better now, and seems to work in IE too, as well as Firefox... The script now makes use of the undocumented delicious JSON callback, which seems to have got rid of the race conditions.)

MiniSearch (which I picked up on via Library Clips) is a neat little search app that allows you to search the text of all the pages linked to from another page.

As an exercise, I've knocked up a web page that loads a list of del.icio.us bookmarked links -via JSON - into the page and then allows you to generate a URL that will perform a Yahoo! search for entered search terms over the JSON loaded links: deliSearch demo - deprecated: use this instead: deliSearch.

It's just as easy to redirect the page to Yahoo! as it is to display the link, of course.

Note that the size of the Yahoo! Search URL is limited. The script takes care of this by adding search URLs until a maximum size limit is reached (I really should pop up a warning to say that not all the URLs in the particular delicious tag are are being used if that's the case).

One way round the size limit would be to encode all the URLs as tinyurl's and then use these (would that work? would Yahoo go to the site aliased by the ?).

On the to-do list for this project, then, is the ability to specify your own user/tag combination. [DONE] This will require me to get my head round the asynch part of AJAX (i.e. build my first proper AJAX app...) because time must be set aside for loading the JSON object before using it to create the search URL. [Details of the fix follow below]

(This post has brought to mind the v unfinished SearchLink pop-up I started to play with some time ago... The idea behind SearchLink augmented pages was to enable a pop-up search box over a link that will allow you to search that page/site for a given search term. I seem to remember using a pop-up script that lost focus too quickly, though...maybe worth revisiting (or can someone email me a fix?)

PS I have just modified the file to load in a JSON object from an arbitrary delicious usr/tag combination (here *DEPRECATED*)). There is a problem in waiting for the JSON object to arrive though...in FF at least, just tell the browser to keep running the script... Does anyone know how to get round this problem, and if so, can you mail me the solution?;-) Thanks... [The fix was to load the JSON object from delicious using a dynamic script load, followed by a second dynamic script load to operate on the loaded object - a working version of the Yahoo search link generator for a search over pages listed in a particular delious tag feed can be found here]

PPS Note to self - here's an RSS2JSON service that supports callbacks... so I guess I can create a client side Yahoo search tool that will do a federated search over links pulled from several social bookmarking systems via their RSS feeds...:-) And that's not all - because I can probably get the results back in JSONic form too via the Yahoo JSON API...

For more, this is Web 2.0 moving into the client, which is gonna make it even more attractive for the hobbiest who doesn't have access to a server for writing/hosting server side web apps:-)

And as the above thought experiment shows, this opens up all sorts of potential for interesting client side search tool mashups (federss'd ('fed-er-essed') search, perhaps ;-)...

Posted by ajh59 at February 18, 2006 01:35 PM