Page 1 of 1
Is Filtering really filtering?
Posted: Wed Nov 05, 2003 6:15 pm
Just downloaded AWASU and am really impressed. I'm on my 10th RSS reader in my quest for the ultimate tool for my needs. AWASU comes close.
I've read and implemented the filtering based on the tutorial, but I'm not getting the results I would expect.
My request would be to filter based on the content of the feed, not just the type of category its assigned to. So you could not only filter the channel Music, but keywords such as "Midnight Oil", or "Australia".
Re: Is Filtering really filtering?
Posted: Wed Nov 05, 2003 9:16 pm
hughgjohnson wrote:My request would be to filter based on the content of the feed, not just the type of category its assigned to. So you could not only filter the channel Music, but keywords such as "Midnight Oil", or "Australia".
Thanks. We are working on our super-duper search engine and things like intelligent search agents will be coming soon.
There is currently a project underway to write a channel hook that gets called when new content is found and put it in a database e.g. MySQL. Also, it would be easy to write a little plugin that does the content searching you want and return the results as a feed.
Channel Hook and Database Population
Posted: Fri Nov 07, 2003 2:49 am
What is the status of this project, as I need a way to put a channel feed into a database? MySql would be fine first and then I would could work with it from there.
Posted: Fri Nov 07, 2003 4:22 am
I have indeed written a channel hook (in Python) that saves any feed to a MySQL database. It's been working for a few weeks, and in the meantime I've been looking at what to do with the accumulating data.
One of the useful features of MySQL is its support in recent versions of FULLTEXT indices. I save feeds with FULLTEXT indexing so you can search them using various criteria.
I finally decided I needed a GUI-enabled program to permit easy searching. I first tried just creating an Awasu channel plug-in to do it, but found it too slow and awkward.
As often happens, on the way to formalizing the whole combination, I started thinking of alternatives. My current "detour" involves monitoring Awasu's progress as it updates channels and saving *all* successful channel updates to the database. This avoids having to specify which channels need the channel hook, but of course means updates for all channels -- whether really interesting or not -- get saved.
Now that this has turned into a public discussion, what of the things I have described would be useful, worthless, etc.? Any other ideas?
Posted: Sat Nov 08, 2003 7:28 pm
I just saw the item below, how do I get access to the source that populates the MySQL database?
There is currently a project under way to write a channel hook for Awasu that adds new content to a database (e.g. MySQL or Berkeley) as it arrives in your feeds. While work is progressing nicely on Awasu's own internal high-powered search engine, this is still an intriguing project since the content will be going into a publicly-accessible database where you can do whatever you want with it. The source will also be available for you to tinker with.
Posted: Sun Nov 09, 2003 12:20 am
I haven't gotten it to a stage where I think it is ready. Please see the posting I made just before yours for general information.