Post Reply
awasu.user
Posts: 88
Joined: Fri Jan 06, 2017 12:50 pm

How use batch processing to detect RSS in links

Post by awasu.user » Thu Feb 08, 2018 4:01 pm

Awasu has nice feature - detection RSS link on website. I use it a lof of time. On some exotic pages is not posible find RSS link in easy way. Awasu is solution, but I have troubles. I get spreadsheet with above 1000 news website. I have to found RSS on each.

Question:
1. How batch processing new url to use Awasu inbuild function to detect RSS link? (Perfect solution is open file and inbuild process Awasu one by one and show info about found new RSS or 404).

User avatar
support
Site Admin
Posts: 3033
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Re: How use batch processing to detect RSS in links

Post by support » Thu Feb 08, 2018 7:36 pm

There isn't really a way to automate this in Awasu, so you'll have to do it yourself. Fortunately, Python to the rescue - the process of detecting feed URL's is called "auto-discovery", and a Google search turns up a few possibilities e.g. this.

awasu.user
Posts: 88
Joined: Fri Jan 06, 2017 12:50 pm

Re: How use batch processing to detect RSS in links

Post by awasu.user » Thu Feb 08, 2018 8:04 pm

I think about it, but when I get RSS link how next process them fx.

Code: Select all

page_first.com/feed/
pages_second.com/rss/
...
pages_n.com/feed/
to add them as source to Awasu? Is a posibility load rss feed URLs (more than one at time) to run New Channel wizard for each?

User avatar
support
Site Admin
Posts: 3033
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Re: How use batch processing to detect RSS in links

Post by support » Thu Feb 08, 2018 8:20 pm

Use the $/channels/create API call for each one.

If you really want the New Channel Wizard to come up, call $/channels/create instead.

awasu.user
Posts: 88
Joined: Fri Jan 06, 2017 12:50 pm

Re: How use batch processing to detect RSS in links

Post by awasu.user » Fri Feb 09, 2018 10:07 am

If I send post data for each channel in format:

Code: Select all

<channel type="standard">
    <feedUrl> RSS_url </feedUrl>
</channel>
when I make API call is it start new channel wizard with RSS_url? Awasu be will wait until finish wizard started before? I don't want crash Awasu when I start adding above 1000 channels from script. I would set folder for channel manually. Not all feeds will be in the same folders

User avatar
support
Site Admin
Posts: 3033
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Re: How use batch processing to detect RSS in links

Post by support » Fri Feb 09, 2018 10:20 am

If you call $/channels/create, the Channel Wizard won't show at all, the channel will just be created (with default settings).

If you want to set folders, you can specify them in the XML you send when creating the channel. Call $/channels/list to see what the XML looks like, then send the same thing to Awasu when you create new channels.

Alternatively, don't set any folders, and just configure them manually after you've created them all.

User avatar
kevotheclone
Posts: 243
Joined: Mon Sep 08, 2008 7:16 pm
Location: Elk Grove, California

Re: How use batch processing to detect RSS in links

Post by kevotheclone » Sat Feb 10, 2018 8:24 am

Sadly some companies have Atom/RSS feeds listed on a web page and don't use the "auto-discovery" mechanism, this New York Times page is one example.
You'd need to "web scrape" the page to extract the Atom/RSS feed URLs using a Python module like Beautiful Soup.

Another way to create many channels at once is to import an OPML file.
You could write some code using the above mentioned Python module feedfinder2 or Beautiful Soup and create an OPML file for each page scraped. Then import the OPML file(s) into Awasu.
Awasu even has an API call $/channels/import to start the OPML import dialog. ::-):

You can find some OPML files to import in the left and center columns of this page: http://feedshare.net/popular/, the right column is a list of Atom/RSS feeds not OPML.

User avatar
support
Site Admin
Posts: 3033
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Re: How use batch processing to detect RSS in links

Post by support » Sat Feb 10, 2018 9:15 am

kevotheclone wrote:
Sat Feb 10, 2018 8:24 am
Another way to create many channels at once is to import an OPML file.
Actually, that's a pretty clever idea :clap: Awasu impresses even me, sometimes... :-)

Import from OPML also has an option to create the imported channels in a specific folder, so as long as you want to put the new channels in the same folder, this would be a quick and easy way to do it.

Post Reply

Return to “Awasu - General Discussion”