View unanswered posts | View active topics

Reply to topic  [ 6 posts ] 

Joined: Fri Jul 09, 2010 8:11 am
Posts: 15
Post More flexibility in the deletion of feed with errors
Hi there,

My Awasu runs with more than 2.500 feeds and, obviously, a lot of them are dead after a few months ...

So, is there any possibility to do this in 2 steps ?

1. create a report with feeds which didn't receive any updates for 1 month (in order to check manually on the website itself what's the matter.)
2. delete automatically those feeds

Other question :

Is there any possibility to configure the "clean up channels" option? Because I don't consider as a big error a feed with only 3 errors since this morning.

Because at the moment, I need to do it manually with this "clean up channels" and it takes a long long time ...

Thanks in advance.


Tue Feb 28, 2012 6:53 am
Profile
Site Admin
User avatar

Joined: Fri Feb 07, 2003 8:48 am
Posts: 2887
Location: Melbourne, Australia
Post Re: More flexibility in the deletion of feed with errors
Lossadil wrote:
So, is there any possibility to do this in 2 steps ?

1. create a report with feeds which didn't receive any updates for 1 month (in order to check manually on the website itself what's the matter.)
2. delete automatically those feeds

The threshold for no recent content the Clean-up channels dialog uses can be configured by opening your CONFIG.INI file and changing the NoRecentContentInterval value (number of days).

If you want to identify these channels and delete them programatically, you'll need to write a script that uses the Awasu API. The easiest way would probably be to do this:
(*) iterate through all the .CHANNEL files and identify channels that haven't received new content in a while. Look for the LastNewItemTime value (a time_t value) that indicates when a new item was last received.
(*) Then use the Awasu API to delete those channels.

Lossadil wrote:
Is there any possibility to configure the "clean up channels" option? Because I don't consider as a big error a feed with only 3 errors since this morning.

Yes. In your DEBUG.INI file, add the following:
Code:
[Debug]
ErrorModeThreshold = ...

where ... is the number of consecutive update errors a channel must get before it goes into error mode (defaults to 3).


Tue Feb 28, 2012 7:28 am
Profile WWW

Joined: Fri Jul 09, 2010 8:11 am
Posts: 15
Post Re: More flexibility in the deletion of feed with errors
Great thanks ... It works great ...

For script ... good idea, I will test it ...


Tue Feb 28, 2012 8:13 am
Profile
User avatar

Joined: Mon Sep 08, 2008 3:16 pm
Posts: 223
Location: Elk Grove, California
Post Re: More flexibility in the deletion of feed with errors
support wrote:
The threshold for no recent content the Clean-up channels dialog uses can be configured by opening your CONFIG.INI file and changing the NoRecentContentInterval value (number of days).

Wow, as much as I've learned about Awasu, I still have a lot to learn.

Sorry Guys I've had this sitting tin my Drafts since Feb. 29"

I've been way too quiet on the Awasu Extension front, so as an exercise for me in Python and the Awasu API I whipped up this little script:

Code:
#-------------------------------------------------------------------------------
# Name:        Awasu Channel Cleanup
# Purpose:     Delete's Channel's
#
# Author:      kevotheclone
#
# Created:     28/02/2012
# Copyright:   (c) kevotheclone 2012
# Licence:     public domain
#-------------------------------------------------------------------------------
#!/usr/bin/env python

import argparse
import os
import urllib2
import win32api

# Constants
dirPath = "M:\Documents\Awasu\MyData\Channels"
awasuApiUrl = "http://localhost:2604/channels/delete?token=ABC123&id="

def getChannelsWithErrors( threshold ):
    errorChannels = []

    for filename in os.listdir( dirPath ) :
        fullPath = os.path.join( dirPath , filename )
        val = int( win32api.GetProfileVal( "State" , "NConsecutiveUpdateErrors" , 0 , fullPath ) )
        chanId = int( win32api.GetProfileVal( "Config" , "LocalChannelId" , 0 , fullPath ) )

        if val > threshold :
            errorChannels.append( { "ChannelFilePath" : fullPath , "ChannelID" : chanId , "NumberOfErrors" : val } )

    return errorChannels

def displayChannelsWithErrors( errorChannels ):
    for channel in errorChannels :
        print "Channel ID: " , channel["ChannelID"] , channel["ChannelFilePath"] , "# of errors: " , channel["NumberOfErrors"]

def deleteChannelsWithErrors( errorChannels ):
    for channel in errorChannels :
        chanId = str( channel["ChannelID"] )
        try:
             response = urllib2.urlopen( awasuApiUrl + chanId )
             win32api.DeleteFile( channel["ChannelFilePath"] )
        except urllib2.URLError, e:
            print e.reason

def main():
    parser.add_argument( "-t" , "--threshold" , type=int , default=4 , \
                         dest="threshold" , action="store" , \
                         help="The positive numeric threshold of tolerable Channel errors" )
    parser.add_argument( "-d" , "--delete" , default=False , dest="delete" , \
                         action="store_true" , help="Switch to delete Channels above the theshold" )
    args = parser.parse_args()

    errorChannels = getChannelsWithErrors( args.threshold )
    displayChannelsWithErrors( errorChannels )
    if args.delete:
        deleteChannelsWithErrors( errorChannels )

if __name__ == '__main__':
    main()

There's a couple of hard coded constants that could be parametrized (one could make a call with the Awasu API to get the Channel Files' paths) and I think you need Python 2.7 or newer for the for argparse library, but it works.


Mon Mar 12, 2012 3:55 am
Profile
Site Admin
User avatar

Joined: Fri Feb 07, 2003 8:48 am
Posts: 2887
Location: Melbourne, Australia
Post Re: More flexibility in the deletion of feed with errors
Very cool, thanks :clap:

One thing I noticed, in deleteChannelsWithErrors(), it's not necessary to delete the .CHANNEL file. For reasons too boring to go into, Awasu doesn't delete this file when a channel is deleted but instead sets a flag, then deletes it the next time it starts up.

Come to think of it, if you wanted to be 100% correct, you should also check this flag when trying to identify candidate channels, so that you don't display channels that have already been deleted in the current session.


Wed Mar 14, 2012 11:06 pm
Profile WWW
User avatar

Joined: Mon Sep 08, 2008 3:16 pm
Posts: 223
Location: Elk Grove, California
Post Re: More flexibility in the deletion of feed with errors
support wrote:
...so that you don't display channels that have already been deleted in the current session.

That's why I started deleting the .CHANNEL files. I noticed that while the call to the Awasu API worked in deleting the Channel from Awasu, if you immediately ran the script again it would list the same Channels.

I didn't know there was a flag set and that the Channels would be deleted the next time Awasu starts. Once again still more stuff to learn.


Thu Mar 15, 2012 4:03 am
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 6 posts ] 

Who is online
Users browsing this forum: No registered users and 2 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to: