[jerrymartin]

I've thought about this but here's the reasoning behind why things are the way they are. There's no such thing as a "disabled" channel, what you're referring to is a channel that has been configured to not update automatically at regular intervals. Hence, when you do an "update all channels", it updates since it's still "active", not "disabled". Now, I could add a switch that marks a channel as "disabled" and never updates but why on earth would someone want to do this? I know I sometimes stay subscribed to blogs that have been abandoned since I want to keep the archived content but updating the feed is harmless since it has no effect. Awasu is an information gathering (and processing) tool, so why would someone want to have a channel but not update it? The only reason I can think of why someone would want to be able to disable channels is if they wanted to update them only manually, but why would you want that? And more to the point, this is such a corner case, is it really worth adding more switches and flags to an already cluttered UI to support it?

Again troubleshooting and trying to find the feed that is clogging the system, and feeds that regularly get errors.

Also, it is pretty time consuming to find out which channels you mistakingly set to full content and keep data months. Maybe a settings report could solve this one.

[jerrymartin]

I've thought about this but here's the reasoning behind why things are the way they are. There's no such thing as a "disabled" channel, what you're referring to is a channel that has been configured to not update automatically at regular intervals. Hence, when you do an "update all channels", it updates since it's still "active", not "disabled". Now, I could add a switch that marks a channel as "disabled" and never updates but why on earth would someone want to do this? I know I sometimes stay subscribed to blogs that have been abandoned since I want to keep the archived content but updating the feed is harmless since it has no effect. Awasu is an information gathering (and processing) tool, so why would someone want to have a channel but not update it? The only reason I can think of why someone would want to be able to disable channels is if they wanted to update them only manually, but why would you want that? And more to the point, this is such a corner case, is it really worth adding more switches and flags to an already cluttered UI to support it?

Again troubleshooting and trying to find the feed that is clogging the system, and feeds that regularly get errors.

Also, it is pretty time consuming to find out which channels you mistakingly set to full content and keep data months. Maybe a settings report could solve this one.

[support]

it would be nice if I could run Awasu with a "NoChannelUpdates" command line switch so that none of the Channels would update

If I need this, I usually just pull the network cable out My Awasu is configured to detect if an internet connection is present, and if it isn't, no channels get updated. If that's a little extreme, you can always configure the delay before updating channels after starting the program (Advanced tab of the Program Options).

[support]

it would be nice if I could run Awasu with a "NoChannelUpdates" command line switch so that none of the Channels would update

If I need this, I usually just pull the network cable out My Awasu is configured to detect if an internet connection is present, and if it isn't, no channels get updated. If that's a little extreme, you can always configure the delay before updating channels after starting the program (Advanced tab of the Program Options).

[support]

1. Let's say you have 500 feeds and a few are just taking forever to update. So you want to cancel and maybe turn those off until you figure out what the problem is.

Channel updates will eventually time out, they don't get stuck forever, nor will they impact other channels.

2. If you're trying to troubleshoot #1 above because you don't know which ones are giving you the trouble you might want to go through a few of them to one by one to see what is up. You may want to do that by folder or by highlighting the ones you want to update.

You can identify channels that are having trouble by checking the Error Log. Anyway, there're only a limited amount of things you can do from within Awasu, either a channel updates in a reasonable amount of time, or it hangs and eventually times out. If the latter happens, then what do you do? It's a problem with your ISP, or the remote web server, whatever, but Awasu will continue to work properly.

My problems may be unique because I'm not just reading the news, I'm sending emails, reports, xml, etc to my website.

Not really. You were talking about controlling channel updates but sending emails and uploading files are done by channel reports which run on their own schedule. I can certainly understand why people might want to do these manually but the process of downloading content to populate these reports should be done automatically, there's no reason to do it manually if Awasu is doing it for you automatically every 5 minutes!

My problem right now is simply that my update is taking way too long to finish, and I'm always wanting to cancel and turn off the channel I think might be causing it.

Again, yes a channel might take a while to update (or timeout), but since it doesn't impact anything else, is there really a problem?

As I am writing this my Awasu updates just kicked in, and because I have so many channels Awasu sucks up a lot of my cpu and sometimes I can't do anything else while its running.

It's not really a CPU issue. Awasu will use all the CPU that's available but if someone else wants it, it will relinquish it. What you're seeing is disk contention - Awasu uses the disk a lot and Windows has never been very good at this kind of thing. If you're on a laptop with slow disks, or your disk is fragmented or nearly full, these kind of things will make the problem worse.

I plan to have a lot more then 500 so I'm going to be putting Awasu to the test.

If you're going to have a large installation, you need a system that's going to be able to handle it

[support]

1. Let's say you have 500 feeds and a few are just taking forever to update. So you want to cancel and maybe turn those off until you figure out what the problem is.

Channel updates will eventually time out, they don't get stuck forever, nor will they impact other channels.

2. If you're trying to troubleshoot #1 above because you don't know which ones are giving you the trouble you might want to go through a few of them to one by one to see what is up. You may want to do that by folder or by highlighting the ones you want to update.

You can identify channels that are having trouble by checking the Error Log. Anyway, there're only a limited amount of things you can do from within Awasu, either a channel updates in a reasonable amount of time, or it hangs and eventually times out. If the latter happens, then what do you do? It's a problem with your ISP, or the remote web server, whatever, but Awasu will continue to work properly.

My problems may be unique because I'm not just reading the news, I'm sending emails, reports, xml, etc to my website.

Not really. You were talking about controlling channel updates but sending emails and uploading files are done by channel reports which run on their own schedule. I can certainly understand why people might want to do these manually but the process of downloading content to populate these reports should be done automatically, there's no reason to do it manually if Awasu is doing it for you automatically every 5 minutes!

My problem right now is simply that my update is taking way too long to finish, and I'm always wanting to cancel and turn off the channel I think might be causing it.

Again, yes a channel might take a while to update (or timeout), but since it doesn't impact anything else, is there really a problem?

As I am writing this my Awasu updates just kicked in, and because I have so many channels Awasu sucks up a lot of my cpu and sometimes I can't do anything else while its running.

It's not really a CPU issue. Awasu will use all the CPU that's available but if someone else wants it, it will relinquish it. What you're seeing is disk contention - Awasu uses the disk a lot and Windows has never been very good at this kind of thing. If you're on a laptop with slow disks, or your disk is fragmented or nearly full, these kind of things will make the problem worse.

I plan to have a lot more then 500 so I'm going to be putting Awasu to the test.

If you're going to have a large installation, you need a system that's going to be able to handle it

[support]

trying to find the feed that is clogging the system

But they don't really clog the system. Feeds will eventually timeout and cancel themselves.

and feeds that regularly get errors

You can identify these using <i>Tools|Clean up channels</i>.

Also, it is pretty time consuming to find out which channels you mistakingly set to full content and keep data months. Maybe a settings report could solve this one.

I've been toying with this idea while thinking about how to allow multiple channels to be edited at the same time. But in the meantime, a quick grep through the <tt>.CHANNEL</tt> files will tell you what you need to know...

[support]

trying to find the feed that is clogging the system

But they don't really clog the system. Feeds will eventually timeout and cancel themselves.

and feeds that regularly get errors

You can identify these using <i>Tools|Clean up channels</i>.

Also, it is pretty time consuming to find out which channels you mistakingly set to full content and keep data months. Maybe a settings report could solve this one.

I've been toying with this idea while thinking about how to allow multiple channels to be edited at the same time. But in the meantime, a quick grep through the <tt>.CHANNEL</tt> files will tell you what you need to know...

[support]

As I am writing this my Awasu updates just kicked in, and because I have so many channels Awasu sucks up a lot of my cpu and sometimes I can't do anything else while its running.

BTW, there are ways to slow Awasu down and throttle the amount of work it's doing. It obviously means a round of channel updates will take longer to complete but it will make your computer a bit more usable while it happens.

Let me know if you want the necessary incantations...

<i>EDIT: Already done </i>

[support]

As I am writing this my Awasu updates just kicked in, and because I have so many channels Awasu sucks up a lot of my cpu and sometimes I can't do anything else while its running.

BTW, there are ways to slow Awasu down and throttle the amount of work it's doing. It obviously means a round of channel updates will take longer to complete but it will make your computer a bit more usable while it happens.

Let me know if you want the necessary incantations...

<i>EDIT: Already done </i>

[kevotheclone]

I probably shouldn't post in my current condition, I just drank a large glass of carrot juice, so if I say anything goofy it's the Vitamin A talking...

...just pull the network cable out

I love simple solutions! My firewall has a "Stop all Internet Activity" panic button, that I could use. Why didn't I think of that!

Once an update starts, could jerrymartin just "pull the network cable out" to stop update activity? Will Awasu stop attempting to update any Channels that haven't started the update process yet, or is it too late?

Also regarding speeding up Awasu, if a system has enough free RAM I don't suppose there's any way of safely improving performance by setting up a RAM disk? Will setting my %TEMP% directory to a RAM disk help?

I know I could write a simple batch file that could copy my "local data" and "roaming data" directories to the RAM disk prior to running Awasu (with a modified DEBUG.INI file) and then when I shut down Awasu, copy everything back to my hard drive, but I'd be nervous about this, as if my PC hiccups while Awasu is running I could lose all of the updates for this session.

I don't have any performance problems with my home PC, but of course when I run in PORTABLE mode it's a little slower, and I've tried to think of ways to speed it up.

[kevotheclone]

I probably shouldn't post in my current condition, I just drank a large glass of carrot juice, so if I say anything goofy it's the Vitamin A talking...

...just pull the network cable out

I love simple solutions! My firewall has a "Stop all Internet Activity" panic button, that I could use. Why didn't I think of that!

Once an update starts, could jerrymartin just "pull the network cable out" to stop update activity? Will Awasu stop attempting to update any Channels that haven't started the update process yet, or is it too late?

Also regarding speeding up Awasu, if a system has enough free RAM I don't suppose there's any way of safely improving performance by setting up a RAM disk? Will setting my %TEMP% directory to a RAM disk help?

I know I could write a simple batch file that could copy my "local data" and "roaming data" directories to the RAM disk prior to running Awasu (with a modified DEBUG.INI file) and then when I shut down Awasu, copy everything back to my hard drive, but I'd be nervous about this, as if my PC hiccups while Awasu is running I could lose all of the updates for this session.

I don't have any performance problems with my home PC, but of course when I run in PORTABLE mode it's a little slower, and I've tried to think of ways to speed it up.

[support]

Once an update starts, could jerrymartin just "pull the network cable out" to stop update activity? Will Awasu stop attempting to update any Channels that haven't started the update process yet, or is it too late?

It's too late, although it will cause any existing downloads to terminate immediately. Channels that are already queued up to download will also start, but then fail immediately.

Will setting my %TEMP% directory to a RAM disk help?

Awasu generates a lot of temp files so it will help, but probably not much.

You can avoid having to hack your TEMP environment variable by setting up your <tt>DEBUG.INI</tt> file like this:

Code: Select all

[Debug]
TempFileDir=...

if my PC hiccups while Awasu is running I could lose all of the updates for this session.

This is only an issue for any config changes you make, downloaded content can safely be lost since Awasu will just download it again the next you run it.

I don't have any performance problems with my home PC, but of course when I run in PORTABLE mode it's a little slower, and I've tried to think of ways to speed it up.

I've done quite a bit of performance testing of external drives over the years and you're probably limited by the speed of USB. In this case, turning the search index off will probably make a huge difference.

[support]

Once an update starts, could jerrymartin just "pull the network cable out" to stop update activity? Will Awasu stop attempting to update any Channels that haven't started the update process yet, or is it too late?

It's too late, although it will cause any existing downloads to terminate immediately. Channels that are already queued up to download will also start, but then fail immediately.

Will setting my %TEMP% directory to a RAM disk help?

Awasu generates a lot of temp files so it will help, but probably not much.

You can avoid having to hack your TEMP environment variable by setting up your <tt>DEBUG.INI</tt> file like this:

Code: Select all

[Debug]
TempFileDir=...

if my PC hiccups while Awasu is running I could lose all of the updates for this session.

This is only an issue for any config changes you make, downloaded content can safely be lost since Awasu will just download it again the next you run it.

I don't have any performance problems with my home PC, but of course when I run in PORTABLE mode it's a little slower, and I've tried to think of ways to speed it up.

I've done quite a bit of performance testing of external drives over the years and you're probably limited by the speed of USB. In this case, turning the search index off will probably make a huge difference.

[kevotheclone]

Taka, I'll try to test the RAM disk in the next couple of weeks and give you some timed results. "TempFileDir" setting in the DEBUG.INI file... I guess the old saying is still true: "Where there's Awasu, there's a way!"

Also, it is pretty time consuming to find out which channels you mistakingly set to full content and keep data months. Maybe a settings report could solve this one.

Jerry, I have some Application Plugins that I'll release in the next couple of months that will provide reports on your Channel settings. I'm not planning on trying to include/display every Channel setting though; but they will be open source. So polish up your Python coding skills, have a few drinks and you'll be able to tweaks them into displaying exactly what you want.

Also, there's another Awasu user named gschoepp who's got over 1,500 Channels, gschoepp if you're listening, any performance tuning advice for jerrymartin?

[kevotheclone]

Taka, I'll try to test the RAM disk in the next couple of weeks and give you some timed results. "TempFileDir" setting in the DEBUG.INI file... I guess the old saying is still true: "Where there's Awasu, there's a way!"

Also, it is pretty time consuming to find out which channels you mistakingly set to full content and keep data months. Maybe a settings report could solve this one.

Jerry, I have some Application Plugins that I'll release in the next couple of months that will provide reports on your Channel settings. I'm not planning on trying to include/display every Channel setting though; but they will be open source. So polish up your Python coding skills, have a few drinks and you'll be able to tweaks them into displaying exactly what you want.

Also, there's another Awasu user named gschoepp who's got over 1,500 Channels, gschoepp if you're listening, any performance tuning advice for jerrymartin?

[kevotheclone]

Jerry, you mentioned a couple of times that you are troubleshooting problem feeds; I've got another approach that may work for you.

Download, install and get familiar with an HTTP Proxy/Sniffer. I've used Fiddler for several years and it's pretty good. It can log all of the HTTP calls made by any software on your PC, not just your web browser.

So start Fiddler, and in Fiddler's status bar you'll see a section that may say "All Processes", "Web Browsers", "Non-Browser", or "Hide All"; keep clicking it until it says "All Processes" just to be safe. Start Awasu and update all your Channels, be patient, wait for the update to complete, and then check the logs in Fiddler. Look for anything that has a value other than "200" in the "Result" column; check its URL value to see if it's one of your feeds. Mind you, not all values other than "200" indicate an error, but "200" means "OK" (the 500 series is bad). You can find out what each code value means here: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

I know this may seem a little "lower level" than you probably wanted to get into, but I think if you bite the bullet and use this approach you'll find the problem feeds in one session. Get to know Fiddler or another HTTP Proxy/Sniffer and you'll be happy you did.

[kevotheclone]

Jerry, you mentioned a couple of times that you are troubleshooting problem feeds; I've got another approach that may work for you.

Download, install and get familiar with an HTTP Proxy/Sniffer. I've used Fiddler for several years and it's pretty good. It can log all of the HTTP calls made by any software on your PC, not just your web browser.

So start Fiddler, and in Fiddler's status bar you'll see a section that may say "All Processes", "Web Browsers", "Non-Browser", or "Hide All"; keep clicking it until it says "All Processes" just to be safe. Start Awasu and update all your Channels, be patient, wait for the update to complete, and then check the logs in Fiddler. Look for anything that has a value other than "200" in the "Result" column; check its URL value to see if it's one of your feeds. Mind you, not all values other than "200" indicate an error, but "200" means "OK" (the 500 series is bad). You can find out what each code value means here: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

I know this may seem a little "lower level" than you probably wanted to get into, but I think if you bite the bullet and use this approach you'll find the problem feeds in one session. Get to know Fiddler or another HTTP Proxy/Sniffer and you'll be happy you did.

[support]

Taka, I'll try to test the RAM disk in the next couple of weeks and give you some timed results.

Just timings by themselves might be a bit misleading since how long things take depends very much on how many new items are coming in, how big they are, if they really are new or are revisions, how much metadata processing needs to be done on them, etc. All other things being equal (which they never are), you might want to run a quick SQL query on the database to count how many items are in the database before and after...

[support]

Taka, I'll try to test the RAM disk in the next couple of weeks and give you some timed results.

Just timings by themselves might be a bit misleading since how long things take depends very much on how many new items are coming in, how big they are, if they really are new or are revisions, how much metadata processing needs to be done on them, etc. All other things being equal (which they never are), you might want to run a quick SQL query on the database to count how many items are in the database before and after...

[support]

I know this may seem a little "lower level" than you probably wanted to get into

It might be easier just to analyze the Activity Log. Even if this window is not open, everything is still recorded in the <tt>AwasuActivity.log</tt> file in your user's <tt>Logs</tt> sub-directory.

Awasu logs the start of every channel update, and when each one ends (successful or not) so it wouldn't be hard to write a script that analyzed this log file to track average update times, which channels were taking unusually long to complete, which channels were hung, etc. This might make a nice application plugin...

[support]

I know this may seem a little "lower level" than you probably wanted to get into

It might be easier just to analyze the Activity Log. Even if this window is not open, everything is still recorded in the <tt>AwasuActivity.log</tt> file in your user's <tt>Logs</tt> sub-directory.

Awasu logs the start of every channel update, and when each one ends (successful or not) so it wouldn't be hard to write a script that analyzed this log file to track average update times, which channels were taking unusually long to complete, which channels were hung, etc. This might make a nice application plugin...

[kevotheclone]

Of course! Awasu's Activity & Error logs, why didn't we mention them earlier?

It's been a long time since I had to look at the Awasu's Activity & Error logs, so I completely forgot about them.

I think the point of all these troubleshooting tips, is to not try to stop Awasu from updating in order to track down a bad feed; instead sit down for a dedicated troubleshooting session:

1) Start Awasu
2) Update all Channels (be patient, grab your favorite musical instrument and play "Giant Steps" while you wait)
3) Inspect Awasu's Activity & Error logs and/or an HTTP log

Now you can determine the full list of bad feeds all at one time.

[kevotheclone]

Of course! Awasu's Activity & Error logs, why didn't we mention them earlier?

It's been a long time since I had to look at the Awasu's Activity & Error logs, so I completely forgot about them.

I think the point of all these troubleshooting tips, is to not try to stop Awasu from updating in order to track down a bad feed; instead sit down for a dedicated troubleshooting session:

1) Start Awasu
2) Update all Channels (be patient, grab your favorite musical instrument and play "Giant Steps" while you wait)
3) Inspect Awasu's Activity & Error logs and/or an HTTP log

Now you can determine the full list of bad feeds all at one time.