Page 1 of 1

MonitorURL pluggin timeout

Posted: Wed Mar 08, 2006 4:53 pm
by markmorgan
Whenever I use the MonitorURL pluggin I get the following message

*** ??? Unable to retrieve from http://www.blahblahblah.com/index.htm (status = "Time-out") ??? ***

Any ideas? :?:

Mark Morgan.

Posted: Thu Mar 09, 2006 3:39 am
by abwilson
That should mean it tried to retrieve the page but gave up after not getting a reply from the site after waiting a certain length of time (30 seconds, I believe).

Posted: Thu Mar 09, 2006 8:24 am
by markmorgan
But this happens even when the system is quiet? And the website can easily be pulled up in Internet Explorer.

Could it be a proxy error? Does the code detect and use the proxy server correctly?

Mark

Posted: Fri Mar 10, 2006 12:40 am
by abwilson
The plugin is written in Python and uses the standard Python library routines, which do support any (IE defined) proxy.

What is the/a site you are having trouble with?

Posted: Fri Mar 10, 2006 10:34 am
by markmorgan
The site in question is http://www.kv-63.com/pages/1/page1.html

I have tried it on my home machine, where there is no proxy server getting in the way, and it works fine.

I don't know PHP myself but coding up a simple HTTP GET request in Perl

Code: Select all

my $ua = LWP::UserAgent->new;
my $page = $ua->request(HTTP::Request->new(GET => $url);


fails with an HTTP 500 as it does not use the proxy server unless you do some extra programming. Using LWP simple it works though as this function does the proxy handling for you

Code: Select all

my $page = get($url)


Maybe there are similar issues in PHP?

Mark.

Posted: Sat Mar 11, 2006 2:52 am
by abwilson
Sigh... It is likely my code. Despite what the Release Notes probably still say, the latest version of MonitorURLs does not use a standard Python library to retrieve Web pages, but rather does it using a new library I wrote to allow parallel (asynchronous) reads -- meaning that the plugin can fetch from multiple sites in parallel, rather than sequentially as in the original versions. This improves performance and allows the plugin to finish quicker. (The same new library is used in the CombineFeeds plugin.)

As you might expect from this lead-in, my new library deals with a few special return codes (some redirects, etc.) but doesn't know anything about proxies.

Sorry, but I don't know when I'll have a chance to see about enhancing the library.

Taka, any other proxy suggestions? rssextender (formerly RSS Proxy) deals only with RSS 2.0 feeds, as best I can tell, so shouldn't be suitable for an arbitrary URL retrieval.

markmorgan, how about a version of MonitorURLs (in Perl or whatever) that will handle proxies correctly? :)

Allan

Posted: Tue Mar 14, 2006 1:47 pm
by markmorgan
Is the previous synchronus version available for download as an interim solution?

Thanks,
Mark.

Posted: Tue Mar 14, 2006 8:22 pm
by abwilson
Yes, though I've had to redo it a bit, since my build procedures also changed a while ago. Please send me (abwilson at awasu.com) your e-mail address and I'll send you a .zip file with the "sequential" version of MonitorURLs to try.

Thanks

Posted: Tue Mar 21, 2006 11:11 am
by markmorgan
Sorry it took a while, I have finally got around to trying it.

It gave an XML parse error and the feed contained this error log (rather than XML)

Code: Select all

The script caused an error:
Traceback (most recent call last):
  File "C:\Python24\lib\site-packages\py2exe\boot_common.py", line 69, in ?
ImportError: No module named linecache
Traceback (most recent call last):
  File "<install zipextimporter>", line 1, in ?
ImportError: No module named zipextimporter
Traceback (most recent call last):
  File "seqMonitorURLs.py", line 22, in ?
ImportError: No module named ConfigParser


Thanks,
Mark.

Posted: Tue Mar 21, 2006 7:34 pm
by abwilson
Going back to a previous version after my build procedures had changed proved more difficult than expected. I am sending you an updated build; please try it.

Thanks

Posted: Mon Mar 27, 2006 9:33 am
by markmorgan
Many thanks! :P

I can confirm that it is working fine now.

Cheers,
Mark