[Sderby] squid proxy servers?

[Andrew White]

> the problem with requests is - its just a request,  the proxy doesn't
> know how large the response is going to be.
>
> Directing requests via a secondary outbound connection can be achieved
> by using two web browsers with different proxy settings.  Then its up
> to the user to decide whats big and what isn't and use the appropriate
> browser.
>
> I've not thought or seen an automated way to do this..
[snip]

The only thing I can think of is having a custom proxy sitting just before
the squid proxies, and doing a HEAD request on the URL, checking the
size, and using whichever proxy is best for that particular request.

This shouldn't be too hard to accomplish with a little bit of perl or
something similar.

in fact,

lynx -source -head <URL> | grep Content-Length |awk '{print $2}'

will return the size (use the best proxy for general/small content
to get this), and then you'd just want to set and export the environment
variables

http_proxy, https_proxy, ftp_proxy, etc to whichever is best suited
for this size of content. You'd need to stick the output of the head
request in a variable, and have some logic for working out which to use
(expr would do this).

just use lynx -source <URL> to actually return the content.

Not sure how fast all this'd be. It should be a bit faster in perl,
and I'm not sure how well it'd work at all. But it's a start