Home News Forum Articles
  Welcome back Join CF
You are here You are here: Home | Forum | Simple view on how proxies work (inc DNS)


You are currently viewing our boards as a guest which gives you limited access to view most of the discussions, articles and other free features. By joining our Virgin Media community you will have full access to all discussions, be able to view and post threads, communicate privately with other members (PM), respond to polls, upload your own images/photos, and access many other special features. Registration is fast, simple and absolutely free so please join our community today.


Welcome to Cable Forum
Go Back   Cable Forum > Virgin Media Services > Virgin Media Internet Services > Webspace, E-Mail & Browsing Issues

Simple view on how proxies work (inc DNS)
Reply
 
Thread Tools
Old 12-03-2004, 19:37   #1
Busy Admin
 
Paul M's Avatar
 
Join Date: Oct 2003
Location: Nottingham
Age: 45
Services: VM Phone : Sky+ Multiroom : VM Cable (20 Mbps)
Posts: 14,487
Paul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny star
Paul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny star
Send a message via ICQ to Paul M
Simple view on how proxies work (inc DNS)

Since it seems to confuse people a lot I thought I'd try and explain it in simple terms;

Techies - feel free to point out any mistakes.


First of all - without any proxies - when a client wants a web page ;

1. It does a dns lookup of the required domain using its own (locally) defined dns servers.
2. It sends the http request on tcp port 80 to the destination servers IP address.
3. This request is routed over the network direct to the specified server IP.
4. The requested page is then returned, by the server, direct to the client.


Now if you have a manually defined proxy (such as in IE, on port 8080) - when a client wants a web page ;

1. It does a dns lookup of the proxy servers IP using its own (locally) defined dns servers.
2. It sends the http request on tcp port 8080 to the proxy servers IP address.
3. This request is routed over the network direct to the specified proxy server IP.
4. The proxy server checks if it has a valid copy of the requested page in its cache.
5. If the proxy has the page it returns it directly back to the client, Job done.
6. If the proxy does not have the page it does a dns lookup of the requested domain using it's own defined dns servers.
7. The proxy sends the http request on tcp port 80 to the destination servers IP address.
8. This request is routed over the network direct to the destination servers IP.
9. The requested page is then returned, by the destination server, directly back to the proxy server.
10. The requested page is then returned, by the proxy server, direct to the client.


Now if you have no manually defined proxy, but NTL's proxies are active - when a client wants a web page ;

1. It does a dns lookup of the destination servers IP using its own (locally) defined dns servers.
2. It sends the http request on tcp port 80 to the destination servers IP address.
3. This request is intercepted by NTL's routers and re-directed to one of a group of transparent proxy [cache] servers - the decision on which proxy server gets used is (usually) based on the requested domain.
4. The chosen proxy server checks if it has a valid copy of the requested page in its cache.
5. If the proxy has the page it returns it directly back to the client, Job done.
6. If the proxy does not have the page it does a dns lookup of the requested domain using it's own defined dns servers.
7. The proxy sends the http request on tcp port 80 to the destination servers IP address.
8. This request is routed over the network direct to the destination servers IP.
9. The requested page is then returned, by the destination server, directly back to the proxy server.
10. The requested page is then returned, by the proxy server, direct to the client.

The final scenario is called transparent because the client is unaware that its request as been "hijacked" and still thinks it is talking directly to the destination server.

From the above it can be seen that if you are having web browsing problems and proxies are in use (transparent or otherwise) then ;

1. The DNS servers the proxy is using are just as important as yours - if they are duff then you are in trouble. (However, even if the proxy cannot resolve or contact the destination server - you can still access the requested pages if the proxy has a valid copy of them in its cache)

2. Doing a tracert to the destination server will not show you the route your request is taking as it won't go via any proxy in use.

This is not a precise and exact description of how it all works but I hope it makes it clearer.

To workout if you are using an NTL transparent proxy - use this simple link

Pem.
__________________
DigiGuide Click here for a real, interactive, tv guide.
Paul M is offline   Reply With Quote
Old 12-03-2004, 22:39   #2
cf.mega pornstar
 
danielf's Avatar
 
Join Date: Jun 2003
Posts: 10,237
danielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered stars
danielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered stars
Re: Simple view on how proxies work (inc DNS)

Cheers for that. I think that makes sense I have a question though. What is a 'valid copy' of a page. I.e. How does the proxy decide if the page it has cached is recent enough? Do all pages carry a time stamp (last modified date/time), and does the proxy query the requested site for the time stamp of the requested page?
__________________
Oh Oh Ranje
We houden allemaal van jou, de spelers en het rood wit blauw.
------------------------------------------------------------------------
I wish I knew where I was going, cause I have to get back as well.
danielf is offline   Reply With Quote
Old 12-03-2004, 22:41   #3
Busy Admin
 
Paul M's Avatar
 
Join Date: Oct 2003
Location: Nottingham
Age: 45
Services: VM Phone : Sky+ Multiroom : VM Cable (20 Mbps)
Posts: 14,487
Paul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny star
Paul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny star
Send a message via ICQ to Paul M
Re: Simple view on how proxies work (inc DNS)

All pages effectivley have an expiry date & time.
__________________
DigiGuide Click here for a real, interactive, tv guide.
Paul M is offline   Reply With Quote
Old 12-03-2004, 22:52   #4
cf.mega pornstar
 
danielf's Avatar
 
Join Date: Jun 2003
Posts: 10,237
danielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered stars
danielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered stars
Re: Simple view on how proxies work (inc DNS)

Quote:
Originally Posted by pem
All pages effectivley have an expiry date & time.
Ah, that makes sense, as I was wondering if the proxy had to query for a more recent page, which would increase traffic and defeat the purpose of the proxy unless you only have to retrieve the time stamp of the most recent page.

I presume this expiry time/date is set differently depending on the type of page? (very low for a forum like this) and high(er) for less dynamic pages)?

Or is it just based on recent 'refresh rates'?
__________________
Oh Oh Ranje
We houden allemaal van jou, de spelers en het rood wit blauw.
------------------------------------------------------------------------
I wish I knew where I was going, cause I have to get back as well.
danielf is offline   Reply With Quote
Old 12-03-2004, 22:54   #5
cf.mega poster
 
Join Date: Sep 2003
Location: Leics
Age: 29
Services: none
Posts: 5,641
Chrysalis has reached the bronze age
Chrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze age
Re: Simple view on how proxies work (inc DNS)

pem even when a port is specified my pages are still going through the transperent proxies, the guy who told us this in the other thread was right.
Chrysalis is offline   Reply With Quote
Old 12-03-2004, 23:01   #6
Busy Admin
 
Paul M's Avatar
 
Join Date: Oct 2003
Location: Nottingham
Age: 45
Services: VM Phone : Sky+ Multiroom : VM Cable (20 Mbps)
Posts: 14,487
Paul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny star
Paul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny star
Send a message via ICQ to Paul M
Re: Simple view on how proxies work (inc DNS)

Quote:
Originally Posted by danielf
Ah, that makes sense, as I was wondering if the proxy had to query for a more recent page, which would increase traffic and defeat the purpose of the proxy unless you only have to retrieve the time stamp of the most recent page.

I presume this expiry time/date is set differently depending on the type of page? (very low for a forum like this) and high(er) for less dynamic pages)?

Or is it just based on recent 'refresh rates'?
Believe me - you would not believe how complicated it is for a proxy to determine if it's copy is current ("fresh"). There are pages and pages of specifications and http headers to control it.


Quote:
Originally Posted by Chrysalis
pem even when a port is specified my pages are still going through the transperent proxies, the guy who told us this in the other thread was right.
Sorry, I don't follow what you are saying. Can you explain ?
__________________
DigiGuide Click here for a real, interactive, tv guide.
Paul M is offline   Reply With Quote
Old 12-03-2004, 23:01   #7
Banned
 
zendawn's Avatar
 
Join Date: Nov 2003
Location: Sussex
Posts: 732
zendawn is a jewel in the roughzendawn is a jewel in the roughzendawn is a jewel in the roughzendawn is a jewel in the roughzendawn is a jewel in the rough
Re: Simple view on how proxies work (inc DNS)

Does CTRL+Refresh Overide the proxy in order to get an updated refresh??
zendawn is offline   Reply With Quote
Old 12-03-2004, 23:05   #8
Busy Admin
 
Paul M's Avatar
 
Join Date: Oct 2003
Location: Nottingham
Age: 45
Services: VM Phone : Sky+ Multiroom : VM Cable (20 Mbps)
Posts: 14,487
Paul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny star
Paul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny starPaul M has a nice shiny star
Send a message via ICQ to Paul M
Re: Simple view on how proxies work (inc DNS)

Quote:
Originally Posted by zendawn
Does CTRL+Refresh Overide the proxy in order to get an updated refresh??
Erm ........ if client and proxy follow the rules - Yes.

Do they all follow the rules ...... do they ****
__________________
DigiGuide Click here for a real, interactive, tv guide.
Paul M is offline   Reply With Quote
Old 12-03-2004, 23:05   #9
cf.mega pornstar
 
danielf's Avatar
 
Join Date: Jun 2003
Posts: 10,237
danielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered stars
danielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered starsdanielf is seeing silvered stars
Re: Simple view on how proxies work (inc DNS)

Quote:
Originally Posted by pem
Believe me - you would not believe how complicated it is for a proxy to determine if it's copy is current ("fresh"). There are pages and pages of specifications and http headers to control it.
Ah, the joys of optimisation/efficiency
__________________
Oh Oh Ranje
We houden allemaal van jou, de spelers en het rood wit blauw.
------------------------------------------------------------------------
I wish I knew where I was going, cause I have to get back as well.
danielf is offline   Reply With Quote
Old 12-03-2004, 23:08   #10
cf.geek
 
Join Date: Aug 2003
Location: Asleep down in the server room
Age: 44
Posts: 516
tkiely is a name known to alltkiely is a name known to alltkiely is a name known to alltkiely is a name known to alltkiely is a name known to alltkiely is a name known to alltkiely is a name known to alltkiely is a name known to all
Send a message via MSN to tkiely
Re: Simple view on how proxies work (inc DNS)

just ditch them, the're not there for your convenience!

mind you, i like playing with them at work. but that's not for here!!!!!!!!!!
__________________
www.f2s.com They used to be good b4 Pipex got 'em

now on BT who are em.........pretty good...sorry
tkiely is offline   Reply With Quote
Old 12-03-2004, 23:49   #11
cf.geek
 
Join Date: Jun 2003
Services: Weddings, christenings, barmitzvahs and funerals.
Posts: 859
ntl customer has a reputation beyond reputentl customer has a reputation beyond reputentl customer has a reputation beyond reputentl customer has a reputation beyond reputentl customer has a reputation beyond reputentl customer has a reputation beyond reputentl customer has a reputation beyond reputentl customer has a reputation beyond reputentl customer has a reputation beyond reputentl customer has a reputation beyond reputentl customer has a reputation beyond reputentl customer has a reputation beyond reputentl customer has a reputation beyond repute
Re: Simple view on how proxies work (inc DNS)

How long to the proxies keep stored pages for?
ntl customer is offline   Reply With Quote
Old 12-03-2004, 23:59   #12
cf.geek
 
Join Date: Aug 2003
Location: Asleep down in the server room
Age: 44
Posts: 516
tkiely is a name known to alltkiely is a name known to alltkiely is a name known to alltkiely is a name known to alltkiely is a name known to alltkiely is a name known to alltkiely is a name known to alltkiely is a name known to all
Send a message via MSN to tkiely
Re: Simple view on how proxies work (inc DNS)

Quote:
Originally Posted by ntl customer
How long to the proxies keep stored pages for?
generally, until they change or expire. Some will make checks on a site for change on a daily basis to see if there have been changes that they should cache
__________________
www.f2s.com They used to be good b4 Pipex got 'em

now on BT who are em.........pretty good...sorry
tkiely is offline   Reply With Quote
Old 13-03-2004, 00:26   #13
cf.mega poster
 
Join Date: Sep 2003
Location: Leics
Age: 29
Services: none
Posts: 5,641
Chrysalis has reached the bronze age
Chrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze age
Re: Simple view on how proxies work (inc DNS)

well right now pem I am running through my own proxy on port 3333, but the ntl transperent proxy is still intercepting my pages, I will look for the post that explains this and paste it here.
Chrysalis is offline   Reply With Quote
Old 13-03-2004, 00:33   #14
cf.mega poster
 
Join Date: Sep 2003
Location: Leics
Age: 29
Services: none
Posts: 5,641
Chrysalis has reached the bronze age
Chrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze age
Re: Simple view on how proxies work (inc DNS)

http://forum.nthellworld.co.uk/showp...2&postcount=89

shows the info

but right after i retrieved this post the ip shown in my sig is no longer ntl proxy server.

I think it is working after all
Chrysalis is offline   Reply With Quote
Old 13-03-2004, 01:28   #15
cf.mega poster
 
Join Date: Sep 2003
Location: Leics
Age: 29
Services: none
Posts: 5,641
Chrysalis has reached the bronze age
Chrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze ageChrysalis has reached the bronze age
Re: Simple view on how proxies work (inc DNS)

when I use IE or firefox it doesnt use ntl proxy but if I use MYIE2 it does, very weird especially as MYIE2 uses IE core.

--edit--

fixed it

Last edited by Chrysalis; 13-03-2004 at 01:34.
Chrysalis is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


All times are GMT +1. The time now is 20:25.


Links
Google
 
Web www.cableforum.co.uk


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO 3.1.0
Copyright © 2003 - 2008, Cable Forum.
(s204569790.onlinehome.info)