Web Site is not Accessible by its IP?
-
How come some websites can be accessed by IP Addresses and some cannot e.g. Google is accessible through --> http://74.125.19.99/[^] but msn isn't --> http://207.68.173.76/[^] need help because currently working on a DNS Resolver a component for Distributed Web crawler to save those expensive round trips. Thanx in advance !
AliAmjad(MCP) First make it Run THEN make it Run Fast!
-
How come some websites can be accessed by IP Addresses and some cannot e.g. Google is accessible through --> http://74.125.19.99/[^] but msn isn't --> http://207.68.173.76/[^] need help because currently working on a DNS Resolver a component for Distributed Web crawler to save those expensive round trips. Thanx in advance !
AliAmjad(MCP) First make it Run THEN make it Run Fast!
When a page is requested, the domain name is resolved into the IP number so that the request is sent to the correct server. The server then locates the web site that is configured to listen to that IP number and to that domain name. A web site can be configures to respond to an IP only without the need of a domain name.
Experience is the sum of all the mistakes you have done.
-
When a page is requested, the domain name is resolved into the IP number so that the request is sent to the correct server. The server then locates the web site that is configured to listen to that IP number and to that domain name. A web site can be configures to respond to an IP only without the need of a domain name.
Experience is the sum of all the mistakes you have done.
But when the IP address of the Server is correct and sent along with the path where the page is located then why it is not displaying the correct page. How can i handle such a situation because I've to resolve each discovered URL and if the resolver failed to get the correct information then I've to issue the request with the unresolved address which not only consumes the bandwidth but also effects the performance of the crawler. Thanks for your helps man !!!
AliAmjad(MCP) First make it Run THEN make it Run Fast!
-
But when the IP address of the Server is correct and sent along with the path where the page is located then why it is not displaying the correct page. How can i handle such a situation because I've to resolve each discovered URL and if the resolver failed to get the correct information then I've to issue the request with the unresolved address which not only consumes the bandwidth but also effects the performance of the crawler. Thanks for your helps man !!!
AliAmjad(MCP) First make it Run THEN make it Run Fast!
AliAmjad wrote:
But when the IP address of the Server is correct and sent along with the path where the page is located then why it is not displaying the correct page.
Because there can be several web sites that respond to the same IP, and they differ only by the domain name.
Experience is the sum of all the mistakes you have done.
-
AliAmjad wrote:
But when the IP address of the Server is correct and sent along with the path where the page is located then why it is not displaying the correct page.
Because there can be several web sites that respond to the same IP, and they differ only by the domain name.
Experience is the sum of all the mistakes you have done.
-
Then what should be the correct approach to access the right thing any article here or anywhere that can help me to solve this issue ???
AliAmjad(MCP) First make it Run THEN make it Run Fast!
-
What are you trying to do, really? Why are you "resolving" the URLs? What's wrong with the URL as it is?
Experience is the sum of all the mistakes you have done.
As i told you earlier that I am working on a Distributed Web Crawler and a web crawler must use the Domain Name Service (DNS) to map the web server’s host name into an IP address because if i have discovered say 50 urls from a site e.g. http://www.example.com/ then every request issued to each URL will require a DNS lookup to resolve that domain name which is an overhead a performance reduction or leak etc. what you say !!!
AliAmjad(MCP) First make it Run THEN make it Run Fast!
-
As i told you earlier that I am working on a Distributed Web Crawler and a web crawler must use the Domain Name Service (DNS) to map the web server’s host name into an IP address because if i have discovered say 50 urls from a site e.g. http://www.example.com/ then every request issued to each URL will require a DNS lookup to resolve that domain name which is an overhead a performance reduction or leak etc. what you say !!!
AliAmjad(MCP) First make it Run THEN make it Run Fast!
Well, there's not much that you can do. A domain name is not the same as an IP. Sometimes you can use the IP instead of the domain name, sometimes not, and only the server knows. DNS services are usually cached, so a repeated DNS lookup will usually only go outside the local computer the first time. Have you actually determined that the DNS lookups is even a peformance problem?
Experience is the sum of all the mistakes you have done.
-
Well, there's not much that you can do. A domain name is not the same as an IP. Sometimes you can use the IP instead of the domain name, sometimes not, and only the server knows. DNS services are usually cached, so a repeated DNS lookup will usually only go outside the local computer the first time. Have you actually determined that the DNS lookups is even a peformance problem?
Experience is the sum of all the mistakes you have done.
Yes indeed and I've also read many research paper on this technology and Altavista's Marcator also has this component and their research also shows a significant performance improvement by implementing this very component.
Guffa wrote:
DNS services are usually cached, so a repeated DNS lookup will usually only go outside the local computer the first time.
Can you please tell me or explain it a bit ! Thanks for your help man !!
AliAmjad(MCP) First make it Run THEN make it Run Fast!
-
Yes indeed and I've also read many research paper on this technology and Altavista's Marcator also has this component and their research also shows a significant performance improvement by implementing this very component.
Guffa wrote:
DNS services are usually cached, so a repeated DNS lookup will usually only go outside the local computer the first time.
Can you please tell me or explain it a bit ! Thanks for your help man !!
AliAmjad(MCP) First make it Run THEN make it Run Fast!
AliAmjad wrote:
Yes indeed and I've also read many research paper on this technology and Altavista's Marcator also has this component and their research also shows a significant performance improvement by implementing this very component.
Well, to get that to work you have to send the request differently. You can send the request directly to the IP, but it has to contain the "unresolved" URL.
AliAmjad wrote:
Can you please tell me or explain it a bit !
DNS in Windows XP[^] Configuring the DNS Resolver Cache in Windows XP[^] Adjust Windows XP DNS Cache Settings[^]
Experience is the sum of all the mistakes you have done.
-
AliAmjad wrote:
Yes indeed and I've also read many research paper on this technology and Altavista's Marcator also has this component and their research also shows a significant performance improvement by implementing this very component.
Well, to get that to work you have to send the request differently. You can send the request directly to the IP, but it has to contain the "unresolved" URL.
AliAmjad wrote:
Can you please tell me or explain it a bit !
DNS in Windows XP[^] Configuring the DNS Resolver Cache in Windows XP[^] Adjust Windows XP DNS Cache Settings[^]
Experience is the sum of all the mistakes you have done.
-
How come some websites can be accessed by IP Addresses and some cannot e.g. Google is accessible through --> http://74.125.19.99/[^] but msn isn't --> http://207.68.173.76/[^] need help because currently working on a DNS Resolver a component for Distributed Web crawler to save those expensive round trips. Thanx in advance !
AliAmjad(MCP) First make it Run THEN make it Run Fast!
A single IP can also cater to multiple websites (domains), in the case of shared hosting. Isn't it?
Vasudevan Deepak Kumar Personal Homepage
Tech Gossips
Regional Weblog (in Tamil) :: Voicing for the Society
Yesterday is a canceled check. Tomorrow is a promissory note. Today is the ready cash. USE IT.