obvious
28-06-2003, 21:15
I've tried quite hard to get some answers from ntl about the newsgroups. Here's a run down of the story so far.
I just wanted to put this up somewhere public.
Ben from the Fault Management Team had this to say in ntl.announce at the beginning of november 2002 :-
We have experienced massive and unexpected growth on the News service. This is primarily as a result of a massive increase in the size of binary files being posted by the internet community. In the past these binary files were predominantly photographic images, now we are seeing video files, entire CD images and entire DVD images being posted. I think we are starting to see the impact of larger volumes of broadband connected customers out there on the w.w.w. - posting a CD or DVD on dialup would never have been practical.
To quantify this about 12 months ago the daily inbound news feed was carrying about 150 Gigabytes of data per day and we were serving about 1 Terabyte of data per day out to our customers. This has now increased to about 500 Gigabytes of data inbound each day and in excess of 4 Terabytes of data being read by customers each day. We have a major (just under £1 million) upgrade of the News Servers planned for Q1 2003. We are looking at whether we can bring any of those upgrades forwards. To try an alleviate the burden on the News servers we have enabled NNTP (News) caching on some centralised Inktomi Caches and we have carried out a minor software upgrade to the News Readers to provide additional concurrent connections to the server.
Other ISP's appear to be experiencing similar challenges in expanding their news servers and keeping ahead of the demand curve so this problem is not unique to ntl.
regards,
Ben
Fault Management Team
Technical Support Bureau
from: ntl.announce
I brought this up with Ian Jeffers 17th January 2003
ntl customers have been having trouble accessing the ntl news servers for over a year now. The service levels have dropped, the number of concurrent users who may log in has dropped twice, the number of connections each user may have to the servers has also been reduced. This would be bad enough in itself but additionally, ntl has provided news server access to virgin.net and tesco.net customers (without the limits imposed on ntl customers) thus further decreasing the service level. We have been told repeatedly that there are plans to upgrade the news servers. Then we hear that the plans have been shelved due to lack of money.
1. When the ntl news servers will be fully operational?
2. What level of completion/retention you would expect to be able to provide?
Ian Jeffers' responses
many thanks for your email. I appreciate your frustration with regards the news groups. I know this is less than satisfactory service, please accept my apologies.
While this is outside of my direct area I have been liaising with our Internet team to see how we can improve the service and what the time scales are. This impacts my customers and I want to get the situation resolved. I am sure you do not need me to tell you that the explosion in news groups and in particular the sharing of large files has caught all the ISPs out - as Ben alluded to in his posting.
If you would kindly give me a couple of days I will attempt to come back with a more detailed answer and hopefully some indication of a solution.
Regards
Ian Jeffers
Falling on from my last note I have had a chat with our Internet Product Director, Bill Goodland to get some more details on our short and medium term plans.
Here's a summary of the position -
Short Term
Our Internet Technology Division have introduced a pair of load balanced servers dedicated to the text only newsgroups (which represent by far the largest proportion of newsgroups accessed). The text only newsgroups are now fully accessible and are offering good stability and access.
Improvements have also been made to the front end caching software - at present on a limited trial basis. Where deployed, this has led to a 50% efficiency improvement in network connections. This has the potential to double the amount of concurrent transactions our news service will support. We are therefore taking steps to implement these changes on a wider scale.
Medium Term
Newsgroup data volume is doubling year on year. Downloading files such as music and video clips (binary) has significantly increased the loading on the servers. Acknowledging this we are undertaking the following actions :-
Subject to a business case and approval from the Board some overall newsgroup infrastructure improvements are planned including:
Improving the efficiency of the storage software.
Introduction of new caching storage to the front end.
Increase the storage capacity of the back end servers.
We expect the medium term news group improvement could be made before the end of Q1. But I stress that these are subject to approval from our Board. That said I am using your feedback and similar comments from other customers to highlight the need to make these improvements quickly.
I appreciate the opportunity to give you some more detailed information and thanks for the feedback. I will do my best to try and get this resolved to your satisfaction as soon as possible.
Ian Jeffers
Well thanks for trying Ian but ntl's promises have not been kept.
I just wanted to put this up somewhere public.
Ben from the Fault Management Team had this to say in ntl.announce at the beginning of november 2002 :-
We have experienced massive and unexpected growth on the News service. This is primarily as a result of a massive increase in the size of binary files being posted by the internet community. In the past these binary files were predominantly photographic images, now we are seeing video files, entire CD images and entire DVD images being posted. I think we are starting to see the impact of larger volumes of broadband connected customers out there on the w.w.w. - posting a CD or DVD on dialup would never have been practical.
To quantify this about 12 months ago the daily inbound news feed was carrying about 150 Gigabytes of data per day and we were serving about 1 Terabyte of data per day out to our customers. This has now increased to about 500 Gigabytes of data inbound each day and in excess of 4 Terabytes of data being read by customers each day. We have a major (just under £1 million) upgrade of the News Servers planned for Q1 2003. We are looking at whether we can bring any of those upgrades forwards. To try an alleviate the burden on the News servers we have enabled NNTP (News) caching on some centralised Inktomi Caches and we have carried out a minor software upgrade to the News Readers to provide additional concurrent connections to the server.
Other ISP's appear to be experiencing similar challenges in expanding their news servers and keeping ahead of the demand curve so this problem is not unique to ntl.
regards,
Ben
Fault Management Team
Technical Support Bureau
from: ntl.announce
I brought this up with Ian Jeffers 17th January 2003
ntl customers have been having trouble accessing the ntl news servers for over a year now. The service levels have dropped, the number of concurrent users who may log in has dropped twice, the number of connections each user may have to the servers has also been reduced. This would be bad enough in itself but additionally, ntl has provided news server access to virgin.net and tesco.net customers (without the limits imposed on ntl customers) thus further decreasing the service level. We have been told repeatedly that there are plans to upgrade the news servers. Then we hear that the plans have been shelved due to lack of money.
1. When the ntl news servers will be fully operational?
2. What level of completion/retention you would expect to be able to provide?
Ian Jeffers' responses
many thanks for your email. I appreciate your frustration with regards the news groups. I know this is less than satisfactory service, please accept my apologies.
While this is outside of my direct area I have been liaising with our Internet team to see how we can improve the service and what the time scales are. This impacts my customers and I want to get the situation resolved. I am sure you do not need me to tell you that the explosion in news groups and in particular the sharing of large files has caught all the ISPs out - as Ben alluded to in his posting.
If you would kindly give me a couple of days I will attempt to come back with a more detailed answer and hopefully some indication of a solution.
Regards
Ian Jeffers
Falling on from my last note I have had a chat with our Internet Product Director, Bill Goodland to get some more details on our short and medium term plans.
Here's a summary of the position -
Short Term
Our Internet Technology Division have introduced a pair of load balanced servers dedicated to the text only newsgroups (which represent by far the largest proportion of newsgroups accessed). The text only newsgroups are now fully accessible and are offering good stability and access.
Improvements have also been made to the front end caching software - at present on a limited trial basis. Where deployed, this has led to a 50% efficiency improvement in network connections. This has the potential to double the amount of concurrent transactions our news service will support. We are therefore taking steps to implement these changes on a wider scale.
Medium Term
Newsgroup data volume is doubling year on year. Downloading files such as music and video clips (binary) has significantly increased the loading on the servers. Acknowledging this we are undertaking the following actions :-
Subject to a business case and approval from the Board some overall newsgroup infrastructure improvements are planned including:
Improving the efficiency of the storage software.
Introduction of new caching storage to the front end.
Increase the storage capacity of the back end servers.
We expect the medium term news group improvement could be made before the end of Q1. But I stress that these are subject to approval from our Board. That said I am using your feedback and similar comments from other customers to highlight the need to make these improvements quickly.
I appreciate the opportunity to give you some more detailed information and thanks for the feedback. I will do my best to try and get this resolved to your satisfaction as soon as possible.
Ian Jeffers
Well thanks for trying Ian but ntl's promises have not been kept.