Slooooooooooooooooooooooowwww!
-
Hey Tim, The bottleneck is squarely in the SQL server, so although the session count is in the stratosphere, the web server idles along at about 20% CPU load. The SQL Server is pinned during the times you see the site running slow. The path to solving this is clear: either an effective way of reducing the number of SQL queries that are now run (and they are run for just about everything on CP), partial page caching (whole page caching or reverse proxying would force us to remove much of the nice personalization stuff, and I'd hate to see that happen), or a big hardware upgrade. All of these things are underway. Something is in the works to allow display of the whole tree at one pull. The problem here has always been making the site as usable as possible while not completing killing the site's page views (thus advertising potential). Although it's an archaic measurement, page views are still used as a metric to determine a site's importance. David
Ok - I guess blaming everthing on sessions was a bit simplistic. Site's smokin today tho. What's different? Something is in the works to allow display of the whole tree at one pull. The problem here has always been making the site as usable as possible while not completing killing the site's page views (thus advertising potential). Although it's an archaic measurement, page views are still used as a metric to determine a site's importance. Well, yes but if I get disgusted with the prospect of wading through 20 refreshes to get 20 messages and just disconnect (and, sadly, I often do), where does that leave the hit count? And leaving everything server side like that might be part of the reason the SQL server is overworked.
-
Whatever it was..... It seems to be fixed now. I still don't get a few pages immediately and have to hit refresh, but it happens much faster now. Now I can USE the site again. Brad I'm still curious about what caused the problem though!!!!
-
No offense, but I don't buy it. I'm getting < 60ms pings to the server, which tells me there's plenty of bandwidth available. That would mean that the bottleneck is the servers and scripts themselves. It's 4am CST on a saturday here, and the server is still dog slow (upwards of 2 minutes to load a page). Now, I realize that there are a lot of people from around the world that access the site, and it's not 4am everywhere, but geez.. it's a saturday, traditionally codeprojects lightest load is on weekends (if you go by the number of posts). This is what i've noticed: 1) I haven't seen a huge increase in posting in recent weeks. Now, one would think that if traffic increased 40% or more in a few weeks we'd see at least a 40% increase in posts on average (perhaps a little more or a little less). 2) While codeproject has been steadily getting slower over the last few weeks, this massive drop in performance happened (literally) overnight. The only thing that would explain this would be a huge advertising campaign, and I thought I read that this was still weeks away, or a major change in the scripts or queries which cause a bigger performance drop (I suppose hard disk fragmentation might cause some as well) I don't know... if my sysadmin was out mountain climbing and schmoozing with product managers while his site was in seriously degraded performance, I'd fire him (That's a joke, I know chris is hard at work.. maybe he should fire himself ;) )
I don't know... if my sysadmin was out mountain climbing and schmoozing with product managers while his site was in seriously degraded performance, I'd fire him I don't know...if I had access to a free site like this and knew the guy running it was trying to juggle a million things while dealing with demanding readers I'd say "Thanks".
-
Wouldn't it be funny, if eventually we discovered that the slow performance is due to, say .NET itself? Sorry for being politically incorrect (can't help it). ;P
Well, it might be funny but it wouldn't be accurate. ;P Our initial tests have shown that moving to .Net will result in a ~5x improvement in performance of page generation. Throw in web gardens, partial page caching, etc., and it should all make for a nice bump in performance. No, the problem right now is the load on the SQL Server, and the majority of that load is caused by the forum code. David
-
Something else you might look into is MS's web cache software, SWC. I know that Version 3 is in beta right now and is expected to be released soon, but version 2 is available and is supposed to greatly improve the speed by not requiring dynamic content to be generated as often.
SWC (Scalable Web Cache) is only good for static pages, and only had significant benefit for NT 4.0. It cached static pages in memory, using a per-processor memory cache (making the cache per-processor made it faster). W2K already does good memory caching of files, so SWC doesn't add much performance. In any event, SWC is only good for static pages, which won't work for CodeProject because they do ads and all sorts of dynamic stuff. Of course, my preference for improving performance are our products: Active Page Generator for pre-rendering pages and page fragments. And ASPCache for data caching and page fragment caching. John Crim johnc@webgecko.com http://www.webgecko.com
-
I don't know... if my sysadmin was out mountain climbing and schmoozing with product managers while his site was in seriously degraded performance, I'd fire him I don't know...if I had access to a free site like this and knew the guy running it was trying to juggle a million things while dealing with demanding readers I'd say "Thanks".
I don't know, did you not see the fact that I put a smiley in there to indicate it was a joke (and explicitly qualified it afterwards?) Why do you try to take a comment out of it's context and attempt to portray it as something else?
-
Well, it might be funny but it wouldn't be accurate. ;P Our initial tests have shown that moving to .Net will result in a ~5x improvement in performance of page generation. Throw in web gardens, partial page caching, etc., and it should all make for a nice bump in performance. No, the problem right now is the load on the SQL Server, and the majority of that load is caused by the forum code. David
First I have to say that I really LOVE this site, and the forums :-) I have a few questions I really would like you to answar; How many daily sessions are there on this site? How many pagehits? What database are you using? (SQL Server 7.0 I guess) I can see that you are using IIS 5.0... Are the webserver and database server on the same hardware-box? What kind of hardware runs the database server (processors, ram, disks, raid)? How big is the database-file? I mean, I have been working with databases for several years, and I have seen queries that took almost 30 seconds to execute, but when we optimized the SQL code, they executed in under 1 second... No offence, really, I have a great respect for you guys, it's just that I know my way around SQL Server, and don't see why it's so slow... Lately Chriss was talking about a Serverfarm, a database cluster and that kinda stuff, but I guess you haven't implemented it yet... - Anders
-
First I have to say that I really LOVE this site, and the forums :-) I have a few questions I really would like you to answar; How many daily sessions are there on this site? How many pagehits? What database are you using? (SQL Server 7.0 I guess) I can see that you are using IIS 5.0... Are the webserver and database server on the same hardware-box? What kind of hardware runs the database server (processors, ram, disks, raid)? How big is the database-file? I mean, I have been working with databases for several years, and I have seen queries that took almost 30 seconds to execute, but when we optimized the SQL code, they executed in under 1 second... No offence, really, I have a great respect for you guys, it's just that I know my way around SQL Server, and don't see why it's so slow... Lately Chriss was talking about a Serverfarm, a database cluster and that kinda stuff, but I guess you haven't implemented it yet... - Anders
Hey Anders, The big problem is that we are connecting to the databases multiple times for a page, and executing multiple queries where only a single query is really needed. The site simply grew too big too fast (not that that's a bad thing!) and as is often the way, our 'proof of concept' site became our production site. Hence the work being done on rewriting from scratch to do away with all the kludges and do it 'right'. It's a juggling act between ensuring that articles get posted and updates made, and ensuring that we try and get the new site written ASAP. Uwe has been working on rewriting the forums to reduce the number of queries which should take a load off. Hardware is still coming, .NET is still coming - all sorts of things are still coming :) cheers, Chris Maunder
-
First I have to say that I really LOVE this site, and the forums :-) I have a few questions I really would like you to answar; How many daily sessions are there on this site? How many pagehits? What database are you using? (SQL Server 7.0 I guess) I can see that you are using IIS 5.0... Are the webserver and database server on the same hardware-box? What kind of hardware runs the database server (processors, ram, disks, raid)? How big is the database-file? I mean, I have been working with databases for several years, and I have seen queries that took almost 30 seconds to execute, but when we optimized the SQL code, they executed in under 1 second... No offence, really, I have a great respect for you guys, it's just that I know my way around SQL Server, and don't see why it's so slow... Lately Chriss was talking about a Serverfarm, a database cluster and that kinda stuff, but I guess you haven't implemented it yet... - Anders
Hey Anders, Chris has added a site description to the about page. Here's a quick link: http://www.codeproject.com/info/about.asp David
-
I don't know, did you not see the fact that I put a smiley in there to indicate it was a joke (and explicitly qualified it afterwards?) Why do you try to take a comment out of it's context and attempt to portray it as something else?
> did you not see the fact that I put a smiley in there to indicate it was a joke > (and explicitly qualified it afterwards?) Why do you try to take a comment > out of it's context and attempt to portray it as something else? That seems to happen a LOT around here... :( Peace! -=- James.