Site scraper alert: new article layout
-
Hey Luc, I've been working on a who's who related app. I was originally going to do it with Silverlight, but there were cross-domain issues that I didn't want to have to deal with, so I switched over to WPF. It's not that tough (as long as Chris doesn't change the element IDs). Of course, I had to code around the fact that we're scraping web pages instead of just hitting a web service...
.45 ACP - because shooting twice is just silly
-----
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"The staggering layers of obscenity in your statement make it a work of art on so many levels." - J. Jystad, 2001John, I don't mind people scraping the WsW as long as there is no Web Service, however I'd prefer the WsW not to be mentioned in the suggestions forum any more than strictly necessary till then. I trust there is a sleeping dog proverb in English too. :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, and update CP Vanity to V2.0 if you haven't already.
-
I would think he could just draw the partially filled stars dynamically, and just have two pre-drawn stars - a filled one, and an empty one.
.45 ACP - because shooting twice is just silly
-----
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"The staggering layers of obscenity in your statement make it a work of art on so many levels." - J. Jystad, 2001There often are many solutions to a single problem, mentioning one often is what it takes to get it done one way or another. :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, and update CP Vanity to V2.0 if you haven't already.
-
To all those who have written site scrapers because I'm too snowed under to get you your API, a warning: beta.codeproject.com has the latest build and has a new article layout so if you wish to update your code to work with the new markup now's your chance. Upload as soon as I've slept.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
It looks really nice. :)
-
John, I don't mind people scraping the WsW as long as there is no Web Service, however I'd prefer the WsW not to be mentioned in the suggestions forum any more than strictly necessary till then. I trust there is a sleeping dog proverb in English too. :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, and update CP Vanity to V2.0 if you haven't already.
Luc Pattyn wrote:
I trust there is a sleeping dog proverb in English too.
'When Dogs sleep, it is an excellent opportunity to lick your own balls'
------------------------------------ I will never again mention that I was the poster of the One Millionth Lounge Post, nor that it was complete drivel. Dalek Dave CCC League Table Link CCC Link[^]
-
Luc Pattyn wrote:
I trust there is a sleeping dog proverb in English too.
'When Dogs sleep, it is an excellent opportunity to lick your own balls'
------------------------------------ I will never again mention that I was the poster of the One Millionth Lounge Post, nor that it was complete drivel. Dalek Dave CCC League Table Link CCC Link[^]
-
I thought it was; When the dog sleeps, is the best time to get your clothes on and the hell out of the house.
Every man can tell how many goats or sheep he possesses, but not how many friends.
-
Luc Pattyn wrote:
I trust there is a sleeping dog proverb in English too.
'When Dogs sleep, it is an excellent opportunity to lick your own balls'
------------------------------------ I will never again mention that I was the poster of the One Millionth Lounge Post, nor that it was complete drivel. Dalek Dave CCC League Table Link CCC Link[^]
Dalek Dave wrote:
When Dogs sleep, it is an excellent opportunity to lick your own balls
I fail to see the logic in that one, however I'll take your word for it, this once. :laugh:
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, and update CP Vanity to V2.0 if you haven't already.
-
Dalek Dave wrote:
When Dogs sleep, it is an excellent opportunity to lick your own balls
I fail to see the logic in that one, however I'll take your word for it, this once. :laugh:
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, and update CP Vanity to V2.0 if you haven't already.
-
To all those who have written site scrapers because I'm too snowed under to get you your API, a warning: beta.codeproject.com has the latest build and has a new article layout so if you wish to update your code to work with the new markup now's your chance. Upload as soon as I've slept.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
-
To all those who have written site scrapers because I'm too snowed under to get you your API, a warning: beta.codeproject.com has the latest build and has a new article layout so if you wish to update your code to work with the new markup now's your chance. Upload as soon as I've slept.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
It actually feels a little more 'zippy' than the main site. I'm not sure if that's just because it is on a separate web farm or if it is just faster. :) Btw, what happened to the http://alpha.codeproject.com/[^] version?
-
To all those who have written site scrapers because I'm too snowed under to get you your API, a warning: beta.codeproject.com has the latest build and has a new article layout so if you wish to update your code to work with the new markup now's your chance. Upload as soon as I've slept.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
I noticed that when I vote for an article the stars at the top do not update unless I manually refresh the page. I also noticed that when I then bookmarked the article the stars disappeared completely and only reappeared once I refreshed the page. Windows XP SP2 w/IE 7
-
It actually feels a little more 'zippy' than the main site. I'm not sure if that's just because it is on a separate web farm or if it is just faster. :) Btw, what happened to the http://alpha.codeproject.com/[^] version?
Andrew Rissing wrote:
what happened to the http://alpha.codeproject.com/\[^\] version?
After over 10 years of development, Code Project Forever has finally come out of alpha. :-D
-
Please don't mention the WsW ever again. :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, and update CP Vanity to V2.0 if you haven't already.
-
Apart from fixing a dumb HTML mistake John pointed out, no. (so, actually, yes. Slightly)
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
-
Thanks for the advance notice. No problem whatsoever, I'm not scraping article pages at all. FWIW: I think it again looks better, with a more structured title block. I like the stars. Minor comments: 1. there is a bug where sometimes (had it twice in 5 visits), not sure when/how/why, a tooltip (I think it was "Discuss this article") is already showing when the page opens up, dislocating the "print", "bookmark", "discuss", "report" icons. 2. The pink background seems like a step backward, the white one was more modern IMO. I would opt for a white title box, with an orange border, and containing the new content as is (so the previous style and the latest layout). 3. the spacing between "Edit", "Delete", "Get HTML" could be improved (e.g. by adding a space and a nbsp between edit and delete) 4. The stars should support partial fill, i.e. 4.31 and 4.49 would show differently (that would take them to be rendered separately and chosen from a collection of some 6 to 11 different star symbols). :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, and update CP Vanity to V2.0 if you haven't already.
1. I'll keep my eye open for it 2. Pink? Where? It's a light tan here. 3. Will add 4. They do. The stars are a single transparent gif. I draw a fill rectangle behind them. The stars fill up accordingly. It's not as obvious when you are close to a whole unit, but anything between 0.25 and 0.75 will show the effect.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
-
Andrew Rissing wrote:
what happened to the http://alpha.codeproject.com/\[^\] version?
After over 10 years of development, Code Project Forever has finally come out of alpha. :-D
Tentatively out of alpha.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
-
1. I'll keep my eye open for it 2. Pink? Where? It's a light tan here. 3. Will add 4. They do. The stars are a single transparent gif. I draw a fill rectangle behind them. The stars fill up accordingly. It's not as obvious when you are close to a whole unit, but anything between 0.25 and 0.75 will show the effect.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
More on the same: 1. it is 4 out of 7 now; I have screen shots and can reproduce: - using FF3.6, open an article in the current style (www.codeproject.com/...) - move mouse to URL textbox, edit to beta, hit ENTER (so mouse clearly is not within the article at any time) - article shows with either "Discuss this article" or "report this article as inappropriate" tooltip showing I have mailed you a screen shot. 2. OK, it is R=252,G=243,B=226; pretty close to Color.Linen 4. Noticed it now you said it; the intentions are good, however I think it would work better when the empty stars would have a thinner outline. :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, and update CP Vanity to V2.0 if you haven't already.
modified on Tuesday, December 7, 2010 7:16 PM
-
It actually feels a little more 'zippy' than the main site. I'm not sure if that's just because it is on a separate web farm or if it is just faster. :) Btw, what happened to the http://alpha.codeproject.com/[^] version?
Andrew Rissing wrote:
what happened to the http://alpha.codeproject.com/\[^\] version?
Why, they forgot to move it to the new location obviously. :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, and update CP Vanity to V2.0 if you haven't already.
-
More on the same: 1. it is 4 out of 7 now; I have screen shots and can reproduce: - using FF3.6, open an article in the current style (www.codeproject.com/...) - move mouse to URL textbox, edit to beta, hit ENTER (so mouse clearly is not within the article at any time) - article shows with either "Discuss this article" or "report this article as inappropriate" tooltip showing I have mailed you a screen shot. 2. OK, it is R=252,G=243,B=226; pretty close to Color.Linen 4. Noticed it now you said it; the intentions are good, however I think it would work better when the empty stars would have a thinner outline. :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, and update CP Vanity to V2.0 if you haven't already.
modified on Tuesday, December 7, 2010 7:16 PM
1. I think this is due to the stylesheet being loaded from the same server (beta) instad of from s.codeproject.com, meaning the browser is less able to load in parallel. However, it's extremely odd it's only one of the bubbles, not all. I've been playing with it and, unfortunately, can't replicate. I'll keep trying. 4. I originally had very thin stars but they looked really jagged. I'll probably have to redraw them by hand and tweak the anti-aliasing.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
-
1. I think this is due to the stylesheet being loaded from the same server (beta) instad of from s.codeproject.com, meaning the browser is less able to load in parallel. However, it's extremely odd it's only one of the bubbles, not all. I've been playing with it and, unfortunately, can't replicate. I'll keep trying. 4. I originally had very thin stars but they looked really jagged. I'll probably have to redraw them by hand and tweak the anti-aliasing.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
1. that sounds like the definition of a timing hazard. You need synchronization then, not more hardware. :laugh: I always wondered why you had the s-domain. :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, and update CP Vanity to V2.0 if you haven't already.
