our forum has msgs > 8K
-
Hi, Our website provide users with a forum where they can enter messages. So, typically, a user would enter a message, click submit, and their message would be displayed on the forum. The db that stores these msgs uses a varchar field. Now, we're adding a facility to convert any urls that are entered to links by adding 'a href' tags, so that when users see messages with urls, those urls appear as links, and they can click on them. The problem is that if a user enters a message of 8K chars, then we add 'a href' tags to any urls they have, that makes the message length > 8K, preventing us from storing it in our db. (Please note that we do need to add HTML tags for other reasons as well, not just to convert urls to links.) We thought about adding 'a href' tags during pre-render, thus avoiding storing html in the db, but found that doing such string manipulations, while a user is waiting to see their post displayed on the forum, was expensive. Timing tests revealed that the time to render such pages more than doubled. I have solved such problems previously by adding an extra field in the db to store any overflow data, that is, data that appears after the first 8K chars. However, I learnt of sql server 2005's varchar(max) datatype, and 'am wondering if the best way to solve this problem would be to replace the varchar field with varchar(max). Thanks in advance for any advice. Shefali
-
Hi, Our website provide users with a forum where they can enter messages. So, typically, a user would enter a message, click submit, and their message would be displayed on the forum. The db that stores these msgs uses a varchar field. Now, we're adding a facility to convert any urls that are entered to links by adding 'a href' tags, so that when users see messages with urls, those urls appear as links, and they can click on them. The problem is that if a user enters a message of 8K chars, then we add 'a href' tags to any urls they have, that makes the message length > 8K, preventing us from storing it in our db. (Please note that we do need to add HTML tags for other reasons as well, not just to convert urls to links.) We thought about adding 'a href' tags during pre-render, thus avoiding storing html in the db, but found that doing such string manipulations, while a user is waiting to see their post displayed on the forum, was expensive. Timing tests revealed that the time to render such pages more than doubled. I have solved such problems previously by adding an extra field in the db to store any overflow data, that is, data that appears after the first 8K chars. However, I learnt of sql server 2005's varchar(max) datatype, and 'am wondering if the best way to solve this problem would be to replace the varchar field with varchar(max). Thanks in advance for any advice. Shefali
shefali_sinha wrote:
We thought about adding 'a href' tags during pre-render, thus avoiding storing html in the db, but found that doing such string manipulations, while a user is waiting to see their post displayed on the forum, was expensive. Timing tests revealed that the time to render such pages more than doubled.
To be honest, that sounds odd. If you were using a regular expression to detect URIs then you might want to switch to a cheaper option, or consider using a precompiled regex.
shefali_sinha wrote:
However, I learnt of sql server 2005's varchar(max) datatype, and 'am wondering if the best way to solve this problem would be to replace the varchar field with varchar(max).
Yes, that would be a solution. Be aware that memo-style fields are stored differently (physically) to standard fields. I doubt this will make a difference to a typical forum database though.
Mark Churchill Director Dunn & Churchill Free Download:
Diamond Binding: The simple, powerful, reliable, and effective data layer toolkit for Visual Studio. -
shefali_sinha wrote:
We thought about adding 'a href' tags during pre-render, thus avoiding storing html in the db, but found that doing such string manipulations, while a user is waiting to see their post displayed on the forum, was expensive. Timing tests revealed that the time to render such pages more than doubled.
To be honest, that sounds odd. If you were using a regular expression to detect URIs then you might want to switch to a cheaper option, or consider using a precompiled regex.
shefali_sinha wrote:
However, I learnt of sql server 2005's varchar(max) datatype, and 'am wondering if the best way to solve this problem would be to replace the varchar field with varchar(max).
Yes, that would be a solution. Be aware that memo-style fields are stored differently (physically) to standard fields. I doubt this will make a difference to a typical forum database though.
Mark Churchill Director Dunn & Churchill Free Download:
Diamond Binding: The simple, powerful, reliable, and effective data layer toolkit for Visual Studio.Thanks a lot. Haven't had a chance to look into pre-compiles regex, but will look into it. I simply converted a varchar(8000) field to varchar(max) and another ntext field to varchar(max), the latter conversion was for the sake of better perfromance of a varchar(max) vs. ntext, and my world stayed perfect. Testing reveled no problems. To do the conversion in sql server 2005, I used the statement: alter table alter column varchar(max). Just thought I should document this here in case it helps someone else. Shefali
-
Thanks a lot. Haven't had a chance to look into pre-compiles regex, but will look into it. I simply converted a varchar(8000) field to varchar(max) and another ntext field to varchar(max), the latter conversion was for the sake of better perfromance of a varchar(max) vs. ntext, and my world stayed perfect. Testing reveled no problems. To do the conversion in sql server 2005, I used the statement: alter table alter column varchar(max). Just thought I should document this here in case it helps someone else. Shefali
ntext -> nvarchar(max). Otherwise you'll blow away any unicode data....
Mark Churchill Director Dunn & Churchill Free Download:
Diamond Binding: The simple, powerful, reliable, and effective data layer toolkit for Visual Studio.