Most unusable technology award (my nomination - regular expressions)
-
Every now and then I need to solve a problem for which regular expressions looks like it is the "perfect" answer. Today that happens to be validating and extracting UK postal codes from addresses. BUT Every time I try to use regular expressions I find that no-one (especially me) has a clue how to use them and that all "posted" solutions can be demonstrated as flawed and therefore dangerous to use. The sheer complexity of the expressions makes them virtually impossible to read and therefore understand. I post the wikipedia solution to demonstrate my case (GIR 0AA)|(((A[BL]|B[ABDHLNRSTX]?|C[ABFHMORTVW]|D[ADEGHLNTY]|E[HNX]?|F[KY]|G[LUY]?|H[ADGPRSUX]|I[GMPV]|JE|K [ATWY]|L[ADELNSU]?|M[EKL]?|N[EGNPRW]?|O[LX]|P[AEHLOR]|R[GHM]|S[AEGKLMNOPRSTY]?|T[ADFNQRSW]|UB|W[ADFNRSV] |YO|ZE)[1-9]?[0-9]|((E|N|NW|SE|SW|W)1|EC[1-4]|WC[12])[A-HJKMNPR-Y]|(SW|W)([2-9]|[1-9][0-9])|EC[1-9] [0-9]) [0-9][ABD-HJLNP-UW-Z]{2}) Can anyone think of a less usable technology?
www.it-workplace.com
"If a man speaks in a forest where there is no woman to hear him, is he still wrong?"A single Regex is really not sutable for UK post code, (incode and outcode), validation as there are post codes still in use that do not conform to current rules, (GIR 0AA as above). There are 6 valid post code formats plus one invalid one that's in use, I would probably validate each valid, (or invalid an in use), format individually with its own regex as what you've got there is pretty much unreadable... Either that or comment in a reference to the post code standards, (which can be found here...[^]), and apologise profusely to anyone that comes to that monstrosity after you :-)
Rhys "Technological progress is like an axe in the hands of a pathological criminal" "Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe"
-
Every now and then I need to solve a problem for which regular expressions looks like it is the "perfect" answer. Today that happens to be validating and extracting UK postal codes from addresses. BUT Every time I try to use regular expressions I find that no-one (especially me) has a clue how to use them and that all "posted" solutions can be demonstrated as flawed and therefore dangerous to use. The sheer complexity of the expressions makes them virtually impossible to read and therefore understand. I post the wikipedia solution to demonstrate my case (GIR 0AA)|(((A[BL]|B[ABDHLNRSTX]?|C[ABFHMORTVW]|D[ADEGHLNTY]|E[HNX]?|F[KY]|G[LUY]?|H[ADGPRSUX]|I[GMPV]|JE|K [ATWY]|L[ADELNSU]?|M[EKL]?|N[EGNPRW]?|O[LX]|P[AEHLOR]|R[GHM]|S[AEGKLMNOPRSTY]?|T[ADFNQRSW]|UB|W[ADFNRSV] |YO|ZE)[1-9]?[0-9]|((E|N|NW|SE|SW|W)1|EC[1-4]|WC[12])[A-HJKMNPR-Y]|(SW|W)([2-9]|[1-9][0-9])|EC[1-9] [0-9]) [0-9][ABD-HJLNP-UW-Z]{2}) Can anyone think of a less usable technology?
www.it-workplace.com
"If a man speaks in a forest where there is no woman to hear him, is he still wrong?"Regexes are very usable technology - it's just that the syntax is somewhat arcane and it isn't easy to work out. Having said that, a tool like Expresso[^] can really help - it's free, and it examines, explains, generates and helps you create Regular expressions. They do get overused though: I would probably check a postcode against an online address checker rather than use a regex anyway - if only to save errors in data entry of the address.
Ideological Purity is no substitute for being able to stick your thumb down a pipe to stop the water
-
Every now and then I need to solve a problem for which regular expressions looks like it is the "perfect" answer. Today that happens to be validating and extracting UK postal codes from addresses. BUT Every time I try to use regular expressions I find that no-one (especially me) has a clue how to use them and that all "posted" solutions can be demonstrated as flawed and therefore dangerous to use. The sheer complexity of the expressions makes them virtually impossible to read and therefore understand. I post the wikipedia solution to demonstrate my case (GIR 0AA)|(((A[BL]|B[ABDHLNRSTX]?|C[ABFHMORTVW]|D[ADEGHLNTY]|E[HNX]?|F[KY]|G[LUY]?|H[ADGPRSUX]|I[GMPV]|JE|K [ATWY]|L[ADELNSU]?|M[EKL]?|N[EGNPRW]?|O[LX]|P[AEHLOR]|R[GHM]|S[AEGKLMNOPRSTY]?|T[ADFNQRSW]|UB|W[ADFNRSV] |YO|ZE)[1-9]?[0-9]|((E|N|NW|SE|SW|W)1|EC[1-4]|WC[12])[A-HJKMNPR-Y]|(SW|W)([2-9]|[1-9][0-9])|EC[1-9] [0-9]) [0-9][ABD-HJLNP-UW-Z]{2}) Can anyone think of a less usable technology?
www.it-workplace.com
"If a man speaks in a forest where there is no woman to hear him, is he still wrong?"I definitely agree with Slacker on this one. This is when a useful tool becomes an totally unhelpful. Yes regular expressions are very helpful when you can fit the pattern onto one line - any more than one line means another tool needs to be used.
“That which can be asserted without evidence, can be dismissed without evidence.”
― Christopher Hitchens
-
A single Regex is really not sutable for UK post code, (incode and outcode), validation as there are post codes still in use that do not conform to current rules, (GIR 0AA as above). There are 6 valid post code formats plus one invalid one that's in use, I would probably validate each valid, (or invalid an in use), format individually with its own regex as what you've got there is pretty much unreadable... Either that or comment in a reference to the post code standards, (which can be found here...[^]), and apologise profusely to anyone that comes to that monstrosity after you :-)
Rhys "Technological progress is like an axe in the hands of a pathological criminal" "Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe"
-
Every now and then I need to solve a problem for which regular expressions looks like it is the "perfect" answer. Today that happens to be validating and extracting UK postal codes from addresses. BUT Every time I try to use regular expressions I find that no-one (especially me) has a clue how to use them and that all "posted" solutions can be demonstrated as flawed and therefore dangerous to use. The sheer complexity of the expressions makes them virtually impossible to read and therefore understand. I post the wikipedia solution to demonstrate my case (GIR 0AA)|(((A[BL]|B[ABDHLNRSTX]?|C[ABFHMORTVW]|D[ADEGHLNTY]|E[HNX]?|F[KY]|G[LUY]?|H[ADGPRSUX]|I[GMPV]|JE|K [ATWY]|L[ADELNSU]?|M[EKL]?|N[EGNPRW]?|O[LX]|P[AEHLOR]|R[GHM]|S[AEGKLMNOPRSTY]?|T[ADFNQRSW]|UB|W[ADFNRSV] |YO|ZE)[1-9]?[0-9]|((E|N|NW|SE|SW|W)1|EC[1-4]|WC[12])[A-HJKMNPR-Y]|(SW|W)([2-9]|[1-9][0-9])|EC[1-9] [0-9]) [0-9][ABD-HJLNP-UW-Z]{2}) Can anyone think of a less usable technology?
www.it-workplace.com
"If a man speaks in a forest where there is no woman to hear him, is he still wrong?" -
Andrew Wiles wrote:
Can anyone think of a less usable technology?
.Net? J/K :rolleyes: I use regualar expressions, via Perl, all the time at work and it is a great tool. With that said, not all problems can or should be solved with regexes. Just looking at the example you have shown above, without really researching it, looks to me to be the work of an amateur. The rule of thumb with regualar expressions is that the longer and more complex they are, the more they are filled with bugs and bad matches. If your regex is that long, you are doing something wrong or you need to find another solution. my 2 cents.
Just along for the ride. "the meat from that butcher is just the dogs danglies, absolutely amazing cuts of beef." - DaveAuld (2011)
"No, that is just the earthly manifestation of the Great God Retardon." - Nagy Vilmos (2011) "It is the celestial scrotum of good luck!" - Nagy Vilmos (2011) -
If the only tool you've got is a hammer, every problem looks like a nail! ;P Regular expressions are a very powerful tool when it comes to string matching, although as you've already stated the readability does suffer when you try to tackle complex matching operations. Still I think it is far easier than constructing a dedicated parser to accomplish the task even though there are lots of tools out there that help you in doing so: ANTLR, ProGrammar, Lex & Yacc (Bison) etc. I'm not sure though if reading a grammar is going to tickle your fanncy! :-D Cheers, Manfred
"With sufficient thrust, pigs fly just fine."
Ross Callon, The Twelve Networking Truths, RFC1925
Manfred R. Bihy wrote:
If the only tool you've got is a hammer, every problem looks like a nail!
I like that. :thumbsup:
Just along for the ride. "the meat from that butcher is just the dogs danglies, absolutely amazing cuts of beef." - DaveAuld (2011)
"No, that is just the earthly manifestation of the Great God Retardon." - Nagy Vilmos (2011) "It is the celestial scrotum of good luck!" - Nagy Vilmos (2011) -
Every now and then I need to solve a problem for which regular expressions looks like it is the "perfect" answer. Today that happens to be validating and extracting UK postal codes from addresses. BUT Every time I try to use regular expressions I find that no-one (especially me) has a clue how to use them and that all "posted" solutions can be demonstrated as flawed and therefore dangerous to use. The sheer complexity of the expressions makes them virtually impossible to read and therefore understand. I post the wikipedia solution to demonstrate my case (GIR 0AA)|(((A[BL]|B[ABDHLNRSTX]?|C[ABFHMORTVW]|D[ADEGHLNTY]|E[HNX]?|F[KY]|G[LUY]?|H[ADGPRSUX]|I[GMPV]|JE|K [ATWY]|L[ADELNSU]?|M[EKL]?|N[EGNPRW]?|O[LX]|P[AEHLOR]|R[GHM]|S[AEGKLMNOPRSTY]?|T[ADFNQRSW]|UB|W[ADFNRSV] |YO|ZE)[1-9]?[0-9]|((E|N|NW|SE|SW|W)1|EC[1-4]|WC[12])[A-HJKMNPR-Y]|(SW|W)([2-9]|[1-9][0-9])|EC[1-9] [0-9]) [0-9][ABD-HJLNP-UW-Z]{2}) Can anyone think of a less usable technology?
www.it-workplace.com
"If a man speaks in a forest where there is no woman to hear him, is he still wrong?"I think you are slightly over doing it. Isn't
[A-Z]{1,2}[0-9R][0-9A-Z]? [0-9][ABD-HJLNP-UW-Z]{2}enough?
Panic, Chaos, Destruction. My work here is done. Drink. Get drunk. Fall over - P O'H OK, I will win to day or my name isn't Ethel Crudacre! - DD Ethel Crudacre I cannot live by bread alone. Bacon and ketchup are needed as well. - Trollslayer Have a bit more patience with newbies. Of course some of them act dumb - they're often *students*, for heaven's sake - Terry Pratchett
-
Every now and then I need to solve a problem for which regular expressions looks like it is the "perfect" answer. Today that happens to be validating and extracting UK postal codes from addresses. BUT Every time I try to use regular expressions I find that no-one (especially me) has a clue how to use them and that all "posted" solutions can be demonstrated as flawed and therefore dangerous to use. The sheer complexity of the expressions makes them virtually impossible to read and therefore understand. I post the wikipedia solution to demonstrate my case (GIR 0AA)|(((A[BL]|B[ABDHLNRSTX]?|C[ABFHMORTVW]|D[ADEGHLNTY]|E[HNX]?|F[KY]|G[LUY]?|H[ADGPRSUX]|I[GMPV]|JE|K [ATWY]|L[ADELNSU]?|M[EKL]?|N[EGNPRW]?|O[LX]|P[AEHLOR]|R[GHM]|S[AEGKLMNOPRSTY]?|T[ADFNQRSW]|UB|W[ADFNRSV] |YO|ZE)[1-9]?[0-9]|((E|N|NW|SE|SW|W)1|EC[1-4]|WC[12])[A-HJKMNPR-Y]|(SW|W)([2-9]|[1-9][0-9])|EC[1-9] [0-9]) [0-9][ABD-HJLNP-UW-Z]{2}) Can anyone think of a less usable technology?
www.it-workplace.com
"If a man speaks in a forest where there is no woman to hear him, is he still wrong?"mobile phones with camera inside. Apple products. Fritzbox networking stuff. But really: String parsing is a pain in the a**, no matter how it's done. I avoid that as much as possible. and therefor I'm not forced to use regex. But when I do, I struggle my way through it like you did. Coffee and chocolate helps.
regards Torsten When I'm not working
-
I think you are slightly over doing it. Isn't
[A-Z]{1,2}[0-9R][0-9A-Z]? [0-9][ABD-HJLNP-UW-Z]{2}enough?
Panic, Chaos, Destruction. My work here is done. Drink. Get drunk. Fall over - P O'H OK, I will win to day or my name isn't Ethel Crudacre! - DD Ethel Crudacre I cannot live by bread alone. Bacon and ketchup are needed as well. - Trollslayer Have a bit more patience with newbies. Of course some of them act dumb - they're often *students*, for heaven's sake - Terry Pratchett
-
mobile phones with camera inside. Apple products. Fritzbox networking stuff. But really: String parsing is a pain in the a**, no matter how it's done. I avoid that as much as possible. and therefor I'm not forced to use regex. But when I do, I struggle my way through it like you did. Coffee and chocolate helps.
regards Torsten When I'm not working
-
mobile phones with camera inside. Apple products. Fritzbox networking stuff. But really: String parsing is a pain in the a**, no matter how it's done. I avoid that as much as possible. and therefor I'm not forced to use regex. But when I do, I struggle my way through it like you did. Coffee and chocolate helps.
regards Torsten When I'm not working
TorstenH. wrote:
Coffee
Nothing like a good cup of coffee to awaken the coding beast within. :)
Just along for the ride. "the meat from that butcher is just the dogs danglies, absolutely amazing cuts of beef." - DaveAuld (2011)
"No, that is just the earthly manifestation of the Great God Retardon." - Nagy Vilmos (2011) "It is the celestial scrotum of good luck!" - Nagy Vilmos (2011) -
Every now and then I need to solve a problem for which regular expressions looks like it is the "perfect" answer. Today that happens to be validating and extracting UK postal codes from addresses. BUT Every time I try to use regular expressions I find that no-one (especially me) has a clue how to use them and that all "posted" solutions can be demonstrated as flawed and therefore dangerous to use. The sheer complexity of the expressions makes them virtually impossible to read and therefore understand. I post the wikipedia solution to demonstrate my case (GIR 0AA)|(((A[BL]|B[ABDHLNRSTX]?|C[ABFHMORTVW]|D[ADEGHLNTY]|E[HNX]?|F[KY]|G[LUY]?|H[ADGPRSUX]|I[GMPV]|JE|K [ATWY]|L[ADELNSU]?|M[EKL]?|N[EGNPRW]?|O[LX]|P[AEHLOR]|R[GHM]|S[AEGKLMNOPRSTY]?|T[ADFNQRSW]|UB|W[ADFNRSV] |YO|ZE)[1-9]?[0-9]|((E|N|NW|SE|SW|W)1|EC[1-4]|WC[12])[A-HJKMNPR-Y]|(SW|W)([2-9]|[1-9][0-9])|EC[1-9] [0-9]) [0-9][ABD-HJLNP-UW-Z]{2}) Can anyone think of a less usable technology?
www.it-workplace.com
"If a man speaks in a forest where there is no woman to hear him, is he still wrong?"There are people who massage their ego by making the biggest and most labrythine RE possible. X|
-
I think you are slightly over doing it. Isn't
[A-Z]{1,2}[0-9R][0-9A-Z]? [0-9][ABD-HJLNP-UW-Z]{2}enough?
Panic, Chaos, Destruction. My work here is done. Drink. Get drunk. Fall over - P O'H OK, I will win to day or my name isn't Ethel Crudacre! - DD Ethel Crudacre I cannot live by bread alone. Bacon and ketchup are needed as well. - Trollslayer Have a bit more patience with newbies. Of course some of them act dumb - they're often *students*, for heaven's sake - Terry Pratchett
Nearly, but doesn't cover the case where the user has not entered the space (i.e. 'LS1 9EL' vs 'LS19EL'). We also have some other occasional but common variations such as 'LS1_9EL' that we can try to parse for. My understanding is also that whilst this expression will validate the general format of the postcode there are specific exceptions that it does not cover. Unfortunately the task is not one of validating data at point of entry but matching data that has not been properly validated in the first place (>5m records), so refering to a web service such as the BING api is ruled out for performance reasons.
www.it-workplace.com
"If a man speaks in a forest where there is no woman to hear him, is he still wrong?" -
Manfred R. Bihy wrote:
If the only tool you've got is a hammer, every problem looks like a nail!
I like that. :thumbsup:
Just along for the ride. "the meat from that butcher is just the dogs danglies, absolutely amazing cuts of beef." - DaveAuld (2011)
"No, that is just the earthly manifestation of the Great God Retardon." - Nagy Vilmos (2011) "It is the celestial scrotum of good luck!" - Nagy Vilmos (2011)I first saw that line was used in the end-of-mag article for "Language" magazine (an early issue). The article showed how to write a program that called a function that returned the cube of the numbers 1 through 10, doing so in a large number of languages for comparison. Interestingly, the shortest version was one of the Unix shells. I believe the magazine to have gone away to wherever they go to.
"The difference between genius and stupidity is that genius has its limits." - Albert Einstein
"As far as we know, our computer has never had an undetected error." - Weisert
"If you are searching for perfection in others, then you seek disappointment. If you are seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010
-
If the only tool you've got is a hammer, every problem looks like a nail! ;P Regular expressions are a very powerful tool when it comes to string matching, although as you've already stated the readability does suffer when you try to tackle complex matching operations. Still I think it is far easier than constructing a dedicated parser to accomplish the task even though there are lots of tools out there that help you in doing so: ANTLR, ProGrammar, Lex & Yacc (Bison) etc. I'm not sure though if reading a grammar is going to tickle your fanncy! :-D Cheers, Manfred
"With sufficient thrust, pigs fly just fine."
Ross Callon, The Twelve Networking Truths, RFC1925
C# is my hammer! RegEx is one of those tools that was bought a long time ago for a very specific job and now lurks at the back of the toolbox. It only comes out very infrequently so: 1. I have to read the manual each time I use it because I have forgotten the instructions since the last time. 2. I am unwilling to invest the amount of time required to master its use because (refer to 1) I only use it very infrequently. 3. Each time I use it I have a lingering concern that I have left a problem for the next poor soul who has to maintain the solution.
www.it-workplace.com
"If a man speaks in a forest where there is no woman to hear him, is he still wrong?" -
Nearly, but doesn't cover the case where the user has not entered the space (i.e. 'LS1 9EL' vs 'LS19EL'). We also have some other occasional but common variations such as 'LS1_9EL' that we can try to parse for. My understanding is also that whilst this expression will validate the general format of the postcode there are specific exceptions that it does not cover. Unfortunately the task is not one of validating data at point of entry but matching data that has not been properly validated in the first place (>5m records), so refering to a web service such as the BING api is ruled out for performance reasons.
www.it-workplace.com
"If a man speaks in a forest where there is no woman to hear him, is he still wrong?"Andrew Wiles wrote:
doesn't cover the case where the user has not entered the space (i.e. 'LS1 9EL' vs 'LS19EL'). We also have some other occasional but common variations such as 'LS1_9EL'
Okay then:
[A-Z]{1,2}[0-9R][0-9A-Z]?[ _-]{0,1}[0-9][ABD-HJLNP-UW-Z]{2}That'll work for no separator, ' ', '_' or '-'. If you want to validate the content then I would look, if possible, at not using a single regex but one per style.
Panic, Chaos, Destruction. My work here is done. Drink. Get drunk. Fall over - P O'H OK, I will win to day or my name isn't Ethel Crudacre! - DD Ethel Crudacre I cannot live by bread alone. Bacon and ketchup are needed as well. - Trollslayer Have a bit more patience with newbies. Of course some of them act dumb - they're often *students*, for heaven's sake - Terry Pratchett
-
Every now and then I need to solve a problem for which regular expressions looks like it is the "perfect" answer. Today that happens to be validating and extracting UK postal codes from addresses. BUT Every time I try to use regular expressions I find that no-one (especially me) has a clue how to use them and that all "posted" solutions can be demonstrated as flawed and therefore dangerous to use. The sheer complexity of the expressions makes them virtually impossible to read and therefore understand. I post the wikipedia solution to demonstrate my case (GIR 0AA)|(((A[BL]|B[ABDHLNRSTX]?|C[ABFHMORTVW]|D[ADEGHLNTY]|E[HNX]?|F[KY]|G[LUY]?|H[ADGPRSUX]|I[GMPV]|JE|K [ATWY]|L[ADELNSU]?|M[EKL]?|N[EGNPRW]?|O[LX]|P[AEHLOR]|R[GHM]|S[AEGKLMNOPRSTY]?|T[ADFNQRSW]|UB|W[ADFNRSV] |YO|ZE)[1-9]?[0-9]|((E|N|NW|SE|SW|W)1|EC[1-4]|WC[12])[A-HJKMNPR-Y]|(SW|W)([2-9]|[1-9][0-9])|EC[1-9] [0-9]) [0-9][ABD-HJLNP-UW-Z]{2}) Can anyone think of a less usable technology?
www.it-workplace.com
"If a man speaks in a forest where there is no woman to hear him, is he still wrong?"I find the issue with regular expressions is often how they re presented. They can be formatted to be quite neat and intuitive, if one takes the time. Many don't, and unfortunately I fall into the trap too. However, they are so ridiculously useful.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
-
Andrew Wiles wrote:
doesn't cover the case where the user has not entered the space (i.e. 'LS1 9EL' vs 'LS19EL'). We also have some other occasional but common variations such as 'LS1_9EL'
Okay then:
[A-Z]{1,2}[0-9R][0-9A-Z]?[ _-]{0,1}[0-9][ABD-HJLNP-UW-Z]{2}That'll work for no separator, ' ', '_' or '-'. If you want to validate the content then I would look, if possible, at not using a single regex but one per style.
Panic, Chaos, Destruction. My work here is done. Drink. Get drunk. Fall over - P O'H OK, I will win to day or my name isn't Ethel Crudacre! - DD Ethel Crudacre I cannot live by bread alone. Bacon and ketchup are needed as well. - Trollslayer Have a bit more patience with newbies. Of course some of them act dumb - they're often *students*, for heaven's sake - Terry Pratchett
-
I find the issue with regular expressions is often how they re presented. They can be formatted to be quite neat and intuitive, if one takes the time. Many don't, and unfortunately I fall into the trap too. However, they are so ridiculously useful.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
