How to maintain an external C++ DLL developed with Literate Programming?
-
Today I received the source code for a DLL developed by a small company for us. The source code was part of the deal, and one of it's purposes was that we should be able to continue maintenance efforts once the main functionality of the library is done, and no more big issues remain. Unfortunately, I've found the source code to be written auto-generated as CWEB, using a Literate Programming tool. That means these files cannot be directly read by the compiler, nor interpreted with the integrated tools of the IDE. I've requested the 'tangled' C++ source files (i. e. processed by ctangle.exe into .h and .cpp files), to be able to at least read them into my UML tool, but that code has all comments and blanks removed, so is not in a suitable form to be read in the IDE either. In the UML tool I now have several hundred classes, structs, typedefs and enums all entangled into one huge diagram. The information I get from this is nicely summarized into one word: HELP! What I currently have is: - 359 auto- generated CWEB files (.w extension, some with a .hw extension, and a few others), in one and the same directory (actually I believe some of these are in fact not generated and don't belong there) - these CWEB files have comments and proper formatting, but are unreadable by my tools - alternately 245 C++ files (no idea why there are less of these) - C++ files have no comments or blanks or tabs (but do have linebreaks at least) - There's no 1-1 relation between source files and classes (neither for the CWEB files nor the C++ files): the total number of classes is around 300-500. Apparently some files contain dozens of classes, others none at all. In short, it's a total mess. I'm now at a loss what to do about it: I can no longer trust the original programmer to maintain it (there already has been significant doubt before), but I have no idea how to take care of it myself either. Since the whole of it is just generated code, I'd have to look at the original code used to generate the .w files, but I am neither familiar with Literate Programming, nor with the tools available for them. Also I have the nagging feeling that I won't be able to use any of the tools that I know if I wanted to work with that code. I don't see any sense in refactoring the generated code into managable packages. But what else can I do? At this point the only alternative I see is rewrite the entire library from scratch X| Maybe someone with experience in Literate Programming can give me an estimate as to how much time it
-
Today I received the source code for a DLL developed by a small company for us. The source code was part of the deal, and one of it's purposes was that we should be able to continue maintenance efforts once the main functionality of the library is done, and no more big issues remain. Unfortunately, I've found the source code to be written auto-generated as CWEB, using a Literate Programming tool. That means these files cannot be directly read by the compiler, nor interpreted with the integrated tools of the IDE. I've requested the 'tangled' C++ source files (i. e. processed by ctangle.exe into .h and .cpp files), to be able to at least read them into my UML tool, but that code has all comments and blanks removed, so is not in a suitable form to be read in the IDE either. In the UML tool I now have several hundred classes, structs, typedefs and enums all entangled into one huge diagram. The information I get from this is nicely summarized into one word: HELP! What I currently have is: - 359 auto- generated CWEB files (.w extension, some with a .hw extension, and a few others), in one and the same directory (actually I believe some of these are in fact not generated and don't belong there) - these CWEB files have comments and proper formatting, but are unreadable by my tools - alternately 245 C++ files (no idea why there are less of these) - C++ files have no comments or blanks or tabs (but do have linebreaks at least) - There's no 1-1 relation between source files and classes (neither for the CWEB files nor the C++ files): the total number of classes is around 300-500. Apparently some files contain dozens of classes, others none at all. In short, it's a total mess. I'm now at a loss what to do about it: I can no longer trust the original programmer to maintain it (there already has been significant doubt before), but I have no idea how to take care of it myself either. Since the whole of it is just generated code, I'd have to look at the original code used to generate the .w files, but I am neither familiar with Literate Programming, nor with the tools available for them. Also I have the nagging feeling that I won't be able to use any of the tools that I know if I wanted to work with that code. I don't see any sense in refactoring the generated code into managable packages. But what else can I do? At this point the only alternative I see is rewrite the entire library from scratch X| Maybe someone with experience in Literate Programming can give me an estimate as to how much time it
Are the
.wand.hwfiles needed to rebuild the DLL? Can you run the C++ files through some sort or beautifer to put indentation and spaces back in?"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Show me a community that obeys the Ten Commandments and I'll show you a less crowded prison system." - Anonymous
-
Are the
.wand.hwfiles needed to rebuild the DLL? Can you run the C++ files through some sort or beautifer to put indentation and spaces back in?"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Show me a community that obeys the Ten Commandments and I'll show you a less crowded prison system." - Anonymous
1. No. The only additional 'information' they contain are the comments. But then I would be better served by the original files used to generate them: they too are illegible to my tools, specifically as they do use lots of symbols that may not even be printed correctly without the right editor, but at least that code is at least theoretically meant to be human-readable 2. Yes: manually I can just press Ctrl-A, Ctrl-K, Ctrl-F. it's just that I don't feel like doing that for 245 files, and definitely not again when there is an update and the entire set of files gets regenerated. But even if there will be no updates, reformatting does not add the comments back in, nor remove the annoying compiler directives that make VA Outline all but unreadable in larger files. :( Besides, even formatting doesn't change the fact that the code wasn't generated with readability in mind...
-
Today I received the source code for a DLL developed by a small company for us. The source code was part of the deal, and one of it's purposes was that we should be able to continue maintenance efforts once the main functionality of the library is done, and no more big issues remain. Unfortunately, I've found the source code to be written auto-generated as CWEB, using a Literate Programming tool. That means these files cannot be directly read by the compiler, nor interpreted with the integrated tools of the IDE. I've requested the 'tangled' C++ source files (i. e. processed by ctangle.exe into .h and .cpp files), to be able to at least read them into my UML tool, but that code has all comments and blanks removed, so is not in a suitable form to be read in the IDE either. In the UML tool I now have several hundred classes, structs, typedefs and enums all entangled into one huge diagram. The information I get from this is nicely summarized into one word: HELP! What I currently have is: - 359 auto- generated CWEB files (.w extension, some with a .hw extension, and a few others), in one and the same directory (actually I believe some of these are in fact not generated and don't belong there) - these CWEB files have comments and proper formatting, but are unreadable by my tools - alternately 245 C++ files (no idea why there are less of these) - C++ files have no comments or blanks or tabs (but do have linebreaks at least) - There's no 1-1 relation between source files and classes (neither for the CWEB files nor the C++ files): the total number of classes is around 300-500. Apparently some files contain dozens of classes, others none at all. In short, it's a total mess. I'm now at a loss what to do about it: I can no longer trust the original programmer to maintain it (there already has been significant doubt before), but I have no idea how to take care of it myself either. Since the whole of it is just generated code, I'd have to look at the original code used to generate the .w files, but I am neither familiar with Literate Programming, nor with the tools available for them. Also I have the nagging feeling that I won't be able to use any of the tools that I know if I wanted to work with that code. I don't see any sense in refactoring the generated code into managable packages. But what else can I do? At this point the only alternative I see is rewrite the entire library from scratch X| Maybe someone with experience in Literate Programming can give me an estimate as to how much time it
-
1. No. The only additional 'information' they contain are the comments. But then I would be better served by the original files used to generate them: they too are illegible to my tools, specifically as they do use lots of symbols that may not even be printed correctly without the right editor, but at least that code is at least theoretically meant to be human-readable 2. Yes: manually I can just press Ctrl-A, Ctrl-K, Ctrl-F. it's just that I don't feel like doing that for 245 files, and definitely not again when there is an update and the entire set of files gets regenerated. But even if there will be no updates, reformatting does not add the comments back in, nor remove the annoying compiler directives that make VA Outline all but unreadable in larger files. :( Besides, even formatting doesn't change the fact that the code wasn't generated with readability in mind...
Was there a specifications or requirements document agreed upon by this other company and yours? In other words, were you expecting to receive "plain" C++ files in the end?
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Show me a community that obeys the Ten Commandments and I'll show you a less crowded prison system." - Anonymous
-
Today I received the source code for a DLL developed by a small company for us. The source code was part of the deal, and one of it's purposes was that we should be able to continue maintenance efforts once the main functionality of the library is done, and no more big issues remain. Unfortunately, I've found the source code to be written auto-generated as CWEB, using a Literate Programming tool. That means these files cannot be directly read by the compiler, nor interpreted with the integrated tools of the IDE. I've requested the 'tangled' C++ source files (i. e. processed by ctangle.exe into .h and .cpp files), to be able to at least read them into my UML tool, but that code has all comments and blanks removed, so is not in a suitable form to be read in the IDE either. In the UML tool I now have several hundred classes, structs, typedefs and enums all entangled into one huge diagram. The information I get from this is nicely summarized into one word: HELP! What I currently have is: - 359 auto- generated CWEB files (.w extension, some with a .hw extension, and a few others), in one and the same directory (actually I believe some of these are in fact not generated and don't belong there) - these CWEB files have comments and proper formatting, but are unreadable by my tools - alternately 245 C++ files (no idea why there are less of these) - C++ files have no comments or blanks or tabs (but do have linebreaks at least) - There's no 1-1 relation between source files and classes (neither for the CWEB files nor the C++ files): the total number of classes is around 300-500. Apparently some files contain dozens of classes, others none at all. In short, it's a total mess. I'm now at a loss what to do about it: I can no longer trust the original programmer to maintain it (there already has been significant doubt before), but I have no idea how to take care of it myself either. Since the whole of it is just generated code, I'd have to look at the original code used to generate the .w files, but I am neither familiar with Literate Programming, nor with the tools available for them. Also I have the nagging feeling that I won't be able to use any of the tools that I know if I wanted to work with that code. I don't see any sense in refactoring the generated code into managable packages. But what else can I do? At this point the only alternative I see is rewrite the entire library from scratch X| Maybe someone with experience in Literate Programming can give me an estimate as to how much time it
Ouch! One of our guys suggested some 3 years ago, that we should specify what Tools, and Platforms should be able to handle the Documentation provided for a Project. In the end, we included it reluctantly in our Doc Spec. (It may do No Good, but it doesn't do any harm, except from possibly adding a bit to the price.) I suspect that you now wish that that guy worked for you. I Suggest you still speak to the programmer (small company). If he is half decent, and is looking for new business, and is interested in keeping up a reputation, he should be willing to assist. If at the otherhand, he takes the attitude of 'Gotche by the Goolies, I have the Real Source Code, Come to me for Ever and Ever for mantenance', that would be deception. I take it that you have already 'at least' Part paid him. I do not know under what jurisdiction the contract was signed. Here in the UK you would have various legal options. If you specified in your Contract that you require the source code for software maintenance purposes, you can reasonably expect to receive them in a format that 'your' tools can accept. Seeing that in this relationship he would be classed as the 'expert', it would be incumbent on him (her?) to discuss with you at an early stage as to the format details of the documentation provided. It may also not be possible to generate Human Readible Source Code that can be loaded into an IDE, from the Source Script your developer wrote for his Code Generator. In that case, Cut your Losses (and Sue if Possible). Best to speak to a Solicitor at this stage. Sorry to hear about your Trouble, Regards, :(
Bram van Kampen
-
Stefan_Lang wrote:
using a Literate Programming tool.
Why can't you just learn and use that tool?
I considered it. I really did, although I wasn't happy about that. I spent several hours to learn about it, skimming articles on the issue, with a special focus on the things I knew to be a weakness in the current library. As matters stand, every single article I've specifically searched for either didn't address the keywords I searched for, addressed them in a different context, or put a rather negative view on the whole context of LP. I am now pretty sure that 1. LP is detrimental to team development 2. ignores interoperability with most modern SW development tools and therefore in the long run will in fact produce low quality code by modern standards, in spite of it's proclaimed goals to produce high quality. 3. Has no consideration whatsoever regarding performance, and in fact, due to automated code generation makes it near impossible to care about that. (performance is crucial for our purposes, and we made a point of that) 4. Does not prevent the programmer to produce bad code: It didn't take me long to spot some pretty horrible things in the code we got. I've yet to find out if these come from generated code or are in fact coded into the original algorithm. But in the end that does not matter. Judging by the articles I looked into, I got the impression LP never really got outside universities. And it seems for very good reasons. I'm not going to put effort into changing that.
-
Was there a specifications or requirements document agreed upon by this other company and yours? In other words, were you expecting to receive "plain" C++ files in the end?
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Show me a community that obeys the Ten Commandments and I'll show you a less crowded prison system." - Anonymous
The contract just says that we should be able to build it with our IDE. We're still on a friendly basis, so I suppose we could ask for some help, but I'm afraid we need a lot more than 'some' help.
-
Ouch! One of our guys suggested some 3 years ago, that we should specify what Tools, and Platforms should be able to handle the Documentation provided for a Project. In the end, we included it reluctantly in our Doc Spec. (It may do No Good, but it doesn't do any harm, except from possibly adding a bit to the price.) I suspect that you now wish that that guy worked for you. I Suggest you still speak to the programmer (small company). If he is half decent, and is looking for new business, and is interested in keeping up a reputation, he should be willing to assist. If at the otherhand, he takes the attitude of 'Gotche by the Goolies, I have the Real Source Code, Come to me for Ever and Ever for mantenance', that would be deception. I take it that you have already 'at least' Part paid him. I do not know under what jurisdiction the contract was signed. Here in the UK you would have various legal options. If you specified in your Contract that you require the source code for software maintenance purposes, you can reasonably expect to receive them in a format that 'your' tools can accept. Seeing that in this relationship he would be classed as the 'expert', it would be incumbent on him (her?) to discuss with you at an early stage as to the format details of the documentation provided. It may also not be possible to generate Human Readible Source Code that can be loaded into an IDE, from the Source Script your developer wrote for his Code Generator. In that case, Cut your Losses (and Sue if Possible). Best to speak to a Solicitor at this stage. Sorry to hear about your Trouble, Regards, :(
Bram van Kampen
Thanks for your response. I'm afraid that legal action won't get us anything much - what we need most is get our work done, and no amount of jurisdiction will help us achieve it. And anyway, that's something my superiors will have to decide. In the meantime I'm still stuck with the problem of making things work. That's why I posted here, in the hopes someone with experience in LP might have an idea about how to deal with it.
-
I considered it. I really did, although I wasn't happy about that. I spent several hours to learn about it, skimming articles on the issue, with a special focus on the things I knew to be a weakness in the current library. As matters stand, every single article I've specifically searched for either didn't address the keywords I searched for, addressed them in a different context, or put a rather negative view on the whole context of LP. I am now pretty sure that 1. LP is detrimental to team development 2. ignores interoperability with most modern SW development tools and therefore in the long run will in fact produce low quality code by modern standards, in spite of it's proclaimed goals to produce high quality. 3. Has no consideration whatsoever regarding performance, and in fact, due to automated code generation makes it near impossible to care about that. (performance is crucial for our purposes, and we made a point of that) 4. Does not prevent the programmer to produce bad code: It didn't take me long to spot some pretty horrible things in the code we got. I've yet to find out if these come from generated code or are in fact coded into the original algorithm. But in the end that does not matter. Judging by the articles I looked into, I got the impression LP never really got outside universities. And it seems for very good reasons. I'm not going to put effort into changing that.
Stefan_Lang wrote:
1. LP is detrimental to team development...I got the impression LP never really got outside universities
Possible I suppose. However Literate Programming is an idiom, not a tool. The problem here is the tool.
Stefan_Lang wrote:
4. Does not prevent the programmer to produce bad code: It didn't take me long to spot some pretty horrible things in the code we got.
Not sure what that means. Generated code doesn't have to be pretty. Myself I prefer that it makes an attempt to be so, but it isn't a requirement. And actually there are specific things that I expect from generated code, such as repeating the same code over and over again in different places. Now it could be that the tool generates bad code (nothing to do with pretty.) Or it could be that the use of the tool in the first place was incorrect. That latter problem can be corrected by learning how the tool works and then correcting the input source so the generated code is better. The former problem can normally only be solved by fixing the tool.
Stefan_Lang wrote:
3. Has no consideration whatsoever regarding performance, and in fact, due to automated code generation makes it near impossible to care about that. (performance is crucial for our purposes, and we made a point of that)
That is a process problem not a technological problem. If your company contracted performance critical code to a third part and didn't stipulate performance criteria then then failure is in the outsourcing process and has nothing to do with the implementation. The performance failure could have resulted regardless of the implementation methodology and actually, with no performance criteria in the contract, probably would have (presuming not trivial code.)
-
Thanks for your response. I'm afraid that legal action won't get us anything much - what we need most is get our work done, and no amount of jurisdiction will help us achieve it. And anyway, that's something my superiors will have to decide. In the meantime I'm still stuck with the problem of making things work. That's why I posted here, in the hopes someone with experience in LP might have an idea about how to deal with it.
Hi, Well, At Least you have Superiors. (I've Not). That means that your paycheck is secure. Well, look at this one, You're given an impossible problem, (which you cannot solve), You're sacked because of incompetence. In walks this stranger after you're sacked, looks at the documents provided by the contractor, and says, 'YESS' I can Work with this!, and Solves the Problem. Take care, that that was not a Stooge, planted to look for your job! I Don't know your Circumstances, who you are or what you do!. My first reaction on reading your post was 'Shurely SoMeOne is trying to pull a Scam' 'Literal Code' essentially comprises a Program, still compilable under 'C' or 'CPP', which would be produced as an intermediate file to the final compiler. It should come with a header File.Equally Abstract. It is Human Readible to the extent that it can be reproduced in print( it does not contain 'Awquard' Characters, such as \b(bell) etc. Human Readible Text means just that.: That if it is printed, you can read all chars. It does not mean that what is printed makes Human Sense
Bram van Kampen
-
Stefan_Lang wrote:
using a Literate Programming tool.
Why can't you just learn and use that tool?
Hi, It Is 'UnLearnable' as I understand it, This OP was presented for his employer by one of those 'Intermediate' File Generators, produced by one of those intractable .NET,C#, Matrix, or other devilish software tool. The File is written in CPP, but Un Documented. Written in a format, easy to generate by a code generator, not intended for Human Consumption. Example:
class ABCDEF110{ABCDEF110();~ABCDEF110();protected: V ABCDEF110(PQR ST){V=ST,YZ} };It deserves a Decent Decompiler before you start on this one. That will take a Lot of Time and Efford. Someone is on the Take Here, probably Not the OP,Probably the Outside Contractor.Bram van Kampen
-
Stefan_Lang wrote:
1. LP is detrimental to team development...I got the impression LP never really got outside universities
Possible I suppose. However Literate Programming is an idiom, not a tool. The problem here is the tool.
Stefan_Lang wrote:
4. Does not prevent the programmer to produce bad code: It didn't take me long to spot some pretty horrible things in the code we got.
Not sure what that means. Generated code doesn't have to be pretty. Myself I prefer that it makes an attempt to be so, but it isn't a requirement. And actually there are specific things that I expect from generated code, such as repeating the same code over and over again in different places. Now it could be that the tool generates bad code (nothing to do with pretty.) Or it could be that the use of the tool in the first place was incorrect. That latter problem can be corrected by learning how the tool works and then correcting the input source so the generated code is better. The former problem can normally only be solved by fixing the tool.
Stefan_Lang wrote:
3. Has no consideration whatsoever regarding performance, and in fact, due to automated code generation makes it near impossible to care about that. (performance is crucial for our purposes, and we made a point of that)
That is a process problem not a technological problem. If your company contracted performance critical code to a third part and didn't stipulate performance criteria then then failure is in the outsourcing process and has nothing to do with the implementation. The performance failure could have resulted regardless of the implementation methodology and actually, with no performance criteria in the contract, probably would have (presuming not trivial code.)
jschell wrote:
Generated code doesn't have to be pretty.
As it turned out the .w files were not generated after all: they're all hand-written just as they are. But I was talking about code quality anyway, not prettyness.
jschell wrote:
failure is in the outsourcing process
Agreed. I could elaborate on that, but I'm not here to complain over past failures (ok, that as well, but I'm almost done on that part ;) ), but to get advice what I could do to fix it.