To script or not to script...
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
Why not both? :-D
This space intentionally left blank.
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
Which one will you look at in six months after it's become a critical production process and be able to understand/adapt more easily?
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
You say potata...[^] My take on these sorts of issues is use whatever you are comfortable with. The advantage of the script is that it won't need recompiling if you need to make a change the logic is easily viewed by opening the script in notepad. The advantage of the compiled code app is that the debug facilities in the IDE may be better. You say potata..
“That which can be asserted without evidence, can be dismissed without evidence.”
― Christopher Hitchens
-
Which one will you look at in six months after it's become a critical production process and be able to understand/adapt more easily?
I started with scripting for adaptability, but, now I'm leaning towards a console app for expanded control (garbage cleanup, better logging, debugging, etc). It WILL be a mission critical app to the users once in full production. Ugh...
-
I started with scripting for adaptability, but, now I'm leaning towards a console app for expanded control (garbage cleanup, better logging, debugging, etc). It WILL be a mission critical app to the users once in full production. Ugh...
I'm not sure what any of those things have to do with deciding between a compiled or scripted solution.
-
I'm not sure what any of those things have to do with deciding between a compiled or scripted solution.
Started with the scripted solution because it's quick and should have worked. However, despite destroying objects, there is still an issue with something not being released, but, of course, the scripting app won't tell me what it is. If scripting worked, I could edit in place and move on. Since it is causing issues, I will be moving to an app (.NET based) where I can control the environment better. And, what does one have to do with the other and this response? Prefer the simple and elegant, but moving to the more involved, but probably better in the long run. And.. continued to try to get scripting working after original post.
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
I'd say write a script. It is what the scripting languages are there for. Size of your script (depending on the scripting language) will be much smaller (thus less effort) than the app. It should be much quicker to execute too (depending on the scripting language and your scripting skills). In the long run you can build yourself a scripting tool kit. Advantages of writing a script for the current project will possibly not show yet but will definitely show in the projects to come. When you require the use of a scripting language in the future, you'll have scripts performing script related tasks and apps performing app related tasks and not a unmanageable toolkit consisting of both scripts and apps performing only script related tasks. I agree with GuyThiebaut that you won't have advantages of debugging... thus would suggest sanity checks throughout your script (a good practice even if you're writing a compiled app).
"Program testing can be used to show the presence of bugs, but never to show their absence." << please vote!! >>
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
Have you thought of doing a best of both? Personally I'd have an app that can read the input file and then use scripted functions for the reformatting.
speramus in juniperus
-
I started with scripting for adaptability, but, now I'm leaning towards a console app for expanded control (garbage cleanup, better logging, debugging, etc). It WILL be a mission critical app to the users once in full production. Ugh...
-
Started with the scripted solution because it's quick and should have worked. However, despite destroying objects, there is still an issue with something not being released, but, of course, the scripting app won't tell me what it is. If scripting worked, I could edit in place and move on. Since it is causing issues, I will be moving to an app (.NET based) where I can control the environment better. And, what does one have to do with the other and this response? Prefer the simple and elegant, but moving to the more involved, but probably better in the long run. And.. continued to try to get scripting working after original post.
Your main question is related to maintainability of the solution imo. Who is going to maintain it? What technology (script language) do you intend to use? Who else can handle this technology in your team? What is your rollout strategy like? Do you require formal testing and signoff for example... Just to name a few... For me; scripts are easier to roll out than executables. And .NET is not a data friendly solution if all you need is CSV formats to be handled.
Cigarettes are a lot like hamsters. Perfectly harmless, until you put it in your mouth and light it on fire.
-
I'd say write a script. It is what the scripting languages are there for. Size of your script (depending on the scripting language) will be much smaller (thus less effort) than the app. It should be much quicker to execute too (depending on the scripting language and your scripting skills). In the long run you can build yourself a scripting tool kit. Advantages of writing a script for the current project will possibly not show yet but will definitely show in the projects to come. When you require the use of a scripting language in the future, you'll have scripts performing script related tasks and apps performing app related tasks and not a unmanageable toolkit consisting of both scripts and apps performing only script related tasks. I agree with GuyThiebaut that you won't have advantages of debugging... thus would suggest sanity checks throughout your script (a good practice even if you're writing a compiled app).
"Program testing can be used to show the presence of bugs, but never to show their absence." << please vote!! >>
R. Erasmus wrote:
I agree with GuyThiebaut that you won't have advantages of debugging... thus would suggest sanity checks throughout your script (a good practice even if you're writing a compiled app).
Not ok... even scripts should get a complete test set. Especially as it is going to be important in production. Don't cut corners there, you'll regret it.
Cigarettes are a lot like hamsters. Perfectly harmless, until you put it in your mouth and light it on fire.
-
R. Erasmus wrote:
I agree with GuyThiebaut that you won't have advantages of debugging... thus would suggest sanity checks throughout your script (a good practice even if you're writing a compiled app).
Not ok... even scripts should get a complete test set. Especially as it is going to be important in production. Don't cut corners there, you'll regret it.
Cigarettes are a lot like hamsters. Perfectly harmless, until you put it in your mouth and light it on fire.
Agreed, if its going to be production code then it needs to be unit tested which makes the decision for you. Scripts are good for adhoc operations tasks but not for a production system that's going to evolve and go through a number of versions
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
When I process CSV files this way I usually just do it all in SQL with temp tables. If you have to do a lot of string manipulation then procedural code might be better, but otherwise processing the CSV with a stored procedure is generally going to be fast and easy to maintain. Otherwise I don't see much difference between a script and a compiled app, other than possibly speed. But if you can do it in set operations with SQL rather than with loops, that's going to be a huge speed gain right there.
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
For me, everything rounds to what would be easier, if the script language (or environment) have everything I need then I do a script, if the compiled app would be easier to create, then I go for the compiled app. As a rule of thumb, if the script will end up with more than a hundred of lines, then a compiled app may be better.
CEO at: - Rafaga Systems - Para Facturas - Modern Components for the moment...
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
Try Power Shell script if like
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?
I have several apps that do this and I have used scripts, console apps and now I pretty much create services. 1. Unless you control the box the script is on, some one is going to open up the script in a text editor just to look at it. Come you know you have. Ever think you can make it better with just a little tweak? 2. If people log into the box with different credentials and run your console app. Oops different results - must be a bug, not a security feature. 3. With a service I can control the user the app is executing under, and people are less likely to mess with it.
-
I have an third party application that will be processing CSV files and pushing data to a data store. However, due to limitations in the application, the CSV files need to be pre-processed to add some additional data, reformat some fields and split the original file into smaller files. The question is: to write a script to process it daily or to write a compiled app... Either way, the solution will be running as a scheduled task on a server. Thoughts? Opinions?