Workflows and batch files
-
Out of curiosity, do any of you (in a professional capacity, as in, the company you work for) have various automated jobs that run at night the still execute batch files? And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails? Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
Latest Article - Building a Prototype Web-Based Diagramming Tool with SVG and Javascript Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
One of the last places I worked at botched a mainframe migration to J2EE and runs both together. The J2EE works all day queuing data for batch processing. At night the (80's era) scheduler runs the jobs, and if one fails, everything fails. The jobs loads the data needed by the J2EE for the next day. It's a mess, and about half the IT staff is always on-call at night. Oh, and there's no notification system. One person must log in at about 3:00 am to see if everything worked. This is a government agent with a $1.1B budget!
-
Out of curiosity, do any of you (in a professional capacity, as in, the company you work for) have various automated jobs that run at night the still execute batch files? And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails? Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
Latest Article - Building a Prototype Web-Based Diagramming Tool with SVG and Javascript Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
Batch files? Absolutely, love 'em, love 'em, love 'em. Can't get my head around PowerShell. This[^] is good, if you want '.bat++'. In fact, it offers a way superior command line to cmd.exe in general.
Paul Sanders http://www.alpinesoft.co.uk
-
Out of curiosity, do any of you (in a professional capacity, as in, the company you work for) have various automated jobs that run at night the still execute batch files? And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails? Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
Latest Article - Building a Prototype Web-Based Diagramming Tool with SVG and Javascript Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
Using batch files here. And the batch code is actually pretty sophisticated (at least after I've took it upon me to make it work properly, the Linux guys who created that batch mess really didn't know what they're doing and by their own account, they don't want to know what they're doing).
-
Out of curiosity, do any of you (in a professional capacity, as in, the company you work for) have various automated jobs that run at night the still execute batch files? And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails? Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
Latest Article - Building a Prototype Web-Based Diagramming Tool with SVG and Javascript Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
-
Out of curiosity, do any of you (in a professional capacity, as in, the company you work for) have various automated jobs that run at night the still execute batch files? And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails? Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
Latest Article - Building a Prototype Web-Based Diagramming Tool with SVG and Javascript Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
We run an overnight process that updates a database with information from a Finance system and this was done using a .NET exe file which is started by Windows Task Scheduler. The exe file accesses the source DB in read only mode and the target DB in update mode. It runs each database update in the right order and provides log files and emailed error, progress and diagnostic reports. I know that if the last report has been emailed then all is well otherwise where it fails is in the logs it leaves behind. It does occasionally fail but that is usually following a target db change that has not been fully tested.
-
Out of curiosity, do any of you (in a professional capacity, as in, the company you work for) have various automated jobs that run at night the still execute batch files? And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails? Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
Latest Article - Building a Prototype Web-Based Diagramming Tool with SVG and Javascript Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
You may check WexFlow or may I suggest you Jenkins ?
-
Out of curiosity, do any of you (in a professional capacity, as in, the company you work for) have various automated jobs that run at night the still execute batch files? And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails? Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
Latest Article - Building a Prototype Web-Based Diagramming Tool with SVG and Javascript Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
Our company uses [Jenkins](https://jenkins.io/)[[^](https://jenkins.io/)]. Builds can run over night, or be initiated at any time. Jenkins can build out of GitHub, deploy to a VM (for eg.) and report success/failure results to Slack.
Cheers, Mike Fidler "I intend to live forever - so far, so good." Steven Wright "I almost had a psychic girlfriend but she left me before we met." Also Steven Wright "I'm addicted to placebos. I could quit, but it wouldn't matter." Steven Wright yet again.
-
Out of curiosity, do any of you (in a professional capacity, as in, the company you work for) have various automated jobs that run at night the still execute batch files? And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails? Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
Latest Article - Building a Prototype Web-Based Diagramming Tool with SVG and Javascript Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
batch files. Kind of. yes sort of. Usually powershell scripts with fairly good transaction processing roll back included that emails successes and failures. We try to avoid the whole Rube Goldberg Machine setup where if one little thing goes wrong the whole contraption doesn't work. It does seem to happen once in awhile. What really chafs me is when it is forced on you by the application vendor. "Oh we have this timer service you setup to run this process. It needs to run this one first and then the second one. It usually takes 15 minutes except when it doesn't. It should be ran at night and no it doesn't email. you just have to manually check it. We recommend setting them to run 2 hours apart. No don't run anything else at that time. Opps sorry that latest patch caused the first one to run for 3 hours. Now everything is messed up. Yes you will need to rerun them during the day. Yes it will be down while this completes. sorry" Yep, we have had that conversation once. Not fun.
To err is human to really mess up you need a computer
-
You may check WexFlow or may I suggest you Jenkins ?
I also agree with using Jenkins and it is free
-
Out of curiosity, do any of you (in a professional capacity, as in, the company you work for) have various automated jobs that run at night the still execute batch files? And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails? Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
Latest Article - Building a Prototype Web-Based Diagramming Tool with SVG and Javascript Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
At work we use a product called "Cybermation" , its formal name is "Enterprise Scheduling System (EPS)". At first I hated it, then I thought, hey, it beats 5,000 tasks, or windows services. A long term goal I have, is building something like it. The way it works with Active Directory is nice. It puts a small db on the target prod server, and credentials are cached. In short, something like this is really beneficial, because we have 10's of thousands of jobs running on this. Informatica is also funneled into this, as well as a lot of other big products.
Where there's smoke, there's a Blue Screen of death.
-
Out of curiosity, do any of you (in a professional capacity, as in, the company you work for) have various automated jobs that run at night the still execute batch files? And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails? Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
Latest Article - Building a Prototype Web-Based Diagramming Tool with SVG and Javascript Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
At work we use a product called "Cybermation" , its formal name is "Enterprise Scheduling System (EPS)". At first I hated it, then I thought, hey, it beats 5,000 tasks, or windows services. A long term goal I have, is building something like it. The way it works with Active Directory is nice. It puts a small db on the target prod server, and credentials are cached. In short, something like this is really beneficial, because we have 10's of thousands of jobs running on this. Informatica is also funneled into this, as well as a lot of other big products. Another Scheduling product I see i SKYBOT , it looks more modern.
Where there's smoke, there's a Blue Screen of death.
-
Out of curiosity, do any of you (in a professional capacity, as in, the company you work for) have various automated jobs that run at night the still execute batch files? And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails? Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
Latest Article - Building a Prototype Web-Based Diagramming Tool with SVG and Javascript Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny Artificial intelligence is the only remedy for natural stupidity. - CDP1802
I tend to create SQL Server Agent jobs (C# console programs); with a scheduled start. The "dashboard" works well. Jobs can be any type. Easily restarted. No agent for the SQL Server Express version though.
"(I) am amazed to see myself here rather than there ... now rather than then". ― Blaise Pascal
-
Marc Clifton wrote:
And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails?
Automated a build at a previous company, but that excludes the whole chain-fail thingy; you simply check the errorlevel-variable.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
We have enough Control-M batch jobs that we have a small team supporting them.