The way software is pasted together these days
-
Glued together from a web of dependencies, development "organized" via agile methodology. How long before we just fire software packages out of a t-shirt cannon? "Hold my beer, I've got a version update to deploy" I am really uncomfortable with the state of software development these days. Dependency hell didn't used to be a distributed problem. I suppose I'm just getting old and this is the new normal. Eventually we'll *need* AI just to keep our import hierarchies straight. :~
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
People complain about dependency hell, but they forget the alternative being that you would have to write everything yourself from scratch. I'm sure for some people this is fine ("HTTP server? I'll do it myself!") but for a large majority of people this isn't going to cut it. I use Java and Maven to manage my dependencies. If I choose to upgrade my dependencies because it's overdue, does it cause me headaches? Absolutely! I might have to spend an hour or two carefully adjusting versions until everyone is happy. But the trade-off is that I haven't had to build entire frameworks from scratch, and would I even want to do that if I could? Absolutely not.
-
People complain about dependency hell, but they forget the alternative being that you would have to write everything yourself from scratch. I'm sure for some people this is fine ("HTTP server? I'll do it myself!") but for a large majority of people this isn't going to cut it. I use Java and Maven to manage my dependencies. If I choose to upgrade my dependencies because it's overdue, does it cause me headaches? Absolutely! I might have to spend an hour or two carefully adjusting versions until everyone is happy. But the trade-off is that I haven't had to build entire frameworks from scratch, and would I even want to do that if I could? Absolutely not.
The problem is not the presence of dependencies, but the number of dependencies. Instead of using a single framework and then writing the things missing in this framework, many programmers will import a package for each minor piece of code. Leaving aside the security issues (without reading the code, how do you know that the package doesn't contain malicious code in addition to the useful stuff?), the maintenance overhead of so many packages tends to be prohibitive. I would prefer to use fewer packages, and not have the headache of working out whichg version of X is compatible with Y, etc. I won't comment on the practice of some developers of making breaking changes to their code - either by renaming functions or by changing the functionality of existing functions. Any such comments would not be KSS.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows. -- 6079 Smith W.
-
People complain about dependency hell, but they forget the alternative being that you would have to write everything yourself from scratch. I'm sure for some people this is fine ("HTTP server? I'll do it myself!") but for a large majority of people this isn't going to cut it. I use Java and Maven to manage my dependencies. If I choose to upgrade my dependencies because it's overdue, does it cause me headaches? Absolutely! I might have to spend an hour or two carefully adjusting versions until everyone is happy. But the trade-off is that I haven't had to build entire frameworks from scratch, and would I even want to do that if I could? Absolutely not.
I mean, hey, people obviously weren't having enough trouble managing local dependencies, so now we've flung them across the Internet. So when your guy making a Widget control goes through a messy divorce, maybe one of the things he does is pull his project. Then what? Like I said, dependency hell didn't used to be a distributed problem.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
Glued together from a web of dependencies, development "organized" via agile methodology. How long before we just fire software packages out of a t-shirt cannon? "Hold my beer, I've got a version update to deploy" I am really uncomfortable with the state of software development these days. Dependency hell didn't used to be a distributed problem. I suppose I'm just getting old and this is the new normal. Eventually we'll *need* AI just to keep our import hierarchies straight. :~
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
Eee-yah. :thumbsup:
-
I mean, hey, people obviously weren't having enough trouble managing local dependencies, so now we've flung them across the Internet. So when your guy making a Widget control goes through a messy divorce, maybe one of the things he does is pull his project. Then what? Like I said, dependency hell didn't used to be a distributed problem.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
What are the options then? You either: * Build your own libraries from scratch and maintain them yourself, potentially taking weeks or months of work * Manage your own dependencies with version control by manually downloading whatever you need and having them copied into your projects * Use dependency managers like NuGet, Maven, Gradle, npm etc You make it sounds like there is no winning solution. I'd much rather make use of a dependency repository and know I can almost click + collect a solution to whatever problem I have, than spend a long time Googling for an answer and then trial and error'ing until I find a working version. And the issue of people pulling libraries is a non-starter if you're using an appropriate dependency proxy. In Java we have Nexus, which acts as an intermediary which downloads and caches dependencies, so even if they're pulled from the internet (or someone maliciously overrides an existing version) we have constant access to what we need. I'd be more than happy to hear alternatives to dependency/package managers!
-
Glued together from a web of dependencies, development "organized" via agile methodology. How long before we just fire software packages out of a t-shirt cannon? "Hold my beer, I've got a version update to deploy" I am really uncomfortable with the state of software development these days. Dependency hell didn't used to be a distributed problem. I suppose I'm just getting old and this is the new normal. Eventually we'll *need* AI just to keep our import hierarchies straight. :~
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
Earlier this week I needed something to quickly decode a string - I don't know how this string I'm handed has been encoded, but I know it's part of some well-supported standard. Anyway, I did find a NuGet package that had the function to break it apart, but it had its own set of dependencies - it brought along 8 new packages (!)...some of which I was already using, but still - had it not been for time, I would've been very, very tempted to roll my own. I held my breath in disgust and they're now part of my solution file...but every time I look at it, I can't help but go "X|"
-
Glued together from a web of dependencies, development "organized" via agile methodology. How long before we just fire software packages out of a t-shirt cannon? "Hold my beer, I've got a version update to deploy" I am really uncomfortable with the state of software development these days. Dependency hell didn't used to be a distributed problem. I suppose I'm just getting old and this is the new normal. Eventually we'll *need* AI just to keep our import hierarchies straight. :~
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
You know my stance. I roll my own. :-D I haven't yet run into a situation where I wasn't able to write what I need just with what's in .net itself -- or things like Oracle's own .net provider. Third-party solutions don't appeal to me. Not to forget that any third-party packages had to be approved by corporate and that could take months. I will say that I have dabbled with EPPlus and AngleSharp (both fine products), just to see what they offered, but never used them for anything that went into production. I don't recall either having other dependencies. Personally, I see dependencies as red flags.
-
Earlier this week I needed something to quickly decode a string - I don't know how this string I'm handed has been encoded, but I know it's part of some well-supported standard. Anyway, I did find a NuGet package that had the function to break it apart, but it had its own set of dependencies - it brought along 8 new packages (!)...some of which I was already using, but still - had it not been for time, I would've been very, very tempted to roll my own. I held my breath in disgust and they're now part of my solution file...but every time I look at it, I can't help but go "X|"
How do you know it's correct? How well does it react to corrupt data? :-D I would learn the "well-supported standard" well enough to at least make a rough version of the encoder and decoder, and then compare inputs and outputs between the two. Once you have that in place, if they agree and unless it's seriously slower than the reference implementation, just use yours. I'd be interested in knowing more about this "well-supported standard" -- last year I had to implement a JSON parser which could detect and react to certain imperfections in some files we were receiving from a third-party utility.
-
What are the options then? You either: * Build your own libraries from scratch and maintain them yourself, potentially taking weeks or months of work * Manage your own dependencies with version control by manually downloading whatever you need and having them copied into your projects * Use dependency managers like NuGet, Maven, Gradle, npm etc You make it sounds like there is no winning solution. I'd much rather make use of a dependency repository and know I can almost click + collect a solution to whatever problem I have, than spend a long time Googling for an answer and then trial and error'ing until I find a working version. And the issue of people pulling libraries is a non-starter if you're using an appropriate dependency proxy. In Java we have Nexus, which acts as an intermediary which downloads and caches dependencies, so even if they're pulled from the internet (or someone maliciously overrides an existing version) we have constant access to what we need. I'd be more than happy to hear alternatives to dependency/package managers!
How about not using dependencies for simple code? Not using dependencies just because you can. That's a start.
Chris Copeland wrote:
And the issue of people pulling libraries is a non-starter if you're using an appropriate dependency proxy. In Java we have Nexus, which acts as an intermediary which downloads and caches dependencies,
This is actually hilarious to me. It reads like "It's not an issue. You see, it became such an issue that Oracle caches dependencies to prevent it"
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
How do you know it's correct? How well does it react to corrupt data? :-D I would learn the "well-supported standard" well enough to at least make a rough version of the encoder and decoder, and then compare inputs and outputs between the two. Once you have that in place, if they agree and unless it's seriously slower than the reference implementation, just use yours. I'd be interested in knowing more about this "well-supported standard" -- last year I had to implement a JSON parser which could detect and react to certain imperfections in some files we were receiving from a third-party utility.
PIEBALDconsult wrote:
How do you know it's correct? How well does it react to corrupt data? :-D
I don't. And I don't have the resources to investigate. All I know is that it's a library Microsoft wrote, so I have to trust it (as far as I can trust Microsoft). I'm still not going to name names to protect the innocent.
-
PIEBALDconsult wrote:
How do you know it's correct? How well does it react to corrupt data? :-D
I don't. And I don't have the resources to investigate. All I know is that it's a library Microsoft wrote, so I have to trust it (as far as I can trust Microsoft). I'm still not going to name names to protect the innocent.
dandy72 wrote:
a library Microsoft wrote
I'd put money on it then.
dandy72 wrote:
not going to name names
:sigh: I would have welcomed the exercise.
-
Glued together from a web of dependencies, development "organized" via agile methodology. How long before we just fire software packages out of a t-shirt cannon? "Hold my beer, I've got a version update to deploy" I am really uncomfortable with the state of software development these days. Dependency hell didn't used to be a distributed problem. I suppose I'm just getting old and this is the new normal. Eventually we'll *need* AI just to keep our import hierarchies straight. :~
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
honey the codewitch wrote:
with the state of software development these days.
These days? Versus when? Like when they first attempted to re-write the airline reservation system and managed to end up with a failed project and something like 10+ million (or billion) dollar lawsuits? Or when they released a CPU that had a bug in the floating point library. Or the space craft that had to be destroyed after taking off. Humans are fallible. That is true now and always has been. Complexity in no way helps with that.
-
Earlier this week I needed something to quickly decode a string - I don't know how this string I'm handed has been encoded, but I know it's part of some well-supported standard. Anyway, I did find a NuGet package that had the function to break it apart, but it had its own set of dependencies - it brought along 8 new packages (!)...some of which I was already using, but still - had it not been for time, I would've been very, very tempted to roll my own. I held my breath in disgust and they're now part of my solution file...but every time I look at it, I can't help but go "X|"
dandy72 wrote:
Earlier this week I needed something to quickly decode a string
For something similar to which I knew how to write the code, the very best solution I found pulled in hundreds of dependencies. It was part of a much larger application. No way I was comfortable with that so I just rolled my own code.
-
honey the codewitch wrote:
with the state of software development these days.
These days? Versus when? Like when they first attempted to re-write the airline reservation system and managed to end up with a failed project and something like 10+ million (or billion) dollar lawsuits? Or when they released a CPU that had a bug in the floating point library. Or the space craft that had to be destroyed after taking off. Humans are fallible. That is true now and always has been. Complexity in no way helps with that.
Versus when software did not have 150 dependencies for an online guest book
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
dandy72 wrote:
Earlier this week I needed something to quickly decode a string
For something similar to which I knew how to write the code, the very best solution I found pulled in hundreds of dependencies. It was part of a much larger application. No way I was comfortable with that so I just rolled my own code.
-
Glued together from a web of dependencies, development "organized" via agile methodology. How long before we just fire software packages out of a t-shirt cannon? "Hold my beer, I've got a version update to deploy" I am really uncomfortable with the state of software development these days. Dependency hell didn't used to be a distributed problem. I suppose I'm just getting old and this is the new normal. Eventually we'll *need* AI just to keep our import hierarchies straight. :~
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
How about not using dependencies for simple code? Not using dependencies just because you can. That's a start.
Chris Copeland wrote:
And the issue of people pulling libraries is a non-starter if you're using an appropriate dependency proxy. In Java we have Nexus, which acts as an intermediary which downloads and caches dependencies,
This is actually hilarious to me. It reads like "It's not an issue. You see, it became such an issue that Oracle caches dependencies to prevent it"
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
Of course you shouldn't be using dependencies for simple code, that should be a given. But I build a lot of microservice APIs, and each one has different requirements. Of course I'm going to use dependencies for those, including HTTP servers, security etc. You could argue "well if you're using HTTP servers so much, why not just make your own and re-use it for all your projects?" and you're right, I could have done. But why re-invent the wheel by building a brand new library to do exactly what someone else has done for me. The point I'm arguing is simply that before we had reliable dependency package managers, the industry was constantly rolling their own versions of things (even downloading other libraries and then making changes/amendments to meet their own requirements). If you hopped between jobs, you'd be learning an all new framework from the ground up, and each one had their own pros and cons. Nowadays, there's industry standards in adopted tools, so finding a new job is just matching up what skills you have. It sounds to me like your argument is focused on solo development, or for hobbyism. I've worked for several companies as a developer, ranging from small (2-3 team) to medium/large (30+ team, but department of 300+), and can tell you that in each role we used Maven and used a lot of dependencies. Why? Because we were building software that demanded tools that would have made no sense building from scratch when the solutions already existed out there. Try promoting "dependencies are baaaad" when you have to build an inbound rest API that provides credit card processing in real-time with transactional database queries, transaction ingress validation and an external API call to validate the card transaction claims, all while keeping processing speeds to a minimum. > This is actually hilarious to me. It reads like "It's not an issue. You see, it became such an issue that Oracle caches dependencies to prevent it" You're clearly commenting on something you know little about. Nexus is a product created by Apache for anyone (but mostly commercial) to use as a proxy-cache for dependencies, as well as a private distribution channel for internally built and deployed libraries and packages. The fact that is provides security in caching dependencies and preventing malicious overwrites is just a handy part of the tool, and I'd wager most companies working with Maven are using this tool or similar. You can find similar tools for any package manager.
[
-
Of course you shouldn't be using dependencies for simple code, that should be a given. But I build a lot of microservice APIs, and each one has different requirements. Of course I'm going to use dependencies for those, including HTTP servers, security etc. You could argue "well if you're using HTTP servers so much, why not just make your own and re-use it for all your projects?" and you're right, I could have done. But why re-invent the wheel by building a brand new library to do exactly what someone else has done for me. The point I'm arguing is simply that before we had reliable dependency package managers, the industry was constantly rolling their own versions of things (even downloading other libraries and then making changes/amendments to meet their own requirements). If you hopped between jobs, you'd be learning an all new framework from the ground up, and each one had their own pros and cons. Nowadays, there's industry standards in adopted tools, so finding a new job is just matching up what skills you have. It sounds to me like your argument is focused on solo development, or for hobbyism. I've worked for several companies as a developer, ranging from small (2-3 team) to medium/large (30+ team, but department of 300+), and can tell you that in each role we used Maven and used a lot of dependencies. Why? Because we were building software that demanded tools that would have made no sense building from scratch when the solutions already existed out there. Try promoting "dependencies are baaaad" when you have to build an inbound rest API that provides credit card processing in real-time with transactional database queries, transaction ingress validation and an external API call to validate the card transaction claims, all while keeping processing speeds to a minimum. > This is actually hilarious to me. It reads like "It's not an issue. You see, it became such an issue that Oracle caches dependencies to prevent it" You're clearly commenting on something you know little about. Nexus is a product created by Apache for anyone (but mostly commercial) to use as a proxy-cache for dependencies, as well as a private distribution channel for internally built and deployed libraries and packages. The fact that is provides security in caching dependencies and preventing malicious overwrites is just a handy part of the tool, and I'd wager most companies working with Maven are using this tool or similar. You can find similar tools for any package manager.
[
Chris Copeland wrote:
Of course you shouldn't be using dependencies for simple code, that should be a given.
If it's a given then why do so many people do it, in so many projects? It seems the norm rather than the exception.
Chris Copeland wrote:
You could argue "well if you're using HTTP servers so much, why not just make your own and re-use it for all your projects?"
Why would I? This isn't about NIH syndrome. If it was, I would have mentioned it.
Chris Copeland wrote:
The point I'm arguing is simply that before we had reliable dependency package managers, the industry was constantly rolling their own versions of things (even downloading other libraries and then making changes/amendments to meet their own requirements).
Oh you're making a point about dependency package managers? Because I'm not. My point is about people.
Chris Copeland wrote:
You're clearly commenting on something you know little about.
[Then goes on to explain that it's exactly what I said it was]
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
Chris Copeland wrote:
Of course you shouldn't be using dependencies for simple code, that should be a given.
If it's a given then why do so many people do it, in so many projects? It seems the norm rather than the exception.
Chris Copeland wrote:
You could argue "well if you're using HTTP servers so much, why not just make your own and re-use it for all your projects?"
Why would I? This isn't about NIH syndrome. If it was, I would have mentioned it.
Chris Copeland wrote:
The point I'm arguing is simply that before we had reliable dependency package managers, the industry was constantly rolling their own versions of things (even downloading other libraries and then making changes/amendments to meet their own requirements).
Oh you're making a point about dependency package managers? Because I'm not. My point is about people.
Chris Copeland wrote:
You're clearly commenting on something you know little about.
[Then goes on to explain that it's exactly what I said it was]
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
> If it's a given then why do so many people do it, in so many projects? It seems the norm rather than the exception. I'm guessing for convenience? If a library exists out there that accomplishes exactly one of the things that you need your solution to do, why would you not use it? If I wanted to build a quick tool that processed some CSV file and dumped the data in a database, I'd probably leverage the two dependencies I am familiar with which will expedite this process, cutting my development time down from hours to, likely, less than an hour. And you're probably thinking "well this is my point, people just jump straight to using dependencies and that's why we're in this mess", and absolutely, you're right. But again, people also don't want to be re-inventing the wheel and wasting hours or days building a new library that does exactly what another library does. > Why would I? This isn't about NIH syndrome. If it was, I would have mentioned it. Then what is your argument here? I get you're saying that reliance on dependencies can make projects messy, and that maybe we're overly dependent on them, but what's the alternative? Does everyone go back to writing their own libraries and storing them in version control? It's like an anarchist saying "the government is terrible and we should abolish them!" Okay, if I agree that the government is terrible, what is the alternative? > Oh you're making a point about dependency package managers? Because I'm not. My point is about people. Well package managers are where people get their dependencies, unless they're downloading the sources and building them themselves. People have unified repositories for downloading the libraries they need, rather than ye olde days when you'd manually download DLLs, or checkout code and build from source, and store in version control. People choose to use these package managers because they can Google "c# asp.net validation nuget" and be presented with packages which provide validators for ASP.NET modal objects. > [Then goes on to explain that it's exactly what I said it was] Except it isn't exactly what you said. Oracle never developed anything, it's an Apache tool which is used by businesses and freelancers for storing and caching dependencies on a node within an infrastructure to reduce network-load in both build pipelines and on internal networks. The fact that it provides additional security by preventing malicious version overwrites, and retains dependency versions even when removed from the source distribution channel
-
> If it's a given then why do so many people do it, in so many projects? It seems the norm rather than the exception. I'm guessing for convenience? If a library exists out there that accomplishes exactly one of the things that you need your solution to do, why would you not use it? If I wanted to build a quick tool that processed some CSV file and dumped the data in a database, I'd probably leverage the two dependencies I am familiar with which will expedite this process, cutting my development time down from hours to, likely, less than an hour. And you're probably thinking "well this is my point, people just jump straight to using dependencies and that's why we're in this mess", and absolutely, you're right. But again, people also don't want to be re-inventing the wheel and wasting hours or days building a new library that does exactly what another library does. > Why would I? This isn't about NIH syndrome. If it was, I would have mentioned it. Then what is your argument here? I get you're saying that reliance on dependencies can make projects messy, and that maybe we're overly dependent on them, but what's the alternative? Does everyone go back to writing their own libraries and storing them in version control? It's like an anarchist saying "the government is terrible and we should abolish them!" Okay, if I agree that the government is terrible, what is the alternative? > Oh you're making a point about dependency package managers? Because I'm not. My point is about people. Well package managers are where people get their dependencies, unless they're downloading the sources and building them themselves. People have unified repositories for downloading the libraries they need, rather than ye olde days when you'd manually download DLLs, or checkout code and build from source, and store in version control. People choose to use these package managers because they can Google "c# asp.net validation nuget" and be presented with packages which provide validators for ASP.NET modal objects. > [Then goes on to explain that it's exactly what I said it was] Except it isn't exactly what you said. Oracle never developed anything, it's an Apache tool which is used by businesses and freelancers for storing and caching dependencies on a node within an infrastructure to reduce network-load in both build pipelines and on internal networks. The fact that it provides additional security by preventing malicious version overwrites, and retains dependency versions even when removed from the source distribution channel
Chris Copeland wrote:
why would you not use it?
Because there's a cost associated in terms of relying on other people's code. You have things like trust, control over the direction of it, bugfixes on your schedule rather than theirs, and a myriad of other reasons. Using a dependency is a *decision process* and I'm glad you asked this question because it basically answers a question you asked further down, to wit - "what are you arguing?" so I'll address that here.
Chris Copeland wrote:
why would you not use it?
If you have to ask that question, then it sounds a whole lot like you don't acknowledge that there is effort, work and compromise in managing external resources, not just internal resources. There are plenty of reasons not to use dependencies depending on what it is, and the situation. And this attitude of ignoring that is exactly the problem. Dependency hell is real. Now it's a distributed problem which makes it worse.
Chris Copeland wrote:
Well package managers are where people get their dependencies, unless they're downloading the sources and building them themselves.
It doesn't matter how people get them. What matters is how and when they use them. See above.
Chris Copeland wrote:
. Oracle never developed anything, it's an Apache tool which is used by businesses
Oh Apache, not Oracle. My mistake. Guess that renders my entire argument invalid then.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix