How do you maintain code quality / complexity?
-
As I am starting my new job at EA, working on the Frostbite Editor, which is easily a 1000 man years job if I were to hazard guess... The codebase is very large! And the parts I am working on are very messy... Trying to study one little bit of functionality... the code ping pong between many different classes which all work on the same data at the same time, it's very challenging to grasping it all. One might call it some sort of spaghetti code. Now.. it might seems inevitable on large project with large team... But, at the risk of being blind folded by fanboyism, I think Microsoft.NET API code looks quite neat and simple. And this is a large project API too, 20 years in the making! By a large corporation! So.. how did they do it? How does one push back against the growing complexity?
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
Best resolved as follows: 1 - Ctl-A 2 - DEL
"The difference between genius and stupidity is that genius has its limits." - Albert Einstein
"If you are searching for perfection in others, then you seek disappointment. If you seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010
-
honey the codewitch wrote:
There are functions used in oleaut32.dll routinely that I swear nobody at microsoft actually knows how they work, or maybe even how to call them properly anymore.
COM is probably one of the best understood technologies at Microsoft. Many of the old COM/OLE guys are still around. Although Don Box[^] left two months ago. The only API that I know of that might have been lost is the DirectUI code. When the team was dissolved apparently some of the source code disappeared. There was like 1 guy in building 88 with some DirectUI header files last I heard.
LOL oleaut32.dll has a few functions for mangling typelibs that nobody uses anymore, but I had to emulate at one point. I never finished that part because nobody at MS could tell me what they did. I even contacted people involved in the Windows OS team when I was on it.
Real programmers use butterflies
-
Best resolved as follows: 1 - Ctl-A 2 - DEL
"The difference between genius and stupidity is that genius has its limits." - Albert Einstein
"If you are searching for perfection in others, then you seek disappointment. If you seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010
such helpful wisdom in your words. :sigh:
-
LOL oleaut32.dll has a few functions for mangling typelibs that nobody uses anymore, but I had to emulate at one point. I never finished that part because nobody at MS could tell me what they did. I even contacted people involved in the Windows OS team when I was on it.
Real programmers use butterflies
Hmmmm,
honey the codewitch wrote:
I even contacted people involved in the Windows OS team
I am pretty sure that the mangling is controlled by the interface definition language[^] so you probably should have asked someone on the compiler team instead. There use to be some non-public MIDL reserved keywords on our internal build tools. I'm not sure if that's still a thing.
-
Hmmmm,
honey the codewitch wrote:
I even contacted people involved in the Windows OS team
I am pretty sure that the mangling is controlled by the interface definition language[^] so you probably should have asked someone on the compiler team instead. There use to be some non-public MIDL reserved keywords on our internal build tools. I'm not sure if that's still a thing.
The IDL facilities use the typelib facilities i'm talking about. The compiler team did not create oleaut32.dll
Real programmers use butterflies
-
The IDL facilities use the typelib facilities i'm talking about. The compiler team did not create oleaut32.dll
Real programmers use butterflies
honey the codewitch wrote:
The IDL facilities use the typelib facilities i'm talking about. The compiler team did not create oleaut32.dll
No, we are in a grey area here... oleaut32 has it's origins in VB5. I believe it was distributed with the VB5/VB6 runtime prior to Windows XP SP1. What time frame are you referring to? Was this back when you worked at the company?
-
honey the codewitch wrote:
The IDL facilities use the typelib facilities i'm talking about. The compiler team did not create oleaut32.dll
No, we are in a grey area here... oleaut32 has it's origins in VB5. I believe it was distributed with the VB5/VB6 runtime prior to Windows XP SP1. What time frame are you referring to? Was this back when you worked at the company?
It was old by the time I was working on the Windows team in about 2001. oleaut32 is a system DLL - i could have sworn it predated VB5 because of OLE automation which predates it, but my memory isn't so reliable these days. oleaut32.dll provides system implementations of things like IDispatch and ITypeLib. It's basically the guts that make Automation work inside of what is now known as ActiveX** but prior to that was simply called OLE (object linking and embedding) ** I'm simplifying a little bit. OLE and ActiveX both exist, and ActiveX was sort of intended as OLE pared down for web downloads at first but then it became the de facto way to develop new components.
Real programmers use butterflies
-
It was old by the time I was working on the Windows team in about 2001. oleaut32 is a system DLL - i could have sworn it predated VB5 because of OLE automation which predates it, but my memory isn't so reliable these days. oleaut32.dll provides system implementations of things like IDispatch and ITypeLib. It's basically the guts that make Automation work inside of what is now known as ActiveX** but prior to that was simply called OLE (object linking and embedding) ** I'm simplifying a little bit. OLE and ActiveX both exist, and ActiveX was sort of intended as OLE pared down for web downloads at first but then it became the de facto way to develop new components.
Real programmers use butterflies
-
Their departments are semi-autonomous. Their quality kind of depends on the department, but in general they follow the same basic practices, some just better than others. When they do it well, they are really good. They do regression, stress testing, they instrument for code coverage. They have bug counts and charts of counts and goals on the hallways outside of developers quarters. They really do put an emphasis on it. It's why I am so not happy with their windows updates. They can do better. Something happened since I left. :mad:
Real programmers use butterflies
-
As I am starting my new job at EA, working on the Frostbite Editor, which is easily a 1000 man years job if I were to hazard guess... The codebase is very large! And the parts I am working on are very messy... Trying to study one little bit of functionality... the code ping pong between many different classes which all work on the same data at the same time, it's very challenging to grasping it all. One might call it some sort of spaghetti code. Now.. it might seems inevitable on large project with large team... But, at the risk of being blind folded by fanboyism, I think Microsoft.NET API code looks quite neat and simple. And this is a large project API too, 20 years in the making! By a large corporation! So.. how did they do it? How does one push back against the growing complexity?
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
It sounds like no one had a good idea how to structure this code cleanly, so it has gotten out of hand. There is undoubtedly a fair bit of inherent complexity in it, made much worse by artificial complexity. As a newcomer to the team, you probably lack the influence to change things, especially the culture that produced this. Focus on gaining the respect of your colleagues and understanding as much of the existing code as possible. Not so much on how it works, but what it does. Think about how you would refactor or even rewrite some or all of it. If a crisis eventually occurs, you will then be ready to propose a path forward. Hardly any of your colleagues--quite possibly none--will have thought about this. If you present a cogent redesign, it will likely get adopted, and you will have the opportunity to clean up the mess if that's the kind of thing that appeals to you. In the meantime, there's the question of how long you're willing to work on convoluted code, waiting for a crisis, before it becomes so painful that you need to transfer to another group. In a large company, it is often possible to find an internal transfer, and you may be able to look at other groups' code to determine if their grass is greener.
Robust Services Core | Software Techniques for Lemmings | Articles
The fox knows many things, but the hedgehog knows one big thing. -
In C#/.NET I've found the use of partial classes to be fantastic for that. For example, I'll have MyList.IEnumerable.cs MyList.ICollection.cs MyList.IList.cs And each file implements the named interface. In C++ your options are ... messier. There's no way to split code up in C++ that I know of that doesn't add to complexity.
Real programmers use butterflies
Don't C++ allow implementation of multiple abstract classes?
-
Don't C++ allow implementation of multiple abstract classes?
Yes, but multiple inheritance comes with its own complexities. I wouldn't recommend it if your goal is to reduce complexity. This is triply true when templates are also involved.
Real programmers use butterflies
-
Quote:
Something happened since I left
You left, obviously! ;-)
- I would love to change the world, but they won’t give me the source code.
I thought that would have improved things. :confused:
Real programmers use butterflies
-
As I am starting my new job at EA, working on the Frostbite Editor, which is easily a 1000 man years job if I were to hazard guess... The codebase is very large! And the parts I am working on are very messy... Trying to study one little bit of functionality... the code ping pong between many different classes which all work on the same data at the same time, it's very challenging to grasping it all. One might call it some sort of spaghetti code. Now.. it might seems inevitable on large project with large team... But, at the risk of being blind folded by fanboyism, I think Microsoft.NET API code looks quite neat and simple. And this is a large project API too, 20 years in the making! By a large corporation! So.. how did they do it? How does one push back against the growing complexity?
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
Super Lloyd wrote:
Now.. it might seems inevitable on large project with large team... But, at the risk of being blind folded by fanboyism, I think Microsoft.NET API code looks quite neat and simple. And this is a large project API too, 20 years in the making! By a large corporation!
First thing that came to mind is "[EA Game Name] [Year++]". That's how it works, right? They keep going back to the same codebase, someone adds something to make it bigger/uglier, and they release a new version in the established franchise...
-
Super Lloyd wrote:
the parts I am working on are very messy...
Free tip : If I were you, I'd be very careful with such statements on the internet about your current job, especially since you mentioned your employer's name and even the projects you are working on. Be unspecific, Loungers know where you work :-)
Excellent point. But in reality, I don't think details (like the actual company name) have to be known. If something's been evolving for 20 years and being worked on by a large team...yeah, us developers will all automatically assume it's grown to become downright nasty.
-
Working on several generations of the same project for 25 years almost I see that there is no straightforward solution for this. Setting up code standard does not help, code review does not help... There is no way to 'clean' an old and complex code but to rewrite it from ground up - it will probably cost more in time than the original investment combined over years and will left you with a buggy code to test from zero...
"The only place where Success comes before Work is in the dictionary." Vidal Sassoon, 1928 - 2012
Yes.. I suspect it is unavoidable...
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
-
Super Lloyd wrote:
How does one push back against the growing complexity?
If I may venture an opinion: sometimes you don’t. Sometimes that complexity reflects what I call “battle-scarred code”. Old code that has seen many battles is rarely clean but it contains that protection against a divide by 0, and can handle that weird situation we encountered 5 years ago and so on. Really great companies, like Microsoft, might also have someone like Raymond Chen to maintain some kind of oral history project but in most places you just have to find yourself all the good stories hidden inside the code. And the bad ones too. Anyway, I learned to respect the battle-scarred code.
Mircea
Mircea Neacsu wrote:
someone like Raymond Chen
I remember some Microsoft blog video some years ago, and cant remember if was talking to him, that essential majority of the .net documentation (at least 2005 - 2015 maybe) was passed through this one person, hence provided great consistency across the documentation. As to Frostbite, I wonder how much gaming industry project to project rehire cycle has an impact, and the get it out the door. If you have somthing like World of Warcraft, where new players coming in over the years, then fixing "old" parts are relevant to user support. Also go read [Object-oriented programming is dead. Wait, really?](https://thenextweb.com/news/object-oriented-programming-is-dead-syndication) posted in articles the other day, might be relevant to how people thought they needed to program the system over the years vs reality of programming it.
-
Regular review, and long design cycles. Lots of dogfooding by other developers. All of which is expensive. Also, actually having a documentation and technical writing team working *with* the development team helps both parties make better stuff. And then in terms of implementation I think they had an eye toward creating a "reference implementation" from the jump so there was some feeling that your code was going to be looked over by the public to keep people honest. This is all just feels and vibes I got working from working on .NET and Visual Studio teams at Microsoft at points, nothing concrete. But I will say this: Microsoft threw a ton of money at implementing the .NET BCL precisely because it was public facing. I don't know how public facing and broadly distributed the Frostbite editor source code is, but I do know EA is a big company and *can* produce clean code if they want to. It just costs more in terms of time (and thus money) - and trust me when I say that it's a hassle to have to code to standards that such necessarily requires - as well it should be since it's almost always harder to do the Right Thing(TM)
Real programmers use butterflies
honey the codewitch wrote:
But I will say this: Microsoft threw a ton of money at implementing the .NET BCL precisely because it was public facing.
This is the key point. Friends who worked in the gaming industry mostly burned out due to the insane time-to-market "schedules". The entire point was to get the programs working so they could be published, and it didn't matter how programs worked as long as they worked. Once a version was published, and sometimes before a version was published, it was on to the next version. There was no time to fix the errors of rushed coding unless something conflicted with something else in the next version. Sure, this happens in other industries, but gaming is IMO the poster child for propagating and maintaining bad code.
-
honey the codewitch wrote:
But I will say this: Microsoft threw a ton of money at implementing the .NET BCL precisely because it was public facing.
This is the key point. Friends who worked in the gaming industry mostly burned out due to the insane time-to-market "schedules". The entire point was to get the programs working so they could be published, and it didn't matter how programs worked as long as they worked. Once a version was published, and sometimes before a version was published, it was on to the next version. There was no time to fix the errors of rushed coding unless something conflicted with something else in the next version. Sure, this happens in other industries, but gaming is IMO the poster child for propagating and maintaining bad code.
That makes total sense to me. I mean, not that it works that way, but that gaming would be the one to do it. The pressure to market is just overwhelming with games. It's a constant race against competitors, against newer hardware, against major shopping holidays - it's ridiculous. I don't think there's a way to fix the code without fixing the above.
Real programmers use butterflies
-
As I am starting my new job at EA, working on the Frostbite Editor, which is easily a 1000 man years job if I were to hazard guess... The codebase is very large! And the parts I am working on are very messy... Trying to study one little bit of functionality... the code ping pong between many different classes which all work on the same data at the same time, it's very challenging to grasping it all. One might call it some sort of spaghetti code. Now.. it might seems inevitable on large project with large team... But, at the risk of being blind folded by fanboyism, I think Microsoft.NET API code looks quite neat and simple. And this is a large project API too, 20 years in the making! By a large corporation! So.. how did they do it? How does one push back against the growing complexity?
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
The only time I've ever had much success with that sort of thing (and I'm guessing this isn't an option here) is by sitting everybody down, writing a detailed specification for what it's *supposed* to do, then redesigning the whole thing from scratch - properly - and rewriting it. In the long term, this will actually save time, in that from that moment on, half the development effort won't be spent refactoring old code... but it's a tough job to sell something like that - tough in proportion to the size of the existing codebase, that is.
