Actually we build using the VC compiler, just not inside VS or a VS project. That system doesn't scale well for our needs. While this is an interesting approach I'm still favoring the monitoring of the file activity external to the compiler for serveral reasons: 1. We have C/C++ and C# projects in our tree, and possibly other languages as well that I haven't had to deal with yet. 2. We deal with a number of different building tools that may not have output options like the more mature C/C++ compilers do. 3. Since I have a number of differnt build tools I need to track dealing with each one separately and maintaining the output processing code for handling each tools output sounds like a big task and fragile Taking a more build tool agnostic approach to gathering this information I feel will make the tool more reliable and take less maintenance work. I won't be trying to track down all of the differnt build tools we use and the necessary command line options to gather the information. And then have to figure out how to process each tools output to gather the information I need. Adam

Hmm, I'm not sure how hard it would be to create something that would properly process it statically. There is some thinging here that needs to be done around tracking geneology, but once I figure that out I would thing this shouldn't be too hard, but then again I always get blindsided by some of the small details I didn't see when I was thinking about a problem at a higher level. :-) Although I don't think all of the files that need to be processed are C/C++ files, there are resource files as other types of files. And these projects aren't built in VS, they're built in our own build system base off of nmake. Adam

I hope you don't mind I'm using this group as a sounding board. I'm posting my thoughts in part to help solidify them in my head and then also to see if anyone out there has any interesting observations or experience that can help. :-) I have been thinking about what I and my customers would like to get from this tool and have come up with a great many pieces of information that I could/should track. But I'm thinking that I just need to find the core pieces of information and then use queries against the information store to figure out the rest. For example, some of the things I'd like to be able to do are: 1. Determine build order of files/projects 2. Determine which projects can be built simultaneously 3. See all the dependancies of file X/project Y 4. See all the dependancies on file X/project Y 5. Find duplicate files 6. Which projects create which files and then which other project then consume those files. I have some ideas on what to track I just need to implement it and see how it works. The more I think about this issue I think the harder part will be to actually collect this information. Ideally I'd be collecting this information while multiple instances of the compilation tools are executing at once. I'm having some fun wrapping my head around having a single process monitoring multiple processes and maintaining the build ordering/hiereachy inside and outside of a project. I'm wondering if it would be best to do this in two passes. Then there's the question of what's the best design for processing all of those file open/creation transactions. My current thought is to have one thread per compilation process. I only expect to have 4-8 processes going at once, so that doesn't seem like it should be to resource intesive. And just so you know this is the biggest project I've ever tackled, so I'm a bit nervous about getting my approach organized before I start and then making sure I'm starting in the correct place. Thanxx for your input, Adam

Hello there, I'm starting up a little personal project that I'm not too sure what the best way to tackle is. I'm hoping someone could give me a few pointers and/or some ideas on questions I should answer before I dig in. The problem I'm trying to solve to determine file and project dependancies in a big sprawling codebase with tens of thousands of files in it. This codebase has evolved over time and can be quite unwieldy. There are many sub projects inside the build tree and lots of interdependancies between these projects. I want to be able to map these depenencies out. I found a library that will allow me to moniotr file activity, for example file opens and creates. So my thought is if I run a full clean build and monitor and record all of the file activity of a build I can generate a dependency graph of the entire project. This will also allow me to be able to determine what I should build and in which order when I want to build a tiny piece of the tree. I'm sure there will be other interesting things I can do with this information. I'd also like to try and visualize the entire project and maybe create a file change heat map from it. My quandry is how best to record the file activity so that I can build a depenecy tree. Ideally I'd like to be able to do this in a multi-threaded way, as our build system can utilize multi-proc machines and have multiple files building at once, and be able to tell which files need to be built before others and which files are grouped in a project and so on. My current proposed approach is to record all file activity to a logfile and the post process it to generate the depenecy graphs. I'm still a little hazy about all the data that I need to record. I'm currently thinking I'll figure that out as I go when I find I'm missing some important information. Any pointers or thoughts would be greatly appriciated. :-D Thanxx, Adam

Hello there, I'm starting up a little personal project that I'm not too sure what the best way to tackle is. I'm hoping someone could give me a few pointers and/or some ideas on questions I should answer before I dig in. The problem I'm trying to solve to determine file and project dependancies in a big, mature, sprawling codebase, with tens of thousands of files. This codebase has evolved over time and can be quite unwieldy. To build a small tool I need to build most of the build tree because it's almost impossible to figure out what all my dependancies are. So I found a library that will allow me to watch file activity, for example file open, creates, and so forth. So my thought is if I run a full clean build and monitor and record all of the file activity of a build and can generate a dependency graph of the entire project and/or be able to automatibally create a command file that to build any project I want and have it build everything I need in the proper order. I'm sure there will be other interesting things I can do with this information. I'd also like to try and visualize the entire project and maybe create a change heat map from it, but looking at the source server info. My quandry is how best to record the file activity so that I can build a depenecy tree. Ideally I'd like to be able to do this in a multi-threaded way and be able to tell which files need to be built before others and which files are grouped in a project and so on. My current proposed approach is to record all file activity to a logfile and the post process it to generate the depenecy graphs. I'm still a little hazy about all the data that I need to record. I'm currently thinking I'll figure that out as I go when I find I'm missing some important information. Any pointers or thoughts would be greatly appriciated. Thanxx, Adam