Dependency Management

If ever there was a problem that just wouldn’t die it has to be inter-project dependency management. If your code depends on external libraries, it’s pretty simple to pick whichever solution you prefer – either grabbing a version from a repository or checking jars into your source control. However, if you depend on a project that you control, it gets so much messier.

If the projects are small, it’s probably a good idea to just set up the build system to build them all together – effectively making them separate projects even if for development purposes you can just build this bit or that bit and utilize precompiled versions of the dependencies.

Once the projects get to a certain size though it simply takes too long to go recompiling everything unless you really need to. So you want to set up a repository for your internal projects that you can just grab builds from. That’s great until you’re out on the road and want to compile – suddenly that repository isn’t so accessible. The other problem is you always want to be able to reproduce the build with the exact same contents – including the exact build of your dependencies. This implies that you have the particular version number used checked into source control and that you can get that build again – even if your repository goes belly-up (worst case you have to rebuild the dependencies from source control too).

Of course, if you have a particular build number checked into source control, you’re not building against the very latest so if things break you won’t know about it until much later when you upgrade. This is where Ephox is at and it’s just not good enough. If you always build against the very latest though, it’s hard to tie things down when you’re testing for a final release.

There’s a couple of things we can do to improve the situation:

1. When you depend on some behavior or API in another project, write a test for it in that project and give it an obvious comment describing which project needs that behavior. Then if the behavior or API changes, the project just won’t build and someone will have to go and fix it.
2. I think we need to set up a nightly build that takes the Gump approach. It should build every single project, always using the very latest version of it’s dependencies, then if the build passes, it should automatically commit the version identifier for the dependencies so that when the developers come in the next day they’ll all develop against that build (which they can pull from the repository). This neatly avoids the development delays of having to checkout and build dependencies, while still keeping things as up to date as possible and providing fairly rapid feedback if things break.

That leaves just the problem of doing offline builds – something that’s not been a major problem for us before but on a recent project has been a key requirement. The solution we have for that project, which seems to work quite well, is to have the build pull in the dependencies and cache them where they aren’t deleted when you do a build clean. Then, simply provide a property that can be set to tell the build to work completely offline and just use the cached version. There’s still a few glitches with it but generally it’s working really well.

So if someone wants a project while I’m away on my honeymoon, come talk to me this week and I’ll give you an overview of how I think we can make it work.