The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

dependencies and makefiles

When I'm making a command line program, or a test harness, I just use the command line compiler. But that requires a makefile, which I have to manually maintain. In particular, I have to manually go through source code to find all the .h files and manually add them as dependencies to every single file.

Is there some sort of program out there that isn't a IDE will just traverse a hierarchy of files, parse the C and C++ files, figuring their .h dependencies, and auto-generate a nice makefile to build the whole thing?
Art Wilkins
Sunday, July 09, 2006
 
 
It seems that doing "gcc -M sourcefile.c" will generate the dependency line you need for a makefile to the standard output.

So I guess the idea is to write a script that calls this on all the files present in the hierarchy.
Art Wilkins
Sunday, July 09, 2006
 
 
Then delete eveything that isn't in your src tree, since -M will return all the standard headers as well.

Post it when you've got it, sounds handy.
Lenny
Sunday, July 09, 2006
 
 
Turns out that -MM skips the system headers...

I found some makefile makers here and there on the net but they are either so simple that they don't handle dependencies, or they are so complex I have no hope of configuring them.
Art Wilkins
Sunday, July 09, 2006
 
 
The complex one I am thinking of is this:

http://sourceware.org/automake/

Which requires the following library:

http://ftp.gnu.org/gnu/autoconf/

Downloading both of them gives you megabytes of source files with nonsensical documentation and nonworking configure scripts. But this is supposed to be the 'standard' way of doing it. I am sure I could hire someone to work on this for a month and figure it out but by that time I might as well just go back to manually writing the makefiles by hand.
Art Wilkins
Sunday, July 09, 2006
 
 
(The pdf manual explaining how to install and configure automake is 144 pages long... I am looking for something to make my life easier, not harder, is what I want to tell the gnutards!)
Art Wilkins
Sunday, July 09, 2006
 
 
Incidentally, that's what kills me about the whole GNU/Linux thing.  I'm a Windows programmer, so when I want to learn how to write something on Linux, I picked up a book on automake and autoconf.  Now, I'm hearing that CMake is the way to go. 

It seems like every time you go to use something, you find out about a dependency that you require (libyourbehind.thetimes.so), or, behold, now there's a new mega cool thing you should be using instead.

Arggh!

This is also why I hated Java so much.  You write an app and someone says, "that's cool but you should have used the AWT."  Then 2 months later or whatever, it's the Swing library.  (Or did I get those backwards-- I don't know.)

Give me Microsoft Visual Studio any day, and C++.
Meganonymous Rex Send private email
Sunday, July 09, 2006
 
 
OK, downloading cmake...
Art Wilkins
Sunday, July 09, 2006
 
 
Ha - it installed 20MB of files... but where?

Off to find a manual and try to figure out what it just did.
Art Wilkins
Sunday, July 09, 2006
 
 
Ah, there is a 250 page book that explains how to use it and is available for a mere $48.
Art Wilkins
Sunday, July 09, 2006
 
 
Hm, it seems to do stuff. This might work. Thanks.
Art Wilkins
Sunday, July 09, 2006
 
 
Here is my tip for you guys. After installing it, type this:

cmake --help-html > help.html

That causes it to choke up a copy of its own documentation for you to read.
Art Wilkins
Sunday, July 09, 2006
 
 
There are other make file tools available, scons for example.  I used it to build a real world application with under five minutes worth of reading to solve problems that you mention in this very thread.  Ruby seems to be the in thing these days, so if python isn't your scripting poison, I'm sure you can find a few make alternatives over at Ruby Gardens or Rubyforge...Or with any other scripting langauge of your liking.

As for using any GNU tools, I believe autoconf is the most convoluted and complicated tool to coerce a solution out of but then, when you consider what it is doing across platforms that can vary so widely, it makes life easy for others who have minimal experience in software configuration and build processes; one understands the complexity required from the actual maintainer to use it.
Passing Through
Sunday, July 09, 2006
 
 
autoconf is horrible. Being a Linux guy I have occasionally felt the urge to use it. My advice is don't. Unless the project is horribly complex or is going to be ported to 10 different version of Unix you are better off manually maintaing a Makefile and writing your code in a portable way.

DigiTemp (http://www.digitemp.com) runs on most flavors of Unix and I still only use a Makefile with a different target for each system that changes a define.

Brian
Brian C. Lane Send private email
Sunday, July 09, 2006
 
 
Thanks for all the pointers. I've got cmake working, it's actually pretty straightforward and seems to do exactly what I want. It's pretty cool.
Art Wilkins
Monday, July 10, 2006
 
 
> autoconf is horrible

I thought it was just me.  I had read the first few chapters of an O'Reilly book and was like...wha the fuh?? I do not have much professional experience programming on UNIX, though, so I assumed that I was just being naive/idiot. 

Good news on cmake, Art.  I have been hearing about it a lot lately-- and I think that this months Linux Users Journal (or Dr. Dobbs-- one of the two) has a brief article on using that tool.
Meganonymous Rex Send private email
Monday, July 10, 2006
 
 
Creating a makefile for GNU make that works is possible, but it's a bit of a faff. However, you don't need to be a make expert to start with -- though you may end up one :-|

To make it pretty much work, you need to mark each .c file as producing a .d file and a .o file, and run gcc with the -M switches. (I used -MD and -MT, off hand I don't recall why -- I'm looking at the last makefile I wrote, but I don't have gcc installed.)

Your makefile also needs to include all the relevant .d files. This is the key: when including, GNU make will try to ensure the included files are up-to-date first, and so run the steps to generate the .d files. It will then include them, and do the make proper. (As it happens, in this case, making the dependencies does the compiling, so when doing the 'proper' make it will only need to link, but that's not really important.)

The only other thing you need to do is surround your includes with a check for the build target, because you don't want it to try to compile your files when you're doing a "make clean".

(For my part, I turned all this into a macro, to make it easy to type out.)

Virtually all of my ideas came from here:

http://www.canb.auug.org.au/~millerp/rmch/recu-make-cons-harm.html

This provides a good guide to making a working makefile, in terms of making the dependencies work.
Tom_
Monday, July 10, 2006
 
 
The aforementioned macro, for whatever it's worth. (I've manually fixed it up in the forum text box with '\'s -- hopefully correctly.)

Firstly, set up the BUILD_NAME variable. I generate this from build type (release, debug, etc.) and names of libraries (e.g., "debug-unopt-wxgtk"), but it doesn't really matter.

The build name is used to generate the name of the final binary, and the folders used to store the .o, .s and .d files.

Once you've generated the build name, this snippet goes next:

---8<---

ALL_OBJS:=
ALL_DEPS:=
CTAGS_FILES:=

INTERM_DIR:=$(BUILD_NAME)
BIN_NAME:=./pgm-name-$(BUILD_NAME)

define template
$(1)_INTDIR:=$$($(1)_SRCDIR)/output/$(INTERM_DIR)

$(1)_FULL_CFLAGS=$$($(1)_CFLAGS) -MD -MT\
  "$$($(1)_INTDIR)/$$*.d"

$(1)_MKDIR=mkdir -p "$$($(1)_INTDIR)"

$$($(1)_INTDIR)/%.s:$$($(1)_SRCDIR)/%.$(2)
    $$($(1)_MKDIR)
    $(CC) $$($(1)_CFLAGS) -dAp -o "$$@" -S "$$<"

$$($(1)_INTDIR)/%.d $$($(1)_INTDIR)/%.o:$$($(1)_SRCDIR)/%.$(2)
    $$($(1)_MKDIR)
    $(CC) $$($(1)_FULL_CFLAGS) -o\
  "$$($(1)_INTDIR)/$$*.o" -c "$$<"

$(1)_OBJS:=$$(patsubst %,$$($(1)_INTDIR)/%,$$($(1)_NAMES:.$(2)=.o))
$(1)_ASMS:=$$($(1)_OBJS:.o=.s)
$(1)_DEPS:=$$($(1)_INTDIR)/*.d

.PHONY: $(1)_clean
$(1)_clean:
    $(RM) $$($(1)_OBJS) $$($(1)_DEPS)

.PHONY: $(1)_cleanall
$(1)_cleanall: $(1)_clean
    $(RM) -R $$($(1)_SRCDIR)/output

ALL_OBJS:=$$(ALL_OBJS) $$($(1)_OBJS)
ALL_DEPS:=$$(ALL_DEPS) $$($(1)_DEPS)
CTAGS_FILES:=$$(CTAGS_FILES) $$(patsubst \
  %,$$($(1)_SRCDIR)/%,$$($(1)_NAMES)) $$(wildcard \
  $$($(1)_SRCDIR)/*.h) $$(wildcard $$($(1)_SRCDIR)/*.inl)

endef

---8<---

Then, for each component in your program you list the compile flags (_CFLAGS), source folder (_SRCDIR) and the .c files (_NAMES) you're using, all with a prefix corresponding to component. (The "component" notion is for your benefit alone; the makefile doesn't generate .a files or anything like that, it just produces .o files and links them all together as one at the end.)

For example, the HL component, which I'll pretend is the only one:

---8<---
HL_CFLAGS:=$(STD_CFLAGS) -IHostShared
HL_SRCDIR:=./HostLinux
HL_NAMES:=\
    Host.c\
    HostGfx.c\
    HostSnd.c\
    HostInp.c\
    HostTmr.c
---8<---

(Note STD_CFLAGS; this is just a var I set up that holds some standard flags. The exact contents aren't so important, it is for common stuff like -D and -O options.)

You then expand the macro once for each component:

---8<---
$(eval $(call template,HL,c))
---8<---

(For debugging, you can use "$(warning $(eval ...))" to print the expansion on stderr, saved me no end of heartache that did :)

And then you link at the end:

---8<---
$(BIN_NAME): $(ALL_OBJS)
    $(CC) $(LDFLAGS) -o $(BIN_NAME) $(ALL_OBJS)
---8<---

(Again, LDFLAGS is just a var to hold linker flags. Exact contents aren't important; it just has the -l options you want and so on.)

For cleaning support, add some targets:

---8<---
.PHONY: clean
clean:HL_clean

.PHONY:cleanall
cleanall:HL_cleanall
---8<---

Ctags for your project and some select system headers ("make tags"):

---8<---
tags: $(CTAGS_FILES)
    @echo "ctags..."
    @ctags\
  $(shell find "/usr/X11R6/include/" -name "*.h")\
  $(shell find "/usr/local/include" -name "*.h")\
  $(CTAGS_FILES)
    @echo "  ctags done."
---8<---

And finally, the including mechanism that I mentioned before, along with the target check:

---8<---
ifneq ($(MAKECMDGOALS),clean)
ifneq ($(MAKECMDGOALS),cleanall)
-include $(ALL_DEPS)
endif
endif
---8<---

And to make it all easy, have your first (default) target in the makefile as this:

---8<---
default_target:$(BIN_NAME) tags
---8<---

And just type "make" :)

(This wasn't that much fun to write, nor did I end up any smarter because of it, but life's too short to spend time on manual "make depend" etc. And once I was up and running with this, along with some vim keyboard mappings for F7 etc., it was not really any harder to use and maintain than doing the equivalent with Visual C++; what you lose in drag-and-drop, you gain in not having to fight those tiny project properties text boxes!)
Tom_
Monday, July 10, 2006
 
 
I'm still laughing at "gnutards", LOL.
~Eric
Monday, July 10, 2006
 
 
Art Wilkins wrote:

"Downloading both of them gives you megabytes of source files with nonsensical documentation and nonworking configure scripts. But this is supposed to be the 'standard' way of doing it."

To be fair, automake and cmake were created to solve a much bigger problem then simply automating dependency tracking. Their real function is to build makefiles that will work on multiple platforms and that will accommodate multiple compilers and multiple versions of the "standard" C libraries. If you don't need the multi-platform stuff, then you may be using a sledge hammer to kill a gnat.

I found that the documentation for automake is like a lot of UNIX documentation: on first reading it is nearly incomprehensible. However, after reading a couple of O'Reilly books, being tutored by an automake guru, and writing a couple of simple examples, it begins to make sense. After a year of daily use the documentation will seem perfectly clear, and you won't be able to think of any way of stating matters more clearly.

If you do end up needing to use automake, the "Goat" book (http://sourceware.org/autobook/) is a useful alternative to the GNU documentation.
Charles E. Grant Send private email
Monday, July 10, 2006
 
 
"If you don't need the multi-platform stuff, then you may be using a sledge hammer to kill a gnat."

No argument that it's a gigantic thing that solves huge industrial sized problems.

But it's also the standard 'answer' to the question.

cmake has the advantage that it makes simple things easy and hard things possible.

In automake, everything is possible... if you can figure it out. Likewise, programming a custom solution in assembly is also a powerful solution as long as one is willing to invest the time needed.

Many open-source tools share a design philosophy of 'everything you need, all from the command line' coupled with documentation that consists of out-of-sync man files that consist solely of a 260k long dump describing all the features of each of 7800 switch options.

By the time something is this complex, it needs something other that switches and/or configuration files.

cmake is cool because it goes out and finds the information it needs and makes the right decisions about things.

This is extremely rare to see in an open source tool and its designers are to be commended. They are obviously pragmatists.
Art Wilkins
Monday, July 10, 2006
 
 
I meant to say "each of 7800 switch options, arranged alphabetically."
Art Wilkins
Monday, July 10, 2006
 
 
The trick with dependencies is to cache them in dependency files.

You have a rule which generates a .d file for each .cpp .h pair. This contains the output from something like a gcc -MM

You then include this in your makefile, usually by doing something like a wildcard on your .cpp files and then a patsubst to turn it into a list of .d files.

If they don't exist, but your generic ".cpp -> .d" rule does, GNU make will execute that and restart the make from that point, meaning you don't need a seperate "make depend" cycle.

The real trick in this is to process your .d files to make them themselves depend on the .cpp file and the .h files it includes. This way it will correctly remake the dependency list before evaluating remaking the .o file

There's a paper which is a reasonably good guide to this stuff;

http://www.canb.auug.org.au/~millerp/rmch/recu-make-cons-harm.html
Katie Lucas
Tuesday, July 11, 2006
 
 
Thanks.
Art Wilkins
Tuesday, July 11, 2006
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz