The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Social Software: Xoreax Incredibuild

Having used Incredibuild, a parallel distributed build environment, here at work for a while, I notice that it has some shortcomings in its social engineering aspects.  While it is not designed solely to facilitate social interaction in the same way that IM chat clients or web sites like Friendster are, it nevertheless, by virtue of being a distributed team build environment, comes with unavoidable social interaction overhead built-in.

However, it is quite clear that the designers have not had a lot of experience with social software design, nor any background in game theory.

The idea behind Incredibuild is that multiple members of a software development team all install the Incredibuild client on their machines, and then when any team member wants to build a project, the software farms out the build to all other clients which are available.

A client is considered available if it is enabled by the person using the machine (a toggle in the client UI) and if it is not already working on some other build.  There are also factors of CPU load taken into consideration by the system when assigning distributed build tasks.

There are various legitimate reasons for disabling your client.  However, and this is where the social misengineering comes into play, there is no penalty applied for disabling your own client.  You can still request builds, and have the other clients perform work for you, even if your client is not willing to perform work for them!

In game theory terms, there is no penalty at all for defecting, and essentially no payoff for cooperating.  The software is designed such that only other members of your team get a payoff when you cooperate, and likewise they suffer a penalty when you defect.

The result of this is that in my team of about 20 developers, roughly half disable their clients all the time, on principle.

If this software had been designed with social engineering in mind, one or both of the following features would have been added, at theoretically next to no development cost:

1) A disabled client does not get to request build support from other (enabled) clients.


2) Each user can set a flag locally to prevent their client from accepting build requests from other disabled clients.

Either of these options would dramatically increase the incentive for every individual to enable their own client.  It would still be possible, of course, to leave one's client disabled most of the time, and only enable it in order to build.  But this is a lot more work than just leaving it enabled, and thus less likely to occur.

I did bring up this issue with Xoreax support, and their response was basically that no one else had mentioned this, so they had no business case to implement these features.  But they'd keep it in mind in case the demand arose...
Steve Keppel-Jones Send private email
Tuesday, March 14, 2006
Right, the whole thing is extremely easy to "beat" since you can reap the rewards without giving anything back to the community.

Then, if this spirals out of hand, the whole system becomes useless... but Incredibuild still has all that money from the licenses purchased.

So, why should Incredibuild enforce a bunch of whiners?
Tuesday, March 14, 2006
> game theory terms

Are you sure you have modeled the problem correctly in a game theory sense? Are there really defectors and cooperators in a work setting?
son of parnas
Tuesday, March 14, 2006
I do believe that the game theory approach is entirely valid.  Formal game theory applies to more than just tic-tac-toe.  In any situation where there is an interaction between two or more people, or rabbits, or software agents, the fundamentals of game theory are at work.  An interaction takes place, and the outcome can be harmful or beneficial to any number of the participants.  The results of each interaction are taken as inputs to the next interaction whenever there is a repeated interaction situation, which occurs in many real-world instances.  These are the most interesting types of games.

There are all kinds of corollaries to this, related to the size and anonymity of a population, and the resulting tendencies to cooperate or defect, but I won't go into that now.

Anyway, in the case of Incredibuild, just as in any social situation, you are considered a "cooperator" when you take actions which benefit the rest of the group, regardless of whether you personally benefit or not - i.e. when you enable your build agent to help others.  You are considered a "defector" when you take actions which are at the expense of one or more others - i.e. when you disable your build agent.

In many real-life situations, there is an immediate benefit to defecting, and usually a smaller benefit to cooperating.  But in the long run, being known as a defector generally has greater negative consequences than the benefit that you reaped by defecting in the first place, while cooperating tends to have a smaller but more consistent benefit for the individual and the group.  So the interesting questions are: Who chooses to cooperate, and who chooses to defect, and why?  This is what game theorists study.

Naturally the Incredibuild designers don't get any immediate benefit themselves from applying game theory principles to their software - as was noted, they have their license revenue and they're happy.  But given a choice between two similar products, one which has the social engineering right and one which doesn't, I would pick the correctly-designed one every time.  The problems here are that, first, there isn't really anything else on the market which is just like Incredibuild except for being socially engineered correctly, and second, I do not get to make purchasing decisions where I'm currently working.  So Xoreax is in luck, this time around.
Steve Keppel-Jones Send private email
Tuesday, March 14, 2006
>> Anyway, in the case of Incredibuild, just as in any social situation, you are considered a "cooperator" when you take actions which benefit the rest of the group, regardless of whether you personally benefit or not <<

I think the personal benefit that results from running the utility on your desktop when everyone else on the team is doing it, is the prevention of an unpleasant meeting with your manager in his office.

"Joe, don't you *want* to be a team player here at IniTech?  Because it sounds like you failed to ask yourself what would be good for the company."
example Send private email
Tuesday, March 14, 2006
My game would be there are a lot of other players in the company whos machines do virtually nothing all day. Don't bog down my machine, which I actually use, to do builds. Suck up your resources elsewhere.

So I think developers are behaving quite rationally.
son of parnas
Tuesday, March 14, 2006
"Anyway, in the case of Incredibuild, just as in any social situation, you are considered a "cooperator" when you take actions which benefit the rest of the group, regardless of whether you personally benefit or not"

I think this is what son of parnas is getting at. In a work setting, you and your co-workers aren't autonomous agents, you're all working (in theory) on the same project, or at least for the same company.

So does it make sense to "defect" at all, when the lot of you will prosper or fail together based on the success of the project overall? Are you really getting ahead by defecting if it harms the team that you're working on?

And if you don't actually gain anything from defecting, then how does game theory apply?

The defect/cooperate dynamic seems like it'd be more apt if you had, say, two unrelated companies that shared an Incredibuild license between them. But how often does that happen?
Ian Schreiber Send private email
Tuesday, March 14, 2006
>  Ian Schreiber

Ian said it much better than I could.
son of parnas
Tuesday, March 14, 2006
Don't know how well Incredibuild monitors "idle time". Condor (a computational cluster management system) is quite good at it -- with its default, it doesn't kick in while I'm using my computer, and takes ~5 seconds to evacuate running tasks when I want to use it again after not using it (hence giving away my resources).

I think you're going too far with applying game theory to incredibuild. Taking ad absurdum, introducing bugs vs. fixing them is the same. The real reward/loss comes from meeting with your manager / client / whoever.
Ori Berger Send private email
Tuesday, March 14, 2006
Oh no, I don't think I'm going too far at all with the application of game theory to Incredibuild.  At any given moment, each individual who has a client installed has two choices with respect to that client: enable it or disable it.  Cooperate or defect.  The advantage to defecting is that your machine is never going to be busy doing a build for someone else.  This is a measurable advantage - Incredibuild is definitely not a completely invisible background agent.  It uses up resources that you could otherwise be using yourself, it doesn't always immediately go away if you start running something else (it always finishes the current compile task first, which can take on the order of a minute), and especially in the laptop environment that most of us run here, it means that your CPU fan and disk drive are going to be running a lot more than they otherwise would.  This causes noise and other distractions on your desk.

So some people in my group simply choose to defect - disable their clients - all the time.  This is despite the obvious point made earlier that the whole group is, in fact, working towards a common goal, and therefore it is in the group's best interest if every individual leaves their build client enabled all (or almost all) of the time.

There is no management pressure to have everyone's client enabled, and only a little to install the client in the first place (the obvious motivation there is that you cannot do a distributed build yourself unless you install the client).  So once the client is installed, for every individual, it is quite clear that their own work will proceed faster with the client disabled.  Not much faster, but every little bit helps.  And in any environment where individual progress is measured, which is most of the companies I've worked for (despite the fairly logical arguments that such measurement is actually harmful), there is a clear incentive to do your own work faster, even (or especially) at the expense of your coworkers.

Dysfunctional?  Absolutely.  But that's a separate topic.

So it is quite obvious to me that group usage of Incredibuild is a textbook game, and the designers haven't gotten it right.  They barely even know that they're playing, never mind what the rules are.
Steve Keppel-Jones Send private email
Wednesday, March 15, 2006
Then it sounds like the issue is more that Incredibuild inflicts noticeable pain on its users that gives them reason to defect. The solution isn't to use game theory to convince users to cooperate; the solution is to enhance the user experience so that there's no longer a reason to do so.

If someone wants to use their own spare CPU cycles for something, then Incredibuild should lower its own cycles accordingly, right away. As for laptops making more noise, if a significant number of their customers work on laptops they should allow a special configuration that uses fewer cycles than normal (or something) so that the fan doesn't make any extra noise.

Just using spare CPU cycles that you're not using yourself anyway really shouldn't be causing you pain. If it is, then THAT'S what they need to fix.
Ian Schreiber Send private email
Wednesday, March 15, 2006
Well, it may very well be that Incredibuild puts more load on a system that it should.  But the fact remains that a distributed build system cannot be completely invisible.  Distributed builds require substantial amounts of network traffic, disk traffic, and CPU time - there is no way around this.  And especially in Windows, there is essentially no way to make something as resource-hungry as a compile unnoticeable to a user of a desktop system.  As long as the load is greater than zero, there will always be an incentive to disable it.  And unless the social aspects of the software are correctly designed, there will be no incentive to re-enable it.

The whole point that Joel made in his article on social software engineering is that most people barely even know that there is anything to know about it.  This is roughly the state that GUI design was in, back in the 1980s.  And comments made in this thread serve very well to reinforce this statement.  Now, twenty years later, some of the principles that underlie a good GUI design are starting to become well-known, although not by any means ubiquitous.  Similarly, perhaps in another twenty years, the principles that underlie social aspects of software interaction may become well-known.

My point here is that Incredibuild is just not leading the way by any stretch of the imagination...
Steve Keppel-Jones Send private email
Thursday, March 16, 2006
It seems clear that the Xoreax can either:
* Try to improve the efficiency of their distributed compile system through lots of hard work, caching and other optimisations
* Use social design principles to encourage users to keep their client turned on, thus making sure any company using their software has a larger and thus faster compile system available to it

The win for Xoreax seems clear here. If a competitor's program uses social engineering to make sure people actually LET you compile on their laptops, then it will build programs an awful lot faster than Incredibuild does.

If the boss has to get involved, it seems to me that the software has failed - isn't this the social equivalent of a bug workaround?
Mark O'Connor Send private email
Friday, March 24, 2006
Well, at least there's one other person who understands what I'm getting at - thanks, Mark.

As another followup to one of Ian's comments, game theory has nothing to do with whether a group to which an individual belongs is working towards a goal that would benefit that individual as well.  In fact, the classic conundrum of the Prisoner's Dilemma (a textbook game if there ever was one) and the Tragedy of the Commons (game theory applied to resource exploitation) is that by defecting, and thereby maximizing short-term personal gain, all of the individuals that comprise a group are actually cooperating more or less unwittingly in a longer-term strategy that causes immense harm to the entire group.  So there is nothing contradictory about applying game theory to a situation such as an office.

To those who, like Ian, don't believe that game theory applies to Incredibuild in an office environment, allow me to quote the definition of game theory from Wikipedia: "a branch of applied mathematics that studies strategic situations where players choose different actions in an attempt to maximize their returns."  There is no way that you can exclude Incredibuild from this definition.  I choose actions in my work environment that maximize my returns - doing the work that I have been assigned in as expeditious a manner as possible.  [1]  Since my work involves the standard software development edit-compile-test cycle, the faster I can run this cycle, the more productive I will be - and this means minimizing the number of other tasks that my machine is performing, such as helping other people to run their own builds, scanning for viruses, executing viruses and other malware, etc. etc.

In other words, by choosing actions that maximize my productivity and hence my returns, including enabling or disabling my Incredibuild agent, I am executing exactly the definition of the subject of game theoretical study.

[1] For the purposes of this discussion I will exclude people whose definition of maximal return is advancement within their organization at the expense of anyone who is in their way, without contributing anything directly measurable to the organization's overall returns - so-called corporate sociopaths, or even psychopaths.
Steve Keppel-Jones Send private email
Friday, April 07, 2006

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz