The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Professionalizing the development practice

Hi all

I'm working in a large research project, on which many partners develop different pieces of a large software ecosystem with many components. The interfaces are mostly based on web services, but we also have some API libraries we have developed. The project has just released, with a great pain, the first release of the code and we plan to have at least two other major releases.

After near a year and a half, we have found that our development process is not "professional", mostly when it comes to the integration of components (i would say that some components are developed in a very professional way).

What I mean by "not been very professional, is that we have not a real methodology nor a proper set of tools to integrate the code, make testing and track bugs.

I've been commissioned to propose some changes in our current practices, while other guy is doing a research for supporting tools.

I've been gathering some best practices mostly related to design, testing and continuous integration (a practice I've personally never used).

I would appreciate any suggestion regarding to ANYTHING we should consider in our practices.

Thanks is advance
Pablo Chacin Send private email
Wednesday, March 05, 2008
 
 
Come up with a way to document and maintain specs, use cases/stories, etc.  These don't have to be huge 200 page things.  Often for a feature a 1-2 page spec is good enough.  The challenge is keeping them up to date with changes, and requires good document management and discipline.

Lots of the more agile-type methodologies scoff on deep documentation and say "the design is the code", which is only true to a point. 

When you're trying to explain to the marketing guy how something is going to work, he wants to see a one page higher level spec, rather than know you implemented it with three levels of abstraction in Spring.

Plus, when the time comes to hand off the project, or add team members, you have a huge tool to help get people up to speed.
Dan Fleet Send private email
Wednesday, March 05, 2008
 
 
As Dan said, documentation is a must. Figuring out just what documentation you need and the order you should write it in, however is much easier said than done. To borrow from his example, it could be that your marketing guy is the old-fashioned handshake type and he's not going to read anything you write for him.

I'd take his advice to start thinking about a storage/search mechanism such as a wiki, a blog, paper copies, specialized databases, etc.

I'd also get the team together, schedule a few days for it if need be, and just write down all of the problems you had developing the project. I.e., did your schedule slip? Were key decision makers involved too late? Were the requirements poorly communicated? Did it take longer to learn a technology than planned? Were you short on resources like computers? And so on.

Once you have a list of problems, look for "root causes". If your schedule was bad because you had no training in preparing good schedules, then the real problem is (arguably) lack of training and experience, not the schedule itself.

If you exclude the problems that can be blamed on specific individuals such as the executive who spent all of his time on the golf course and never answered his phone, the rest of the set should indicate where you need to focus your documentation and where you should work on improving your development practices.

There are a lot of "management tools" out there that will help you make sure you've covered everything, such as CMMI, Six Sigma, ITIL and so on, but I do not recommend you spend any time reading up on those tools until you've at least identified some common themes among your problem areas. Introducing a management tool where its not needed, has a way of quickly making things hellish.
TheDavid
Wednesday, March 05, 2008
 
 
Document the interfaces between the components (I assjme this is what you mean by "integration of componants"). Make them living documents that will evolve during development, but they should be documented in sufficient detail up front that the developers on both sides of the interface feel comfortable that they know what is going on and can see the weak points.
Anominal
Wednesday, March 05, 2008
 
 
Hello Pablo,

By "professionalizing" the development effort, I take that to mean controlling it, thus making the development, integration and testing effort goes more smoothly. Since you have already had your first release, what you need to do is document the various component's interfaces (APIs) and what those components do. You could use something as simple as MS Word or Excel documents since they are a defacto standard. Make them accessible to all and have the developers of the components enter the pertinent info themselves (list component I/O, individual I/O info, overall component functionality), thus saving you the effort of tracking it down. Doing this first step will allow you to know what you currently have and set the stage for what needs to be done next.

Good luck!
Mark Meyer Send private email
Wednesday, March 05, 2008
 
 
<quote>
Lots of the more agile-type methodologies scoff on deep documentation and say "the design is the code", which is only true to a point.
</quote>

As someone who has done virtually all agile development for the past 7 years, let me say I do not "scoff" at documentation. I only prefer to have has little as is necessary to get the job done. If that means 10 binders full, so be it.

I would say that none of us now know enough to be able to provide the OP with anything close to useful advise. Before I'd even dare to do that, I'd want to know more about "not professional" means.

From the sounds of it, I tend to infer that the individual components were developed without much pain but that the integration of these components wasn't as easy. Speculating even further, I would guess that this is due to the fact that integration took place near the end.

My "in the dark" advice would be to "front end" integration. Integrate as early as you possibly can, as soon as there is some sort of build of the components. The components alone are worthless so have the integration drive development.

The good news is, you don't need more documentation to do that. :)
Bruce
Wednesday, March 05, 2008
 
 
Good point. To me, professional and efficient are not synonyms.

If your interfaces are well designed and well documented, yes, you can assemble your components efficiently.

If you can assemble your components efficiently but you couldn't explain to anyone why you chose that particular method, you are unprofessional. Err...  that's an incendiary accusation so let me clarify.

Imagine getting a set of instructions that say press button 5A, flip switch 22, press buttons GER, ING, POT, SYN in order, toggle switch FLIR to the PRIME setting, wait two seconds, then toggle FLIR to the GO setting and press the green button simultaneously.

One of the first "stupid" questions someone will ask is why can't they just toggle FLIR to GO and press the green button? Why do they have to do all that extra stuff?

The solution in this case is not just a list of interfaces and APIs. I would also have liked to see a design document or a functional spec that at least explains why you're using the components you're using.

When Paolo said he had problems integrating all of these components at the last minute, one of the thoughts that occured to me was that maybe his people didn't know they needed these components until then.
TheDavid
Wednesday, March 05, 2008
 
 
"As someone who has done virtually all agile development for the past 7 years, let me say I do not "scoff" at documentation. I only prefer to have has little as is necessary to get the job done."

As much as you need, and no more. I'll agree to that.  Except of course, I find that to implement something requires less documentation than it takes to actually maintain something, particularly if you switch developers through the project, or switch project ownership.

I also make no claim that agile (or Agile) developers all scoff at documentation, just that the overwhelming impression I get from various agile development reading I've done (and practiced) errs on the side of too little, which I see as being riskier than having than too much.

Disclaimer: I do variations of agile development myself, but I like to have things clearly written down at a conceptual level.  I don't like to draw giant UML diagrams of classes, but I like to see components, draw an interaction diagram, write user stories, and describe what a component is supposed to do. 

I also only document library or object APIs ahead of time in a general sense unless forced, because APIs change daily during a development cycle.
Dan Fleet Send private email
Wednesday, March 05, 2008
 
 
+1 to Bruce for integrating early

Create interfaces and stub out components if they are going to take a while to complete.

Integrate often.  This will help if you have a version control tool that does automatic merges such as subversion or cvs.  Check-in your code at least once a day if possible, or once every couple of days at the most.  Update your code from the version control repository daily as well.

For a new developer with the necessary tools involved one such be able to get the head revision from the repository and compile and run the code without modifying anything except for maybe setting up some local config variables.
Ted
Thursday, March 06, 2008
 
 
Hi guys

Thanks for your valuable advices. Some confirm that we did some things right but other terrible bad. For instance, we didn't build mock ups of modules so integration took too long to start.

Regarding documentation, I'm just starting to adopt the agile methodologies formally, but it seams that we did it right just by intuition.

A couple of things I would like to explore more are
1. Continuous integration. Some guys in the project had insisted on that, but I'm afraid that with so many components, most of them still in a flush, it will be problematic to make frequent commits without screwing everything up.

I found and interesting page about software configuration  management patters (http://www.scmpatterns.com/book/pattern-summary.html) and I think that some of the proposed patterns, like having personal branches for developers, a branch for stabilization of current version, a branch(or branches) for next version and so forth, is a better approach. Have any of you experience doing this?

The other idea I would like to explore is using a blog for  the development team. I've used it but mostly as a personal log, sharing it rarely with others. I would propose to have a bog to keep track of what is going on with the development of diverse components.

Thanks again
Pablo Send private email
Thursday, March 06, 2008
 
 
<quote>
1. Continuous integration. Some guys in the project had insisted on that, but I'm afraid that with so many components, most of them still in a flush, it will be problematic to make frequent commits without screwing everything up.
</quote>

So, here's an observation that may or may not be correct.

Basically, you can't do team development as a bunch of isolated coders. Are you trying to do this?

Do not work on components. Work on features. Features will cross the integration boundaries and force you to face integration early. It will also force you to face your code synchronization and build issues, if you have any.
Bruce
Thursday, March 06, 2008
 
 
<quote>
Basically, you can't do team development as a bunch of isolated coders. Are you trying to do this?
</quaote>

Not really. Each component has a well integrated team, but those teams are rather isolated because they are at different  organizations and geographical locations.

However, there is a strong interaction among components, what makes integration testing complex.

I think that the idea of working on features, not components could be a good idea but would require a complete reorganization of the development across organizational boundaries, which I don't believe could be easy.

Pablo
Pablo Send private email
Friday, March 07, 2008
 
 
<quote>
1. Continuous integration. Some guys in the project had insisted on that, but I'm afraid that with so many components, most of them still in a flush, it will be problematic to make frequent commits without screwing everything up.
</quote>

_this: is exactly what you should fix early instead of waiting for the final disaster.
Daniel
Tuesday, March 11, 2008
 
 
Have a look at www.selenic.com/mercurial, the official web site of hg DVCS. You will find a wiki and an e-book with some  usage patterns that may help you (download the book and read it THREE !!!)

Since it is a Distribuited VCS (and not centralized as CVS) EVERY DEVELOPER has its own repository, you can merge changes from any repository in any moment.

The flow of changes may vary from commit-early-and-often (and break the build) to commit-only-when-feature-is-ok but since this is a PRIVATE repository it's up to the developer... then when changes are ready they can push to other repositories...

You may setup an "integration" repository, a "stabile" repository, a "test" repository... whatever you like, but remember that (for mental sanity) you have to PLAN your hg usage before starting creating repository and you have to obey to the rules you set....

hg also highly simplify the branch/merge stuff...
francesco Send private email
Wednesday, March 12, 2008
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz