The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

From Idea to Implementation

Okay, this is kind of a general open ended question intended to spark some discussion in the hope that we can come up with better ideas than we have now, but I don't hold out hope that we will discover _*the*_ Idea.  I have been kind of wrestling with it for a couple of weeks now.  How can we (the human race, programmers) get from idea to implementation faster?

Some things I have considererd:

- Other programming paradigms  (OOP, Functional)
- Higher level abstraction languages (Common Lisp influence here)
- Larger, more complete, more bug free and well documented libraries
- Code Generation
- More advanced pattern recognition (see the patterns in large scale programs, as opposed to patterns in objects or methods).  We can use these patterns to produce an entire application for a specific market sector.
- ??

What am I missing?
Joshua Volz Send private email
Sunday, July 31, 2005
> What am I missing?

A solution that will get you there :-)

Everything you describe is an incremental improvement at best. It's only when we have a mechanical means intelligent enough to create software that we'll see real productivity improvements.
son of parnas
Sunday, July 31, 2005
I agree that my suggestions above are only incremental improvements.  However, my goal is only to be able to produce a working program (possibly that I can later refactor into a beautiful and easy to alter program) in the shortest amount of time possible.  In my mind, this means in a work week.  I think if we got it to that point, I would say we have accomplished our goal of moving from idea to implementation faster. 

Implementation includes, in my mind, everything that is required to have a functioning business based on it.  That means if you wrote winfoms software, you need an installer.  You probably need a website, w/ some mechanism for purchasing your service/product.  An implementation is big enough to do something useful, but small enough that we can get it done quickly (in my case, before I get bored or distracted - Yes, ADD is a problem). 

We need to stop rewriting things that already function too.  That would be useful in terms of getting things done quickly. 

I am currently employing code reuse (OOP, outside component purchases, existing libraries, code examples on the internet, etc.) and code generation to get things done quickly.  I am also trying out the concept of writing a trash version that works, and then refactoring it.  Mostly my time improvements have come from realizing that while I could write everything myself from nothing, it is far faster to let someone or something else do it.
Joshua Volz Send private email
Sunday, July 31, 2005
If you're willing to forego compatibility at first, you might try Apple stuff - Cocoa, Quarts, Webcore, Audio Core, Video Core, and Data Core might help. I am by no means an Apple advocate - hell, I don't even own an Apple, but it looks like they've got a lot of useful stuff out of the box.
Ben Atkin Send private email
Sunday, July 31, 2005
Oops, I meant Quartz. My spelling is horrible today.
Ben Atkin Send private email
Sunday, July 31, 2005

> What am I missing?

Basically, you are missing the whole point.

No matter how fast you can produce a piece of rocket science software, if it doesn't solve a customer problem; what's the point?

Your goal is pointless, unless it serves a greater purpose: to satisfy your customers.
Hakan Send private email
Sunday, July 31, 2005
While I might not have been clear about it before, I already have customers who already have needs.  I want to see if we can together find ways to improve the speed with which we can fill those needs.  Sure, before I said "idea" which could have been interpreted as just something I thought up devoid of other human contact.  I _meant_ once I already have the "customer need" I want to get to the customer fulfillment quicker. 

Let's even say that it was just my own idea, if I could do it very quickly, it might be worth doing it just to test the market.  I believe this is called rapid prototyping in some development plans.  If producing a prototype only takes you a week, why not try out an idea?
Joshua Volz Send private email
Monday, August 01, 2005
There are many areas where politics and finance killed technological improvements. Therefore there seems to be a rich history of ideas to mine in all sorts of areas, if they could only be found. One that I know of is the book "Interactive Programming Environments," a collection of papers from authors of various programmer's apprentices and whatnot. (Tellingly, the only things which survived are Kernighan/Mashey on Unix and Stallman on Emacs -- both things that could be cheaply and easily obtained, and they were both relatively primitive.)

The (at best) 2-d nature of code is a problem. IDEs are usually enablers of the old text interface, as opposed to "structured editing" like Interlisp (and it's current incarnation, Simonyi's Intentional Programming), which allowed you to operate on units of sourcecode rather than text.

One thing I like is to grab kind of generic code from somewhere, and shape it into what I want. I find that lowers any demands on my memory or reference books. Perhaps a paradigm could be created around this.

Lisp lets you manipulate sourcecode easily -- why not an IDE that lets you manipulate itself easily? It could conform to your problemspace. Or a primitive low-level coding example: there are certainly things that programmers visualize that are essentially invisible, like function call stacks. This could be taken out of the sole realm of debuggers and error stacktraces.

Someone mentioned the interesting way Minsky programmed. He'd write an expression and executed it, which immediately threw an error because some function was missing. So he'd write that function, execute the code again, and again fill in whatever was missing when the environment complained. That sounds like a nice interactive way to program that might receive special support.
Tayssir John Gabbour Send private email
Monday, August 01, 2005

The initial list you produced focuses on the technical aspects of what can be changed. To dramatically shorten development time I suspect you need to look at it from a management point of view.

Add more people (provided it is managed properly)

Adding the right people.

Adding the right people and managing the job so they are doing what they are good at.

But, no matter what you do to shorten timescales, the cost is going to go up. (Unless you are prepared for quality to drop).

The most productive environments I have ever worked in included a Documentation team. An Engineer would product the initial version of the document, and hand it over to the doc team where they would tidy diagrams, sort out the contents/index and organise the review and necessary updates. To them, it didn’t make sense to have an expensive engineer wasting time wrestling with Word trying to line up headers when they could be doing something else more productive. Unfortunately everywhere else I have worked since, developer productivity and efficiency is an alien concept.
Ian H. Send private email
Monday, August 01, 2005
Suggestion summary:

Tayssir -

* Produce an IDE that is better than 2D, and that works with units of code instead of units of text. 
* Develop interesting ways to order coding so that it reveals the most information. 
* Grab code from somewhere and mold it to what you want.

Ian H / Hakan -

* The business stuff is what takes the most time, focus on improving the productivity there. 

These are all very good ideas, some of which I have tried, some of which I am in the processing of trying.  It does seem that non-core programming is on par with all the other things that go on in order to bring a product to market, in terms of time required.  While I had originally starting this discussion with the notion of attempting to improve the core development time, I do see that another place where lots of time improvement can be made is with automation or parallel processing of the business stuff.  Have graphics and the website ready when the core product is done.  Have the installer ready when the core winforms product is done.  Have the production web server installed and tested when the webforms product is done. 

Since the goal is to produce something cool and make money with it, we should give equal time to the making money step.  That's my personal yard stick for success of a product, and I don't think I am alone in that notion. 

I am not sure if we need another post to discussion the business stuff.  I think I will just make one.  It will be in Business of Software under "From Idea to Implementation to Sales".  Link to be posted soon.
Joshua Volz Send private email
Monday, August 01, 2005
I meant _core_ programming is on par with the other things in terms of time consumption. 

I started an area for us to dicuss the ways we can improve the business portions of software production.

Have at thee!
Joshua Volz Send private email
Monday, August 01, 2005
Fastest way to speed up development:

Reduce features.

The first time I saw that, it floored me.  It's sooo simple, yet so seldom done.

Pick the low hanging fruit.

This solves another problem : How do I get my users to upgrae to the new version?
Mr. Analogy {uISV} Send private email
Monday, August 01, 2005
Read Fred Brooks "Mythical Man-Month" -- preferably the 1996 version.  First written in 1976, updated in 1986, and then with a commentary added in 1996, Fred summarizes the state of the art from 1976 to 1996.

All of the issues you raise were raised in that book -- scary, isn't it?

The maximum leverage on this problem seems to be implementing some pictoral version of a program, which can be "automatically" converted into code.  Note this has been a "Holy Grail" of software development since they got graphics screens.

Current approach is trying to leverage UML for this.  Personally, I think the current UML is too vague and ambiguous for this purpose.

Ultimately, I think SOME graphic standard to code generation will evolve.  But we're not there yet.
Tuesday, August 02, 2005
"The maximum leverage on this problem seems to be implementing some pictoral version of a program, which can be "automatically" converted into code"

That does seem like the most likely solution, but I doubt it'll happen for 30 years.

The construction business works that way:

1.  Everything is drawn pictorially (blueprints).
2. Standard components are reused (bricks, 2x4s, etc.)

But the real world is awfully inconsistent.  The shape of each problem is different, just as the shape of each yard is different.

Hey, I own my software company. I'm all for making programming ridiculously easy, but I don't see it happening, as Fred B pointed out.
Mr. Analogy {uISV} Send private email
Tuesday, August 02, 2005
Mr. Analogy:

> That does seem like the most likely solution,
> but I doubt it'll happen for 30 years.

Be careful about your predictions :)

Five years later someone might quote you and put on this list:

My favorite is:

"I see little commercial potential for the Internet for at least ten years".

1994, Bill Gates

Cheers ;)
Hakan Send private email
Tuesday, August 02, 2005
Yeah, I've wondered why the programming world hasn't advanced for decades (unless you like type theory...) I think the reason is that the current model is of evil geniuses having some labs where a couple of them might come up with a holy grail of UIs or methodologies, and revolutionize the world.

So these academics are the real developers, and normal programmers just become users.

The problem is, this model should likely be a little more like giving normal programmers powerful tools that let them easily program their own programming environments, and that would drive innovation. Eventually people would see what they have in common and that's where standardization would come in; or academics could pore over the patterns they see, to make advances.
Tayssir John Gabbour Send private email
Wednesday, August 03, 2005
2D and 3D modelling that is automatically converted to code kind of assumes that objects are used, or am I missing the point?  I would think it would be hard to represent a function accurately as an object.  I suppose they will come up with something. 

I am not necessarily trying to make programming easy, I think that conceiving of a program, and then conceiving of how to accomplish that program's implementation will probably not be *easy*.  I am saying, once you have the idea and the design, how can we best get that design implemented. 

I agree with the last comment about letting programmers program their working environments.  The problem is that that already exists.  It's called Lisp, and it was invented in 1958.  Not very many people use it. 

Good discussion, let's keep things moving along.....
Joshua Volz Send private email
Wednesday, August 03, 2005
I gave it some further thought (I am waiting for my copy of Mythical Man Month) and I came up with three general ways through which we can solve problems quicker.  I am not saying these are the only ways.  What I came up with:

-Reductionism:  Reduce what we are doing.  Cut out features.  The best form of this is cutting out bugs.  I know we would all love to write an entire program and compile it once, and have that work, and then have no logic bugs when we run every feature in there.  Imagine the time savings!  I am not sure that is possible.  Some people like to remove documentation from the plan in order to accomplish reductionism.  With tools I think we could work towards that asymptotic minimum of time used.  I do think that by using careful strategy in our programming time and in our business decisions, we can do a lot to improve the time from idea to implementation using this methodology.
-Parallelism:  Get someone else to help us.  This is the point of hiring other people.  Yes, we have discussed that hiring others increases task switching time and communicate time, therefore lessening the effect of the 2nd person, but I think it is still a useful tool when used properly.  Don’t become the next IBM as a MicroISV, but maybe consider having someone sell the product while you are writing the product and bug fixing. 
-Memoization:  Do work, and then never do it again because you have (or remember) the results.  This often comes in the form of building libraries for MicroISVs.  Do you really need to implement email functionality more than once?  This has the effect of making you very good at a particular type of product.  If you already have all or most of the functionality for that type of product built, you can more easily put out something usable.  I would not understate the effect of having already debugged libraries from which to build your new application (bottom up programming in disguise?). 

Each of these methodologies I don’t consider to be a great boon to MicroISV productivity.  Used in combination, I think they could lead to a good decrease in the time from idea to implementation.  If you can beat your competitor by even 10% in terms of time, that is 10% more time you have to sell your product.  I think also, that such an advantage would snowball into a larger and larger advantage.  If you can always produce the same feature (idea) in 10% less time then you are eventually going to have 10% more (or better) features than your competitor. 

The question:  what general problem solving strategies am I missing?  What assumptions am I making that are not necessarily true?  If we figure this out, I think I can safely say that I will take everyone out to ice cream.
Joshua Volz Send private email
Saturday, August 06, 2005

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz