The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Creating an XML "standard" for exchanging/sharing data

For a new product we are considering developing we would like to use an XML file to act as a means to share certain specific data that basically make up a template that the system will use.  We would like it so users can share these templates between themselves and possibly have 3rd parties also create these for their products that our app will help manage.

With that said we want to make something as "industry standard" as possible.

Does anyone have any tips or advice on what should be included for such an open standard XML file as far as structure, contents, etc or any advice on how best to create something such as this?
ML
Tuesday, June 28, 2005
 
 
Industry standards are a pain to create from whole cloth if you attempt to do it in a politically correct fashion and obtain input from the industry.  You wind up in committee meetings, arguing over which data must be included versus which does not, and in what format, and so on.

I would strongly suggest you first research existing file formats in the industry and see if there is anything already there that is a de facto standard.  If all you have is flat file formats, it's not horrible to create XML from that.

If there are a slew of formats around, and you want to try to make the "One True Format", you will have to hash out field sizes, dependencies, and worst of all, encodings.

You could just try to take boatloads of sample data from a wide range of sources and create a single database from it.  Many database systems can then export a viable XML schema from that.  Use that as a starting point, but you may have to tweak it to make it more friendly.

Incidentally, yes, been there, done that.  Pharmaceutical data is nuts.
Aaron F Stanton Send private email
Tuesday, June 28, 2005
 
 
Oh, regarding the PITA bit about committees:  It's really really slow, and you don't always get the buy-in you thought you would.  It might be easier to just do it yourself and then shop it to other companies - show them betas of it and ask them for advice.
Aaron F Stanton Send private email
Tuesday, June 28, 2005
 
 
Not really wanting to take it to the committee level.  (I was once on an EDI committee and don't want to go there again) Basically no standard currently exists for what we are developing so what we would like to do is try to create something that "might" be accepted.  If it is not no big loss but we just want to try and make something fairly usable and open in the hopes others might make use of it as well.

Really just looking for info on what makes a good XML standard, ie things that it should have over and above say a format that would only be used by your app exclusively and never used by "outsiders".
ML
Tuesday, June 28, 2005
 
 
Well, other than paying more attention to naming conventions (trying to make things as semantically descriptive as possible, so that the XML is as human parsable as possible) and giving extra thought to redundancy, use of attributes vs. properties (always contentious, but if you can get what seems to be a logical divide...) that's about it I would say for the actual format.

Where I think your make/break point is will be the documentation/API notes. If you document the format well, then it will be less effort to use your format than to go off and roll your own. On the flip side, if you don't provide good docs and specs, people will just come up with their own representation.

So make the barriers to using your format, especially mental/learning barriers, as small as possible, and hope that gravity does the rest...
Andrew Cherry Send private email
Tuesday, June 28, 2005
 
 
Another thing is you might want to put some effort into creating some XSL for it, too.  You may find that what looked like a reasonable XML format requires horrible XSL to massage it into reasonable output.  If you do both hand in hand you might be able to get an XML format that is useful both for your app and for XSL.
Aaron F Stanton Send private email
Tuesday, June 28, 2005
 
 
Oh, yes I second the bit on documenting it.  Include tons of sample data (nothing critical to your business, obviously, but reasonable fake data) and ways to get the data in and out of the format.  Also, you can demo the XSL I mentioned to show how reasonable data can be prettied up.
Aaron F Stanton Send private email
Tuesday, June 28, 2005
 
 
I wish someone would/could create a universal standard for names and addresses of customers. In my little corner of the world, everyone has their own and we burn mountains of time mapping between them all. In the simple case of John Doe it is not that tricky, the complications is when its John and Jane Doe, or John and Jane in trust for Billy, etc. etc. Then you get into the need (in our industry anyway) to have multiple addresses for vacation addresses, copies of stuff to accountants, etc. etc. Blech.
NetFreak Send private email
Tuesday, June 28, 2005
 
 
I wrote some of the XML "industry standards" for metadata at the Library of Congress.  (ie.  My name is in them first.)

First, research to see what's out there, why people are/aren't using it.

Next, try to fix those things and/or keep the good things.

Next, get a v0.1 out there.  Send it to a few people in the industry who you trust.  DON'T send it to everyone.

Once you get these people's feedback, draft a v0.2  Repeat this cycle a few times until there are just style issues left... make sure the commentors have bought it.

THEN make the thing public and LOUDLY thank your commentors for their VALUABLE INPUT and INSIGHT.

If you do all of this well, they will be your champions.
KC Send private email
Tuesday, June 28, 2005
 
 
If you want third parties involved, then providing a validator somewhere would be good, either online or via a trial version.  Make your standard the path of least resistance for third party developers and you might end up with the defacto standard.

Wednesday, June 29, 2005
 
 
Who cares about XML? It is a means, not an end in itself.
Keep it simple (low threshold). XML might help there because there is huge tool support and middelware around it. For the rest, provide value. What would a third party gain by using your schema?

It's all cost/benefit. Make the balance positive from the "others" pov.
Just me (Sir to you) Send private email
Wednesday, June 29, 2005
 
 
One big problem you'll run into is the Not Invented Here syndrome.  One of three things will happen - If you are dealing with an established company, they will already have their own proprietary data format, and even if you can demonstrate that your format is better, they still have to deal with the cost of changing their existing code base to use it.  Second, if the company is new, there is a good chance that some hotshot cowboy coder is going to think he can do a better job than you and code it himself anyway.  Third, there's a really good chance that unless you market the hell out of it, companies won't know about it and by the time a small company is a big company and you're looking at case number one.

After a year and a half of sitting on a committee to standardize pharma lab data formats, I actually thought to ask how much buy in the format was going to get from the companies participating.  Basically all the large companies, the ones swinging the biggest sticks and making the roughest demands of the format, all said they weren't going to switch to it because of the cost to alter their preexisting code base.  It really made me wonder why we were listening to them, since they lacked the commitment to use it themselves.  Sure, their experience was valuable, since they could present many cases that small companies had never encountered but eventually would, but still...
Aaron F Stanton Send private email
Wednesday, June 29, 2005
 
 
One place I used to work had a VP on an industry board that was standardizing interchange formats.

This VP flatly stated to us that his role on the committee was to actively obstruct progress so as to maintain their vendor lockin. Non-functional interchange meant that nobody could get reasonable data out of the system.

I always felt kinda slimy going into that place for work.
Chris Tavares Send private email
Wednesday, June 29, 2005
 
 
I remember when I was working at Avaya, Cisco was participating in a standards commitee for the provision of "phantom power" for IP telephones.

After the standards had been settled and the other companies began production Cisco decided it would be "better" to do use different pins.

Since Cisco is the leading company in network switches it was able to use the standards process to delay the ability of their competitors to produce a phantom power IP phone.

So in summary, in any industry: standards help the little guys and big companies will protect their position. It's normal.

-Andrew
Team Bob Send private email
Thursday, June 30, 2005
 
 
Appart from all the politics, make liberal use of "any" and "anyattribute" to make your schema as "forward compatible" as possible, but be aware of the "simplicity" tradeoff, so weigh the pro/cons for your particular market.
Just me (Sir to you) Send private email
Thursday, June 30, 2005
 
 
You may want to look at RDF: http://www.w3.org/RDF/ . The XML syntax for it isn't pretty, but you're not limited to XML, there's a wide selection of implementations that do the XML bits for you, and it's more suited towards expressing semantics of the elements. 3rd parties can integrate their RDF vocabularies with yours in standardized ways in which there is no standard for a plain XML+namespaces approach (e.g., using RDF Schema to express relationships). An example of elements that can be used as RDF to describe resources in general can be found at http://dublincore.org/documents/dcmi-terms/#H2 .
Mark Cidade Send private email
Thursday, June 30, 2005
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz