The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

A new web storage design?

Background: One of the problems with all of these online services that store data for you like online e-mail, spreadsheets, calendars and photo albums is that you have to trust them to store it safely. Every time you join one, the risk increases that one of the online services you use will be compromised. Amazon S3 storage webservice is a way for a website to use a 3rd party to provide and guarantee storage, but it is still managed by the website, not the user, so if their website was compromised the malicious entity might still (though perhaps less likely) access the data of all their custtomers.

So here is the question: does it exist or is there discussion of a situation where you as the user would have your own online storage account which you could provide to websites such that they would gain access to a segment of your online storage (may be this is the vision of Amazon S3 and I just don't know it). I am not saying they would gain access to any of your other storage; they would have their own segmented area to use. You would still be trusting that they would not store any of your data on their own server but just temporarily pass it through their application while you were using their website. But the point would be that they would have no reason to store it and then there would be less risk that any malicious party would get access to your data if their site was compromised. They would put the specific version of your storage segment credentials in a cookie on your browser rather than on their server, so they would not even jeopardize a table of customer credentials (albeit segment specific) on their server.

This would also mean that you as a user could effectively completely unregister from a website by deleting that website's segment from your storage service. A great benefit to you of having a single secure and reliable place to manage all of your online presence. Each website might use encryption and proprietary formats to store their data there, so it does not mean you could otherwise inspect it, but at least you could manage it as a whole.
Ben Bryant
Thursday, January 18, 2007
Yes, this has been discussed before, and the two arguments against it are firstly, there's nothing preventing you from sending the site contaminated data, and secondly, there's no real financial incentive for them to do so.

Let's say I had my own online storage unit, and I've asked to save my wishlist, my credit card information, my purchasing history in my unit instead of theirs. I could conceivably hack into my unit and alter the details to say that I never received a shipment or that Amazon owns me credit. By moving the data to their servers, they can take some precautions against me altering it.

With respect to the second argument, say we did come up with a mutual encryption scheme. My service goes down. I attempt to buy a book. Amazon can't access my info. They just lost a sale, plus we might have to do the customer support dance figuring out what went wrong. It's actually cheaper for Amazon to maintain the data themselves instead of building a support network to handle whatever problems we introduce.

Yes, we'd like more control over our data, but it's like going to a restaurant and paying for a meal with a credit card - you've got to trust they won't make a copy of the card.
Thursday, January 18, 2007
TheDavid, thanks! If anyone has links or keywords for finding those other discussions let me know!

>I could conceivably hack into my unit and alter the details to say that I never received a shipment or that Amazon owns me credit.

Well, Amazon would not store any data there that would be considered its own such as status of shipments or balances. But definitely they would use it for your profile which does mean they would have extra work to validate it every time they accessed it. I am thinking too of sites that deal more exclusively in your personal data like your documents and photos. Some of which could make it a feature of their solution that you could gain independent access to your files.

You're right it is in the website's interest to control and store your info, and there is no direct incentive to use this model. But this would be such a compelling storage model for users that they would be happier about joining websites that supported it and once there were enough adopters the tide would turn so that all websites would be expected to support it. It would have to start off as optional for some types of websites, so it would be a lot of work for websites, but if some of the big ones bit the bullet, and support was added to common web development toolkits, it could happen.

> it's like going to a restaurant and paying for a meal with a credit card - you've got to trust they won't make a copy of the card.

Actually that is a good example of the same model at work. The credit card company, like the storage webservice is a 3rd party that takes responsibility for your information. The store does not have an incentive to copy your credit card number because then they would be liable. Yes there can be bad stores and there can be bad or careless websites, but I don't think this model lets them do you more harm than they could otherwise do you.
Ben Bryant
Thursday, January 18, 2007
That is kind of a neat idea, though, thanks for sharing it.

Not directly related but in my world (stocks & bonds) it is mind-boggling how much data is redundant/repeated amongst a zillion different systems (ie. within the same brokerage firm) with all the inevitable inconsistencies and timing hassles, not to mention the megadollars spent interfacing them all, usually in suboptimal ways. I bet most industries are the same.

I have always thought it would be awesome if someone could come up with the be-all-and-end-all storage solution, and then all the ISV's could plug into it.

Then again it would also be cool if there were peace in the world and no hunger... also not likely to happen.
Greg Send private email
Thursday, January 18, 2007
A dream, yes, but I'm thinking 10 years from now, not immediate. Online storage is just starting to become a big thing, most people still have a lot on the PC, but with e-mail and photos, more handheld devices to access your data from, plus file sharing and other apps looking to get in on it a lot more is getting stored online. With this will eventually come more webservice APIs, like Amazon's, and with that more pressure to let the user manage his online information via that mechanism rather than having it spread everywhere with every mom and pop online company.

And yes, the thing I like about this idea is that even if it starts off as a completely segmented thing where each application only looks in its own area, it would be easy to create segments which multiple applications could be given access to, e.g. your personal address info so you don't have to type it in again. And by not holding onto the credentials, the web applications do not risk exposing all their customers.
Ben Bryant
Thursday, January 18, 2007
And not just mom and pops, I just saw this:

"T.J. Maxx and Marshalls chains, said on Wednesday that the computer systems that process its customer transactions had been breached and customer information had been stolen.",1397,2085267,00.asp
Ben Bryant
Thursday, January 18, 2007
I like this idea, except instead of storing the data on a remote server, I want it stored on my local drive (maybe a USB drive).  Then I KNOW where the data is, can back it up or take it with me, etc, etc.  Almost a Microsoft Wallet idea (I think).
Friday, January 19, 2007
I think the more useful concept is being able to control who has permission to access your data, regardless of where it is. I'd like to see something like public key cryptology used where retains the data, but they can't access it without my private key.

Subsequently, if someone broke into or they sell their data, I still get "notified" in some fashion that they're trying to read it and ideally, I can decline to provide my key. In the T.J.Maxx example, I know I've provided info to T.J.Maxx, and I'm ok with them using it, but if suddenly asks me to verify the same stolen info, I can block them from using it.

Unfortunately, I don't think there's a way to prevent them from caching my key, or making a local copy of the decrypted data?
Friday, January 19, 2007
What if Amazon failed? (either for technological or business reasons). If ALL your data (aka your customer's data) is outsourced and one day the Amazon lights go off, your entire business is toast. This is one of the downsides of web-based apps (compared to desktop). ALL your customers would be pissed/going elsewhere, all on the exact same day.

Amazon can't go under? (Worldcom and Enron looked good pre-death, so it's not always easy to spot!) I'm also skeptical of their business model for S3. How are they able to charge so little that other websites are selling off servers and moving to S3 ??? I'm not sure it's sustainable.

Having said all that, I think it's a decent idea. Maybe someone can write an API on top of amazon S3 that would make this really easy to do (who knows - maybe Amazon will ???)
Warren Stevens Send private email
Monday, January 22, 2007
Amazon failing is a legitimate concern but really a problem for the online storage market to solve since their goal is to make you feel secure about your data. But your point assumes that all the companies eggs and yours are in one online storage basket. I assume there will be lots of players in this market before websites start hooking into them. It would even be possible to use your own storage at home (like was mentioned above) if you exposed the same storage webservice API as the big name providers using some off-the-shelf tool. So if the API is standardized you would have all sorts of places to implement your online-accessible storage.

I think that the concept of allowing the end-user to control his online storage may be compelling enough that as online storage grows in usage, this scenario could happen. But it will take some adjustments to work the kinks out of it -- a lot of good points have been made here about the problems with it.
Ben Bryant
Monday, January 22, 2007

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz