The Joel on Software Discussion Group (CLOSED)

A place to discuss Joel on Software. Now closed.

This community works best when people use their real names. Please register for a free account.

Other Groups:
Joel on Software
Business of Software
Design of Software (CLOSED)
.NET Questions (CLOSED)
Fog Creek Copilot

The Old Forum

Your hosts:
Albert D. Kallal
Li-Fan Chen
Stephen Jones

Web “Security” Question

Somehow, I’ve only recently become involved in making things available over the web, so sorry if this question is naïve.

Suppose you have “” that is meant for public consumption, and you do whatever you can to make it available to interested parties and search engines.  You also add, etc - and there could be many instances of somewhatprivatestuffX that link to each other, but are not linked from the main site.

How secure is somwhatprivatestuffX?  I’m guessing it would not appear on search engines – but do people do some sort of dictionary attack to find this stuff?

And if an external site links to somewhatprivatestuffX, I guess everything that was linked internally would eventually become available to the search engines.

Is all this right, and what would you do if this were to happen?  Protecting somewhatprivatestuffX with passwords would help – I think – but I would prefer not to do that.
Somewhat Interested in Security
Monday, October 11, 2004
If you don't protect it with passwords, then assume someone will find it, because they will.
Brad Wilson Send private email
Monday, October 11, 2004
"Security through obscurity is no security at all"

Put it this way - would you hide $10,000 under a bush in your back yard because nobody would ever look there?

Philo [MSFT] Send private email
Monday, October 11, 2004
Thanks for the responses - I'm new to this.

But - How would they find it? 

What is the difference between finding “” and “” plus a password?  If there isn't much difference, and if we make X difficult to dictionary (and do people do that?), what’s the problem?

This stuff is not hyper-sensitive, but if it were to appear on Google it would be a problem.

Requiring passwords would be tough on this one.  I'm asking - how would anyone find it?
Somewhat Interested in Security
Monday, October 11, 2004
Dictionary attacks are virtually unheard of against web sites - about the closest technique is hackers looking for well known paths (i.e. the IIS hacks generally try for dozens of well known IIS paths). Presumably in the scenario you're envisioning directory browsing won't be allowed, and you're not going to name the directory in a way that might collide with a known exploit.

Having said that, the security of such a design is limited because URLs traditionally are not intended to be secure - People don't treat URLs as secure and often freely distribute them, and additionally your browser happily passes the URL on as well (i.e. you add a link to some site on that page, give the secret page link to your secret associate, and they click on the link. Their browser happily reports to the linked site that it came from /yoursecretpageXXX/page.html).

Just to go back to a common quote seen above:

"Security through obscurity is no security at all"

I definitely agree with the gist of this -- the risk in such a security technique is that it distracts you from doing the right thing, instead opting for the easy solution of obscurity. However, having an unlinked path is definitely a form of weak security - it's basically like a 1-part authentication (where the subdirectory is a "password" of sorts). Couple it with some credible security like username/password and you're a little further ahead of username/password alone.
Dennis Forbes Send private email
Monday, October 11, 2004
"I'm asking - how would anyone find it?"

-You link somewhere from the site, and for whatever reason the linked site publishes a "where people came from" page that links back to your site. Even if they don't any site admin looking at the logs sees the URLs.

-You have directory browsing on which allows people to basically enumerate the available directories.

-A user trusted with the URL posts it in a newsgroup, or in an email to a friend, or whatever, and it spreads from there.

-You forget to configure functionality of your site, for instance search, so it is returning hits against the secret page

-You cleverly decide to stop spiders from archiving it by adding an explit exclusion in your robots file :-) 

Disallow: /MySuperSecretDirectoryStayAway
Dennis Forbes Send private email
Monday, October 11, 2004
setting up a password with basic authentication is often quite simple. not particlarly secure, but no spiders (should) go through it, as they're not interested in hacking in to your system. it's basically a door locked with a flimsy lot: to get in, you've got to break the lock, which indicates malicious intent.
mb Send private email
Tuesday, October 12, 2004
I can see two probelms with the tentative design, but I can't assess their releveance, since I don't know the context:

1) URL history: If users access the site from someone elses, or a public computer, the URL will reside in the URL history of the browser used. Passwords will not be exploitable this way.

2) Sniffing: Any network sniffer will easily pick up both URLs, passwords and content if the connection is not encrypted.

Unless you have (really) good reasons not to, I stronlgy advise to use both SSL/TLS and password protection. As a previous poster said: it's not hard at all.

Henrik Sidebäck Send private email
Tuesday, October 12, 2004
Once again Philo states it the most simply.  [Philo, welcome back.]

Even if YOU don't link to it, all it takes is one of your users posting a link somewhere and the GoogleBots will find it.  Then, if people find it interesting, others will link to is and then you start moving up in the rankings...
KC Send private email
Tuesday, October 12, 2004
While I'm glad that Philo is back, he didn't "state it most simply" (unless by that you mean "vague unsubstantiated statements"). In fact the "security through obscurity is no security at all" wisdom is oft misused in the same way that "a foolish consistency is the hobgoblin of little minds" is misquoted.

To draw from Philo's specific example, the $10,000 hidden in the back yard, let's pretend that you're in organized crime and you can't put your money in the bank (damn FBI!), and you're too lazy to head to the store and buy a vault (or maybe it's a small enough amount of money that you can tolerate a loss). So what is more secure - putting the $10,000 in a box marked "money box", or hiding it under a large rock in your garden? Obviously the obscurity of hiding it under a rock is improved over putting it in an obvious location. Even if you tell your, err, associates that the money where the money is hidden (and they might have loose lips and tell others), the fact remains that you're still in a better position than simply putting it in an obvious location. It's obvious that the OP should research better security, but saying that their approach is no security at all is inane.
Tuesday, October 12, 2004
Jones, to make the analogy complete I think we'd need to add that there is a neighbor who has been known to regularly look into the garden to see what is going on.

When blogs have trackback links on their pages, this is the equivalent, no?
Scot Send private email
Tuesday, October 12, 2004
No you had a couple of associates stand around with bats while you planted it. It is impossible for a neighbour to see where you hid it.

A better analogy would be that your associates then write the location down to help them remember, but leave their directions carelessly in public, or maybe they wrote it on a hotel paper pad and it indented all of the pages below with the directions. In either case data carelessness (though oversight or ignorance) reveals the secret.
Tuesday, October 12, 2004
Jones is right on the security issue.

But oh my gosh, if you don't want only some people to access data (covers everything) then do the following:

1) require valid-user who connects SSL.
2) only return the data over an SSL connection.
3) make sure your box is secure so people don't find other ways to get the data.

There are people (dumb people) who will just try urls at websites. Smart people will have spiders do that for them ... Also, even smarter people will just filter network traffic. That would be interesting. Where would you setup apps to capture network traffic ... I'm not that smart but I've done it on my local lan ... if you can listen to traffic on your local lan don't you think there are people who are listening to the big LANS ... interesting thing is that a lot of companies think about the above 2 and think that because they have their boxes behind some corporate firewall that all is safe yet there is an awful lot of traffic still going on with sensitive data such as through email, ftp, rsync, etc. (how many people encrypt their email ... rsync can be done through ssh but how many do) ... anyhow, trust nothing, always be diligent.
Tuesday, October 12, 2004
Jones, to clarify -
"Security through obscurity is no security at all" - simply relying on "oh they'll never find this" IS NOT SECURITY. Not by any stretch of the imagination. As several others have posted, the problem is that either through effort or accident, someone will most likely find it. That's the logic behind changing passwords every so often - because you should assume they will become compromised.

My analogy requires no modification - if you had $10,000 in cash, would you put it under a bush in your back yard? I believe the average rational person would answer an emphatic "No way!" But why? Have you ever had a trespasser in your back yard? (Probably not) If you had, what were the odds they went digging through your bushes (slim and none) Etc. And yet, we *know* that simply putting that money outside with no other protections is foolhardy because if someone DOES find it, by whatever means, it's gone. The money has distinct value, so we put a little more effort into buying a safe, or putting it in the bank, or at least hiding it inside the house where there's a lock on the front door and we're always aware of visitors.

If the data on the web pages are of value, then they are worth the effort to apply a little security. If not, then why make the effort to ask the question?

Philo [MSFT] Send private email
Wednesday, October 13, 2004
--simply relying on "oh they'll never find this" IS NOT SECURITY--

But it _IS_ security, albeit weak security, and that's my, and others', point. Waving your hand and saying that it isn't security is quite simply incorrect. Could someone find it? Of course they could, but it's better than having it in the free and open.

Regarding the money, simply saying "it's money therefore it should be in the bank" is again a flawed analogy. $10,000 is a fair amount of money, and you might go to lengths to secure it, but not all money is $10,000 worth. I have a pile of change sitting in my desk drawer here at work. I have done virtually nothing beyond the superficial (putting it in my drawer where it is out of sight) to protect it. Would I do this with $10,000? Of course I wouldn't. In the case of the OP's question they have to decide if they have some change, or $10,000 worth, because simply saying that everything is $10,000 is absurd.

If you truly believe that a large part of society relies upon security through obscurity, usually coupled with real security, to great effect, I encourage you to go to your local bank and take some pictures, ask some questions about what kind of vault they use, and ask for the blueprints for the building and questions about the networking method that their security system uses.
Wednesday, October 13, 2004
To give another example of why I find that mantra irritating, some time back I worked on a system where we had to expose SQL Server on the Internet (long story...). Despite following all best practices and keeping up to date on every patch, we _additionally_ decided to put it on a non-standard port as well (because we controlled both ends). Invariably this led to inapplicable rote repetition of "Security through obscurity is no security at all!" as if it were some sort of mantra. It was as if the anti-obscurity folks believe that obscurity actually undermines additional security methods. Of course any rational observation proves that to be absurd, and if slammer were a day-0 worm that came out before the MS Patch, we would have been safe (worms don't bother nmapping, nor do casual hackers. Real hackers do but it makes it very easy to detect them and close them off -- just like lots of security relies upon an additional layer of obscurity to allow the possibility of catching people trying to look through the obscurity).
Wednesday, October 13, 2004
Jeez, pay 10 bucks for another domain name like

and have that heavily passworded.

Alternatively, have

and that goes to a different server.
Wednesday, October 13, 2004
Domain names aren't secret - by definition they're in the public databases, which you can procur by sending in a signed letter. Subdomains are a little more secure but then many people have their DNS configured to do a zone dump to anyone who wants it.
Wednesday, October 13, 2004
"To give another example of why I find that mantra irritating"

You find the mantra irritating because it's not directed at you. YOU understand threat modeling and security practices. The mantra is directed at people who ask "is hiding my front door key under my front mat enough to make my house pretty secure?"

Yes, the house is MORE secure than if you simply left the door unlocked, but not very much more. The mantra is to convince people who ask those kinds of questions that no, they really haven't added much security to their system at all - there are still a LOT of vulnerabilities, and they should probably work on understanding them.

Simple mantras are for beginners who don't even understand the rules, much less how or when to break them. "A silly consistency is the hobgoblin of small minds" is a mantra that you can feel free to break when you understand what it's saying.

But when you can justify it, it's not really a *silly* consistency any more, is it?

As for the change on your desk - I'm assuming we're not talking about $10,000 in change, are we? If not, then you've changed the conditions of the problem. My point was about $10,000 (which is why I said "$10,000" and not "some money"). Nobody ever asks "do you think this $1.25 in change will be safe in my desk?" because the amount of money really isn't worth the breath to ask. It's when you start expending energy to ask if a safeguard is sufficient to protect your data that I've got to start wondering aloud if you shouldn't be putting that effort into some more security instead of just hoping. :-)

Philo [MSFT] Send private email
Wednesday, October 13, 2004
If you explicity disallow spiders to access to the path (via robots.txt) everyone will be able to know that the path exists. Otherwise, if only one of your visitors have the google toolbar installed, google will find.

So, only possibility I can think of is to disallow robots on the whole site.
David Send private email
Thursday, October 14, 2004
The robots standard does allow you to disallow a wildcard and then allow specific paths, and of course alternately if the search engine has no way to know about a link to get from A -> B, it'll never index B. The problem is ensuring that it never knows about B.
Thursday, October 14, 2004

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz