Web hosting – Historical Perspectives

hmtl/ or public_html/, DNS, what is IP or name based virtual hosting, and what's this SNI stuff? Why does it say, “It Works!”?

Self-hosting was always easy. You simply harness the awesome power of a UNIX machine with BIND and an Apache server, and dump all those AngelFire files into the default directory. Then you register a domain name from Internic for $35/yr and give them the names and IP addresses for the glue records to point at your AUTH nameservers.

Easy Peasy!

And then you realize that for all the work you just did, you could be hosting a couple of hundred websites. After all, you've got an AMD K6-2/350 with a whopping 16MBytes of RAM and that /24 they handed you gratis a few years back!

Most websites just sit there, waiting very quickly to have their files served, and the money just keeps rolling in at $15/mo for each website, but you're beginning to run out of IP addresses to alias and bind to your 3Com 3c905 NICs.

But with HTTP 1.1 there's a new potential, it might be a little tricky at first though...

Yes, back when SNI first enabled us to move away from IP based VirtualHost containers in Apache to the Name Based VirtualHost where many websites shared a single IP address, there were some interesting issues, such as the default VirtualHost container being served if you didn't also specify non RFC compliant hostnames as aliases.

For example, there was a trend at the time to ignore proper usage of hostnames, with “www” generally being used as the traditional defacto, default hostname for a host delivering HTTP. i.e.,(http://www.SLD.TLD).

Similarly, you would usually name your first FTP server “ftp”, and the protocol was specified as “ftp”, yielding ftp://ftp.SLD.TLD, and gopher would be gopher://hostname.SLD.TLD in the address bar of your browser. Other protocols followed that same “protocol://hostname.4LD.3LD.SLD.TLD format, these child zones being a very common practice in universities in the .EDU and .MIL TLDs, as well as the .UK, .FR, .AU, .US, and some other ccTLD's (Australia has recently dropped that whole requirement for .com.au and .net.au finally lol).

So, although it was merely convention to name your hosts after the services they were delivering, like “www, www2, www3, Etc.”, or ftp, gopher, Etc., Those were only conventions and you could name them whatever you wanted.

But you weren't supposed to use the canonical domain name as the hostname!

Nevertheless, as the “web” became more ubiquitous and average users, oblivious to other protocols, began to ask the question, “why do I need to type in all those extra characters for www.ibm.com (like http://www3.ibm.com) instead of just ibm.com?“, in conjunction with the popular trend of browsers defaulting to “http://” protocol (unless you manually specified another protocol), many providers started to accommodate those hosting customers by breaking with the RFCs and pointing DNS A RRs to “@” or “SLD.TLD” – the canonical domain name, akong with another A record for “hostname.SLD.TLD” (usually still using www for the first website).

Now, when you query a DNS server for a hostname, it returns an IP address, no problem so far. And with IP based hosting it really doesn't matter what you call your website in your Apache server's VirtualHost container – it's just going to return a web page for the server at that IP address, coz that's all it's being asked: “Do you have HTML to send me for the IP address on this HTTP server?”, And then it sends the page – coz there's only one website per IP address!

But with HTTP 1.1 and SNI it's a compound question:

“Do you have HTML to send me for the IP address with the hostname of 'www.SLD.TLD' on this HTTP server?”

If yes, then serve the page. If not however, it gets a little weird. The default VirtualHost container is usually the first one defined in httpd.conf, so on many web servers that's the page that says – “It Works!” lolz.

So, in DNS you had to have two A Resource Records (RRs) one for say, 'www', and one for '@', or you could use a CNAME RR if you're one of those lame-o boneheads (bad form – don't use CNAME's!), AND... You also either need to specify two <Named VirtualHost> containers pointing to the same directory, or just one with an Alias directive.

NOTE: you 'could' point to two different directories, but in 99% of all cases, why the heck would you do that?

I dunno why, but maybe coz many hosting providers didn't understand how to write a DNS zonefile in BIND or maybe coz they didn't grasp that the browser was actually now asking for an actual hostname with it's request to the IP address that httpd was listening on, but this resulted in a lot of “no joy” when people neglected to type in the “www”, resulting in delivery of either someone else's website or the comical “It Works!” page (ROFLMAOPMP).

Adding insult to injury, the opposite could be the case when leaving out the hostname and just typing in the domain into the address bar, depending on how httpd.conf was configured – many knowledgeable sysadmins resisted this the because it wasn't RFC compliant, but eventually (the customer's money is always right), most default websites today just serve the canonical domain name as the main (or maybe even the only) hostname in a VirtualHost container in Apache with an alias to the 'www' hostname (or server block on Nginx.

As a little epilog, that came back to bite hosting providers, and moreso, their customers, in the last decade or so when the evil Cloudflare and ever popular WordPress came into prominence – To enforce their predatory vendor lock-in practices, Cloudflare offered free CDN services to people with websites... But only for one hostname per domain, AND, that hostname HAD TO BE “www”, lolz...

Well, after a couple of decades of dumbing down the user and people setting the main name of their WordPress sites to, “SLD.TLD”, this would cause, in the best scenario, to have their websites served to most people without the (so-called) benefit of Cloudflare's CDN... Unless you could retrain every user on the entire planet to type in the actual hostname (www), OR... pay Cloudflare lolz.... 🤘🤡🤘

Well my friends, That's all I've got right now, I hope you found that a bit amusing in a sort of nostalgic kinda way.

I can be reached on Matrix at:

@tallship:matrix.org

via XMPP at:

tallship@jabber.org

and in the Fediverse at: @tallship@social.sdf.org

#tallship #ramblings #lamentations #humor #geek_pr0n #BOFH #comedy

.