artlung: a personal website ... december 2001
JOE CRAWFORD'S HOME OF ENIGMATIC NARCISSISM AND MISCELLANY

December 20th, 2001

In case you came here looking for a reference to netflix based on this…

In case you came here looking for a reference to netflix based on this, well, I like looking at peoples’ robots.txt files.

Truth be told, I learn a great deal by looking at how sites do their thing. I remember visiting amazon a long time ago and seeing how they generate extra url stuff and wondering what it was. I distinctly remember opening and closing browsers to see how I could mess with it and see if I could fool it. I also love tweaking url strings to see if sites still work when the urls are messed with. As you see what happens as you add and remove elements you get a sense for what’s happening “under the hood.”

I suppose my behavior is an amplified version of a a monkey with a twig poking at anthills.

Leave a Reply