Net weirdness

[altivo]

As most of you know, I edit a monthly newsletter for my handspinning guild. The newsletter goes online in HTML format and people read it from there. For four years I've been loading those files into my personal webspace at Earthlink without any problems.

This morning I got a warning from Earthlink that my webpages were at 80% of their transfer quota for the month of October, and if access continued at the same rate, they would be cutting them off for the rest of the month. They went on to suggest a number of ways to fix this, not all of which involved removing content or buying more bandwidth.

WTF? Why the sudden surge? I have no idea. They do provide a reporting tool, and it shows a huge number of hits on the newsletter from last February. It appears that there was a sudden surge of interest in that particular issue starting in May, and increasing to a giant peak on September 26, followed by a slow drop to the present level, which is still unsustainable. There have been over 8000 hits on that file since October 1.

There's nothing unusual about it that I can see. It contains stuff of little interest to non-members, just a schedule of meetings, a discussion of the upcoming meeting's agenda, a few notices and a brief discussion article (about two paragraphs) on the Alaskan musk ox and its fiber product, qiviut. I can only guess that it's the qiviut article that is getting all this attention. A search of Google under qiviut or musk ox(en) doesn't find this at the top of the hit list, though, or even in the first few pages. Google doesn't report any other page that has linked to it, either.

I've relocated that one issue to another site, and redirected to it. The redirection stub is less than 200 bytes and should stop the hemorrhage from Earthlink, I hope. I'm still left with the mystery of who is reading this and why.

For the curious, the relocated content is viewable here.

Comments

Uh...

[vakkotaur.livejournal.com]

I get:

The page you navigated to does not exist.

Re: Uh...

[altivo.livejournal.com]

Apparently you have to be signed into a Google or Gmail account to see it. It displays for me when I'm signed in, but not when I am logged out.

[vakkotaur.livejournal.com]

Oy, yet another thing Google is being evil about.

[altivo.livejournal.com]

Well, in this case you really aren't missing much. While I have no problem with non-members seeing the newsletters, this stampede to look at one in particular is making trouble for me. I'm looking for another way to host all of them.

[altivo.livejournal.com]

I think it's fixed now. It's a bug in Google groups. Even though the page says it is set for public viewing by default, it really isn't. You have to set it to private and then back to public to get it to become visible.

[vakkotaur.livejournal.com]

It is indeed visible now. Interesting questions about achieving desired colors, as well as the text about the musk ox.

[altivo.livejournal.com]

I thought perhaps it was the photo of the baby musk ox that was attracting all the attention. But as far as I can tell, nothing seems to have linked to it.

[hrrunka]

If it's not a link it could be just a high ranking for some popular search string. ot something that's easy to figure out unless you get some suitably-digested referrer logs, though...

The one time my local astronomical society's website went, well, astronomical (from about 10 or 20 MB a day to a peak of almost 2GB in a day) it was down to the Venus Transit, and a link from ESA. Thankfully, once the transit was over the traffic very quickly dropped back to more normal levels.

[altivo.livejournal.com]

I'm working on getting referrer information. They don't make it readily available for the individual user "free" web pages though. Hey, I pay for access, but evidently not enough. I'm going to have to put Google analytics scripts into the newsletter pages or something like that.

[farhoug.livejournal.com]

Yep, it looks like Google ate your newsletter...

Maybe it's some errant spider indexing the web page, and it has gotten stuck in a loop? Though I guess Earthlink doesn't give that much details about the visitors. How much bandwidth they provide, by the way?

[altivo.livejournal.com]

Evidently you have to be signed into Gmail or another Google account in order to see it.

[farhoug.livejournal.com]

It might need joining the group too, at least the simple Gmail account login isn't enough for it...

[altivo.livejournal.com]

The group options are set to NOT require membership in order to view messages or pages. Perhaps, though, you need to log into Google groups using your Gmail address and password.

[farhoug.livejournal.com]

Joined the group, switched to Firefox, used the Groups login and went through "My groups" to make sure it's not a caching issue, and it's still giving the same error message. Curious thingie...

[altivo.livejournal.com]

Now that's really weird. Can you see this page?

http://groups.google.com/group/hollow-tree-spinners/web/hts-newsletters

[farhoug.livejournal.com]

Yep, that works properly, as do the images you've uploaded today (the dyed sheep, etc.)

Maybe the page hasn't propagated throughout the Google's servers yet? Especially if it works for you... Thus the usual fix, try and reload the page again tomorrow? :-)

[altivo.livejournal.com]

That's the only answer I can think of. Google has an awful lot of servers, obviously, and instantaneous propagation is a lot to expect. I see you signed up as a group member, does that mean I get to teach you to spin? ;p

I wonder if their servers run Windoze...

[farhoug.livejournal.com]

Yeah, I even tried "Tor"-ing to Germany and France, they can't get through either. We'll see how it goes...

I could weave something nice out of CAT-5e... though I'd have to get some other colors than the usual beige-grey. I wonder how one dyes PVC cable... :-)

[altivo.livejournal.com]

I think you'd have to paint the PVC rather than dyeing it.

I'm still not convinced that propagation is the issue. I just tried on another machine where I'm not logged in. That machine has the same effective IP address and routing (behind the same firewall.) Even there, I can't see the new page, but I can see the old one.

[farhoug.livejournal.com]

They might still assign a random server for each client at the Google end of internets, regardless of the location or routing. There's plenty of sneaky tricks for load balancing. But it does seem a bit odd...

Maybe the propagation phase itself has failed, after the initial successful post?

[altivo.livejournal.com]

I think it's working now. It's apparently a known bug in Google groups. Even though the option is set for the page to be viewable by anyone, it isn't. You have to set it to private and reset it to viewable before it works.

[farhoug.livejournal.com]

*nodnods* Now it's working properly. ^^

[flamekist.livejournal.com]

I can see it without logging into anything at all.

[altivo.livejournal.com]

OK, that helps confirm Far's hypothesis. He's in Finland, and the server Google is connecting him to doesn't recognize the new file yet. This one is an older file, though it gets modified once a month.

Can you see the original file in question? The one I linked to at the end of the main post?

[soanos.livejournal.com]

I am in Finland, too, and in the same ISP area. I can't see the link in the original post, but I can see the articles in the link posted in a reply.

I am not signed on my G-mail. I haven't joined any groups either. :)

[altivo.livejournal.com]

Yeah, there's something funky about the way Google groups is handling this. I guess we wait until tomorrow to see if it's still broken.

[soanos.livejournal.com]

That was a fast reply. Do you have something that notifies you about replies? :)

[altivo.livejournal.com]

I get an e-mail for every comment/reply, yes. It's in the preferences for LJ.

[soanos.livejournal.com]

I see. I only use Hotmail at the moment so it is a bit inconvenient for me. Sorry about spamming your inbox. :)

[altivo.livejournal.com]

It's not spam. I'm set up to handle it. And I *like* hearing from you. :)

I think the link is fixed now.

[soanos.livejournal.com]

Yay! \o/ *hugs*
At least someone likes me :D

Yes, the link is working properly now, without logging in into anything.

[altivo.livejournal.com]

*yays and hugs back*

[flamekist.livejournal.com]

Yep, I can see the original one in the main post. My apologies for the two-hour delay for an answer.

[altivo.livejournal.com]

Thanks for checking. No reason I should expect you to be hanging around just to test my stuff. :)

Actually, it was in fact broken earlier and I found the fix in between then and when you looked.

[altivo.livejournal.com]

Oops. I failed to answer your bandwidth question. I didn't even know I had a limit until now, but apparently it's 1 GB per user per month. One of their suggested solutions was to spread the data out over multiple users, since my monthly payment entitles me to up to eight e-mail accounts and eight different web sites, each of which gets 10 MB of storage and 1 GB of transfer. Pretty paltry, these days, but I don't pay a lot either. Their service isn't worth a lot, frankly.

[farhoug.livejournal.com]

Yeah, it's not much, but your pages are pretty lean too. I guess those habits come easy when living behind a modem line... ;-)
Hopefully you'll find out what's eating the bandwidth, it's not really handy to have to spread pages around the net, even if it's somewhat transparent to the user.

I'm paying about 30 euros per year for my "site", it's not doing much anything but holding some various web projects under progress, and of course my main email (and some friends and family too.) Usual bandwidth used is about 100-200 MB/month, out of 20GB. (Though I guess it's mostly my email client pulling the mail status through POP, I should clean up the old stuff from there... ^^;)

It's a bit unreliable, probably down a day or two in a year, but it's cheap too. That's good enough reason for me to stick with that service.

[altivo.livejournal.com]

Heh. "Lean" pages are easy. Just don't use stupid flash stuff, which is what seems to be poisoning every page on the web these days.

[farhoug.livejournal.com]

Yeah, people don't bother to think things through these days anymore, I still see BMP pics in the net every now and then, and otherwise poorly compressed images. Every web-dev should live a month with just a modem line to get these things, I guess.

I did one set of pure Flash pages once, for a local peer support group. I managed to squeeze almost every page under 100k, it was interesting to try to keep things small. =)

[altivo.livejournal.com]

Yeah, flash doesn't really favor "small".

The problem with the web-dev types is, and always has been, that they assume everyone has the latest, fastest processor, with gobs of RAM and a high speed connection. Why? Because that's what they have themselves, and anyone with less isn't worth bothering with anyway.

[duskwuff.livejournal.com]

Flash is actually surprisingly good at generating small files if you design your content with that in mind. Most people who use Flash (stupid graphic designers, mostly) don't, is all. It's entirely possible to do well, though: one app I put together with Flash a while back (a Telnet client, essentially) was just over 50 KB.

[altivo.livejournal.com]

Umm, why would you write a telnet client in flash? Isn't that like pounding in a screw with a hammer?

[duskwuff.livejournal.com]

This particular bit was a talker/MUCK client. The default Telnet app on most machines (both Windows and others) isn't particularly well-suited for use in an environment where you may get input while you're typing, and I wanted something easier for users to get started with than a downloadable client.

I considered using Java, but, given that well, over 99% of users have Flash installed compared to Java's 75%... it's an easy choice in that light, really.

[altivo.livejournal.com]

I wonder what percentage of users have flash installed but use an add-in like flashblocker to keep it from running. That's what I do. Overall I think flash is the biggest timewaster there is on the net, and that it's used mostly for dumb things. I particularly hate the way it will make a whole page unusable and often lock up the browser while some huge flash file is downloaded.

I'm surprised that the figure on java is that low. Most operating systems seem to install java by default now right along with flash. Java has the same irritating propensity for locking up a browser while an applet loads, but at least the files tend to be much smaller. I've also had no problems with flash apps crashing firefox, where I've had terrible and consistent difficulties with flash doing just that. Usually it happens when you try to leave a page with flash junk on it before all the stuff has fully downloaded. Bang, firefox vanishes completely. Restart firefox and it wants to reconnect you where you left off, which in turn will crash again unless you let it download all the flash or tell it no, start from nothing again and lose all your open tabs. The problem has been reported many times to both Adobe and firefox, but still persists. Because it apparently doesn't afflict the Windows environment, but only Linux and Mac, I guess they don't care.

[altivo.livejournal.com]

Oops. Make that "no problems with java apps crashing firefox, while I've had consistent trouble with flash doing just that..."

[songdogmi.livejournal.com]

My mini-website on Earthlink when I still belonged had a mystery that wasn't so big as yours but still puzzling. I had a list of links to local radio stations and information sources, not that I had any intention of making it The Information Source For Detroit, just for myself. But it consistently got between 100 and 200 hits a month where none of the other pages got more than 10 (which were all from me, pretty much). I never did figure that out.

[altivo.livejournal.com]

Hopefully I'm about to get some answers on this one. I moved the original files to another site, and replaced them with one of those stubs that redirects the browser to the new address. But in the 5 seconds of delay for the redirection, it executes Google analytics code to find out where the request came from. In a day or so, there should be enough data in the trap for analysis.