CBS News | Bush, Atty Powwow Over CIA
Leak | June 2, 2004 22:11:28

cbsnews.com/stories/2004/06/02/politics/main620810.shtml
track this site | 4 links

Duluth News Tribune | 05/14/2004 | Man
fights citation for carrying anti-Bush
sign

duluthsuperior.com/mld/duluthtribune/news/local/8664932.ht m
track this site | 4 links

Stocks mixed on economic news, J&J
deal Alaska's North Slope proposes
pipeline United pilots endorse

September 14, 2004

We've been quietly making some improvements to the beta discussion group software.

Today we rolled out Brett's new full-text search feature. It relies on the database engine to provide full-text search, and we're running Microsoft SQL Server, which has rather poor full-text search capabilities: for example, it requires a manual process to rebuild the index, which we schedule for every 15 minutes, so it won't find anything posted in the last few minutes.

I also added an RSS feed. Originally I wanted to provide full text of all topics and replies in the last three days so that you could use an RSS reader to read the discussion group. Unfortunately that would have resulted in a huge download, and since RSS readers bang on the site every hour or three, our bandwidth usage would have been absurd. So I had to settle for full text of the original topic but not of replies.

And finally we got Summer Intern Ben's excellent Bayesian filtering code working... due to a couple of configuration problems it wasn't running right. The idea is to delete comme nt spam before anyone sees it. It's hard to tell if the filter works yet because it needs more training, but so far it's doing pretty well. If you think comment spam is not a big problem, you haven't moderated a discussion group lately... this is the number one priority for spammers these days, since email filters are starting to work pretty well and spamming a lot of discussion groups is perceived as a good way to trick Google into giving a site prominent placement.

We admit to three strategies to prevent comment spam:

1. Bayesian filtering which can be trained to remove comment spam instantly
2. Not allowing new comments on old posts, so that comment spam can't be hidden in posts which nobody but Google visits any more
3. Using a META tag to ask search engines not to follow URLs from discussion topics. Although this technique prevents comment spam from working it doesn't prevent it from happening because spammers don't seem to particularly care if a given spam works or not.

By "we admit to" I imply that there are other things we do which we don't talk about too much because revealing them would make it that much easier for spammers to work around them, thus reducing the cost of spamming, thus making it more economically feasible.

One side affect of the Bayesian filter is that if it finds a suspicious topic, rather than letting it through, it will flag it for a human moderator. The moderator can then allow it to be posted (which trains the filter) or leave it unshown. The effect of this is that rarely, new posts won't appear until a human approves them. This should happen less and less as the filter learns more.

September 02, 2004

Try out the new discussion group, now in beta!

A few brief comments:

• Yes, there are bugs! Many bugs! Please report them using the link provided.
• It looks kind of plain right now, I know. And I haven't implemented a search feature yet, which is planned.
• It works a lot like the old discussion group.
• There's a new archive section ("Older Topics").
• You can provide a URL with your name when you post. I'm hoping that people who have their own web sites will do so, and that this will make posters a little less anonymous. (Note that these URLs will not be followed by search engines like Google, so they cannot be abused to "steal" PageRank.)

Most of the changes are really on the back end, in the adminitrator's interface and in the abuse- and spam-prevention features. I'm hoping that if all works well, you'll see a lot less spam. And it's now much easier for me to make a new discussion group, something which used to take an hour of work, so I may set up several new groups in the future (if you have a good idea for a topic please post it.)

September 03, 2004

I'm happy to announce that my friend, teacher, and competitor Eric Sink has agreed to host the new Business of Software forum.

Eric has been in the software industry from time immemorial. He built one of the first web browsers, Spyglass and founded the AbiWord project to make an open source word processor. His latest company is SourceGear, a highly successful Inc 500 software company in Champaign, Illinois which makes great source control management software. He writes a column on the business of software for MSDN and has his own weblog where he has been extremely generous with his knowledge of software business. I couldn't have a better host.

September 06, 2004

My Latest Article

Software in the 1980s, when usability was "invented," was all about computer-human interaction. A lot of software still is. But the Internet brings us a new kind of software: software that's about human-human interaction. When you're writing software that mediates between people, after you get the usability right, you have to get the social interface right. And the social interface is more important. The best UI in the world won't save software with an awkward social interface. It 's Not Just Usability

Workspace Quality

Someone posted a nice summary of links to information about the quality of workspace provided to software developers.  “Perhaps it can be useful to other software developers in a position to influence managers or may someday be in a position to make decisions about workspace design themselves.”

CBS News | Bush Ranked 22 Of 53 Pilots | September 7, 2004 20:40:01