Apples to apples: Do comparisons always need to be fair?

I'm becoming increasingly ambivalent about the PC hardware review sites - Tom's Hardware is perhaps one of the better known, but there are a whole bunch of them. This is no doubt partly to do with my changing interests - I'm not quite so involved in hardware as I once was.

But the bigger issue seems to me that the review sites have lost touch with what I do with computers.

Case in point: PC motherboard reviews. I've been playing around with Personal Video Recorder projects for the past 12 or 18 months. In essence, turning a PC into a Tivo workalike. Lots of fun. My latest setup has a big box in a closet with a couple of TV tuners and a big RAID, and a little bitty box beside the TV that has almost no moving parts (there's still one fan I can't seem to shake while still getting decent video output).

It isn't without bugs - but that's what makes it a project.

But one of those bugs seemed, after googling, reading, chatting on email lists and so forth, to be due to DMA issues between a tuner card and the other parts of the system when running large data volume that is generated by a card. in essence, the motherboard or chipset couldn't handle large-volume DMA from multiple sources at once - it caused a reset.

And this got me thinking: if I wanted to pick a great motherboard for home video servers, do the benchmark sites give me any useful information?

The reviews seem to follow a tried and true (and old) formula. Picture of the board. Oh gee, it's green. Isn't that cool. The box is very nice. Look, it has the same slot connectors as everyone. Yup there's room for a large processor heatsink. And so on. Then some benchmarks, which compare this board to others in a raft of performance-critical uses, like spreadsheets, word processing. Okay okay, they do some games as well - very important, the game tests.

But you'll note, no tests at all in which large amounts of bus traffic are driven from an expansion card - all the tests of the system are application-related. And these days, when we're seeing a convergence of home computer and media, application-related use is not the only thing happening.

Another case: I saw a great article - I hate to call it a review - of a couple of desktop PC motherboards designed to take laptop CPUs. The article mentions that these boards are targeting a segment of the market intersted in noise (well, quiet) and power consumption (less of it, that is). Folks like me - I was charmed. "Do processors need to consume over 100 watts?" the author wondered.

Pity was, the article didn't compare any of the boards on the basis of power consumption or noise. The article didn't even report the power consumption or noise of these products. Instead, it included the same old performance comparison.

I think the reason the review folks do this is so they can compare the new product against those they reviewed last week, month, or year. And compare tomorrow's review against this one.

This is nonsensical. In their drive to compare apples to apples, they fail to give the folks for whom these products were built any useful way to select between them.

PS. For those as may be interested, the video server project in question combines MythTV and tuner cards including pcHDTV 3000, AirStar HD5000, and Hauppauge WinTV-PVR-250. The server runs Debian linux and includes a 1.2 TB RAID5 array. The client is a diskless EPIA M10000 based mini-itx system in a Morex 3677 case that boots across the LAN using pxeboot.

[Comments from my previous blog]

1. Sean Kennedy left...

Saturday, 3 December 2005 7:21 am :: http://www.simplemusings.com

Very good observations. I would be interested to know more about your client setup. Can it display HDTV transport streams? Does it output 5.1 sound?

Product management failures: (almost) great coffee

Here's a story of product management gone awry, taken from one of my more self-indulgent pleasures, coffee.

After we recently replaced our coffee machine (ahem), we also bought a new coffee grinder. Our new Krups Burr Grinder does a really good job, much better than any of our previous grinders. The grind is more consistent, and this has to be the quietest grinder I've ever heard (if that isn't an oxymoron). All in all, I love this grinder.

Not to say it doesn't have its faults, and one glaring fault is that the amount of coffee it grinds varies from time to time. It isn't hard to see why, and that 'why' looks to be a textbook case in failure of market segmentation.
The Krups grinder has a little wheel that you use to specify how many cups you'll be grinding, which sets the grind time. This calculation depends on knowing how many beans it will grind through per second. No doubt it works really well on inexpensive coffees, that is, those that come in cans, where the beans are dry on the outside and so slide steadily into the burr.

Better beans, though, or to be more precise, well-roasted beans that are fairly fresh, have oils on the outside. Those oils can make the beans a little tacky, and in this grinder (and others), this tackiness can mean that the beans hang up a bit and the burrs run idle every now and then. A timing-based measure of how much coffee it has ground thus varies, depending on how sticky the beans were that day and on accidents of bean position and so forth. Longwinded explanation, but you can hear it when it happens.

This would have been easy to solve, though no doubt it would have added cost to the grinder. Some rotating vanes or something in the bean hopper would have removed this tendency of the beans to stick, and they would have flowed more steadily into the burr. The ginder even has the vanes, and I thought from the picture that this is what they'd do, but no, they're decoration.

I can see the segmentation error that led to this. The target market for home burr grinders is a sort of mid-market-and-up coffee drinker. But they forgot this target market when deciding whether the hopper design was adequate. 90% of the beans on the general market will work fine in this grinder. But the beans this target market will use won't work as well.

Requirements definition isn't about taking the top priorities from each of a number of different people. I'm reminded of a very old Dilbert cartoon, sales asking engineering for a 23" monitor that fits in a shirt pocket. Requirements definition is about solving most or all the needs of someone - ideally a someone who has a lot of very similar friends. Define a target customer, and delight all folks like that. Take away all their objections to buying. Don't leave them with anything left to think about, or to weigh between your product and someone else's.

Or to put another way, product management is about markets, not customers.

Coming back to the grinder, I do love it. It is head and shoulders above any other grinder I've used, and the only other grinder on the market today that even tempts me costs well over $200 (and it doesn't tempt me that much).

Outlook sucks at IMAP

I've been running my primary desktop on Debian for years, but recently I switched back to Windows for a while to gain daily access to a couple of things that only work there. One such only works in Outlook, so I switched mail clients too.

Outlook claims to support IMAP, but it really sucks. I assume it is the IMAP support - if Outlook does these things when running against Exchange I can't see how anyone uses it. Here are some of the ways it sucks:

1. On initial connection, refreshing the mailboxes and sync'ing the message lists takes a long time. Sometimes 3 or 4 minutes for my email on my mail server (Courier) on my laptop. There is a status dialog, but that is frequently hung.While it is sync'ing there is no response from any interactions - heck, you can't even move the windows on your desktop. The whole thing has the symptom of a single-threaded app trying to do too many things in that thread.
2. Rules. When you open your mail client, it runs your rules against the new messages in Inbox, which for me farm some of the messages off to other folders. In Outlook this doesn't always work for IMAP accounts where the move-to folders are also in IMAP (I haven't tested other cases like local folders). The best way I can figure to explain what is happening is that the mail filtering runs when opening the Inbox, but at this time IMAP hasn't necessarily loaded the complete folder list or sync'd the other folders. Move-to rules show the symptom of failing because the mailbox is invalid, even though the mailboxes exist. You then have to go into the Rules dialog, re-enable the rule (since failing disabled it), and RunRulesNow (across your whole Inbox). This sucks.
3. Spam filtering is desperately primitive. I'm trying an add-on spam filter SpamBayes now, which seems sorta ok but nowhere near as friendly as the native filtering in Thunderbird.
4. Every now and then Outlook goes into a Committing local cache mode in which it just sits for a couple of minutes. Other clients don't do this at all.
5. It is very very slow at negotiating secure SSL connections, compared to other clients I've used.
6. No doubt there is more I've forgotten. For example, I note that I used to use Outlook for two IMAP account (work and personal) and I gave that up as a bad plan. I can't remember why.

With all these negatives, it would seem reasaonable to wonder where the 'love' is in my relationship with Outlook. I'm wondering that too.

I think the reality is actually a love/hate relationship with those who on occasion drive me to Outlook to gain access to some new app or service. Or perhaps just 'hate'.

As for Outlook itself, well, one wonders whether its support for IMAP is deliberately bad. Meets checkbox requirements, but drives folks to Exchange. I'm frequently told I have to learn to control my cynicism. :)

[Comments left on my previous blog]

1. Alan left...

Monday, 12 September 2005 11:37 am :: http://alan.blog-city.com/

I remember the Outlook IMAP support not to be great even around deleting a message. It would put a big line through it, and you then had to do a separate operation to "Expunge" the messages. Very clunky and the only email client i see do this. I can only imagine this was deliberate.

2. Cedric left...

Monday, 12 September 2005 8:56 pm :: http://beust.com/weblog

Remember to expunge your deleted emails regularly, I found that they contribute greatly to IMAP degradation in Outlook. Besides, why keep any emails when you have GMail? :-)

As for your love/hate relationship, I totally share it.

-- Cedric

3. Dmitri Colebatch left...

Tuesday, 13 September 2005 12:02 am :: http://www.colebatch.com/blog

Here here - outlook is the best mail client I've ever used when it comes to user interface, but unfortunately its IMAP support is absolute crap. I'm about to switch all my mail to gmail (I've been using it for lists for ages) as I'm fairly happy with that interface and I'm running out of space on my mail server (o:

4. glen martin left...

Wednesday, 14 September 2005 10:37 am

Yeah, Outlook displays deleted emails by default, but there is an option to hide them. Of course the option is pretty obscure, located in View->ArrangeBy->CurrentView (I don't usually think of a display filter as an instance of ArrangeBy).

Another way Outlook sucks at IMAP is be treating it as a second or third class citizen. For example, there's an option to "Save responses in same mailbox as original email, except for Inbox". But this option doesn't apply to the IMAP Inbox. Additionally, sent messages aren't saved to IMAP Sent, but to local Sent.

It all just adds up to this tedious experience for anyone who has the audacity to try to use standard protocols with this application.

So far as having the best user interface ... well, there are indeed things I like about the interface. But there are some real clunks too.

For example, it marks a message read when you leave, not when you open. When I delete a message, the next message is opened automatically. I can't mark it unread while the message itself is open because it is only market read when I leave. If I don't want to read it now, I have to click on some other nearby message, then right-click on the one I want to mark unread and choose that option. Hopefully I have another nearby message I've already read or don't mind reading right now.

5. David left...

Monday, 10 October 2005 4:12 pm

I too am stuck with Outlook because I require its contacts/tasks/calendar/notes/Palm sync capabilities. To be fair, it does these things extremely well, but it's IMAP support is just horrible. If I used a separate mail client it wouldn't be integrated with the contacts and everything else, so at least for now I'll have to live with Outlook.

6. David left...

Tuesday, 19 September 2006 8:51 pm

The reason Outlook deletes imap messages the way it does is because there is not an imap folder with the delete function. So outlook creates is own method of a two-step deletion process (mark for deletion and commit to deletion). Outlook's biggest imap failure IMO is #2 in this post (rules support)

7. Arizona Web Design left...

Tuesday, 28 October 2008 1:04 am :: http://www.barlamenterprises.com/

Well i was googling for possible Outlook imap fixes and googled "outlook imap sucks" and got to this blog post and I saw it was written in 2005. Rest assured, 2008 now and I have Outlook 2007 installed and it sucks at imap! I wish thunderbird was better :( or there was a better solution. Does entourage have this problems? Maybe i should use get a mac...

8. Andre de Cavaignac left...

Sunday, 9 November 2008 8:27 pm :: http://decav.com

Yup!

After working with the IMAP protocol, I can totally understand why though... Its not pretty, and implementing a client well means a lot of work on the client end.

And by the way, Outlook is a single threaded application, for the most part... Back in the old days, it was built that way and not much has changed because they don't want to break addins. Access to the mailbox all must be done on the UI thread... Oops!

9. Randy Schumaker left...

Tuesday, 4 May 2010 8:40 am

It looks like IMAP support may be getting better in Outlook 2010. http://blogs.msdn.com/outlook/archive/2010/02/06/better-imap-in-outlook-201 0.aspx

Planned obsolescence

Screams aren't a good way to wake up.

Or at least, not one I'd choose.  20 years ago I woke instantly, but nowadays I sort of meander into full attention. Screams from the other end of the house aren't, frankly, all that respectful of my lifestyle. :)

On arrival in the kitchen, I discovered that our coffee machine was on fire. Not smoking, but literally in flames. The plastic was melting, that melted plastic was running across the counter also in flames.

"Things aren't built the way they used to be" has become so commonplace it is more than a little trite. So many products fail pretty much exactly when the warranty runs out that we sometimes talk of planned obsolescence.

I see planned obsolescence as a failure of marketing, or more, as a flawed view of what customer relationship is all about.

Think about it this way. The goal of building a long term relationship is to encourage customers to do business with you again. In essence, to build a sort of annuity of goodwill that continues to generate revenue over time. But if we forget about the word goodwill, and focus on the repeat business, then it starts to make an odd sort of sense to make the product fail and so to force another purchase.

This attitude fails in that early or spectacular product failure doesn't maintain what goodwill the positive experiences with the product may have created.

Without a doubt, I'm not going to buy another of the same coffee machine. And, in fact, my new machine is a completely different brand.

PS. Out of respect the the aforementioend screamer, I'll admit that it was more of a yell than a scream, but I think the text came out

Bayesian filters and infrequent senders

Over 90% of the email I get now is spam. Like many, I've turned to Bayesian filtering to help catch and shunt it aside, but lately I've been thinking about infrequent public correspondents. To whit, Paypal, EBay, and so forth.

At this moment I have 11 messages in my inbox purporting to be from Paypal.  They are all, every one, phishing attempts. By my way of thinking, that's spam.

If I classify them as spam, though, what happens? With no Ham (that is, not spam) from Paypal, how does the learning filter figure out that only a certain class of apparent Paypal messages are Spam? The ones within which there is a 'Click Here' link to a non-paypal IP address? It seems to me as if the filter might just as easily figure that 'From: Paypal' is the culprit. Or emails with <img> links to paypal logos.  When it comes right down to it, the vast majority of the content of this email is quite reasonable, and Paypal could very well send an email that looks a whole lot like this. Which is, of course, the point of phish attempt.  But my filter doesn't necessarily know this.

Next month, or next year, when Paypal really send me an email, will I see it?

What's interesting to me about all this is: what should, can or will Paypal do about it? Skipping the true root cause, that jerks and criminals are sending these phish attempts, as unfixable, the second cause is that I have no Ham from Paypal. The friendly folks at Paypal might reasonably think to send some real messages then, in essence to keep our Bayesian filters 'honest'.  That, of course, requires me to read them, and to ensure the filters know they're not spam or phish attempts. AKA, manual effort on my part.

Another solution might be for me to stop classifying phishes as spam ... but then I'll want another solution to catch phishes.

Or the folks writing trainable (Bayesian or otherwise) filters might figure out a way for me to tell the learning algorithm which bits of any message to disregard.

None of these are great solutions - all require more manual intervention than I appreciate.

Folksonomies vs generated categorizations

Several friends have been pointing me at del.icio.us for quite a while now, which has led me to give some thought to folksonomies.

I'm reminded of a talk at the Web2.0 conference in which Google was demonstrating a research project that performs auto-categorization of search results. The thrust was that when you do a search, rather than just return a flat list of links, Google would examine the context of the search terms within the resulting pages in order to ascertain the category of use. That is, "bismuth" can be a metal or a ship. So Google would return "metal" and "ship" from your search of "bismuth", and you could select the category you meant to get a more precise result.

Folksonomies (aka del.icio.us) are really interesting from the point of view of enabling an architecture of participation around categorization, but will result in a somewhat colloquial categorization on which subsequent viewers can't truly depend. At best. I mean, if we listen to George Carlin ("Toledo Window Box"), we might imagine some folks will tag with "bush", some with "hemp", some with "boo", some with "smoke", some with "weed", some with "guage", some with "grass", some with "tea", some with ... you get the idea. We wind up with a number of variously disjoint islands referring to the same concept. And because the terms are colloquial, what I mean by 'tea' won't be the same as what you mean by 'tea', and you will undoubtably be surprised and amused when you visit some places I've tagged thus.

A folksonomie isn't - can't be - very authoritative. This is probably what Blaise Cronin was referring to: "Undoubtedly, these are the same individuals who believe that the free-for-all, communitarian approach of Wikipedia is the way forward. Librarians, of course, know better." Free-for-all classification is, one suspects, precisely the problem the Dewey Decimal Classification System was designed to address.

All this sets up something of a competition between Google category extraction and del.icio.us. It'll be fun to watch it play out.

DIY magazines

Blaise Cronin* seems confused.
 

Lately, I’ve been wandering around Blogland, and I’m struck by the narcissism and banality of so many personal blogs, of which, if the statistics are to believed, there are millions. ... One wonders for whom these hapless souls blog.

Though he admits

Blogs can keep stories alive, bring them to the surface and propel them into the media mainstream. Take note: a new communicative dynamic is at work in the public sphere.

He closes with:

Undoubtedly, these are the same individuals who believe that the free-for-all, communitarian approach of Wikipedia is the way forward. Librarians, of course, know better.

He's missed making his point apparently by being too concerned with the supply side and not the consumption. He should be talking about the ability of information consumers to create their own magazines.

Many have commented on self-publishing as a democratizer (yup, I'm bastardizing the language again). Blogs, mp3, and other technical advances are reducing the cost of publishing to the point where we mere individuals can publish without the permission of some cigar-smoking executive someplace. It is a great liberator, but it leaves us with the million-monkeys problem. A million monkeys will no doubt produce Shakespeare, but the trick becomes to find the Shakespeare in the volume of material produced.

Librarians, publishers and record companies are filters of information. They choose and organize information to make it more accessible. Because they all try to serve a broad audience, they create filters that aim for a certain breadth. In principle anyway, but more on that in a moment. Without librarians and editors, we use aggregators and Favorites that we set up ourselves to gather information we may want to read.

In effect, we create our own magazine-analogues. DIY magazines.

The challenge in self-selecting content for our private magazines (that is, setting up our own aggregators) is that we can narrowly focus our attention on points of view that we appreciate. How many of us want to read authors we disagree with? With mass-market magazines we can choose to not read an article after seeing the title, or perhaps a call-out box of some sort. We get part of the message anyway, or at least know the message exists.

DIY magazines are our own filters of what content to even become aware of. This can increase the divide between different ideological groups as those groups only read those who think as they do and fail to pull any balancing points of view into their own magazine. Rather than finding common ground, they propel their opinions to extremes by never reading contrary voices. Little groups become their own apparent majorities (at least to themselves) as "everyone says" whatever they want to hear.

Not that there are that many contrary voices anymore anyway. Media consolidation has left us with a few companies (editors in fact), all with similar goals and ideals, controlling our information channels. Read or listen to media outlets around the world (or even just Britain) and you hear many interesting stories about America that somehow American media singularly (pun intended) fail to present. Thank [insert politically correct deity here] for self-publishing.

BTW, Cronin is Indiana University Dean and Rudy Professor of Information Science. I'd be far more impressed if he used this pulpit (well, Chair) to discuss media consolidation vs issues of choice and democracy in a self-publishing world. That is, how to create something better using new technologies, rather than dismiss that which is new out of hand.

PS. Here are a few interesting books in this area. If you look at Breaking The News, be sure to check out the review by Barron Laycock. As I was writing today, I encountered the following quote that provides interesting colour for my point above, from a review of Rich Media, Poor Democracy (emphasis mine):

If your politics lie anywhere to the right of Ralph Nader's, in other words, don't come to this book looking for validation. But for a stimulating, nuanced, and rigorously researched presentation of the case for overhauling the current media regime, look no further.