Testing Times At The Office

POOR Hawthorn. The Hawks will not be playing in next weekend’s Grand Final, having failed last night by a miserable three points to get the better of Collingwood. There will be post-mortems, of course, and the most obvious explanation will be that Jeff Kennett’s leg-weary boys ran low on ergs in the fourth quarter. Few who watched the game, a genuine classic, will disagree with that assessment, but readers of this morning’s Silly will know better: If only the MCG had been officially designated a five-star sustainable sports stadium, Hawthorn’s final-term fitness and work rate would have soared, making vistory inevitable.

This is science we are talking about here, folks -- good, hard, solid, settled science, the sort the Silly and Phage never hesitate to report when field surveys and results, even the most dubious ones, provide just the merest hint of a suggestion they might endorse those newspapers’ jolly green preconceptions. In this instance it is the benefits of working in sustainable office environments, with the Silly reporting the latest news from the Clipboard Kingdom in the story’s first paragraph “The new wave of ultra-green office buildings,” it begins, “is bringing unexpected benefits for their occupants beyond the obvious savings in cost and energy.”*

Overlook the sentence’s cack-handed construction and grammatical imprecision, which are no more nor less than one expects these days from quality journalists and those who teach them.* Instead, examine the, ahem, scientific method the story asserts has established the benefits. After quoting two twenty-something lawyers in Sydney, writer Nick Galvin switches his attention to the south, where he reports the following:

The pair's sentiments are backed by an increasing body of evidence, including a detailed study with another law firm, Oakley Thompson, after it recently moved into a newly refurbished green building in the centre of Melbourne [500 Collins Street]. The study, conducted with the University of Melbourne, found sick days had fallen 39 per cent and the lawyers' billings ratio rose 7 per cent despite the overall hours worked falling 12 per cent. The survey even found the firm's secretaries were typing 9 per cent faster in the new building and with greater accuracy.

Detailed study, eh? Well, not quite. As anyone who examines the actual results will ascertain very quickly indeed, the methods are about as reliable as Hawthorn’s set shots.  Let us take those claims one by one:

SICK DAYS: The study contrasts five years of pre-renovation sickie stats with those for just nine months after the office was awarded its five green stars. The five-year figures were “normalised”, just like tree-ring data, and the post-renovation numbers were not, with the number crunchers asserting they saw no need to do so.

Even so, incidences of illness in post-renovation January exceeded those for the same month’s five-year average. And where there were differences, they were generally slight. March numbers were only a whisker below the five-year norm, as were those for the previous November. Such overall gains as the study claimed to observe are attributable almost solely to three months of somewhat wider divergence (see Chart 3).But given that the post-renovation numbers are based on part of just a single year, it is reasonable to assume that some of the previous years’ pre-renovation figures were just as low.

Logic and surmise are our only guides on this point as, for reasons unexplained, the authors neglect to provide each of those years’ individual trend lines. 

BILLINGS: The claim is that lawyers brought in 7% more fees while working 12% fewer hours. Let the report itself, which Galvin evidently neglected to absorb, explain why that summarised result is pure nonsense.

First, there is the differing natures of the compared samples (underscore added at the Billabong): 

QUOTE #1: The data on which these conclusions were made covered only ten months before the office move, and ten months after the move. Before this period, data was collected manually and the records were not readily comparable. The differences in data collection approaches would also make the comparison questionable.

 Then there is the matter of the survey’s participants:

QUOTE #2: …the pre-occupancy survey was conducted four months before the move, while the post-occupancy survey was three months after the move … It was a voluntary survey and of the total of 20 employees (including partners and solicitors), 12 (60%) agreed to participate. For the survey to obtain meaningful comparisons, the post-occupancy survey should cover the same participants as in the pre-occupancy survey. However, only eight of the 12 original participants responded to the second survey, so that the comparison covers only 40% of the total staff.

But not to worry. Another firm, stockbrokers this time, also was surveyed, with an unnamed senior executive quoted as saying “productivity has gone through the roof.” Alas, there is no way to verify that enthusiasm. As the survey’s authors concede (they do an awful lot of concedin’):

QUOTE #3: … Because Lonsec [the brokerage] only agreed to join the Study after the move, the survey could only be retrospective, with participants directly comparing their new accommodation with their memory of their past accommodation. This is less objective than asking staff to rate a building before a move and then, some time later asking them to rate the building again.

FLYING FINGERS: And finally there is the astonishing claim, the most improbable of them all, that typists are faster and more accurate if tapping away in a green building. As with those busy, busy lawyers, the before-and-after sample was miniscule. An improvement of 9% is claimed, but go to Chart 5 and you will be struck that only five secretaries were tested. Of these, according to Chart 6, one – Typist 4 – was more accurate in her previous surroundings!

There is plenty more in the findings that should have raised reporter Galvin's eyebrows, and it is to be hoped that one of those savvy statistical sorts who make Catallaxy Files so interesting will bring an expertise greater than that of a specialist in Etruscan semiotics to the subject. But even without a more probing analysis, two things stand out as causes for alarm.

The first is that, once again, the Silly has demonstrated its eagerness to swallow and regurgitate more of the green guff its reporters and editors find so congenial, accuracy and insight be damned.

The second is far more alarming and can be found in the survey’s URL, which announces with its “vic.gov.au” that it is now an official document and thus certain, absolutely certan, to be cited when justification is sought for more green handouts or, inevitably, punitive taxes and levies on those employers whose offices are, as yet, uncarpeted with green cant.

If only the Hawks had known what wonders motivated statisticians can work, they would have sent last night’s final score over to Melbourne University and had themselves declared normalised winners. Making stuff up, that's the new normal.


UPDATE: What a pity the survey did not include Ned from the US version of The Office this activist version of The Office in the sample of happy employees. Ignore the preaching at the end, and thanks to Dan Lewis for pointing out the error.

* Buildings do not come in waves. Benefits, unexpected or otherwise, are brought to, or bestowed upon, recipients. Why are the savings in cost and energy obvious? Where is the comma after occupants?