PoliticsHome’s impartiality in doubt

Um, you can’t just include a bunch of people not on the panel just because it fits your line.

Laughable.

Dear Will,

I have seen your “new numbers” on Left Foot Forward for our survey about PoliticsHome’s ownership which we published on Wednesday. Thanks for your interest, but I’m afraid the “revised result” has no meaning whatsoever. I would be grateful if you would remove it.

Your method was simply to add twenty ‘no’ responses to the total and calculate new percentages. This is wrong in the following ways:

1. Each Phi100 result is weighted by party ID to ensure it’s fairness. This means the total numbers of left-leaning, right-leaning, Lib Dem and non-aligned respondents are always represent the same, even, percentages of the total result. This unweighted “new result” would have wildly too many left-leaning respondents as a proportion and is therefore completely out of kilter.

2. You are presuming that no one of the twenty resignations you refer to took part in the survey (incorrect).

3. The panel has a normal membership of 200, of which 70-odd took part on that day. At this point, other than those panellists who have resigned, we have no idea how the remaining 130 members would respond to the question. All it would take would be for more than twenty of those who did not take part that day to take the opposing view and the result moves in the other direction. To try to guess how opinion divides amongst a group you have not surveyed is clearly not a sensible approach.

I will respond to Sunder’s methodological questions later this morning.

Thanks,

Freddie

Hi Freddie,

Thanks for your clarification. I accept points 1 and 2 although presume that some of the 20 who resigned were lib dem or independent (which would dampen the “out of kilter” effect to which you refer).

On point 3, I think it is fair to assume that someone resigning from the Phi100 panel is expressing a public vote of no confidence equivalent to voting “yes” in your poll. It therefore appears that the true number of those confident that Lord Ashcroft’s investment would not “change the editorial line of the website” must lie somewhere between 42% (your number) and 33% (ours).

We have amended the headline accordingly.

Best wishes,

Will

Will

Thanks for the link. I agree with Freddie that you can’t simply aggregate those very public votes of no confidence into a result. I agree with you that full confidence must be 42% or lower of the panel without resignations. (The more important point – is perhaps that with right confidence fairly solid at 78% – almost four in five- is that confidence across the Labour/left, LibDem and non-aligned categories must be somewhere under 20% or one in five, without making allowances for some “boost” to this number because of resignations. Nor can this be said to be a partisan response, when there is no more non-aligned than Labour-leaning confidence).

However, I would also say that Left Foot Forward’s response to criticism can be contrasted rather favourably to that of PoliticsHome on this occasion. Freddie Sayer’s response to the methodological questions is to say they are rational, and then to fail to answer any of them.

This is fine if he is coming close to now saying the insider panel is “just for fun”, as he is making much weaker claims for it than have been made in both recruiting participants and marketing it before. I don’t see how its results could be credibly reportable by any other media organisation on the basis of his response here.

And I think there is a tacit acknowledgement that it is, in its present form, broken in the need to “refresh”

I thought it was unfortunate that he often caricatures the questions put, in order to avoid answering them. The question was not, of course, whether there was an exact methodology for expert opinion. It was whether the liberal-left voices were “broadly credible as broadly representative” to independent expert voices exercising “qualiitative political judgement”.

The idea of inviting expert external opinion to assess that was simply ignored. (I am not an expert:

My post noted that this depends on qualitative political judgment; when PoliticsHome set up the panel, they believed it was important to have expert voices contributing to that. Having lost the confidence of those expert voices, they do not think the same methodology used to set it up as credible is needed to maintain credibility, despite the context.

If I was told that the method of dealing with the refusenik issues is endorsed by leading academic and practioner voices outside PoliticsHome, and that the make-up of the group as it stands has the confidence as “broadly representative” of John Rentoul, John Kampfner, Peter Oborne and Daniel Finkelstein, I would feel that the issue had been satisfactorily dealt with.

The question was whether the panel could sensibly be asked major political questions (funding, constitutional reform) and the (rather silly) reply is they won’t ask people about their relationship with Mr Rawnsley (or, presumably, whether they are funded by Mr Ashcroft too).

Their own results make more – not less – transparency at PoliticsHome urgent, and the decision to plough on on the basis of editorial “pride” without making any attempt to address the perfectly sensible questions very questionable. To mention “transparency” and fail to provide any attempt at external engagement or verification seems to me the worst approach of all.

Sorry – here is the link to what PoliticsHome published as Freddie Sayer’s “response to Phi100 research methodology questions“

Ashcroft has something of a record when it comes to the independence of websites.

http://www.caribbeannetnews.com/breaknews-15371–37-37—breaknews.html

So can we expect a full review of the Baron’s business affairs in Turks and Caicos and Belize on this site in the future. Including an explanation of why he and his companies made cUS $10m of loans (look at the evidence to the Auld Commission – still available on the web) to Michael Misick the former Prime Minister of T&C who did not have sufficient observable income to repay the loans and is now being prosecuted for corruption?

I wait with bated breath!