Anonymous user
Talk:Permutation test: Difference between revisions
made a note of the task specification change
(→Difference in results?: Using rational arithmatic.) |
(made a note of the task specification change) |
||
| (6 intermediate revisions by 4 users not shown) | |||
Line 73:
--[[User:Sluggo|Sluggo]] 02:06, 4 February 2011 (UTC)
===Using rational
I wondered about the accuracy of results after it was queried above and ran the Python solution but with rational numbers as input to see what effect this may have.
Line 88:
--[[User:Paddy3118|Paddy3118]] 03:22, 4 February 2011 (UTC)
:Both sets of results above appear to be incorrect. Do you believe the latter is correct, and if so, can you support it by listing statistics comparable to those above? I maintain that if you investigate it, you'll find that many of the large group of alternative means that should be exactly equal to the empirical mean are off by a little and therefore are counted among those that are greater or lesser. --[[User:Sluggo|Sluggo]] 23:43, 4 February 2011 (UTC)
:Using rational arithmetic, I get:
:<lang>under: 87.1972%
over: 12.8028%</lang> when I subtract mean control effects from mean treatment effects (experimental result is 1379 divided by 90), and I get
:<lang>under: 13.1417%
over: 86.8583%</lang> when I subtract mean treatment effects from mean control effects (experimental result is -1379 divided by 90).
:Have I made a logical error? --[[User:Rdm|Rdm]] 15:33, 4 February 2011 (UTC)
:: No. The problem is that it seems to be an ill-conditioned problem. IEEE arithmetic is getting in the way and causing all sorts of trouble. That's really nasty. The only good way to deal with this is to change the task so it doesn't have the problem, since the magic needed to fix it is evil and problem-specific (multiplying through by 100 pushes the figures into the stable range). Ugh. –[[User:Dkf|Donal Fellows]] 15:53, 4 February 2011 (UTC)
:::Actually, I was asking about how the large and small value reversed (between the under and over categories) in those two cases. --[[User:Rdm|Rdm]] 17:14, 4 February 2011 (UTC)
===Using integer arithmetic ===
Rationals are a bit of overkill when the denominator is always the same. I just posted a version using all integer arithmetic. The problem with floats is those 313 cases where the differences are equal. Anything that causes the difference to be off by the tiniest bit can cause them to be mis-categorized. I don't know enough statistics to know how this is typically handled. For this task, it might be enough to change the task description to provide experimental results as an integer score from 0 to 100. While we're at it, we could change the task to specify that the difference is treatment-control. That would eliminate the double solution.—[[User:Sonia|Sonia]] 20:12, 4 February 2011 (UTC)
:I'm in favor of amending the task as you recommend, and I'll do it if no one objects in the next few days or does it first. --[[User:Sluggo|Sluggo]] 23:46, 4 February 2011 (UTC)
== Name of task ==
Line 95 ⟶ 115:
: "Permutation test" is the conventional name for this test according to the Wikipedia, even though it may be a misnomer. --[[User:Sluggo|Sluggo]] 02:24, 4 February 2011 (UTC)
== Change in specification ==
In keeping with the developing consensus of opinion, I have changed the task specification to use integer test data, and specified that the difference in means is to be calculated by subtracting the control group mean from the treatment group mean. The correct results should be <math>12.80%</math> and <math>87.20%</math>. --[[User:Sluggo|Sluggo]] 21:05, 10 February 2011 (UTC)
| |||